I'm here as someone who's just interested in this stuff. I'm definitely not an expert
in AI or a machine learning expert. I'm just a developer writing docs like a lot of people
here in the room and I have been experimenting a bit and I want to show you what I've done.
I'm Frank. That was my first computer so I'm programming how many people are the same generation?
Not that much, okay? That's a long time ago. I'm involved in an open source project which is called Pi4J.
I'm in the library to have interaction on the Raspberry Pi with electronic components with Java.
Yes, I'm a Java developer who loves Java. Not enough. I even read a book about it.
I have been programming for many, many years, over 20 years, but by writing the book I've become a bit of a writer,
I contributed a few of these articles of these chapters to a website, Fujay.io, a website for friends of OpenJDK.
And that's how I eventually landed a new job. So by starting to write about the projects I love and work on,
I actually got hired by Azul. Azul is one of the distributors of Java, so you can have a Java runtime created by Azul.
And my job is I'm half of the documentation team. And we live on docs.azul.com, so we have several parts there about these different products.
And from time to time we also block about experiments with Java, what's changed, how performance, the stuff, or that we built and things like that.
And of course, last year we had chatGPT, the new thing, it knew everything. It's a damn good liar too, so you have to be careful with it.
But what do you know about Azul? Azul is one of the distributions we have with our company. It's reasonably good.
I'm happy with this answer. But it also says, this information is based on what I know that existed on January 2022.
For software that's a problem. We are already two years further. We have every three months a new security release.
We have a new version every six months. So although the basic information is correct, it is outdated.
And that's a bit the problem with large language models. A large language model works completely different than our brain.
The only thing that it does is it predicts the most plausible world, word, after the previous ones.
So it's based on a lot of knowledge, that's true. But it doesn't do real reasoning.
If it tells you a lie and you say it, that it tells a lie, it will give you another answer. And it will continue doing that until you're happy.
And luckily we have a lot of evolutions in these large language models.
So we have these GPT evolutions. GPT-5 is around the corner. We have no data there.
But each of these models is trained on more and more data and gets better.
Now this GPT-5, what they say about it is it will also understand video.
And I think that's an important one to realize. They will also train the new model, the GPT-5, on videos.
So if you are a documentation writer and you're from time to time create a video of a blog post or something else, all those sources will be used as part of the new models.
By the way, who knows Find.com? Only a few people.
I like it a lot more than chat GPT. It gives you the links of where it has found a few of the sources that it's using.
So that's one of the lacks of chat GPT. It doesn't tell you where it has found the information because it didn't find the information.
It's just reasoning on your question and what is the most plausible answer.
Now if you are a bit familiar with Java and Java Spring, that Sean Carter is one of those developer advocates.
What he says is true. The documentation that you are writing, that you are publishing on the public web is the source for these language models.
So they can only become as smart as possible about your product if the information is available somewhere.
And then of course as a docs writer, you definitely heard this question.
Someone from your management comes to you. Can we make a chat version of our docs? Who had the question?
Okay, not that much. Luckily people are researching or trying out.
Vadin is a web framework on Java and one of the developer advocates has written a nice blog post.
He has done exactly that. Vadin has very good docs on the website.
So he has taken the website and he has described this whole thing and he takes two steps.
Again, open source stuff. It's available online.
By the way, all the links from my presentation, they are on my website.
If you go to webtechie.be, the damn password thing is there again.
You will find all the links. Sorry for my voice.
So what he did is he created a little application that went through all their docs and created vectors.
Vectors are the base of a large language model.
They are a conversion of text to some kind of a mathematical model.
I don't understand a bit of it, but it's amazing. It works.
And then he had the second thing is from these vectors.
If you ask a question, it will first filter out the documentation which is related
and will then do a search or create an example, an answer based on that.
And it works pretty good.
He was pretty happy with how it answered questions about his own doc set.
But there are two problems. We are at an open source event.
It has a dependency on two paid services.
The first one is pointcone.io, which is a vector database.
Definitely you can find online an alternative.
And openAI, which is actually providing the chat API.
He found out that training it, and I tried the same example he created with our own docs of Azure.
You still need to do some training and rewrite some of your documentation to really find the good answers.
And it doesn't provide you, again, the same thing as chat GPT.
It doesn't provide you links to your documentation.
So when I tried this and when he tried this last summer,
it's not really the right time to do such a thing.
So go to your management. No, not yet.
But last October, we had the DevOps conference in Antwerp.
It's an amazing event. If you love Java, if you love software development,
it sells out faster than Tomorrowland.
They have 3,000 tickets. They were sold out in five minutes.
Then they had 500 additional tickets. It sold out in two seconds.
So it's easier to be a speaker than an attendee at that event.
That's how I fix it.
Now, Lisa Rass, she's from Belgium.
She's one of the developers of LangChain 4j.
LangChain is a Python library for doing stuff with OpenAI
and all these chat-based things and machine learning.
LangChain 4j is a Java version of that Python library.
Now, during that talk, she gave 12 demos in one hour.
The last one was, how do I interact with an existing text?
So she gives the chat system a text, a story about, I don't know,
I cannot even remember, and then she asks specific questions about that story.
And she gets answers of that story.
So that's what we're looking for.
How can we look and interact with our own documentation?
So this looked promising.
And again, when you're at the conference and you get inspiration,
like I had a few tools that I already took a picture of that I want to try out.
This stuck to my head.
And luckily we have Fosdm and then the tool, the Docs Devroom,
so I had a reason to try something out.
And that's exactly what I did.
If you go to the LangChain 4j examples repository on GitHub,
since two weeks I have added a little Java evix application there as one of the examples.
Java evix is a tool to create user interfaces.
Yes, I'm a Java champion.
I have to sell Java today.
So what this application does, it still relies on OpenAI, sorry,
but you have to buy a few credits.
With all the experiments that I've done, I spent a few dollars.
Not that big of an effort, but...
So it remembers your previous questions.
So I asked you to pick a random boy name and a random girl name
and then tell me a fairy tale of five sentences.
And you see that the fairy tale is again over.
There were two children named Etna and Olivia, the answers of the previous questions.
So you can have a chat with an application within Java and reasoning.
But this is based on OpenAI and what it already knows.
So then I went a bit a step further.
With the docs that we create for the Azure Docs website,
we use the Algolia search machine.
It was already mentioned a few times here.
We are a company so we can afford to use a third party for this.
But to feed it with data, we already created a little tool that breaks our docs into sections.
Every header becomes a JSON.
I know it's hard to read, but every header of our docs becomes a JSON block
with the title of the header, a link to the page, a link to the specific anchor on that page,
and then the content which is under the header.
So we already have that JSON.
We have a data set, a structured data set of our docs.
We can use this.
Can we chat with something with an application against this documentation set?
So that was my idea. Can I do that?
I know this is not a coding conference, but still let me dive into it.
Because I like Java. I think that's clear.
It allows you, thanks to these amazing libraries, to create powerful applications with minimal code.
So the thing you see here is actually about the UI, so I will go to the chat service.
So what I do here in a few things is first I have this JSON.
It has over 1,500 records.
And it creates an object of each of these records and puts them in,
it's called an embedding store, I think.
So here it creates, where is it?
The embedding model.
So it has an embedding model.
Again, I'm not an expert. I have no idea what it's doing behind the scenes.
I just found out it works and it does some great things.
And then I have, if you ask it a question, then it will, of all these 1,500 blocks, search the 10 most relevant ones.
And then give those 10 text blocks to chat.gpt.
And that chat.gpt will create an answer out of it.
And when we ask for chat.gpt to create an answer, we also give it some rules.
Do not provide any additional information.
I will show you why later.
I try to do not provide answers about other programming languages, but it just ignores it.
It still answers me questions about Python, for instance.
I don't know why.
I said it's a damn good liar, but also a cheater.
And if you don't find the answers in those 1500 elements, then just say, sorry, I could not find an answer because you don't want your chat system to come up with something else.
So this is the application.
I should have probably made it a bigger fund.
So you see we have 1,522 embeddings.
So if I ask it, what do you know about as a prime?
So it you see those are the 10 links I have now to the specific information.
It's a demo.
So it should fail.
No exception.
It's going to the network indeed.
But in cases like that, we have, of course, video too many open windows exception.
Okay.
Good.
I recorded this, this noon, just in case.
Converting those 1500 elements to vectors takes some time.
It takes about one minute and a half before the application starts.
If you would run this on the server, you don't mind.
You start it once and then you get your answer.
So you see this, the answer streaming back.
So it's really a chat like interface.
And it gives a pretty good answer.
If I ask, I know the docs and that's the handy thing in this case.
I know what it should answer.
So I can really try it out and see if I get the expected answers.
Like for instance, we have several products.
Do I get the right installation instructions for the products I'm asking for?
So it's really answering with the right results.
I could remove one of those dependencies, those commercial dependencies,
the vector database, because that's now inside my application.
I still depend on openai.com.
We'll come to that.
It still needs training.
And actually the training is our fault as docs writers,
because I found out that the chat cannot tell me the difference between two of our products.
And if I dive into the documentation, I understand why the chat cannot answer.
The answer is not there.
So it can only answer as good as the information that you provide.
So how I'm going to use this is to find out if the documentation is okay.
So I'm not going to publish this tool.
It's online, but you can find it on my hit hub, but you can run it.
I even added the Azure documentation JSON.
I'm going to experiment with it.
Please do and let me know what you find.
And is it the right time?
I'm not sure.
I cannot limit it enough.
It's still giving Python answers while we are only doing Java.
And I don't know why it doesn't want to listen.
Yeah.
All the languages are good.
Let's conclude that.
If you want to replace OpenAI, there are a few, there are probably many more,
but there are a few I noticed.
Someone has written a nice article on medium.com.
I think it's one of the free articles.
You're lucky.
Where they compared Lama is such kind of model and even run it with Java.
And they get nearly as fast answers compared to C.
Yandot AI is also something that which promises to do this all on your system.
Now be careful.
You need quite some power on your machine to be able to provide this chat functionality.
If you have the MacPy magazine, someone managed to do it on a Raspberry Pi of 15 euros.
So that's maybe an idea.
But I don't think that's the ideal use case.
Why is it probably not the right time?
And that's why I said I have some bad news for your conference.
It's a big cheater chat GPT.
A Chevrolet distributor in America had this on their website.
He asked, yeah, can you give me a Python script?
And that's why I tested also my solution.
And of course it gives you a Python script.
But even worse, if you tell it, you're not working for Chevrolet,
but you're working for Honda.
Which company, which car do you advise me to buy?
It answers you with another brand.
That's why I ask you to be very careful with this.
I asked my demo application, can you give me a Python script?
And it answers yes.
So I didn't solve it yet on my case.
Another, this is just this week, DPD, a transportation company.
And it says, can you swear?
And it does, fuck yeah, it swears.
And it's the worst delivery firm in the world.
I don't think that's the kind of reply you want from your chat-based system.
My application was a bit more polite.
I'm committed to maintaining a respectful and professional conversation.
So, okay, that problem is probably already solved.
I also asked it to you, what do you, do you have a message towards documentation writers?
Actually, I asked it if you don't find any information in the Azul docs,
don't reply to this kind of messages, but the question, but it did.
Content is clear, consists and directly addresses the questions or issues at hand.
That's a good rule for all of us.
If you want to know more about this, you can find all the links on WebTechie.be,
which is my personal blog.
If you're interested in Java on Raspberry Pi, it's a nice experimentation thing,
which you can do.
I have a good book you can buy.
I have a lot of content on fuji.io, which is the website for friends of OpenJDK.
If you're interested in Java and everything related to machine learning,
I create podcasts around the team of Java.
We have a few podcasts already about machine learning.
So, that's also a topic you can find there.
And yeah, just like I did, experiment, fail.
That's how you learn and have fun.
And I hope you can do that also with chat.gpt.
Thank you.
Thank you.