For the general player base.

This patch fixed a major issue where the llama2 model fails majorly at misclassification of the user prompt of a certain type. That part has been taken up by mistral instead.

Aaaand after 2 weeks of a lot of coffee, back pain and brain ache, here's the new update.

At first it may seem that this version doesn't do much, but keep talking with the bot and you will find out :)

To-do: Install three models in ollama. [VERY IMPORTANT]

1. nomic-embed-text (ollama pull nomic-embed-text)
2. llama2 (ollama pull llama2)
3. mistral (ollama pull mistral)

And go to your game folder (.minecraft/config) and delete the old settings.json5 file.

In game, change your selected language model to llama2 for the best performance.

TLDR: The bot is way more intelligent now, can remember past conversations, can store current conversations and then use the data for long term storage, has gotten better at function calling like movement and block detection, no fixed responses, no need of fixed prompt checking, it all happens,

Also the connection to the language model is only made when the bot is spawned in game, thus saving a lot of memory when the bot is not in the game.

Eventually I intend to bring support for custom models on the ollama database.

dynamically, hehe.

Next patch will include more minecraft interaction, such as mining, improved movement, collision/obstacle detection, etc.

Accuracy for detecting correctly what the player wants to say: somewhere between 95-99%

---

For the nerds.

So, for the tech savvy people, I have implemented the following features.

LONG TERM MEMORY: This mod now features concepts used in the field AI like Natural Language Processing (much better now) and something called

Retrieval Augmented Generation (RAG).

How does it work?

Well:

We convert the user input, to a set of vector embeddings which is a list of numbers.

Then physics 101!

A vector is a representation of 3 coordinates in the XYZ plane. It has two parts, a direction and a magnitude.

If you have two vectors, you can check their similarity by checking the angle between them.

The closer the vectors are to each other, the more similar they are!

Now if you have two sentences, converted to vectors, you can find out whether they are similar to each other using this process.

In this particular instance I have used a method called cosine similarity

Cosine similarity

Where you find the similarity using the formula

(x, y) = x . y / |x| . |y|

where |x| and |y| are the magnitudes of the vectors.

So we use this technique to fetch a bunch of stored conversation and event data from an SQL database, generate their vector embeddings, and then run that against the user's prompt. We get then further sort on the basis on let's say timestamps and we get the most relevant conversation for what the player said.

Pair this with function calling. Which combines Natural Language processing to understand what the player wants the bot to do, then call a pre-coded method, for example movement and block check, to get the bot to do the task.

Save this data, i.e what the bot did just now to the database and you get even more improved memory!

To top it all off, Llama 2 is the best performing model for this mod right now, so I will suggest y'all to use llama2.

In fact some of the methods won't even run without llama2 like the RAG for example so it's a must.