Yes they are very integrated into the environment of my game, they are aware of as many things as I can think of, such as their body shape, the clothes they are wearing, the equipment they have, a description of their abilities, inventory, they have over 120 unique commands and infinite variety in their actions based on the environment, they are aware of the player’s abilities. They also have procedurally generated hobbies, interests, and backstory based on a seed derived from their name. This makes the generated data the same for that name everytime it is created.
The main focus currently is expanding the commands and possible interactions. I’ve currently disabled external AI APIs for the local model’s testing. The entire AI made fully in ROBLOX studio can be interacted with at this link, Lumina & Darkness: Testing Server - Roblox
I’m currently creating an animated hair library for the next few days, and have temporarily disabled the procedural world generator to focus on the local AI.
The local AI is a Retrieval based chatbot that generates data based on its surroundings and character. The local response is given to the player immediately then it is injected into the system message of the LLM along with its surroundings, memories, long-term memories, identity and also this new action context system that focus’s on the AI actions and situation,
The local AI has several specialized nodes that are called context databases that are scored based on the input is encoded into a synonym, noun, reflection and antonym array, antonyms decrease the score of the entry, synonyms represent words of similar meaning, nouns reflect the most important section of the input and reflection flips the tense to such as “I” reflects to → “you” and emotional tonality. Then score is divided by the weight (amount of words in the entry) and activated with math.log to create a scoring curve. I have tested various other calculations and have most success from the current scoring algorithm using sigmoid and log.
In short it’s RAG, and retrieval based chatbot and yes I have explained this before.
You ask it “What are your hobbies?” ->chatbot successfully responds with its hobbies–node is procedurally generated hobbies
“How are you doing?” → respond with how it’s doing–emotional state node
"What’s your name? → responds with an introduction and it’s name.–greetings node
“What do you see?”-> respond with an entry from the awareness node
all these nodes are organized via a context matrix that is based on the entries score, repetition and its bias function which either increases the weight of the node decreases it with several levels of intensity. This is used to maintain the flow of the conversation, an example is the greetings node decreases in weight after it is called because it mainly handles introductions, this has been tuned to human preferences, and has contributed to major improvement in quality of the outputs.
They are also connected via their context matrix which tries to find an entry that is related to the input and the connected nodes output.
ok the text filter works there was a documentation on it. I published my game and I guess I will see how it goes.
1 Like
Share the code! or did the text filter function I shared work?
Also, I highly recommend you filter input to the model, if you want to filter the output I would suggest doing it one sentence at a time and concatenating the result.
For the output i just filtered the whole thing so sometimes its all tags. I just made a basic one from the documentation…
I always use AI for creating string parsing functions. I would suggest the prompt “Write a function in the context of Luau in ROBLOX that takes a string and separates into a table of sentences and returns it.”
then filter each sentence, with your function and if it returns false then table.remove(sentences,i) .
And example of how the output looks is this
Query “Who are you?”
Output={
[1] = {
["directory"] = "Personal",
["emotion"] = "Polite",
["match"] = "I like to view myself as a admirable Valkyrie. I carry a Magical Heart Staff that was presented to me by my friend.",
["score"] = 1.707025003849642
},
[2] = {
["directory"] = "Awareness",
["emotion"] = "Epic",
["match"] = "I see you before me now. I can sense my ally, Arch- Mage Jaylene.",
["score"] = 1.3244
},
[3] = {
["directory"] = "Emotions",
["emotion"] = "Epic",
["match"] = "I'm feeling incredible! I have a lot of goals and dreams that inspire me.",
["score"] = 1.03405
},
[4] = {
["directory"] = "Greetings",
["emotion"] = "Happy",
["match"] = "I'm delighted to meet you.",
["score"] = 0.6124000000000001
},
[5] = {
["directory"] = "Therapist",
["emotion"] = "Question",
["match"] = "What do you think?",
["score"] = 0.4796
},
[6] = {
["directory"] = "Judgement",
["emotion"] = "Question",
["match"] = "Don't you think your strength is very weak?",
["score"] = 0.3485
},
[7] = {
["directory"] = "Database",
["emotion"] = "Scary",
["match"] = "As the veiled Goddess, I represent the mysteries of life, death and the realms beyond mortal ken. My secrets unfold gradually to initiates proving themselves worthy.",
["score"] = 0.05
}
}
Some entries don’t meet the accuracy threshold and are affected by the personalities’ mood.
It’s really quite good.
Any updates to the system? I have not checked this thread for a while.
This project was designed as an example of using the libraries I published, it is complete. No updates currently but the wiki feature is compatible with this model to interact with books. Procedural Book Shelf Filling Algorithm + Library [Open-Sourced]
There have been some new similarly sized LLMs released recently that are designed to be tool users. I have a bunch of tools I was not able to include with this project. But once I find an Open-Source model that can do tool function calls like Chatgpt. Once that time comes I’ll publish the agentic tools I developed for GPT-4.
Some planned updates are coming to the Boids library which include more 3-D models of birds, dragons. Additionally, the emotes should be expanded soon to include more emotes.
Also, the open sourced awareness is about to be upgraded to include planning using this new algorithm I wrote using the library to generate plans and goals given the environment. It’s very interesting, I look forward to publishing it soon.
1 Like
What is a tool function? Also isn’t GPT-4 paid.
Also If you’re interested in exploring other models there has been a wave of new releases on hugging face including Google Gemma 3, Meta LLama 3 versions of all their models and a few others. May be worth checking out.
1 Like
how do i do that. do i just add the api url?
