But i am using mistral AI 7b, however the response table that i get in return says that the post async didn’t go well.
(i slightly edited the code in the meantime)
local HttpService = game:GetService("HttpService")
local endpoint = "https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3" -- Replace with your actual endpoint
local apiKey = "Bearer ffaasfdasfaf" -- Replace with your actual API key
local function format_response(str)
-- find the last assistant response in the string
local start = string.find(str, "<|assistant|>", nil, true)
if not start then return str end -- If "<|assistant|>" is not found, return nil
-- Find the last occurrence by searching backwards from the end of the string
local last_start = start
while start do
last_start = start
start = string.find(str, "<|assistant|>", start + 1, true)
end
-- Calculate the end of the last "<|assistant|>" tag
local finish = string.len(str)
-- Extract the response after the last "<|assistant|>"
local response = string.sub(str, last_start + 13, finish)
-- Return the response
return response
end
function query(input, system_message,history)
local system="<|system|>\n "..system_message.. "</s>\n"
-- {
if history==nil then
history=""
else history="<|user|>\n "..history
end
local npcdata={
inputs = system..history.."</s>\n<|user|>\n "..input.."</s>\n<|assistant|>\n",
max_new_tokens = 256,
do_sample = true,
temperature = 1.2,
top_k = 30,
top_p = 0.90
}
local response = HttpService:RequestAsync({
Url = endpoint,
Method = "POST",
Headers = {
["Content-Type"] = "application/json",
["Authorization"] = apiKey
},
Body = HttpService:JSONEncode(npcdata),
})
print(response)
local result=HttpService:JSONDecode(HttpService:JSONEncode(npcdata))
print(response)
return response,history--HttpService:JSONDecode(format_response(response.Body),
end
task.delay(1, function()
query("What color is an apple?", "" ," You are a companion who's avatar has an AI that is beside the player as their party member in a fantasy RPG game. ")
end)
OOh you are asking for the api endpoint? I already left that somewhere on this thread, not to mention it’s in the source file.
I have a copy of that particular module in workflow right now. The endpoint is
local endpoint = "https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3" -- Replace with your actual endpoint
Also quickly delete your post because you leaked you api key.
Also zephyr endpoint is
local endpoint= "https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta"
Zephyr has much shorter context window, but it’s a bit more charming than mistral.
In my custom code I 1 shot a answer from zephyr, then complete the response with mistral.
This endpoint dosent work either sadly… I still get the cannot post error.
1 Like
Mistral you have to go to the model page i linked and agree to the terms of use for that model. Some groups like google, facebook, and others request that you accept the terms of use before you can use the model.
Looking at the documentation endpoint urls are constructed like this
ENDPOINT = https://api-inference.huggingface.co/models/<MODEL_ID>
Model id being a placeholder for the components shown above. The code worked when it was made, and so far everyone else has had no issues.
To clarify the linked model is
Be sure to agree to the terms of use to use that particular model.
Ohh ok i must have used the wrong endpoint i presume. Now i get way less errors. Now i only get an authentication error saying that my token is invalid
1 Like
Yeah api’s are confusing at first but it gets better. I left instructions on how to get a new token in the 2nd post on this thread. if that helps you, good luck with your project, feel free to reach out.
if you are logged in to huggingface you can use this link as a shortcut to the tokens page.
https://huggingface.co/login?next=%2Fsettings%2Ftokens
Make sure your token is read only for this use case.
Hey! Have you ever made a AI fully in Roblox Studio?
I dont know if you have addresses that question before, sorry.
If so, what was It? And how did It works?
Yes they are very integrated into the environment of my game, they are aware of as many things as I can think of, such as their body shape, the clothes they are wearing, the equipment they have, a description of their abilities, inventory, they have over 120 unique commands and infinite variety in their actions based on the environment, they are aware of the player’s abilities. They also have procedurally generated hobbies, interests, and backstory based on a seed derived from their name. This makes the generated data the same for that name everytime it is created.
The main focus currently is expanding the commands and possible interactions. I’ve currently disabled external AI APIs for the local model’s testing. The entire AI made fully in ROBLOX studio can be interacted with at this link, Lumina & Darkness: Testing Server - Roblox
I’m currently creating an animated hair library for the next few days, and have temporarily disabled the procedural world generator to focus on the local AI.
The local AI is a Retrieval based chatbot that generates data based on its surroundings and character. The local response is given to the player immediately then it is injected into the system message of the LLM along with its surroundings, memories, long-term memories, identity and also this new action context system that focus’s on the AI actions and situation,
The local AI has several specialized nodes that are called context databases that are scored based on the input is encoded into a synonym, noun, reflection and antonym array, antonyms decrease the score of the entry, synonyms represent words of similar meaning, nouns reflect the most important section of the input and reflection flips the tense to such as “I” reflects to → “you” and emotional tonality. Then score is divided by the weight (amount of words in the entry) and activated with math.log to create a scoring curve. I have tested various other calculations and have most success from the current scoring algorithm using sigmoid and log.
In short it’s RAG, and retrieval based chatbot and yes I have explained this before.
You ask it “What are your hobbies?” ->chatbot successfully responds with its hobbies–node is procedurally generated hobbies
“How are you doing?” → respond with how it’s doing–emotional state node
"What’s your name? → responds with an introduction and it’s name.–greetings node
“What do you see?”-> respond with an entry from the awareness node
all these nodes are organized via a context matrix that is based on the entries score, repetition and its bias function which either increases the weight of the node decreases it with several levels of intensity. This is used to maintain the flow of the conversation, an example is the greetings node decreases in weight after it is called because it mainly handles introductions, this has been tuned to human preferences, and has contributed to major improvement in quality of the outputs.
They are also connected via their context matrix which tries to find an entry that is related to the input and the connected nodes output.
ok the text filter works there was a documentation on it. I published my game and I guess I will see how it goes.
1 Like
Share the code! or did the text filter function I shared work?
Also, I highly recommend you filter input to the model, if you want to filter the output I would suggest doing it one sentence at a time and concatenating the result.
For the output i just filtered the whole thing so sometimes its all tags. I just made a basic one from the documentation…
I always use AI for creating string parsing functions. I would suggest the prompt “Write a function in the context of Luau in ROBLOX that takes a string and separates into a table of sentences and returns it.”
then filter each sentence, with your function and if it returns false then table.remove(sentences,i) .
And example of how the output looks is this
Query “Who are you?”
Output={
[1] = {
["directory"] = "Personal",
["emotion"] = "Polite",
["match"] = "I like to view myself as a admirable Valkyrie. I carry a Magical Heart Staff that was presented to me by my friend.",
["score"] = 1.707025003849642
},
[2] = {
["directory"] = "Awareness",
["emotion"] = "Epic",
["match"] = "I see you before me now. I can sense my ally, Arch- Mage Jaylene.",
["score"] = 1.3244
},
[3] = {
["directory"] = "Emotions",
["emotion"] = "Epic",
["match"] = "I'm feeling incredible! I have a lot of goals and dreams that inspire me.",
["score"] = 1.03405
},
[4] = {
["directory"] = "Greetings",
["emotion"] = "Happy",
["match"] = "I'm delighted to meet you.",
["score"] = 0.6124000000000001
},
[5] = {
["directory"] = "Therapist",
["emotion"] = "Question",
["match"] = "What do you think?",
["score"] = 0.4796
},
[6] = {
["directory"] = "Judgement",
["emotion"] = "Question",
["match"] = "Don't you think your strength is very weak?",
["score"] = 0.3485
},
[7] = {
["directory"] = "Database",
["emotion"] = "Scary",
["match"] = "As the veiled Goddess, I represent the mysteries of life, death and the realms beyond mortal ken. My secrets unfold gradually to initiates proving themselves worthy.",
["score"] = 0.05
}
}
Some entries don’t meet the accuracy threshold and are affected by the personalities’ mood.
It’s really quite good.
Any updates to the system? I have not checked this thread for a while.
This project was designed as an example of using the libraries I published, it is complete. No updates currently but the wiki feature is compatible with this model to interact with books. Procedural Book Shelf Filling Algorithm + Library [Open-Sourced]
There have been some new similarly sized LLMs released recently that are designed to be tool users. I have a bunch of tools I was not able to include with this project. But once I find an Open-Source model that can do tool function calls like Chatgpt. Once that time comes I’ll publish the agentic tools I developed for GPT-4.
Some planned updates are coming to the Boids library which include more 3-D models of birds, dragons. Additionally, the emotes should be expanded soon to include more emotes.
Also, the open sourced awareness is about to be upgraded to include planning using this new algorithm I wrote using the library to generate plans and goals given the environment. It’s very interesting, I look forward to publishing it soon.
1 Like
What is a tool function? Also isn’t GPT-4 paid.
Also If you’re interested in exploring other models there has been a wave of new releases on hugging face including Google Gemma 3, Meta LLama 3 versions of all their models and a few others. May be worth checking out.
1 Like
how do i do that. do i just add the api url?
