Instant Inference possible by Groq, Not from the musktron (they now are called xAI, cuzzz X) because it was already taken by a company that gives free instant inference with the 8B Open Source LLAMA3 from Meta and the Zuck. You just need to get your API key from their “Playground”.
Requirements are simply a FREE api token from Groq (not xAI)
The reason for this api is simple, HTTPService (sucks) doesn’t allow you to stream in data over http protocol. I was thinking to maybe make a simple proxy server which could connect thru Socket.IO or simply websockets. BUT these roblox library are not up to date and dont work with newer socket protocols. So we need INSTANT INFERENCE. this is somehow free as well without need of using a local ollama server that maybe puts out 5 words per second. Also 64 cores are more efficient then using a GPU but thats never mentioned. Im yapping.
results in:
I whish you good luck with making Use of spiritual AI communication for dynamic ROBLOX NPC’s
Also getting this API token is FREE, you must look at their limitations on their website. search for Groq playground on google. there you can test the platform as well as get your requirements.
Get it here:
Enjoy!