could you upload a video of the trained model to see if it works?
1 Like
If you are using the source code beneath the video I posted in the first post, it already have an auto-save feature.
Anyways, if not, just call getModelParameters() to get a copy of the trained weights. Then you can set it back using setModelParameters() to put back the trained weights.
Depending if you want to save the weights online or offline, here are the steps.
Online
Once a copy of weight is made, you can store it directly to roblox’s DataStore.
Offline
Require the Matrix Library you just linked, and call the printPortableMatrix(). You might need to update your matrix library if you don’t see this option. Copy it to text editor.
It’s already in the video of the first post. It shows it works.
Update:
Made the neural network much more faster by removing unnecessary calculations in backpropagation. Instead, it uses forward propagation values and apply it to backpropagation derivatives calculations.
2 Likes
Question, how big can the tables like get for training? Like how fast could the tables fill up before it reaches over the data store limits
That depends on the number of features and the number of training data you have.
Currently, each key can store 4 MB of data for DataStore as stated in Roblox’s documentation.
In my experience, I had stored roughly around 10,000 data with 14 features for my ChaWatcher model training. It was stored as pure table, and not table converted to string.
So, I’m guessing you have plenty of storage that you don’t have to worry about.
Theoretically, lets say that the scripting for my hitboxes and animations e.t.c are all very optimised. How performant is it if I wanted to have, say, 20 NPC’s fighting in the same server?
I’m going to assume that you want to use neural networks with reinforcement learning capabilities.
In that case, it would depends on the number of layers, number of neurons and the activation functions. Smaller neural networks architectures uses less CPU usage, and the least computationally expensive activation function would be the LeakyReLU.
I made a recent library update that removes unnecessary calculations during training. This should give you twice the performance because I have eliminated half of the calculations needed to update the neural networks.
If you still having issues with script exhaustion time, have a look a setWaitDuration() function in BaseModel class as a solution for this.
Anyways, I think it is better for you to see how it performs with your own eyes, and you decide if it is performant or not.
1 Like
Heads up! New update is being created! Expect new kind of experience replays and some refactored code in the next update!
1 Like
Release 1.4:
Released 2 experience replays (PrioritizedExperienceReplay and NStepExperienceReplay) and 2 new neural networks (Double Deep SARSA and Double Deep Expected SARSA)
2 Likes
In short you evaluate a Corpus of text that is formatted into sentences. You can add additional vectors to the model and do a more sohpisticated inference by say incorporating it’s position in a sentence. By saying the words[i]as “i” is its position in the word array, Because this example was just a word frequency model. But when training a neural network you need to be able to feed it features to calculate.
You can use this bag of words model provided insight from Claude2 which intook the code with its 100,000 token window.
-
It’s using a basic bag-of-words model to generate text, which can work to an extent but often lacks coherency. The predictions could be improved by looking at word order and grammar as well.
-
Training the model on various corpora of text is a good idea to expand its vocabulary and knowledge.
-
The overall structure makes sense - training the model, then using it to predict responses. With some refactoring and enhancements to the prediction logic, this could be a decent starting point for a chatbot.
-
Using a probabilistic model to choose words based on frequency is reasonable, but risks repetitive responses. Adding some randomness could make things more interesting.
There are a few different machine learning techniques that could potentially be incorporated into this chatbot model:
-
Recurrent neural networks like LSTMs or GRUs: These are very common for natural language processing tasks like text generation. They can learn complex linguistic patterns and long-term dependencies in text.
-
Convolutional neural networks: CNNs are effective at extracting local patterns and features from text. They could be used to classify the user’s intent and extract keywords to inform the model’s response.
-
Reinforcement learning: The model could be trained to maximize a reward like engagement or coherence, using RL to optimize its dialogue strategy.
The current bag-of-words approach has limitations, so transitioning to neural models like LSTMs or transformers would be a major upgrade. The model size and training process would need to be scaled up accordingly. Overall there are many sophisticated ML techniques that could take this from a basic statistical model to a highly intelligent conversational agent.
Update Release Version 1.5:
-
New Models: ActorCritic, AdvantageActorCritic and AsynchronousAdvantageCritic.
-
New NeuralNetwork Activation Functions: Gaussian, SiLU and BinaryStep.
-
Refactored forwardPropagate() and backPropagate() so that it can be used similarly to PyTorch’s backward().
-
Added DistributedLearning.
4 Likes
Also, updated terms and conditions.
Got my head itching on this new “Mish” activation function. I have added it to the Release 1.5 neural networks. Should perform really good!
2 Likes
Alright folks. I’m getting burned out from adding new features, excessively testing and excessively backup-ing my works.
For the next two-three months, the library will not receive any updates so that I can recharge. I haven’t had some entertainments for like almost a year now since I was so focused on improving this library.
The next update will focus on adding REINFORCE and Dueling DQN.
Any new feature requests will be ignored, but bug reports and questions about the library are always welcomed.
Here’s my to-do list for recharging for those who are interested:
- Play Genshin Impact and explore the new Fontaine map that I have missed.
- Play Blue Archive
- Probably focus on my YouTube channel and add some more video stuff as well.
Thank you very much for all the support you all have given me and the feature requests that made the library’s popularity exploded.
8 Likes
Alright guys, I have realized something.
The self-learning AI (that uses neural networks) takes a lot of time to train and it would be very wasteful to keep on training new models for our own needs. So, I wonder, would you like me to create a repository where we put our pre-trained neural network models that we all can share and modify for our own needs?
Give a like on this post if you want it! (You can reply to it if you want to).
12 Likes
I think that’s a great idea! Sharing an additional functional example would be intuitive as well.
I have disappointing news.
Summary
This library’s neural network is twice as slow than Kironte’s neural network on average.
Here’s the code for benchmarking.
Benchmark.rbxl (200.8 KB)
It’s okay if yours takes… a microsecond longer than somebody elses. We’ll live! You’ve worked so hard on this, and I wouldn’t want you to stress over a small number not being as small as you’d like.
4 Likes
I have a quick question.
I’ve looked at the documentation and I felt kind of lost. I’ve looked into some neural networking tutorials and I kinda get how it works [Forwardpropagation - gives output, backward prop - adjusts weights]. I’m not an expert on neural networking however and I have a question on how I’d go about creating a training model free (So AI that can learn on its own without a training dataset) for NPCs/AI that can do stuff such as learning how to walk. Are there any resources that you’d recommend that I’d look at? Also how would I go about creating the environment for the NPC (Like how would I create an NPC that can manipulate it’s joints while being influenced by roblox’s physics)?