…Wow. I’ll be honest with you, that’s pretty damn impressive. Huge +1 for this.
I may make extensive use of this in my game for certain NPC behaviors. If I do, you’re definitely getting a huge shoutout for it, should the module cater to my precise needs (which, from what reading I’ve done, it most certainly should).
Outstanding work. I’ll be messing around with this for quite some time, I believe.
What makes other ones so bad? Like why would “Identity”, “Binary”, etc… be so bad? I tested alot of them, and alot of them were constantly above 60% correct.
This is because they are either very simple and are not optimized enough to work well, or they are meant for very specific situations, which is why I allow the option to be there anyway.
Usually, ReLU and its subtypes are the ones you need. Sometimes,when you delve much deeper into neural networking, you may need a specific function for some niche AI, so I add that extra bit of customization.
I tried making a simple system which would detect if a number is negative or not with this module-script, and alot of times when I would print the amount of times the neural network would be correct, it would be a value similar to the one before, is this normal?
In small/simple networks, absolutely. This is because with simple problems, it doesn’t take long for any random network to arrive at the same solution in a similar amount of time. Most networks behave like this. The rest are outliers, networks that either have a fatal flaw that causes them to hit a local minimum (as good as it can do) too early, or networks that have good parameters and hit that minimum later on, giving better results.
This is why with the example script I gave, most of the time, you’ll be getting an accuracy of 98%. Sometimes you’ll get 98.7%, sometimes it dies early and you get 70-90%.
Is there anything that I can do about this? I always want my neural network to be above a minimum of 50%, but I don’t want it to constantly have the same wins / “corrects” at one time.
If you mean that you want to get more successful networks given to you at the start, that’s when you use the genetic algorithm. It works surely but slowly. With it, i was able to reach 99.4% in an hour.
This is very cool. Neural networks in Roblox!
However I was thinking about using this in game A.I. How much training data do I need? Is it even practical for use in games at all?
If you can safely say that you can provide thousands of datasets, yes, you can easily train the AI.
If, for example, you want a zombie AI that moves and attacks players, that will take quite some time to train because of the player’s involvement, but is still very possible.
If it is a simple decision making AI that isn’t complex, you can use the genetic algorithm to sort out and breed the best performing networks to create the best one. This is how self driving cars and planes were made on Roblox.
TL:DR, if the AI relies on human input, it will take some effort to train. If the training process can be completely automated (like a car driving down a track), then it is easy and just takes a bit of time to run.
i have a very limited knowledge on how this works and i wanted to make something like this so what i did was open a place then make a module and after that i deleted the place because i had no idea what was going on. but you made it very simple to understand. thank you!
I am currently working on adding support for Recurrent Neural Networks to this library. This would allow networks to take their previous activations in mind when running, making them more appropriate for more complex AIs that move. Once this is done, LSTM (Long Short Term Memory) networks may be feasible.
I have just implemented support for vanilla Recurrent Neural Networks. I have also added better data validation for each function and support for StringValues created by createGenNet() to be substitutes for networks in any function that accepts them!
Just added a genetic algorithm example script for you to try! I promise its a bit more fun than the backpropagation one!
Challenge: Try to reach a 99.999% or 100% success rate with this script while the step is set to 1! If you manage to pull through such a task or get surprisingly close, feel free to use hardCode() to paste the network in the comments below! Any network size or config allowed!
The network above is a 2,2,12,1 version from the first example script, trained 5 million times and achieved a 99.81265% accuracy
Added a couple sections regarding what neural networks are used for and what they are; this way, you don’t need any outside knowledge to start using this library!
Out of curiosity, how does a genetic algorithm work?
Genetic algorithms are very different to every person and situation, but in general, the process works like this.
Create a pool of networks, usually at least 20-50 for enough variation as well as reasonable processing speeds.
Score each network based on the task at hand. Sort all of the networks in order from best to worst.
Kill a certain percentage of the worst networks, in our case, 60%.
Take the best and assign them a chance to mate depending on how well they performed, up to a min and max cap. In our case, the best network has a 75% chance while the worst has a 25%. Randomly pick the parents using these chances and breed them.
To breed them, go through every whole node and have a 50/50 of parent 1’s or parent 2’s version of that node to be implanted.
Once the breeding is done, apply a slight noise to all biases in the new children.
Then, go through every network and randomly mutate all parameters in a set number of nodes, in our case, 2. This excludes the best network.
Rinse and repeat.
This Is a very interesting product, It could help many peapole and maybe some programmers could develop an AI system of this?
Of course. This library eliminates the problem of creating and maintaining complex neural networks, so all the developer now has to do is set up a system where the AI is efficiently given data and trained.
I believe it already has. In my game, I’ve noticed that the AI fire bursts that aren’t really bursts. However, this module could help them be accurate.
