I’m out of ideas. You’ll have to give me the full source code in a private message if you want this fixed.
I’m seriously not picking up any errors for those algorithms in my “Library Lab” place.
That is the full source code 
I mean the whole roblox file. I can’t do proper analysis without the things that are interacting in workspace.
That being said, try adding pcall around :reinforce() first. I want to see if it continues working or not.
It works fine with a pcall
Okay yeah. Definitely the calculation is too slow for the neural network.
You need to add some sort of debounce to prevent this from happening.
could I increase the loop’s task.wait() time? or is there a better alternative
It’s better if you do this
while true do
-- Your code
task.wait()
end
This way you won’t have any idle times and also allows the code to be fully processed.
1 Like
Still errors out even when I task.wait(1) this loop structure
Hmm… I see then. So you’ll have to wrap it around in pcall then. No other way around I suppose. Similar what I did to the sword fighting AIs code.
1 Like
Hmm, the thing is, it doesn’t reinforce a single time, so it just plays the first control over and over again, I don’t think I can use this specific model then, I’ll need to find one that can compute it, which was A2C I think. I’d suppose this is a limitation of roblox Luau, I heard roblox has a way to make computing run smoother, through the usage of Actors, I have not tried it myself but you could experiment with it if you want.
Yep, as expected A2C does work without pcalls, hmm weird.
Well, I kind of tried that and it didn’t give good results. It technically made the calculations far more slower due to the overhead when communicating between actors/.threads.
1 Like
I think I managed to find some good settings nonetheless, the agent seems to not turn into walls anymore with A2C and surprisingly reaches the goal part on the straight stretch
It might look like its only using its first output, but after further testing I found out that that is in fact not true, I don’t know why its so perfect that it doesn’t want to steer away from the goal but, it certainly steers either to the left or the right which is a case of bad reward function on my part.
It is less random, to say the least, might have to do with my epsilon setting.
Okay, I accidentally skipped over some of what you said and fixed the PPO and PPO clip. hopefully it doesn’t give issues now.
1 Like
PPOClip still gives me this 
Oh sorry lol. I forgot to push it to GitHub. Now it should be there.
1 Like
Great improvement I saw, the Agent actually speeds up then tries to slow down by putting its throttle into negative, slowing the car down but not enough so it eventually hits the wall, I will hook up a brake function to let it slow down and see what happens! (PS: not talking abt the video, just trained a fresh agent not too long ago) And I think I figured out why my agent kept turning into the wall a few days ago… you see, my epilson value for reinforcementquicksetup was set above 0 for some reason that made the correct data get manipulated into incorrect data, or so I presume. After setting it to 0, it is much more accurate!
1 Like
Hmm, could I ask if you’re planning on maybe adding Neuroevolution or Genetic Algorithms to your module? I think from my experience, roblox has a tough time trying to keep up with deep learning calculations as you had previously said, Luau’s calculation speed is not up to par with the rest of the script, hence bringing the error into the room. Doing some research, I have seen a very successful AI attempt in roblox, coming from a genetic algorithm within an autonomous Police AI agent structure, I think you’ve seen his twitter posts before but i’ll quote his devforum link regardless,
He has done a brilliant job at creating his piece of AI, which leads me to suggest, because your module has a wonderful neural network setup with matrices and such, would it be possible for you to maybe implement a genetic algorithm or a neuroevolution implementation?
Nope. No plans. The thing with genetic algorithms / neuro-evolution, you have a lot of way to do it. For me to build the code that covers everything is pretty much impossible. Here are the questions related to this issue.
-
Do I choose a single value or a single layer (matrix) to evolve?
-
How do adjust parameters that represents evolution when combining two parameters? Is it choosing maximum, average or minimum values?
-
Do I construct new neural network architecture from an existing one to represent it as evolution?
-
And many more.
There’s so many things you can do, and I think it is best to not implement it for flexibility and easier code maintenance.
1 Like
Hmm, yeah, I agree, it is truly flexible, the amount of ways u can accomplish this is remarkable, apparently scripton rescripted his work 10 times or so, according to his tweets. Well, do you have any tips for using your module to somehow integrate a genetic algorithm/neuroevolution simulation to?