Also sorry for the late reply, was working on a lil sum sum for the vehicle AI.
1 Like
Doesn’t seem like it moves far enough. Try using RandomNetworkDistillation to give the models to have some sort of curiosity.
:3
1 Like
is Double Expected SARSA better than Expected SARSA?
Usually the “Double” variants has better performance than the unmodified ones. Though, the calculations might be slightly slower in exchange for better performance.
1 Like
An error shows up when I try to calculate expectedqvalue
I am using string classes for the vehicle output. It seems ExpectedSARSA only accepts numeric values, how will I classify the output as a throttle or steering output?
EDIT: I don’t think its the best method but I classified forward as the output ‘0.55’ reverse as ‘-0.55’ and left,right as ‘0.5’ and ‘-0.5’ respectively.
This is a bug on my end. Will look into it and make necessary fixes.
1 Like
For now, use SARSA as a place holder algorithm.
1 Like
lmk when its fixed, i’m eager to use it 
This error occurs to me frequently,
I deduced that it was a result of these neuron amounts per layer (specifically layer 1-3) and only go away once neuron amounts in hidden layers are the same as the amount of neurons in the input layer.
But could you explain more in depth why these errors occur and why they sometimes dont occur for some other models? So I don’t make the same mistake again. (I am currently using SARSA)
Very likely something have screwed up the model parameters.
I wrote the library in a way that it is speed performant, at the cost of some important features.
When you place a value at certain index, there is no safeguard that checks whether or not the index is outside of the row / column range.
That being said, I started to see some other issues with my implementation and some of the algorithms will now need major refactoring. The library got a little too large for me to maintain and so I might have missed some details.
But hey, since I am unemployed right now, I can make fixes while I can.
2 Likes
Huh, you’re unemployed now? What happened if you don’t mind me asking? Also, i’ll try to use Q-learning meanwhile you’re fixing it then. I remember there are more models I tried with the same issue, don’t really recall anymore…
Office politics.
This is very valuable project. Don’t you think?
Also, I’ll probably push out a new version later.
1 Like
Sorry to hear that man. Yeah, this project is extraordinary, especially for those looking to get into machine learning, with this project you are still contributing to the global number of people interested in the STEM field, including Machine Learning. Even if you planned for this project to be used by those experienced in Neural Networking, I am sure you caused many to read up on this topic and at least try their hand at machine learning. Once I get a working model out, i’ll be sure to get a video for your marketing!
Need to add one at the right (straight) and one at the left (straight).
1 Like
Hmm, ive been training him for quite a while using DoubleQLearningV2 however for some reason he just doesnt want to correct his mistakes…
This is his BuildModel function.
His environmentVector.
His reward and respawning logic(he is penalised and respawned for touching sidewalks)
What’s wrong with this agent??
Try calling reset() function for ReinforcementLearningQuickSetup whenever it dies.
1 Like
Also, why is it clamped at zero? Shouldn’t it be clamped at a negative number?
I am very (not) smart : ))))))
It still either crashes into the wall repeatedly or reverses and throttles at the same time, I have never seen him go past the first corner at all…