Bro, you need move the addLayer(), setClassesList() and setModelParametersInitializationMode() outside the if statement.
Use generateLayers() inside your if statement instead.
Oh, what does generatelayers() do? reload layers from saved parameters or
Just generates new model parameters. Do note that it will reset the current model parameters if you have any stored inside it though.
Had a bit of fun with the individual agents and managed to teach them to form formations while sticking together!
Well, after having some fun with them, I took a look at some real-life implementations of AI, specifically tesla’s autopilot and got a bit inspired. I understand what inputs I can give the car to stay on the road but… what would I ideally tell it when I want it to drive on the road while actively driving towards their destination? (e.g taking turns to drive towards the destination but avoiding turns that lead away or are unnecessary in order to get to their destination). I couldn’t really find any resources on this but I DID program an A* algorithm, although not perfect it can ideally pathfind for the vehicle, but what I wanted to know was your opinion on what inputs i’d be using to train the model to drive to their destination while staying on the road and making the right turns and what model do you recommend I use if I wanted it to learn how to drive naturally over the course of its training period? (e.g detect other cars lane-switching, braking etc.)
I think this time raw output would be great since there’s a steering and throttle value that can be set from 0 - 1 / -1 - 1, I wouldn’t essentially need any labels or classeslists.
I did come across a couple great courses on self-driving vehicles running off of neural networks but most of them just showed the vehicle sticking to its monotone path instead of taking turns and trying to drive to its destination over the course of multiple separate roads.
First off, I want to mention is that try to avoid using raw values when it is not needed. When you do that the performance will be actually worse. You can quickly look into conclusion for the proof here:
I recommend you do this for the output:
-
Reserve 2 output neurons for controlling the steering specific amount of value for left and right (i.e. 0.1 and -0.1) instead of full amount.
-
Reserve 2 output neurons for controlling throttle value.
-
Make sure to keep track the total amount of change for throttle and steering values, because these will be used for the inputs for the model.
Now to answer for your question for your inputs:
-
Use something similar to the sword fighting AI that marks the target location (i.e. the amount of rotation from the target and the distance from the target).
-
A few raycasts that shoots downwards and some of the raycast have angles. Make sure each raycast have its own input neuron. If the raycast detects the road parts, the value will be 1, otherwise it will be zero. You can also use this to reward the model, depending how many raycasts hits the road parts.
-
Reserve 2 input neurons for the total throttle and steering change. This will be used as a “memory”, so that it knows what’s the current amount of the throttle and steering it’s applying.
-
Reserve some input neurons for raycast shooting at the sides, front and back of the car. Use inverse distance.
For the model, I guess any could do. But if you want a more reckless driving, stick with Q-Learning. Otherwise, stick with expected SARSA or regular SARSA and its variants. The above input and output configuration should work really well with these model. (Don’t ask me the mathematics behind it, you would at least need a month to understand why is that).
Also can you do me a favour? Once you’re done with your stuff, can you give me a 3 minute video for it? I don’t want to see any codes though. I would like to use it for marketing purposes. I will also include credit for your work. It’s fine if you don’t want to.
PS: Can I also see a short video of the formation as well? I’m curious now. 0_o
Thanks for the feedback! Hmm, for the 3 min video i’ll see if I can get a couple good shots of the AI in action and compile them into a little video if I can even manage to train it properly 
.
And for the formations…
The AI is a bit slow at doing them since I only trained them for 15-30 mins but they still manage to form 2 basic things like full-coverage SFL and a square thing that I believe are their attempts at trying to stick together.
Doesn’t seem like it moves far enough. Try using RandomNetworkDistillation to give the models to have some sort of curiosity.
:3
1 Like
is Double Expected SARSA better than Expected SARSA?
Usually the “Double” variants has better performance than the unmodified ones. Though, the calculations might be slightly slower in exchange for better performance.
1 Like
An error shows up when I try to calculate expectedqvalue
I am using string classes for the vehicle output. It seems ExpectedSARSA only accepts numeric values, how will I classify the output as a throttle or steering output?
EDIT: I don’t think its the best method but I classified forward as the output ‘0.55’ reverse as ‘-0.55’ and left,right as ‘0.5’ and ‘-0.5’ respectively.
This is a bug on my end. Will look into it and make necessary fixes.
1 Like
For now, use SARSA as a place holder algorithm.
1 Like
lmk when its fixed, i’m eager to use it
This error occurs to me frequently,
I deduced that it was a result of these neuron amounts per layer (specifically layer 1-3) and only go away once neuron amounts in hidden layers are the same as the amount of neurons in the input layer.
But could you explain more in depth why these errors occur and why they sometimes dont occur for some other models? So I don’t make the same mistake again. (I am currently using SARSA)
Very likely something have screwed up the model parameters.
I wrote the library in a way that it is speed performant, at the cost of some important features.
When you place a value at certain index, there is no safeguard that checks whether or not the index is outside of the row / column range.
That being said, I started to see some other issues with my implementation and some of the algorithms will now need major refactoring. The library got a little too large for me to maintain and so I might have missed some details.
But hey, since I am unemployed right now, I can make fixes while I can.
2 Likes
Huh, you’re unemployed now? What happened if you don’t mind me asking? Also, i’ll try to use Q-learning meanwhile you’re fixing it then. I remember there are more models I tried with the same issue, don’t really recall anymore…
Office politics.
This is very valuable project. Don’t you think?
Also, I’ll probably push out a new version later.
1 Like
Sorry to hear that man. Yeah, this project is extraordinary, especially for those looking to get into machine learning, with this project you are still contributing to the global number of people interested in the STEM field, including Machine Learning. Even if you planned for this project to be used by those experienced in Neural Networking, I am sure you caused many to read up on this topic and at least try their hand at machine learning. Once I get a working model out, i’ll be sure to get a video for your marketing!
