hey, here…
training model 3.rbxl (340.1 KB)
just wait like few min until it start spinning and give the nan’s
Okay, it is replicated. The only strange thing is that when I modified my own file to follow yours, it didn’t run into this issue.
That being said, I think you should change the final layers for all model parameters to have LeakyReLU as their activation function.
Also, start off with new parameters that aren’t stored in datastores.
i already did and still the same thing.
then how much the layer will be to set.
Model:addLayer(3, true, 'LeakyReLU', 0.001)its this what you mean? but the includeRND is off by default and i also tried to change that to “4” and its now better but it still happens but just took longer than before…
and finally it seems back normally when set to 6 from 3/4
but why the highest reward is 0? is this correct? it happens multiple times in a row…
training after like 5-15 min. but after like 20-30 min later it start spinning again but just for short time like 4-7 sec and it back to normal again.
also this library is using native luau operation? if not maybe you can try, it might can gain some performance increases.
after restart the run, it spinning at first start…
im thinking its not the LeakyRLU (RND)
this is my latest file:
training model 3.rbxl (340.9 KB)
so i went redownload the file and only remove jump from the class and change the layer to 6 from 7, it seems to be fine but after few min the run start it start spinning again… and comparing this with previous one, im thinking the previous provide better result because they show the good performance by killing & movement to target, while this one is like just moving randomly and having high minus rewards.
once i tried to enable the includeRND variable, it immediately start spinning at start. i have no idea right now…
i dont know why each model save twicely when end the run.
training model 3.rbxl (340.9 KB)
when using rnd:
+the parameters is all nan
im trying to do some experiment and adding ExperienceReplay but end at this error. i think you forgot to implement “update” function to ReinforcementLearningBaseModel.
that should be categoricalUpdate(). Not sure why I have missed that. I’ll just go and do fixes to the original library.
ProximalPolicyOptimizationClip dont work with UniformExperienceReplay
local function buildActorCriticRLQSModel(ID)
local MainModel = DataPredict.Models.ProximalPolicyOptimizationClip.new()
local AModel = buildActorModel(ID)
local CModel = buildCriticModel(ID)
MainModel:setActorModel(AModel)
MainModel:setCriticModel(CModel)
local MainModelQuickSetup = DataPredict.QuickSetups.CategoricalPolicy.new(60, 0, "Sample")
local ExperienceReplay = DataPredict.ExperienceReplays.UniformExperienceReplay.new(1, 5, 30)
MainModelQuickSetup:setModel(MainModel)
MainModelQuickSetup:setPrintOutput(false)
MainModelQuickSetup:setClassesList(classesList)
MainModelQuickSetup:setExperienceReplay(ExperienceReplay)
table.insert(ReinforcementLearningQuickSetupArray, MainModelQuickSetup)
if includeRND then
table.insert(RNDModelArray, buildRNDModel(ID))
end
return MainModelQuickSetup
endWell technically, it should be like that though if we’re using the existing RL theories. I won’t go full explanation why it should be like that. So stick with variants for Deep Q Learning, Deep SARSA, And Deep Expected SARSA if you really want to use the experience replays
How is your reward “nan”?!?! That could have led to the “nan” model parameters.
I would like to know what PC you’re using.
Yea, I was wondering why you’re catching all these errors while I’m not. There could be difference in how your device handles the numbers.
since im travelling (not in home) im using laptop (ASUS Vivobook S 14 OLED) it using Ryzen 5 7535HS. it doesnt have descrate gpu in the laptop just integrated gpu with 16gb ram ddr5…
i think i had to add return math.max(-1e3, reward) to prevent the nan thingy.
That’s very strange. You pretty much have some of the similar specs with my pc, but you’re the one having a lot of problems… Nobody else in this thread complained about the same thing.
Keep it the same, don’t change anything else.
i just wanted to remove “jump” class… so they focus on attack & movement…
Okay yeah, you can do that. Just make sure the final layers stays as LeakyReLU for all models.
why the RND dont work (break everything)? whenever turn it on and start the run, it make the npc models spinning around…
Rewards generated by RND is likely too high.
i have spent like almost 2 day no result, can u help check?
training model 3.rbxl (341.0 KB)
also does frame time per sec affect to the result? 60 fps & 240 fps…
and you can see that even the reward isnt nan, the parameter still can be nan…
