DataPredict [Release 1.18] - General Purpose Machine And Deep Learning Library (Learning AIs, Generative AIs, and more!)

Resources Community Resources
, ,
#398

so you are saying the more I keep playing , the better in theory they should get.

You don’t have to fight them. It isn’t really necessary. Let the AI fight with each other.

So when the NPC dies, they are loading in stored data that should allow them to be better?

Something like that.

one of them was just turning around in circles, he kept yelling help me , help me!!

Just adjust the code so that it doesn’t load model parameters.

and do they have pathfinding? or how are they finding a target?

They do not use pathfinding. It uses something similar to that of a player’s control.

#399

’
They do not use pathfinding. It uses something similar to that of a player’s control.
’
so will they patrol, and go around walls, where they already know a target npc is at?

#400

Something like that. Though for this part:

where they already know a target npc is at?

Depends on how you code it. For this particular sword-fighting case, it is like 50/50.

#401

can they be programed to go off and find things, like say a chest or a weapon in the workspace?

and whack anyone near them while they patrol?

#402

Yes to all. Though, those AI can be quite bad at long-term goals. The short-term goals like:

whack anyone near them while they patrol?

are fine.

#403

how do you program it to like find a sword?

#404

Please do that yourself. I already provided all the codes for the learning AIs and I am not interested in doing other people’s work for free.

1 Like
#407

Is it possible to create this using this module.

#408

Possible, but:

  • You might want to use raw outputs from Advantage Actor Critic or the Proximal Policy Optimization

  • Expect it will take a long time to train.

1 Like
#412

Update Release 1.14

  • Added GradientMethodBaseModel. LinearRegression, LogisticRegression and NeuralNetwork inherits this instead of BaseModel. This is preparation for distributed gradient descent / ascent.

  • Added DistributedGradients so that the LinearRegression, LogisticRegression and variants of NeuralNetworks can be trained asynchronously. For example, you can make a copy of a model parameters on a client and return back values that are necessary for model parameters update on the server without needing duplicating the client’s model parameters.

  • Renamed DistrbutedLearning to DistributedTraining for better clarity.

2 Likes
#413

Additional Updates for Release 1.14:

  • I have refactored almost all the reinforcement learning models for maintainability. I have removed some parameters and functions like reinforce().

  • Added ReinforcementLearningQuickSetup so that it handles reinforce() instead, but with additional new stuff that you can control.

  • Expect breaking API changes.

  • Additionally, I have updated the tutorials under “Learning AIs” section. Please have a read so that you can make changes to your codes.

4 Likes
#414

hey i tried your Sword Fighting AI Version 3 and is this normal?

image
image1908×391 75.3 KB

after 5 minutes:
image
image1889×425 78.2 KB

im getting lower number and more negative the longer time it train.

#415

Yes, that’s pretty normal. I already made some calculations proofs on how the model parameters can become negative.

If you’re uncomfortable with those negative numbers, you can reset the model parameters or reduce the number of AIs to 2. It should yield a mixture positive and negative numbers.

All you need to worry about if the AI is actually learning or not.

#416

at Sense Script there a action for “AW, AS, WD, DS, rotateLeft, rotateRight” but in MainScript at classesList there not included. also sometimes the model like just rotating and jumping, when i print out the action, mostly does “D” key. also what if my fps goes higher will it be affected?

Edit: when i set limit fps to 1/10 (0.1s) i fell if the model movement seems to be better over 1/60 or higher.

~6 minutes (1/10):


image
image1416×385 51.5 KB

image
image2031×191 40.8 KB

#417

“AW, AS, WD, DS, rotateLeft, rotateRight”

Those keys are removed from the main script so that the training can be faster, you can add them back if you want. It was removed just to prove that the AI does learn things eventually.

model like just rotating and jumping, when i print out the action, mostly does “D” key

Yeah, that is expected to happen. Though currently it should happen way less often as I had focused on reducing it before hand.

also what if my fps goes higher will it be affected?

This is what I’m actually not sure. But looking at your screenshots, it seems like it. I’m planning to make a master thesis that investigate this kind of stuff.

#418

i also recommended to use --!native and --!optimize 2, native use native code generation (mostly better performance) and optimize 2 for disable debugging (live game) and optimize 1 for debugging (studio), so this also can affect to performance too.

Edit: how long this self training pvp took?

1 Like
#419

likely after a half hour, why the model now mostly just jumping where previously spamming swing sword

parameters:

{
	{  -0.16822699042525688,   -0.6812523234147968,  -0.14216823043147495,    -0.20560938981843 },
	{   -0.4932920947216591,  -0.25686042281723653,  -0.18176552147741074,  -0.0421154251307284 },
	{  -0.16061178212812852,   -0.1779255705014777,   -0.4464802214117151, -0.48535976025513927 },
	{   -0.3370262393061145,   -0.2657110226325964,   -0.6479303509706332,  -0.4088173756093041 },
	{  -0.09754697652219911,   -0.0677036885802078,   -0.5697087810690663,  -0.3792300398754359 },
	{   -0.7316973133316927,  -0.09724235503941879,  -0.42209716540705094,  -0.5120988977481272 },
}

{
	{   -0.5462787645810733,  -0.011471729832904773,   -0.7018367017786239,  -0.08644727437162279,   0.06179155466765962,   -0.1611948519813505, -0.7075880453337354 },
	{   0.03770737032365512,   -0.12488889525881229,   -0.5570072731161589,   -0.6625774678769007,  -0.18413168068496508,   0.06032417650997435, -0.6244003782980602 },
	{  -0.31053420998592385,    -0.2523262740989949,   -0.2673833569447544,  -0.02801017537952087,  -0.01242471319447401,  -0.15078727286003188, -0.6007499232786562 },
	{   0.06129832983328676,    -0.4402140928144736,  -0.05390233391513587,  0.028518419188966027,   -0.7281824571987116,  -0.17075485594781545, -0.4686918674282127 },
}

{
	{   -0.6068914728137356,   -0.5611569711528728,   -0.6216079811049421,  -0.4795040956656241 },
	{  -0.10510827905287966,   -0.4968405473071999,   -0.5658942674261749, -0.26465778981174287 },
	{     -0.44718799002743,   -0.2543968079251819,  -0.09564061649044618,  -0.2168078255118253 },
	{    -0.185119042559825,   -0.6835488882621861,   -0.6284902999561925,  -0.5014586756815729 },
	{   -0.2156555223291182,  -0.14705815373336592,   -0.2434440464984957, -0.03581246849981211 },
	{  -0.35883475428671957,   -0.3274551905333717,   -0.6673031528351816,  -0.4199481396692847 },
}

{
	{ 0.05747810714877222 },
	{ -0.3585079205181727 },
	{ 0.11733467638249984 },
	{ 0.11514085567618071 },
}

previous:

image
image1079×420 132 KB

Reinforcement Learning Sword Fighting Version 3.rbxl (196.3 KB)

do i have to reset?

#420

I’ve updated the main script for the version 3. I made minor changes from version 2 that might have led to this.

#421

i also have question, this https://twitter.com/PhoenixSignsRBX/status/1325042971020980224. why he only took like 40 minutes to make a efficiently model pvp? also he seems like have more complex/detailed classes of movements.

1 Like
#422

In that post, you can see that each arena have two AI battle with each other. All those AIs share the same model parameters, which basically means the AIs are fighting itself. Because it is a single model parameters, but a lot of AIs are training it, it learns faster.

In comparison to my sword-fighting AI code, each AIs have its own model parameters, so it trains longer.

Technically, you can do that with DataPredict, but for the purposes of making thing simple and easy to understand, I didn’t implement that.