Devlog #1
Hello! Welcome to my first public development log on the ROBLOX developer forums. Over the next few weeks I will be coding and training a neural network to play an FPS.
To start I will be using Carbon Engine by Titan Technologies for my FPS framework and Marine AI by Y3llow Mustang to accelerate the training process.
I will be creating multiple instances of games as seen in Hide an Seek AI by OpenAI, to accelerate the training process of the neural network.
The neural network will be composed of an input layer, which takes into account the game state, positions of enemies within the line of sight, the character’s position, all of its training data, and surrounding objects. Next the neural network will have 4 hidden layers, 3 of which will be composed of 15 neurons, the last hidden layer will have 10 neurons. Lastly there will be an output layer letting the neural network choosing if it wants to move forward, left, right. backward, aim, shoot, reload, throw a grenade, or other actions normal FPS players are constantly doing.
Setting up training data
To setup training data I will be using the data store service provided by ROBLOX. I will be inserting a table with the data to the service.
First I’ll make a module script that will update, create, and delete the datastore. The format will be:
local data = {
{reward = 0, gamestate = gamestate , finalenemyposx = 0.0, finalenemyposy = 0.0, finalenemyposz = 0.0, playerposx = 0.0, playerposy = 0.0, playerposz = 0.0, surround = surroundingobjects, sight = sight, episode = 0}
}
That will be our data table
Next I will create a module script function to easily access and edit this table.
local DataStoreService = game:GetService("DataStoreService")
local DataStore = DataStoreService:GetDataStore("MyDataStore")
local DataStoreManager = {}
-- Function to save data
function DataStoreManager:SaveData(key, value)
local success, err = pcall(function()
DataStore:SetAsync(key, value)
end)
if not success then
warn("Failed to save data: " .. tostring(err))
end
end
-- Function to load data
function DataStoreManager:LoadData(key, defaultValue)
local success, value = pcall(function()
return DataStore:GetAsync(key)
end)
if success then
if value ~= nil then
return value
else
return defaultValue
end
else
warn("Failed to load data: " .. tostring(value)) -- value contains the error message in case of failure
return defaultValue
end
end
return DataStoreManager
Now we can easily access the training data in and out of the game.
Creating the Neural Network
WARNING MATH AHEAD!
After doing basic research on creating neural networks from scratch I found the main components and concepts you need to understand when creating a neural network from scratch:
-
Neurons: The basic units of a neural network are called neurons. Each neuron receives inputs from other neurons or external sources, applies a weighted sum, and passes the result through an activation function to generate an output.
-
Layers: Neurons are organized into layers, typically consisting of an input layer, one or more hidden layers, and an output layer. The input layer receives the initial data, the hidden layers process the data, and the output layer produces the final result or prediction.
-
Weights and biases: The connections between neurons have weights and biases associated with them. Weights determine the strength of the connection between neurons, while biases help to shift the activation function. Both weights and biases are adjusted during the training process to improve the accuracy of the neural network.
-
Activation functions: Activation functions are applied to the weighted sum of inputs to introduce non-linearity into the neural network. Some common activation functions include the sigmoid function, the hyperbolic tangent (tanh) function, and the Rectified Linear Unit (ReLU) function. We will be using the sigmoid function.
-
Forward propagation: This is the process of passing input data through the neural network to generate an output. Each neuron computes a weighted sum of its inputs, adds the bias, and passes the result through an activation function. This process is repeated for each layer in the network until the output layer is reached.
-
Loss function: The loss function quantifies the difference between the predicted output and the actual output (target) for a given input. Common loss functions include mean squared error (MSE) for regression tasks and cross-entropy for classification tasks.
-
Backpropagation: This is the core algorithm used to train neural networks. It involves computing the gradient of the loss function with respect to each weight and bias in the network. The gradients are used to update the weights and biases, minimizing the loss function and improving the network’s performance.
-
Gradient descent: Gradient descent is an optimization algorithm used to update the weights and biases of the neural network during backpropagation. It adjusts the weights and biases by a fraction of the computed gradients, called the learning rate, to minimize the loss function.
-
Learning rate: The learning rate is a hyperparameter that controls the step size in the gradient descent algorithm. A smaller learning rate results in slower convergence but may lead to more accurate results, while a larger learning rate may lead to faster convergence but potentially less accurate results.
-
Epochs and batches: Training a neural network typically involves passing the entire dataset through the network multiple times, called epochs. Each epoch can be divided into smaller chunks, called batches, which are processed independently and used to update the weights and biases more frequently.
I took a lot of this out of this video for my information. I converted his neural network code to be compatible with roblox and our datastore system, so thanks to him.
The result for the script is:
-- Import DataStoreManager
local DataStoreManager = require(game.ServerScriptService.DataStoreManager)
-- Neural Network Module
local NeuralNetwork = {}
NeuralNetwork.__index = NeuralNetwork
-- Activation function
local function sigmoid(x)
return 1 / (1 + math.exp(-x))
end
-- Derivative of the activation function
local function sigmoid_derivative(x)
return x * (1 - x)
end
-- Initialize the neural network
function NeuralNetwork.new(input_nodes, hidden_nodes, output_nodes)
local self = setmetatable({}, NeuralNetwork)
-- Set the number of nodes for each layer
self.input_nodes = input_nodes
self.hidden_nodes = hidden_nodes
self.output_nodes = output_nodes
-- Initialize weights and biases randomly
self.weights_ih = {}
self.weights_ho = {}
self.biases_h = {}
self.biases_o = {}
for i = 1, hidden_nodes do
self.weights_ih[i] = {}
self.biases_h[i] = math.random()
for j = 1, input_nodes do
self.weights_ih[i][j] = math.random()
end
end
for i = 1, output_nodes do
self.weights_ho[i] = {}
self.biases_o[i] = math.random()
for j = 1, hidden_nodes do
self.weights_ho[i][j] = math.random()
end
end
return self
end
-- Forward propagation
function NeuralNetwork:predict(input_array)
-- Calculate hidden layer outputs
local hidden_outputs = {}
for i = 1, self.hidden_nodes do
local sum = 0
for j = 1, self.input_nodes do
sum = sum + input_array[j] * self.weights_ih[i][j]
end
hidden_outputs[i] = sigmoid(sum + self.biases_h[i])
end
-- Calculate output layer outputs
local output_outputs = {}
for i = 1, self.output_nodes do
local sum = 0
for j = 1, self.hidden_nodes do
sum = sum + hidden_outputs[j] * self.weights_ho[i][j]
end
output_outputs[i] = sigmoid(sum + self.biases_o[i])
end
return output_outputs
end
-- Train the network
function NeuralNetwork:train(input_array, target_array, learning_rate)
-- Forward propagation
local hidden_outputs = {}
local output_outputs = self:predict(input_array)
-- Compute output layer errors
local output_errors = {}
for i = 1, self.output_nodes do
output_errors[i] = target_array[i] - output_outputs[i]
end
-- Update output layer weights and biases
for i = 1, self.output_nodes do
for j = 1, self.hidden_nodes do
self.weights_ho[i][j] = self.weights_ho[i][j] + learning_rate * output_errors[i] * sigmoid_derivative(output_outputs[i]) * hidden_outputs[j]
end
self.biases_o[i] = self.biases_o[i] + learning_rate * output_errors[i] * sigmoid_derivative(output_outputs[i])
end
-- Compute hidden layer errors
local hidden_errors = {}
for i = 1, self.hidden_nodes do
local sum = 0
for j = 1, self.output_nodes do
sum = sum + output_errors[j] * self.weights_ho[j][i]
end
hidden_errors[i] = sum
end
-- Update hidden layer weights and biases
for i = 1, self.hidden_nodes do
for j = 1, self.input_nodes do
self.weights_ih[i][j] = self.weights_ih[i][j] + learning_rate * hidden_errors[i] * sigmoid_derivative(hidden_outputs[i]) * input_array[j]
end
self.biases_h[i] = self.biases_h[i] + learning_rate * hidden_errors[i] * sigmoid_derivative(hidden_outputs[i])
end
end
return NeuralNetwork
In summary:
-
Sigmoid activation function (sigmoid(x)):
The sigmoid function is an activation function that maps input values to the range (0, 1). It is defined as:sigmoid(x) = 1 / (1 + e^(-x))
-
Sigmoid derivative function (sigmoid_derivative(x)):
This function calculates the derivative of the sigmoid function, which is used in backpropagation for updating weights and biases. It is defined as:sigmoid_derivative(x) = x * (1 - x)
-
Forward propagation (NeuralNetwork:predict):
This function computes the output of the neural network for a given input array. It calculates the weighted sum of inputs and biases for each node in the hidden layer, applies the sigmoid activation function, and then repeats the process for the output layer. -
Backpropagation (NeuralNetwork:train):
This function trains the neural network using the input_array and target_array. It first computes the output and hidden layer errors as the difference between the target values and the neural network’s outputs. Then, it updates the weights and biases of the output and hidden layers using the learning rate, the computed errors, and the derivatives of the activation function.
Conclusion
In this devlog I created a training data framework and neural network framework for my upcoming ML project that will allow neural networks to play ROBLOX FPS’s.
In the next devlog I will create the enviorment for the FPS with events. I will also experiment with multi agent interaction and create team based enviorments for the models.
Thank your for reading,
BafoozledBob