Hi, I’m probably not going to get any replies on this post, because it’s complicated. But I’m currently attempting to make a neural network module API thing, because it’s fun and there needs to be more, but I’m having trouble with the math.
Don’t get me wrong, the code is working, there’s not REALLY any errors, but everything is just off. I’ve seen and heard that handling neural networks (learning rate, input layer, activation function type etc…) is hard, and you have to be very fortunate to get the right settings correct on the first try, but I’m not even sure if I’m doing the math right. Let me just show you the code.
The only problem I’m having is with back-propagation. Here’s my code.
function body:Train(desiredOutputs, iteration)
-- get cost sum
local costsum = 0
for i, output in self.Layers[self.Length].Neurons do
costsum += math.pow(output.Activation - desiredOutputs[i], 2)
end
-- initialize changes
local changes = {}
for i, neuron in self.Layers[self.Length].Neurons do
changes[i] = (2 * (neuron.Activation - desiredOutputs[i]))
end
-- get desired changes
if self.Length > 2 then
for layer_num = self.Length, 2, -1 do -- loop through each layer but the first
local _layer = self.Layers[layer_num] -- get current layer
local prev_layer = self.Layers[layer_num - 1] -- get previous layer
for i, end_neuron in _layer.Neurons do -- loop through each neuron in the layer
local z_aDer = activation[end_neuron.ActivationFunction](activation[end_neuron.ActivationFunction](end_neuron.Activation, "Inverse"), "Derivative")
end_neuron.DesiredBias[iteration] += 1 * z_aDer * changes[i]
for j, start_neuron in prev_layer.Neurons do -- loop through weights
start_neuron.DesiredWeights[i][iteration] += start_neuron.Activation *
z_aDer * changes[i]
end
end
local holder_changes = {} -- update changes
for i, start_neuron in prev_layer.Neurons do
holder_changes[i] = 0
for j, end_neuron in _layer.Neurons do
holder_changes[i] += start_neuron.Weights[j] *
activation[end_neuron.ActivationFunction](activation[end_neuron.ActivationFunction](end_neuron.Activation, "Inverse"), "Derivative") * changes[j]
end
end
changes = holder_changes
end
end
return costsum
end
Please, if you understand neural network, just explain to me how it works, because clearly this isn’t. There are no errors.