\HelpClassesNeuralNetwork

Multi-layer Neural Network in PHP

Loosely based on source code by Phil Brierley, that was translated into PHP by 'dspink' in sep 2005

Algorithm was obtained from the excellent introductory book "Artificial Intelligence - a guide to intelligent systems" by Michael Negnevitsky (ISBN 0-201-71159-1)

Example: learning the 'XOR'-function // Create a new neural network with 3 input neurons, // 4 hidden neurons, and 1 output neuron $n = new NeuralNetwork(3, 4, 1); $n->setVerbose(false);

// Add test-data to the network. In this case, // we want the network to learn the 'XOR'-function $n->addTestData(array (-1, -1, 1), array (-1)); $n->addTestData(array (-1, 1, 1), array ( 1)); $n->addTestData(array ( 1, -1, 1), array ( 1)); $n->addTestData(array ( 1, 1, 1), array (-1));

// we try training the network for at most $max times $max = 3;

// train the network in max 1000 epochs, with a max squared error of 0.01 while (!($success = $n->train(1000, 0.01)) && ++$i<$max) { echo "Round $i: No success...


"; }

// print a message if the network was succesfully trained if ($success) { $epochs = $n->getEpoch(); echo "Success in $epochs training rounds!


"; }

// in any case, we print the output of the neural network echo "

End result

"; for ($i = 0; $i < count($n->trainInputs); $i ++) { $output = $n->calculate($n->trainInputs[$i]); echo "
Testset $i; "; echo "expected output = (".implode(", ", $n->trainOutput[$i]).") "; echo "output from neural network = (".implode(", ", $output).")\n"; }

The resulting output could for example be something along the following lines:

Success in 719 training rounds! Testset 0; expected output = (-1) output from neural network = (-0.986415991978) Testset 1; expected output = (1) output from neural network = (0.992121412998) Testset 2; expected output = (1) output from neural network = (0.992469534962) Testset 3; expected output = (-1) output from neural network = (-0.990224120384)

...which indicates the network has learned the task.

Summary

Methods
Properties
Constants
__construct()
export()
import()
setLearningRate()
getLearningRate()
setMomentum()
getMomentum()
calculate()
addTestData()
getTestDataIDs()
addControlData()
getControlDataIDs()
showWeights()
setVerbose()
isVerbose()
load()
save()
clear()
train()
getEpoch()
getErrorTrainingSet()
getErrorControlSet()
getTrainingSuccessful()
$trainInputs
$trainOutput
$trainDataID
$controlInputs
$controlOutput
$controlDataID
No constants found
activation()
derivativeActivation()
$nodeCount
$nodeValue
$nodeThreshold
$edgeWeight
$learningRate
$layerCount
$previousWeightCorrection
$momentum
$isVerbose
$weightsInitialized
$epoch
$errorTrainingset
$errorControlset
$success
N/A
setEpoch()
setErrorTrainingSet()
setErrorControlSet()
setTrainingSuccessful()
fitLine()
getRandomWeight()
initWeights()
backpropagate()
squaredErrorEpoch()
squaredErrorControlSet()
squaredError()
No private properties found
N/A

Properties

$trainInputs

$trainInputs : array

Type

array — Input story

$trainOutput

$trainOutput : array

Type

array — Output story

$trainDataID

$trainDataID : array

Type

array — Training ID

$controlInputs

$controlInputs : array

Type

array — Input control history

$controlOutput

$controlOutput : array

Type

array — Output control history

$controlDataID

$controlDataID : array

Type

array — DataID control history

$nodeCount

$nodeCount : array

Type

array — Amount of nodes

$nodeValue

$nodeValue : array

Type

array — Values of nodes

$nodeThreshold

$nodeThreshold : array

Type

array — Threshold of nodes

$edgeWeight

$edgeWeight : array

Type

array — Edge weights

$learningRate

$learningRate : array<mixed,float>

Type

array<mixed,float> — Learning rate

$layerCount

$layerCount : array<mixed,integer>

Type

array<mixed,integer> — Layers count

$previousWeightCorrection

$previousWeightCorrection : array

Type

array — Fallback position

$momentum

$momentum : float

Type

float — Momentumn

$isVerbose

$isVerbose : boolean

Type

boolean — Is verbose

$weightsInitialized

$weightsInitialized : boolean

Type

boolean — Is inite

$epoch

$epoch : integer

Type

integer — NN epoch

$errorTrainingset

$errorTrainingset : float

Type

float — Error epoch

$errorControlset

$errorControlset : float

Type

float — Control error

$success

$success : boolean

Type

boolean — Success

Methods

__construct()

__construct(array  $nodeCount) 

Creates a neural network.

Example: // create a network with 4 input nodes, 10 hidden nodes, and 4 output nodes $n = new NeuralNetwork(4, 10, 4);

// create a network with 4 input nodes, 1 hidden layer with 10 nodes, // another hidden layer with 10 nodes, and 4 output nodes $n = new NeuralNetwork(4, 10, 10, 4);

// alternative syntax $n = new NeuralNetwork(array(4, 10, 10, 4));

Parameters

array $nodeCount

The number of nodes in the consecutive layers.

export()

export() 

Exports the neural network

import()

import(array  $nn_array) 

Import a neural network

Parameters

array $nn_array

An array of the neural network parameters

setLearningRate()

setLearningRate(array  $learningRate) 

Sets the learning rate between the different layers.

Parameters

array $learningRate

An array containing the learning rates [range 0.0 - 1.0]. The size of this array is 'layerCount - 1'. You might also provide a single number. If that is the case, then this will be the learning rate for the whole network.

getLearningRate()

getLearningRate(integer  $layer) : float

Gets the learning rate for a specific layer

Parameters

integer $layer

The layer to obtain the learning rate for

Returns

float —

The learning rate for that layer

setMomentum()

setMomentum(float  $momentum) 

Sets the 'momentum' for the learning algorithm. The momentum should accelerate the learning process and help avoid local minima.

Parameters

float $momentum

The momentum. Must be between 0.0 and 1.0; Usually between 0.5 and 0.9

getMomentum()

getMomentum() : float

Gets the momentum.

Returns

float —

The momentum

calculate()

calculate(array  $input) : mixed

Calculate the output of the neural network for a given input vector

Parameters

array $input

The vector to calculate

Returns

mixed —

The output of the network

addTestData()

addTestData(array  $input, array  $output, integer  $id = null) 

Add a test vector and its output

Parameters

array $input

An input vector

array $output

The corresponding output

integer $id

(optional) An identifier for this piece of data

getTestDataIDs()

getTestDataIDs() : array

Returns the identifiers of the data used to train the network (if available)

Returns

array —

An array of identifiers

addControlData()

addControlData(array  $input, array  $output, integer  $id = null) 

Add a set of control data to the network.

This set of data is used to prevent 'overlearning' of the network. The network will stop training if the results obtained for the control data are worsening.

The data added as control data is not used for training.

Parameters

array $input

An input vector

array $output

The corresponding output

integer $id

(optional) An identifier for this piece of data

getControlDataIDs()

getControlDataIDs() : array

Returns the identifiers of the control data used during the training of the network (if available)

Returns

array —

An array of identifiers

showWeights()

showWeights(boolean  $force = false) 

Shows the current weights and thresholds

Parameters

boolean $force

Force the output, even if the network is {@link setVerbose() not verbose}.

setVerbose()

setVerbose(boolean  $isVerbose) 

Determines if the neural network displays status and error messages. By default, it does.

Parameters

boolean $isVerbose

'true' if you want to display status and error messages, 'false' if you don't

isVerbose()

isVerbose() : boolean

Returns whether or not the network displays status and error messages.

Returns

boolean —

'true' if status and error messages are displayed, 'false' otherwise

load()

load(string  $filename) : boolean

Loads a neural network from a file saved by the 'save()' function. Clears the training and control data added so far.

Parameters

string $filename

The filename to load the network from

Returns

boolean —

'true' on success, 'false' otherwise

save()

save(string  $filename) : boolean

Saves a neural network to a file

Parameters

string $filename

The filename to save the neural network to

Returns

boolean —

'true' on success, 'false' otherwise

clear()

clear() 

Resets the state of the neural network, so it is ready for a new round of training.

train()

train(integer  $maxEpochs = 500, float  $maxError = 0.01) : boolean

Start the training process

Parameters

integer $maxEpochs

The maximum number of epochs

float $maxError

The maximum squared error in the training data

Returns

boolean —

'true' if the training was successful, 'false' otherwise

getEpoch()

getEpoch() : integer

Gets the number of epochs the network needed for training.

Returns

integer —

The number of epochs.

getErrorTrainingSet()

getErrorTrainingSet() : float

Gets the squared error between the desired output and the obtained output of the training data.

Returns

float —

The squared error of the training data

getErrorControlSet()

getErrorControlSet() : float

Gets the squared error between the desired output and the obtained output of the control data.

Returns

float —

The squared error of the control data

getTrainingSuccessful()

getTrainingSuccessful() : boolean

Determines if the training was successful.

Returns

boolean —

'true' if the training was successful, 'false' otherwise

activation()

activation(float  $value) : float

Implements the standard (default) activation function for backpropagation networks, the 'tanh' activation function.

Parameters

float $value

The preliminary output to apply this function to

Returns

float —

The final output of the node

derivativeActivation()

derivativeActivation(float  $value) : \HelpClasses\$float

Implements the derivative of the activation function. By default, this is the inverse of the 'tanh' activation function: 1.0 - tanh($value)*tanh($value);

Parameters

float $value

'X'

Returns

\HelpClasses\$float

setEpoch()

setEpoch(integer  $epoch) 

After training, this function is used to store the number of epochs the network needed for training the network. An epoch is defined as the number of times the complete trainingset is used for training.

Parameters

integer $epoch

setErrorTrainingSet()

setErrorTrainingSet(float  $error) 

After training, this function is used to store the squared error between the desired output and the obtained output of the training data.

Parameters

float $error

The squared error of the training data

setErrorControlSet()

setErrorControlSet(float  $error) 

After training, this function is used to store the squared error between the desired output and the obtained output of the control data.

Parameters

float $error

The squared error of the control data

setTrainingSuccessful()

setTrainingSuccessful(boolean  $success) 

After training, this function is used to store whether or not the training was successful.

Parameters

boolean $success

'true' if the training was successful, 'false' otherwise

fitLine()

fitLine(array  $data) : array

Finds the least square fitting line for the given data.

This function is used to determine if the network is overtraining itself. If the line through the controlset's most recent squared errors is going 'up', then it's time to stop training.

Parameters

array $data

The points to fit a line to. The keys of this array represent the 'x'-value of the point, the corresponding value is the 'y'-value of the point.

Returns

array —

An array containing, respectively, the slope and the offset of the fitted line.

getRandomWeight()

getRandomWeight(  $layer) : float

Gets a random weight between [-0.25 .

. 0.25]. Used to initialize the network.

Parameters

$layer

Returns

float —

A random weight

initWeights()

initWeights() 

Randomise the weights in the neural network

backpropagate()

backpropagate(array  $output, array  $desired_output) 

Performs the backpropagation algorithm. This changes the weights and thresholds of the network.

Parameters

array $output

The output obtained by the network

array $desired_output

The desired output

squaredErrorEpoch()

squaredErrorEpoch() : float

Calculate the root-mean-squared error of the output, given the trainingdata.

Returns

float —

The root-mean-squared error of the output

squaredErrorControlSet()

squaredErrorControlSet() : float

Calculate the root-mean-squared error of the output, given the controldata.

Returns

float —

The root-mean-squared error of the output

squaredError()

squaredError(array  $input, array  $desired_output) : float

Calculate the root-mean-squared error of the output, given the desired output.

Parameters

array $input

The input to test

array $desired_output

The desired output

Returns

float —

The root-mean-squared error of the output compared to the desired output