Arduino Basics: Neural Network (Part 2) : The Neuron

## 12 August 2011

### Neural Network (Part 2) : The Neuron

The Neuron

The main purpose of the Neuron is to store the values that flow through the neural network. They also do a bit of processing through the use of an Activation function. The activation function is quite an important feature of the neural network. It essentially enables the neuron to make decisions (yes/no and greyzone) based on the values provided to them. The other feature of activation functions is the ability to map values of infinite size to a range of 0 to 1. We will be using the Sigmoid function.

The neuron accumulates signals from one or more connections, adds a bias value, and then sends this value through the activation function to produce a neuron output. Each neuron output will be within the range of 0 to 1 (due to the activation function).

So lets take a look at  a single neuron with 2 connections (+ bias) feeding into it.
As seen below.

connEntry                                                       weight
Neuron.Connection1.connEntry = cEn1           Neuron.Connection1.weight = cW1
Neuron.Connection2.connEntry = cEn2           Neuron.Connection2.weight = cW2
Neuron.Bias.connEntry = bEn = 1 (always)                Neuron.Bias.weight = bW

connExit
Neuron.Connection1.connExit = cEx1
Neuron.Connection2.connExit = cEx2
Neuron.Bias.connExit = bEx

You need to multiply the connEntry value by the weight to get the connExit value.

Connection #1:
cEx1 = cW1  x   cEn1

Connection # 2:
cEx2 = cW2  x  cEn2

Bias:
bEx = bW x 1

As you can see from the Bias calculation, the Bias.connEntry (bEn) value is always 1, which means that the Bias.connExit (bEx) value is always equal to the Bias.weight (bW) value.

The Neuron Input Value is equal to the sum of all the connExit values (cEx1 and cEx2 in this case) plus the bias (bEx), and can be represented by the following formula.

Neuron Input Value:
Neuron Input Value = cEx1 + cEx2 + bEx
or
Neuron Input Value = [cW1 x cEn1] + [cW2 x cEn2] + [bW x 1]

Now that we have the Neuron Input Value, we can put this through the Activation Function to get the Neuron Output Value. This can be represented by the following formula.

Neuron Output Value:
Neuron Output Value = 1 / (1 + EXP(-1 x Neuron Input Value))

We will use this Neuron Output value as an input to the next layer, therefore it automatically becomes a connEntry for one of the neuron connections in the next layer.

Now that we know what a neuron does, let us create a neuron class. See the code below.

#### processing code Neuron Class

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
```/* ------------------------------------------------------------------------
A neuron does all the calculations to convert an input to an output
-------------------------------------------------------------------------  */

class Neuron{
Connection[] connections={};
float bias;
float neuronInputValue;
float neuronOutputValue;
float deltaError;

//The default constructor for a Neuron
Neuron(){
}

//The typical constructor of a Neuron - with random Bias and Connection weights
Neuron(int numOfConnections){
randomiseBias();
for(int i=0; i<numOfConnections; i++){
Connection conn = new Connection();
}
}

//Function to add a Connection to this neuron
connections = (Connection[]) append(connections, conn);
}

//Function to return the number of connections associated with this neuron.
int getConnectionCount(){
return connections.length;
}

//Function to set the bias of this Neuron
void setBias(float tempBias){
bias = tempBias;
}

//Function to randomise the bias of this Neuron
void randomiseBias(){
setBias(random(1));
}

//Function to convert the neuronInputValue to an neuronOutputValue
//Make sure that the number of connEntryValues matches the number of connections

float getNeuronOutput(float[] connEntryValues){
if(connEntryValues.length!=getConnectionCount()){
println("Neuron Error: getNeuronOutput() : Wrong number of connEntryValues");
exit();
}

neuronInputValue=0;

//First SUM all of the weighted connection values (connExit) attached to this neuron.
for(int i=0; i<getConnectionCount(); i++){
neuronInputValue+=connections[i].calcConnExit(connEntryValues[i]);
}

//Add the bias to the Neuron's inputValue
neuronInputValue+=bias;

//Send the inputValue through the activation function to produce the Neuron's outputValue
neuronOutputValue=Activation(neuronInputValue);

//Return the outputValue
return neuronOutputValue;
}

//Sigmoid Activation function
float Activation(float x){
float activatedValue = 1 / (1 + exp(-1 * x));
return activatedValue;
}

}```

When you create a new Neuron, you have the option to create the basic neuron shell with the Neuron() constructor, however, you are more likely to call the 2nd constructor, which allows you to set the number of input connections attached to this neuron.
For example, if I create a neuron using New Neuron(4), then this will automatically
• attach and associate four connections with this neuron
• randomise the weight of each connection
• randomise the bias of this neuron

These are the functions within the Neuron Class:
• getConnectionCount(): returns the number of connections associated with this neuron
• setBias(): sets the Bias of the neuron to a specific value.
• randomiseBias() : randomises the Bias of this neuron
• getNeuronOutput() : values are fed through the neuron's connections to create the neuronInputValue, which is then sent through the Sigmoid Activation function to create the neuronOutputValue.
• Activation() :  is the function used by the getNeuronOutput() function to generate the neuronOutputValue.

Up next:  Neural Network (Part 3)  : The Layer

1. What does (Connection[]) mean in "connections = (Connection[]) append(connections, conn);"? Thanks!

2. what does this section do? could you explain the keywords and meaning so that i can update this for the lastest version of the IDE? thanks.

connections = (Connection[]) append(connections, conn);
}

1. Hi Sean,
Please note - This code was used in the Processing IDE, not Arduino IDE.
It runs on your computer, not on the Arduino.
You can use the Neural network to interact with an Arduino project, but you would need to interface the Arduino with your Computer using a USB cable. You can see an example here.

The addConnection() function is used to ADD a CONNECTION to the neuron.
1. connections is an array of connections
2. conn is the connection that is being added to the array of connections

This function shows how to append a Connection to the array of Connections.