__The Neuron__The main purpose of the Neuron is to store the values that flow through the neural network. They also do a bit of processing through the use of an Activation function. The activation function is quite an important feature of the neural network. It essentially enables the neuron to make decisions (yes/no and greyzone) based on the values provided to them. The other feature of activation functions is the ability to map values of infinite size to a range of 0 to 1. We will be using the Sigmoid function.

The neuron accumulates signals from one or more connections, adds a bias value, and then sends this value through the activation function to produce a neuron output. Each neuron output will be within the range of 0 to 1 (due to the activation function).

So lets take a look at a single neuron with 2 connections (+ bias) feeding into it.

As seen below.

__connEntry weight__Neuron.Connection1.connEntry =

**cEn1**Neuron.Connection1.weight =**cW1**Neuron.Connection2.connEntry =

**cEn2**Neuron.Connection2.weight =**cW2**Neuron.Bias.connEntry =

**bEn = 1**(always) Neuron.Bias.weight =**bW**

**connExit**Neuron.Connection1.connExit =

**cEx1**Neuron.Connection2.connExit =

**cEx2**Neuron.Bias.connExit =

**bEx**You need to multiply the connEntry value by the weight to get the connExit value.

__Connection #1:__

**cEx1 = cW1 x cEn1**

__Connection # 2:__

**cEx2 = cW2 x cEn2**

__Bias:__

**bEx = bW x 1**

As you can see from the Bias calculation, the Bias.connEntry (bEn) value is always 1, which means that the Bias.connExit (bEx) value is always equal to the Bias.weight (bW) value.

The Neuron Input Value is equal to the sum of all the connExit values (cEx1 and cEx2 in this case) plus the bias (bEx), and can be represented by the following formula.

__Neuron Input Value:__**Neuron Input Value**= cEx1 + cEx2 + bEx

or

**Neuron Input Value**= [cW1 x cEn1] + [cW2 x cEn2] + [bW x 1]

Now that we have the Neuron Input Value, we can put this through the Activation Function to get the Neuron Output Value. This can be represented by the following formula.

__Neuron Output Value:__**Neuron Output Value**= 1 / (1 + EXP(-1 x Neuron Input Value))

We will use this Neuron Output value as an input to the next layer, therefore it automatically becomes a connEntry for one of the neuron connections in the next layer.

Now that we know what a neuron does, let us create a neuron class. See the code below.

#### processing code Neuron Class

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

/* ------------------------------------------------------------------------ A neuron does all the calculations to convert an input to an output ------------------------------------------------------------------------- */ class Neuron{ Connection[] connections={}; float bias; float neuronInputValue; float neuronOutputValue; float deltaError; //The default constructor for a Neuron Neuron(){ } //The typical constructor of a Neuron - with random Bias and Connection weights Neuron(int numOfConnections){ randomiseBias(); for(int i=0; i<numOfConnections; i++){ Connection conn = new Connection(); addConnection(conn); } } //Function to add a Connection to this neuron void addConnection(Connection conn){ connections = (Connection[]) append(connections, conn); } //Function to return the number of connections associated with this neuron. int getConnectionCount(){ return connections.length; } //Function to set the bias of this Neuron void setBias(float tempBias){ bias = tempBias; } //Function to randomise the bias of this Neuron void randomiseBias(){ setBias(random(1)); } //Function to convert the neuronInputValue to an neuronOutputValue //Make sure that the number of connEntryValues matches the number of connections float getNeuronOutput(float[] connEntryValues){ if(connEntryValues.length!=getConnectionCount()){ println("Neuron Error: getNeuronOutput() : Wrong number of connEntryValues"); exit(); } neuronInputValue=0; //First SUM all of the weighted connection values (connExit) attached to this neuron. for(int i=0; i<getConnectionCount(); i++){ neuronInputValue+=connections[i].calcConnExit(connEntryValues[i]); } //Add the bias to the Neuron's inputValue neuronInputValue+=bias; //Send the inputValue through the activation function to produce the Neuron's outputValue neuronOutputValue=Activation(neuronInputValue); //Return the outputValue return neuronOutputValue; } //Sigmoid Activation function float Activation(float x){ float activatedValue = 1 / (1 + exp(-1 * x)); return activatedValue; } }

When you create a new Neuron, you have the option to create the basic neuron shell with the Neuron() constructor, however, you are more likely to call the 2nd constructor, which allows you to set the number of input connections attached to this neuron.

For example, if I create a neuron using New Neuron(4), then this will automatically

- attach and associate four connections with this neuron
- randomise the weight of each connection
- randomise the bias of this neuron

These are the functions within the Neuron Class:

**addConnection()**: adds a new connection to this neuron**getConnectionCount()**: returns the number of connections associated with this neuron**setBias()**: sets the Bias of the neuron to a specific value.**randomiseBias()**: randomises the Bias of this neuron**getNeuronOutput()**: values are fed through the neuron's connections to create the neuronInputValue, which is then sent through the Sigmoid Activation function to create the neuronOutputValue.**Activation()**: is the function used by the getNeuronOutput() function to generate the neuronOutputValue.

To go back to the table of contents click here

What does (Connection[]) mean in "connections = (Connection[]) append(connections, conn);"? Thanks!

ReplyDelete