Argentum Solutions, Inc.
Sterling guidance on corrosion and materials degradation
| Potential-pH Diagrams |
 |
|
| Intelligent Tools |
 |
|
 |
|
 |
|
| Corrosion Calculator |
 |
|
| Corrosion Economics Estimator |
 |
|
|
David C. Silverman |
|
The Back-propagation Artificial Neural Network The computing elements are joined together in an array. Two types of networks exist, feed-forward and recurrent. The feed-forward network has links that are unidirectional, e.g. they move information in only one direction through links between layers. The computing element in a given layer receives information from the layer below and passes information to the layer above. The feed-forward network is the type often used when artificial neural networks are employed. That array may be pictured as shown in this figure)
for the
back-propagation network, one type of feed-forward network. Other structures
exist.
This network is pictured as having three layers. The input layer, the lowest layer is written separately from the inputs because sometimes pre-processing of the input values is needed. For example, having all inputs values between 0 and 1 can sometimes help training. The next higher layer is the hidden layer. This layer enables the network to represent non-linear systems (functions). This layer has no direct connection with the outside world, hence its name "hidden". Sometimes more than one hidden layer is present. All inputs are numerical. If one of the required inputs is a verbal description (for example a color or other descriptive characteristic) it must be translated to a numerical value before it can be used in training. There has been a suggestion that any continuous function of inputs can be represented by one appropriately sized hidden layer. Discontinuous functions of inputs can be represented by two appropriately sized hidden layers. Finally, the top layer is the output layer. It also is composed of processing elements the outputs from which communicate with the outside world. The feed-forward artificial neural network has a computational structure in which information proceeds from input to output without cycles. The function of the input values it computes (its output) depends only on the weights. Information cannot flow backwards. In this respect, the feed-forward network is different from the brain which contains numerous back connections. Recurrent networks which are not covered in this tutorial would be closer to mimicking brain function. The structure of the artificial neural network is fixed. The activation function of the artificial neural network is fixed. These characteristics mean that the function which is being represented by the artificial neural network has a specific structure. Different activation functions could create a different functional representation. In addition, the activation functions are non-linear. This observation means that the network itself represents a non-linear function. The weights can be considered as coefficients of this function. When thought of in this manner the process of learning is the process of determining the coefficients that provide a best fit between the input and corresponding output information. Such a process is nothing more than non-linear regression. So, in essence, when training a back-propagation artificial neural network one is using a type of non-linear regression algorithm to find the weights that create the best fit to individual data sets used for the training. The individual weights themselves have no physical meaning. |
|
Previous Page: The Back-propagation Computing Element |
|
David C. Silverman, Ph.D. - Primary Consultant |