Argentum Solutions, Inc.
Sterling guidance on corrosion and materials degradation
| Potential-pH Diagrams |
 |
|
| Intelligent Tools |
 |
|
 |
|
 |
|
| Corrosion Calculator |
 |
|
| Corrosion Economics Estimator |
 |
|
|
David C. Silverman |
|
Probabilistic Artificial Neural Network Probabilistic neural networks are also known as belief networks, Bayesian networks, and knowledge maps. They have the following characteristics:
The implementation in general terms uses the rounding problem in this tutorial to guide the explanation. That problem is as follows:
In simple terms, the Bayes decision rule is a statistical rule in which the strategy chosen from among several available strategies is the one for which the expected value of the outcome is the greatest. In the case of the probabilistic neural network each of these classes has a probability density function and a probability that each member of the inputs will be in that class. The Bayes decision rule compares the products of the probability density function of the class (the probability (chance) of each input feature being a member of that class) with relative frequency that each input vector is a member of that class. In the example with 500 training examples, the comparison is made for each of the 500 input points. The decision rule is to evaluate the probability density function at each input, weight them, and compare them. The probability density function must be estimated. The Parzen estimator is often used because of its simplicity. This estimator places a Gaussian distribution at each of the data points, in this case each of the 500 data points. There are two classes in this example, 0 and 1 (class 1 and class 2). The Parzen estimator for the probability density function for x to lie in class 1 (pd(x|class=1)) is 
(6)where Ntrain-1 is the number of training sets in class 1, N is the dimension (total number) of input nodes, j is a member of class 1, and σ is the variance (~width of the distribution). This estimator models the data distribution as a sum of distributions centered on the data points that lie within the class. In the case of two classes, the ratio of the probabilities for a test value of xtest to lie in either class is 
(7)where the symbols are defined as in equation (6). Taking the limit as σ approaches 0, x test is classified in class 1 if x test has a point in class 1 data which is closer than the closest point in class 2 data and vice versa. This approach is very similar to a nearest neighbor method of which radial basis functions is one example. In terms of the above example, the network has two outputs, one for the class rounding to 0 and the other for the class rounding to 1. Each training set is considered to be a single input. The number of processing elements (containing the probability density functions) in the pattern layer is equal to or greater than the number of training examples. This figure ) .
shows the netwok for the rounding example. Training is fairly rapid.
Notice that equation 7 is represented by two circles for each hidden layer
computing element, one attached to the output corresponding to class 1 and
the other attached to the output corresponding to class 2. These two nodes
are considered as combined in the algorithm. The differentiation is meant
to show choice.
Five hundred training sets were created about evenly divided between those that round to 1 and those that round to 0. The network shown in this figure .
was trained with 500 computing elements. That is, the hidden layer had 500
computing elements. A different set of 100 randomly
generated input-output combinations were used to test the trained networks.
The calculated outputs were between about -.1 and +1.1. The strategy used
to assess error was to assume that if the value is less than 0.5, the
prediction would have been zero and if the value is greater than or equal to 0.5,
the prediction would have been 1. These values were
compared to the actual outputs to assess error. No attempt was made to compare actual values
because that information did not enter into the decision. This figure
)
shows correct and incorrect responses for the probabilistic neural network .
The network had an error rate for the test set comparable to the back-propagation
network with one or two hidden nodes. Note that the training was not optimized so
a relative judgement cannot be made.
|
|
Previous Page: Radial Basis Function Artificial Neural Network |
|
David C. Silverman, Ph.D. - Primary Consultant |