Implement ANFIS

4.3 The proposed research framework

4.3.3 Implement ANFIS

As discussed previously, there are two types that result from combining neural network and fuzzy logic:

(1) Neuro-Fuzzy System (NFS) (2) Fuzzy Neural Network (FNN), and we mentioned the prevalent technique among researchers is NFS; also the most popular algorithm in NFS is the Adaptive Neuro- Fuzzy Inference System (ANFIS). In this thesis ANFIS will be used as a classification technique with modification on the attribute selection process where the IG selection method is used to determine the best attributes that will be suitable for the prediction model, the NFS is unsuitable for large number of

University

Major

Computer science

Computer information system

Software engineering

total

Balqa Applied university

356 71 133 560

Philadelphia university

144 - 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11

771 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 11 1 221

Alzaytonah university

150 69 95 314

Total 656 140 299 1095

attribute due to computational time problem also the attribute added incrementally to the ANFIS to study the effects each attribute on the accuracy of the produced model.

A lot of attributes in the dataset that is used in this thesis contain crisp values that make the prediction process inaccurate; the input attributes in this thesis are converted into fuzzy inputs by the fuzzification process. Fuzzy logic in ANFIS is used to define the weights w, from fuzzy sets, in NN. The w is computed as a membership function for each input.

ANFIS is an adaptive technique between Sagano’s fuzzy inference system and the concept of the neural network approach. The adaptions between the two approaches consist of a set of functions which are represented as a graphical network. The network is constructed from a set of nodes that are distributed into five layers.

To demonstrate how ANFIS works, let us take two inputs variables A1 and A2, and one output variable O1. This inputs variable represents linguistic values such as high, mid and low, and the output value is always crisp value.

Figure 4. 5 ANFIS concept demonstration

As shown in Figure (4.5), the two inputs are represented using the suitable membership function distribution (such as triangle, Gaussian or trapezoidal). In ANFIS the output O is not represented in any membership function distribution, because it uses Takagi and Sugeno’s approach which is unlike the Mamdani approach where the output is represented using membership function distribution. In ANFIS the output is expressed as a function of input parameters:

4.1 O=f (A1, A2)

The output function can be considering as a linear or nonlinear function. The linear function of the input parameter is used most of the time.

We can express the output function in linear wise like the equation 4.2.

4.2 Oi=piA1+qiA2+ri

Where i=1,2, 3…., n

In the equation 4.2, A1 and A2 represents the input parameters, also pi , qi and ri represent the coefficients of the equation. These coefficients update after each iteration of ANFIS using least square error techniques with backpropagation or using any natural inspired techniques like Genetic Algorithm, to get the best value from them. The notation i refers to the number of rules which generate from the linguistic variable of the input parameters.

The ANFIS algorithm comprises five layers as shown in Figure (4.6). Where the linguistic variable is received as input then five steps are done to give crisp value, which is almost similar to the consequence of the rule (i.e. the class employed, unemployed or other). The rules are evaluated according to the training dataset. The layers are:

5. Layer 1: the input is the linguistic values and the output is the computed membership function for each linguistic value.

6. Layer 2: the inputs are the values of the membership function of a particular rule and the output W is the result of applying any t-norm (such as prod. or min) on the output of the previous layer.

7. Layer 3: the input is all the w values of the previous layer and the output is the normalized w value for a specific rule.

8. Layer 4: the input is the input parameters (A1 and A2) and the normalized values that are generated from the rules, and the outputs are

W1O1,w2O2,……,WnOn

where Oi=piA1+qiA2+ri

9. Layer 5: the output of this layer is the result of the summation process on the values that were obtained from layer four.

Figure 4. 6 Adaptive Neuro-Fuzzy Inference System

In this thesis the number of input parameters over 20, but will show here ANFIS architecture for two inputs parameter to simplified the presentation.

In this thesis we used incremental techniques to produce the classifier. We start by three parameters as input to ANFIS, then produce the classifier using the training dataset, then find the accuracy of the classifier using testing dataset. The number of input parameter increased incrementally by one. Our strategy was to build a set of classifiers using different number of input parameters. The computation time and accuracy of each produced classifier recorded and compared with another classification technique that considered as a classification benchmark. In this thesis we try to build classifier with best number of input parameters and the most suitable attributes. The final result that we try to get is the best classifier with better attributes with considers the computational time that needed by ANFIS to produce the classifier.

Figure 4.7 shows the network with two input values x and y, where each input has two linguistic values, A1 , A2 and B1, B2, respectively.

Figure 4. 7 ANFIS architecture

Here the ANFIS layers will be demonstrated in more detail and the customization of it for this research is shown clearly:

Layer 1: The fuzzification process is applied in this layer where each node produces the membership degree for each input label. The linguistic input value A1i is the input to nodei, and the membership function μ(A) is the output Oi,1of node i:

4.3 O

1,i= μ

Ai

(x)

The membership function for the linguistic value A might be used any type of parameterized

μ

function (such as triangle, sigmoid and trapezoidal). In this thesis we apply Gaussian distribution to make the result more accurate. The membership function μ(X):

4.4 ?????

^?

???^???_? ?^??

Select best IG

Add more attributes incrementally

where a, b, c is the parameter set. The bell shapes change according to these values.

Parameters in that layer are called premise parameters.1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Layer 2: In this layer the nodes manipulate each rule using any T-norm such as min or prod operator. In this thesis we use product operation.

4.5

?

_?

? ?

_?

???? ?

_?

???

Similarly, we compute the w2,w3,….,wn.

Layer 3: In this layer the normalized firing strength for each rule is calculated by each node. The result is a normalized firing strength.

4.6

? ?????

_{? ?}^??

????

Similarly, we compute the w2. So the output O3,I is the normalized firing strength .

O

3,1

= ? ????

_?

Layer 4: The output of the nodes in this layer is produced from the multiply of normalized firing strength for each corresponding fired rule.

4.7

?

_???

? ? ?????

_{? ?}

w

here

4.8

?

_?

? ?

_?

? ? ?

_?

? ? ?

_?

Parameters p,q and r in this layer are called consequent parameters.

Layer 5: The sum of all outputs generated from the nodes in layer 4 is the output of this layer in this layer a single node that sums the overall output all nodes in layer 4. The output O6,1is 4.9

?

_???

? ? ?????

_{? ?}

? ? ?????

_{? ?}

The performance of ANFIS mainly depends of the membership distribution function used, as well as the coefficients of the consequence function p, q and r.

The ANIFS algorithm can be tuning or training or in other word, the coefficients of the function can be optimized using any optimization tool like Genetic algorithm or backpropagation algorithm. In this thesis we use Branch and bound algorithm for tuning purposes.