21 ISSN 2085-1944
Kampus ITS Keputih, Sukolilo Surabaya
2
Sekolah Tinggi Teknik Surabaya Ngagel Jaya Tengah 73-77 Surabaya
admin@hansmichael.com 1, hansen1814@yahoo.com 2
ABSTRACT
Hybrid Intelligent Systems are systems that combine several intelligent technologies in order to create new systems that have strength from several systems and cover the weakness from others. Combination Neural Network and Expert System result a Neural Expert System. By combining Neural Network and Expert System, they created a new system that has ability to learn and can give explanation to user about conclusion that have been made. There are several approaches that integrate neural networks and symbolic rules. In this paper, we try to create an approach that can classify well and it has priority of the most necessary information so that it is capable in dealing with incomplete data. Keywords: Neural Expert System, Hybrid
Intelligent Systems, Connectionist Expert System.
1
INTRODUCTION
There are several intelligent technologies, such as: probabilistic reasoning, fuzzy logic, neural networks, and evolutionary computation. Each technology has its strength and weaknesses. We noticed that in many real-world applications, we would need not only to acquire knowledge from various sources, but also to combine different intelligent technologies.
Knowledge in a rule-based expert system is represented by IF-THEN production rules collected by observing human experts. This task, called knowledge acquisition, is difficult and expensive. In addition, once the rules are store in the knowledge base, they cannot be modified by the expert system itself. Expert system cannot learn from experience or adapt to new environments.
Knowledge in neural networks is stored as synaptic between neurons. This knowledge is obtained during the learning phase when a training set of data is presented to the network. The network propagates the input data from layer to layer until the output data is generated. If it is different from
the desire output, an error is calculated and propagated backwards through the network. Unlike expert systems, neural networks learn without human intervention.
In expert systems, knowledge can be divided into individual rules and user can see and understand the piece of knowledge applied by the system. In contrast, in neural networks, one cannot select a single synaptic weight as a discrete piece of knowledge. Here the knowledge is embedded in the entire network. Any change of a synaptic weight may lead to unpredictable results.
An expert system cannot learn, but can explain how it arrives to a particular solution. A neural network can learn, but acts as a black box. Thus by combine the advantages of each technology we can create a more powerful and effective expert system. Learning, generalization, robustness and parallel information processing make neural networks a right component for building a new breed of expert systems.
2
ARCHITECTURE OF NEURAL
EXPERT SYSTEMS
A Neural Expert System can extract IF-THEN rules from the neural network, which enable it to justify and explain its conclusion. Neural networks only can produces weights from training as knowledge. But it cannot explain the solution to user. Thus will be produced IF-THEN rules from the calculation of the weights.
Figure 1 shows the basic structure of a neural expert system. It is consists of five parts:
• Inference Engine:
The heart of a neural expert system is the inference engine. This controls the information flow in the system and initiates inference over the neural knowledge base. A neural inference engine also ensures approximate reasoning.
Figure 1. Neural Expert System’s Architecture
Neural expert systems use a trained neural network in place of knowledge base. The neural network is capable of generalization. In other words, the new input data does not have to precisely match the data that was used in network training. This allows neural expert system to deal with incomplete and noisy data. This ability is called approximate reasoning.
• Rule Extraction:
The rule extraction unit examines the neural knowledge base and produces the rule implicitly from the trained neural networks.
• Explanation Facilities:
The explanation facilities explain to the user how the neural expert system arrives at a particular solution when working with the new input data.
• User Interface:
The user interface provides the communication between the user and the neural expert system.
3
RECENT WORKS
There are two approaches that can be used to draw a conclusion with neural expert system. Each approach have their own strength and weakness.
3.1
First Approach: Negnevitsky
This approach introduced by Negnevitsky in 2002. this approach needs target object, and then system will give questions corresponding to the object. Number of questions that given to user depend on the user answers that given before.
This is uncommon as a classification method, because it needs a class target object wanted to achieve, but the main objective of a classification is to find the target object based on characteristics that are known. However, by knowing what the target object is, the system can provide questions that related to the object, so not
all information are required. Only the most important information are required to draw a conclusion.
How does a neural expert system extract rules that justify its reference? Neurons in the network are connected by links, each neurons has a numerical weight attached to it. The weights in a trained neural network determine the strength or importance of the associated neuron inputs. This characteristic is used for extracting rules.
Then system will determine whether that obtained information is sufficient to draw a conclusion or not. This following heuristic can be applied (Gallant, 1993). An inference can be made if the known net weighted input to a neuron is greater than the sum of absolute values of the weights of the unknown inputs.
3.2
Second Approach: Ioannis
Second approach for neural expert system is introduced by Ioannis Hatzilygeroudis and Jim Prentzas in 2004. Inference engine in this approach is based in backward chaining algorithm. Inference engine uses the working memory, which contains facts required from the user to the inference process.
The hybrid inference engine implements the way neurules co-operate to reach a conclusion. It is based on the firing ratio, a measurement of the firing intention of a neurule, which is similar to the convergence ratio, introduced in (Ghalwash 1998). However it is possible to deduce the output of a neurule without knowing all of the condition values. To achieve this, known-sum and remaining-sum must be defined as follows:
∑
Where E is the set of evaluated conditions, U the set of unevaluated conditions and Ci is the value
of condition condi. So, known-sum is the weighted
sum of the values of the already known or evaluated conditions (inputs) of the corresponding neurule and remaining sum represents the largest possible weighted sum of the remaining or unevaluated conditions of the neurule. If |kn-sum| > rem-sum for a certain neurule, then evaluation of its conditions can stop, because its output can be deduced regardless of the values of the unevaluated conditions. In this case, its output is guaranteed to be '-1' if kn-sum < 0 whereas it is ‘1’, if kn-sum > 0.
Inference Engine
Neural Knowledge Base Rule Extraction
ISSN 2085-1944 So, we define the firing ratio (fr) of a neurule as
follows:
The firing ratio of a neurule is an estimate of its intention that its output will become ‘±1’. Whenever fr > 1, the values of the evaluated conditions can determine the value of its output, regardless of the values of the unevaluated conditions. The rule then evaluates to ‘1’ (true), if kn-sum > 0 or to ‘-1’ (false), if kn-sum < 0.
4
A NEW APPROACH TO NEURAL
EXPERT SYSTEM
On the third section, the approaches of neural expert system are described with their own strength and weaknesses. Now a new approach is introduced to cover the weakness from those approaches. Negnevitsky is good in expert system, but uncommon as a classification method. Ioannis is good in classification, but it cannot determine the priority of information that mostly needed. By combining the benefits from those approaches, it is obtained a new system that can classify well and it has priority of the most necessary information so that it can dealing with incomplete data.
4.1
Algorithm
Algorithm for this approach (hybrid) is similar with the Negnevitsky algorithm. But the main difference is the best attribute always recalculate in each iteration. So the priority of information that needs by the system can be different based on user input. More formally, the inference algorithm is as follows:
1. Initialize array with the weights from each neuron for sorting process.
2. Eliminate attributes that do not contributing. An input is considered to be not contributing if it does not move the net weighted input in the opposite direction (combination weights must be positives and negatives).
3. While the system has not reached a final conclusion and there is unknown information do:
3.1. Search the most important attribute from the weights and the object is valid. The most important attribute is determined by maximum absolute value from the weights.
3.2. Read input from user based on the most important information.
3.3. Update values for the kn-sum and rem-sum for all target objects. If |known value| > unknown value then stop with success. kn-sum=kn-sum+weightQuestionIndex
rem-sum=rem-sum-weightQuestionIndex
3.4. Eliminate objects that do not appropriate with the user input or move the net weighted input in the opposite direction (negative value). If an object is eliminated then the object is set into invalid. Thus it will not be used for determining the most important attribute for the next iteration. 4. Determine conclusion if success, and the target
is an object with the biggest kn-sum. Otherwise if not success then stop (failure) and determine object as unknown.
4.2
Case Study
We use an example of Contact Lens dataset to illustrate the functionalities of this approach. It is a small dataset that consists of 24 instances and four input attributes. There are three values of patient age (young, pre-presbyopic, presbyopic), two spectacles values (myope, hypermetrope), two astigmatics values (no, yes), two tear production rates values (reduced, normal), and three target class of recommended lenses (none, soft, hard). Content of contact lens dataset is shown in Table 2. After pre-processing dataset and training process, knowledge base for neural knowledge base is as follows:
Table 1. Neural Knowledge Base
Table 2. Contact Lens Dataset
After initialize all the weights, then we must eliminate attribute age that do not contributing (algorithm step 2), because all weights in each object are positives and negatives. And finally there are only three input attributes that can be used (spectacle, astigmatic, and tear production rate). Now start calculation for the first iteration.
In the first iteration, the most important attribute is tear production rate (16.83). Then system will ask user to answer what the input for attribute tear production rate. If user answers reduced (-1) then the calculation for kn-sum and rem-sum as follows:
None: kn-sum = -11.86 * (-1) = 11.86 rem-sum = |0.04| + |0.83| = 0.87 kn-sum>rem-sum success Soft: kn-sum = 16.83 * (-1) = -16.83
rem-sum = |7.84| + |-16.83|= 24.67 kn-sum<rem-sum check next iteration Hard: kn-sum = 16.16 * (-1) = -16.16
rem-sum = |-8.25| + |16.16|= 24.41
kn-sum>rem-sum check next iteration Because target object None is success then the system will conclude that:
IF Tear production Rate IS Reduced THEN Recommended Lens is None.
From the rules above, we can check the accuracy that all inputs attribute tear production rate is reduced (12 instances) then Recommended Lens is none (also 12 instances). This make the accuracy is 100%. Now let’s try another input if tear production rate is normal (+1). The calculation for kn-sum and rem-sum as follows:
None: kn-sum = -11.86 * (+1) = -11.86 rem-sum = |0.04| + |0.83| = 0.87 kn-sum>rem-sum check next iteration Soft: kn-sum = 16.83 * (+1) = 16.83
rem-sum = |7.84| + |-16.83|= 24.67 kn-sum<rem-sum check next iteration Hard: kn-sum = 16.16 * (+1) = 16.16
rem-sum = |-8.25| + |16.16|= 24.41 kn-sum>rem-sum check next iteration No
Age of The
Patient Spectackle Astigmatic
Tear Production
Rate
Recommended Lens
1 young myope no reduced none
2 young myope no normal soft
3 young myope yes reduced none
4 young myope yes normal hard
5 young hypermetrope no reduced none
6 young hypermetrope no normal soft
7 young hypermetrope yes reduced none
8 young hypermetrope yes normal hard
9 pre-presbyopic myope no reduced none
10 pre-presbyopic myope no normal soft
11 pre-presbyopic myope yes reduced none
12 pre-presbyopic myope yes normal hard
13 pre-presbyopic hypermetrope no reduced none 14 pre-presbyopic hypermetrope no normal soft 15 pre-presbyopic hypermetrope yes reduced none 16 pre-presbyopic hypermetrope yes normal none
17 presbyopic myope no reduced none
18 presbyopic myope no normal none
19 presbyopic myope yes reduced none
20 presbyopic myope yes normal hard
ISSN 2085-1944 Because all target objects not success, thus
system will check the second iteration. And the most important attribute for the second iteration is astigmatic (-16.83). Then system will ask user again to answer what the input for attribute astigmatic. If user answers no (-1) then the calculation for kn-sum and rem-sum as follows:
None: kn-sum = -11.86 + (0.83) * (-1) = -12.69 rem-sum = |0.04| = 0.04 kn-sum>rem-sum check next iteration Soft: kn-sum = 16.83 + (-16.83)*(-1) = 33.66
rem-sum = |7.84| = 7.84 kn-sum<rem-sum Success
Hard: kn-sum = 16.16 + (16.16)*(-1) = 0 rem-sum = |-8.25| = 8.25 kn-sum>rem-sum check next iteration Target object Soft is success, so system will conclude that:
IF Tear production Rate IS Reduced AND Astigmatic IS No
THEN Recommended Lens is Soft
From the rules above, we can check the accuracy that all inputs attribute tear production rate is reduced and astigmatic is no (6 instances) then Recommended Lens is none (5 instances, 1 instance is recommended lens none). This make the accuracy is 5/6*100%. = 83.33%.
If user input is unknown value, then kn-sum is the average value of the significant factors of all the homonymous condition and rem-sum will not change. For example if input attribute age of patient for target object none is unknown, then kn-sum will be increased as much as the average of all age weights (-11.81 -6.32 -6.03) / 3 = -8.05, but the remaining sum will not decrease, because input attribute age is still unevaluated or unknown.
If user input unknown value, it is no need to check the success state, because the kn-sum never more than the rem-sum. And this can be used to increase performance of the system. For the next iteration, the unknown attribute must be flagged so it does not use for the most important attribute again.
From examples above, we can see that the accuracy of this approach is quite high depends on input that given to the system and can deal with noise and incomplete data.
5
CONCLUSION
In this paper, we presented a new approach to neural expert system. Combination of Neural Network and Expert System will create a more powerful and effective expert system. It combines the advantages of all systems and covers other weaknesses. It is obtained a new system that has ability to learn and can give explanation to the user.
The accuracy of the created rules during the testing process is strongly influenced by the users input. The questions that give to user is influenced by weights obtained from the training process.
The first approach of neural expert system (Negnevitsky) is good in expert system and can determine the priority of information that mostly needed. But it is uncommon as a classification method because it needs target object wanted to achieve.
The second approach of neural expert system (Ioannis) is good in classification and can dealing with incomplete data. But the accuracy of this approach is highly influenced by the combination of user inputs, and cannot determine what the most important information. It is better to know what the most important information so can dealing with incomplete data.
The new approach combines the benefits of both previous approaches. Thus it is obtained a new system that can classify well and it has priority of the most necessary information so that it is capable in dealing with incomplete data
REFERENCES
[1] Hatzilygeroudis, I., & Prentzas, J. (2001) HYMES: A Hybrid Modular Expert system with Efficient Inference and Explanation: Proceedings of the 8th Panhellenic Conference on Informatics, Nicosia, Cyprus, November, 2001 Vol.1 422-431. [2] Hatzilygeroudis, I., & Prentzas, J. (2004)
Integrating (Rules Neural Networks) and Cases for Knowledge Representation and Reasoning in Expert Systems.
[3] Kuswara (2005) Paradigma Sistem Cerdas. [4] Nard B. A., Neural Expert Systems: Survey
Paper for Cheng-568.
[5] Negnevitsky, Pearson Education. Hybrid Intelligent System.
[6] Negnevistsky, Pearson Education. Rule Base Expert System.
[8] Venkat Venkatasubramanian, Sourabh Dash, Mano Ram Maurya, Priyan Patkar
