Introduction to Artificial Intelligence

(1)

Introduction to

Artificial Intelligence

Prof. Dr. Tamer Kawady

Electrical Engineering Dept.

Umm Al-Qura University MAKKAH

(2)

Artificial Neural Networks

(3)

Human Inspired Biological System

(4)

(5)

(6)

(7)

• A neuron receives input from other neurons (typically many thousands).

Inputs sum (approximately). Once input exceeds a critical level, the neuron discharges a spike - an

electrical pulse that travels from the body, down the axon, to the next neuron(s) (or other receptors).

• When the input exceeds a threshold the neuron sends an electrical spike that travels from the body, down the axon, to the next neuron(s)

(8)

(9)

• 200 billion neurons, 32 trillion synapses

• Element size: 10^-6m

• Energy use: 25W

• Processing speed: 100 Hz

• Parallel, Distributed

• Fault Tolerant

• Learns: Yes

• Intelligent/Conscious:

• 1 billion bytes RAM but trillions of bytes on disk

• Element size: 10^-9m

• Energy watt: 30-90W (CPU)

• Processing speed: 10⁹Hz

• Serial, Centralized

• Generally not Fault Tolerant

• Learns: Some

• Intelligent/Conscious:

The Computer vs. The Brain

(10)

 Neural networks technology is not trying to produce biological machine

 but is trying to mimic

nature’s approach in order to

mimic some of nature’s

capabilities.

(11)

It resembles the brain in two respects:

1. Knowledge is acquired by the

network through a learning process.

2. Interneuron connection strengths

known as synaptic weight are used

to store the knowledge.

(12)

• A NN is a machine learning approach inspired by the way in which the brain performs a particular learning task:

• Knowledge about the learning task is given in the form of examples.

• Inter neuron connection strengths (weights) are used to store the acquired information (the training examples).

• During the learning process the weights are modified in order to model the particular learning task correctly on the training examples.

(13)

(14)

A Simple Artificial Neuron

The basic computational element (model neuron) is often called a node or unit. It receives input from some other units, or perhaps from an external source.

Each input has an associated weight w, which can be modified so as to model synaptic learning. The unit computes some function f of the weighted ^sum of its inputs

•Its output, in turn, can serve as input to other units.

•The function f is the unit's activation function.

In the simplest case, f is the identity function, and the unit's output is just its net input. This is called a linear unit.

(15)

●The bias b has the effect of applying a transformation to the weighted sum u

v = u + b

●The bias is an external parameter of the neuron. It can be modeled by adding an extra input.

●v is called induced field of the neuron

b w

x w

v

^j

m j

j



 

 0

0

(16)

Input signal

Synaptic

Summing function Bias

b

Activation function Local

Field

v ^Outputy

x₁

x₂

x_m

w₂

w_m w₁

 

  ( )

Artificial Neuron

(17)

Input signal

Synaptic weights

Summing function

Activation function Local

Field

v ^Outputy

x₁

x₂

x_m

w₂

w_m w₁

 

  ( )

w₀ x₀= +1

•Bias is an external parameter of the neuron. Can be modeled by adding an extra input.

b w

x w

v

^m ^j

j



 

 0

0

(18)

●The choice of activation function determines the neuron model.

Examples:

●step function:

●ramp function:

●sigmoid function with z,x,y parameters

●Gaussian function:









 



 



  



2

2 exp 1 2

) 1

( 





 v  ^v

) exp(

1 ) 1

(v z xv y





 

 

















otherwise ))

/(

) )(

((

if

if )

(

c d a b c v a

d v b

c v a

 v







 

c v b

c v v a

if

if )

(

(19)

c b

a

c d

b

a

Ramp Step

Sigmoid

Gaussian

(20)

1. Nonlinearity

2. Input-Output Mapping 3. Adaptivity

4. Very fast Response (parallel computation) 6. Fault Tolerance

7. Implementability

8. Uniformity of Analysis and Design 9. Neurobiological Analogy

Benefits of ANNs

(21)

The Perceptron

(22)

The Perceptron

• We can connect any number of McCulloch-Pitts neurons together in any way we like.

• An arrangement of one input layer of McCulloch-Pitts neurons feeding forward to one output layer of McCulloch-Pitts

neurons is known as a Perceptron.

(23)

Input layer of

source nodes

Output layer of

neurons

Single-layer Feedforward ANN

(24)

Input layer

Output layer

Hidden Layer

Multi-layer Feedforward ANN

(25)

• Logical AND

x1 x2 y

0 0 0

0 1 0

1 0 0

1 1 1

x1

x2

y 1

1

θ=2

Implementing Gate AND with Perceptron

(26)

• Logical OR

x1 x2 y

0 0 0

0 1 1

1 0 1

1 1 1

x1

x2

y 2

2

θ=2

Implementing Gate OR with Perceptron

(27)

• Logical NOT x1 y

0 1

1 0

x1

y -1 θ=2

2 bias

1

Implementing Gate NOT with Perceptron

(28)

Finding the Weights Analytically

• We have two weights w1 and w2 and the threshold q,

and for each training pattern we need to satisfy

(29)

The Perceptron

(30)

• Logical XOR

x1 x2 y

0 0 0

0 1 1

1 0 1

1 1 0

x1

x2

y

?

Implementing Gate XOR with Perceptron

(31)

Finding the Weights Analytically

• For the XOR network

• Clearly the second and third inequalities are incompatible with the fourth, so there is in fact no solution. We need more complex

networks, e.g. that combine together many simple networks, or use different activation/thresholding/transfer functions.

(32)

Implementing Gate XOR with Feedforward network

(33)

Summary of Applications

• Function approximation

• Pattern recognition

• Clasification

• Signal processing

• Modeling

• Control

(34)

• An important application of neural networks is pattern recognition. Pattern recognition can be implemented by using a feed-forward neural network that has been trained accordingly.

• During training, the network is trained to associate outputs with input patterns.

• When the network is used, it identifies the input pattern and tries to output the associated output pattern.

• The power of neural networks comes to life when a pattern that has no output associated with it, is given as an input.

• In this case, the network gives the output that corresponds to a taught input pattern that is least different from the given pattern.

Pattern Recognition

(35)

• Suppose a network is trained to recognize the patterns T and H. The associated patterns are all black and all white respectively as shown above.

Pattern Recognition

(36)

Since the input pattern looks more like a ‘T’, when the network classifies it, it sees the input closely resembling

‘T’ and outputs the pattern that represents a ‘T’.

Pattern Recognition

(37)

Pattern Recognition

The input pattern here closely resembles ‘H’ with

a slight difference. The network in this case

classifies it as an ‘H’ and outputs the pattern

representing an ‘H’.

(38)

Pattern Classification

• Speech Recognition and Speech Synthesis

• Classification of radar/sonar signals

• Remote Sensing and image classification

• Handwritten character/digits Recognition

• Credit card application screening

• Data mining, Information retrieval

(39)

Control, Time series, Estimation

• Machine Control/Robot manipulation

• Financial/Scientific/Engineering Time series forecasting.

• Inverse modeling of vocal tract

(40)

Optimization

• Traveling sales person

• Multiprocessor scheduling and task assignment

• VLSI placement and routing