PDF Artificial Intelligence: An Introduction - Seoul National University

(1)

Artificial Intelligence: An Introduction Artificial Intelligence: An Introduction

Byoung-Tak Zhang

School of Computer Science and Engineering Seoul National University

E-mail: [email protected] http://bi.snu.ac.kr/

Can Machines Think?

The Turing Test

Computing Machinery and Intelligence [Turing, 1950]

(2)

3

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Chess Playing Chess Playing

화성탐사화성탐사 로봇로봇 소저너소저너

(3)

5

자연자연 언어언어 처리처리

z

다의성

(Polysemy)

4 I keep the money in the bank.

4 I walk along the bank of the river.

z

중의성

(Ambiguity)

4 Time flies like an arrow.

4 I saw a man with a telescope.

z

다양성

(Diversity)

4 She sold him a book for five dollars.

4 He bought a book for five dollars from her.

z

전문가전문가 시스템시스템

(4)

7

웹웹 정보검색정보검색

Text Data

Classification System

Information Filtering System

question user profile

feedback

answer

DB

Location Date

DB Record

DB Template Filling

& Information Extraction System Information Filtering

Information Extraction

filtered data

Text Classification Preprocessing and Indexing

History of AI History of AI

z Early enthusiasm (1950’s & 1960’s)

4 Turing test (1950) 4 1956 Dartmouth conference

4 Emphasize on intelligent general problem solving z Emphasis on knowledge (1970’s)

4 Domain specific knowledge 4 DENDRAL, MYCIN

z AI became an industry (late 1970’s & 1980’s)

4 Knowledge-based systems or expert systems

4 Wide applications in various domains

(5)

9

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Artificial Intelligence (AI) Artificial Intelligence (AI)

Symbolic AI

Rule-Based Systems

Connectionist AI Neural Networks

Evolutionary AI Genetic Algorithms

Molecular AI:

DNA Computing

Research Areas and Approaches Research Areas and Approaches

Artificial Intelligence

Research

Rationalism (Logical) Empiricism (Statistical) Connectionism (Neural) Evolutionary (Genetic) Paradigm

Application

Intelligent Agents Information Retrieval Electronic Commerce Data Mining

Bioinformatics

Natural Language Proc.

Expert Systems

Learning Algorithms

Inference Mechanisms

Knowledge Representation

Intelligent System Architecture

(6)

11

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

지능형지능형 에이전트에이전트

Autonomous Agents

Biological Agents Robotic Agents Computational Agents

Software Agents Artificial Life Agents

Entertainment Agents Task-specific

Agents

Viruses

Applications of Intelligent Agents Applications of Intelligent Agents (1) (1)

z E-mail Agents

4 Beyond Mail, Lotus Notes, Maxims z Scheduling Agents

4 ContactFinder z Desktop Agents

4 Office 2000 Help, Open Sesame

(7)

13

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Applications of Intelligent Agents Applications of Intelligent Agents (2) (2)

z News-service Agents

4 NewsHound, GroupLens, FireFly, Fab, ReferralWeb, NewT

z Comparison Shopping Agents

4 Mysimon, BargainFinder, Bazzar, Shopbor, Fido z Brokering Agents

4 PersonalLogic, Barnes, Kasbah, Jango, Yenta z Auction Agents

4 AuctionBot, AuctionWeb z Negotiation Agents

4 DataDetector, T@T

Computers Meet Biosciences Computers Meet Biosciences

Bioinformation Technology (BIT)

BT IT

(8)

15

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

AI in Life Sciences AI in Life Sciences

Structure analysis Structure analysis

4 Protein structure comparison 4 Protein structure prediction 4 RNA structure modeling

Pathway analysis Pathway analysis 4 Metabolic pathway 4 Regulatory networks Sequence analysis Sequence analysis 4 Sequence alignment

4 Structure and function prediction 4 Gene finding

Expression analysis Expression analysis 4 Gene expression analysis 4 Gene clustering

“Owing to this struggle for life, any variation, however slight and from whatever cause proceeding, if it be in any degree profitable to an individual of any species, in its infinitely complex relations to other organic beings and to external nature, will tend to the

preservation of that individual, and will generally be inherited by its offspring.”

Origin of Species “Charles Darwin (1859)”

Evolutionary Computation Evolutionary Computation : : Nature as Computer

Nature as Computer

(9)

17

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Genetic Algorithms Genetic Algorithms

solutions

1100101010 1011101110 0011011001 1100110001

110010 1110 101110 1110 110010 1010

crossover crossover

mutation mutation

00110 101110 1010

1001 1

00110 0 1001

evaluation evaluation

1100101110 1011101010 0011001001 solutions

fitness computation roulette

wheel

selection selection new

population

encoding

chromosomes

Hot Water Flashing Nozzle (1) Hot Water Flashing Nozzle (1)

Start

Hot water entering Steam and droplet at exit

At throat: Mach 1 and onset of flashing

Application Example 1 Application Example 1

Hans-Paul Schwefel

performed the original

experiments

(10)

19

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Concrete Shell Roof Concrete Shell Roof

under own and outer load (snow and wind)

Height 1.34m

Spherical shape Optimal shape

Half span 5.00m

Savings : 36% shell thickness 27% armation max min

Orthogonal bending strength

ϕ

→ m

Application Example 3 Application Example 3

Cooperating Robots (3) Cooperating Robots (3)

Cooperating Autonomous Robots

Application Example 13

(11)

21

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Co- Co -evolving Soccer evolving Soccer Softbots Softbots (1) (1)

z At RoboCup there are two "leagues": the "real" robot league and the "virtual" softbot league

z How do you do this with GP?

4 GP breeding strategies: homogeneous and heterogeneous

4 Decision of the basic set of function with which to evolve players 4 Creation of an evaluation environment for our GP individuals

Application Example 14 Application Example 14

Co- Co -evolving evolving Soccer

Soccer Softbots Softbots With Genetic With Genetic Programming Programming

Co Co - - evolving Soccer evolving Soccer Softbots Softbots (2) (2)

z Initial Random Population Application Example 14

(12)

23

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Co Co - - evolving Soccer evolving Soccer Softbots Softbots (3) (3)

z Kiddie Soccer Application Example 14 Application Example 14

Co Co - - evolving Soccer evolving Soccer Softbots Softbots (4) (4)

z Learning to Block the Goal Application Example 14

(13)

25

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Co Co - - evolving Soccer evolving Soccer Softbots Softbots (5) (5)

z Becoming Territorial Application Example 14 Application Example 14

1000- 1000 -Pentium Beowulf Pentium Beowulf- -Style Style

Cluster Computer for Parallel GP

(14)

27

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

컴퓨터와컴퓨터와 생명체의생명체의계산계산 속도와속도와 기억기억 용량의용량의 비교비교

( ( Moravec, 1988) Moravec , 1988)

Von Neumann

Von Neumann’ ’s s The Computer The Computer and the Brain (1958)

and the Brain (1958)

(15)

29

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

The Computer and the Brain The Computer and the Brain

- 10 billion neurons (100 trillion synapses) - Distributed processing - Nonlinear processing - Parallel processing - Carbon-based (wet) - Less than 1 million processors

(10 –9 sec each, neuron: 10 -3 sec) - Central processing

- Arithmetic operation (linearity) - Sequential processing

- Silicon-based (dry)

From Biological Neurons to From Biological Neurons to Artificial Neurons

Artificial Neurons

(16)

31

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Activation Function Scaling Function

Output Comparison Information Propagation

Error Backpropagation

Input x

₁

Input x

₂

Input x

₃

Output

Input Layer Hidden Layer Output Layer Weights

Activation Function

Multilayer

Multilayer Perceptron Perceptron (MLP) (MLP)

∈ ∑

−

≡

outputs k

k k

d w t o

E ( ) 2

2 ) 1 ( r

i i

i

w

w E w w

w ∂

− ∂

= Δ Δ +

← , η

x o = f (x )

Application Example:

Autonomous Land Vehicle (ALV) Autonomous Land Vehicle (ALV)

z NN learns to steer an autonomous vehicle.

z 960 input units, 4 hidden units, 30 output units

z Driving at speeds up to 70 miles per hour

ALVINN System

(17)

33

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Neural Nets for Face Recognition

960 x 3 x 4 network is trained on gray-level images of faces to predict whether a person is looking to their left, right, ahead, or up.

Humans and Computers Humans and Computers

Current Computers

What Kind of Computers?

Human Computers

The Entire Problem Space

(18)

35

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

What is

What is the information the information processing principle

processing principle underlying underlying human intelligence?

human intelligence?

Mind, Brain, Cell, Molecule Mind, Brain, Cell, Molecule

Brain

Cell Mind

Mind

(19)

37

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Molecular Basis of Learning and Memory Molecular Basis of Learning and Memory in the Brain

in the Brain

Principles of Information Processing in the Principles of Information Processing in the Brain

Brain

z The Principle of Uncertainty

4 Precision vs. prediction

z The Principle of Nonseparability “UN-IBM”

4 Processor vs. memory z The Principle of Infinity

4 Limited matter vs. unbounded memory z The Principle of “Big Numbers Count”

4 Hyperinteraction of 10 11 neurons (or > 10 17 molecules) z The Principle of “Matter Matters”

4 Material basis of “consciousness” [Zhang, 2005]

(20)

39

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Unconventional Computing Unconventional Computing

z Quantum Computing 4 Atoms

4 Superposition, quantum entanglements z Chemical Computing

4 Chemicals

4 Reaction-diffusion computing z Molecular Computing

4 Molecules

4 “Self-organizing hardware”

Molecular Computing:

BioMolecules

BioMolecules as Computer as Computer

(21)

41

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

DNA Molecules for Information DNA Molecules for Information Storage and Processing

Storage and Processing

DNA memory strands

a t c g g t c a t a

g c a c t

0 0 0

a t c g g t c a t a

1 0 1

t a g c c c g t g a

Writing: make DNA sequences

Reading: hybridization and readout

Bio- Bio -Lab Procedure Lab Procedure

1. Sequence Design and Synthesis 1. Sequence Design and Synthesis

2. Hybridization 2. Hybridization

3. Ligation 3. Ligation

4. Learning (Affinity Separation) 4. Learning (Affinity Separation)

5. Classification 5. Classification Initial Version Space

cs four

staff ee

?

status

?

dept

five faculty

?

floor

Primer Sticky End

cs faculty four

cs faculty five

cs faculty ?

cs staff four

cs ? ?

ee faculty ?

ee ? ?

? ? ?

...

(22)

43

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Molecular Computers vs. Silicon Computers Molecular Computers vs. Silicon Computers

Deterministic Probabilistic

Reproducibility

Very high Ultrahigh

Density

High Low

Reliability

Ultra-fast (nanosec) Fast (millisec)

Speed

Sequential Massively parallel

Parallelism

Fixed (synchronous) Amorphous (asynchronous)

Configuration Communication Medium Processing

2D switching 3D collision

Solid (dry) Liquid (wet) or Gaseous (dry)

Hardwired Ballistic

Silicon Computers Molecular Computers

Molecular Information Processing Molecular Information Processing

z MP4.avi

(23)

45

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

x8 x9

x12 x1

x2

x3

x4

x5

x6

x7 x10

x11 x13 x14 x15

y

= 1 x15

=0 x14

=0 x13

=0 x12

=1 x11

=0 x10

=1 x9

=0 x8

=0 x7

=0 x6

=0 x5

=0 x4

=1 x3

=0 x2

=0 x1

=1

y

= 0 x15

=0 x14

=1 x13

=0 x12

=0 x11

=0 x10

=0 x9

=1 x8

=0 x7

=0 x6

=0 x5

=0 x4

=0 x3

=1 x2

=1 x1

=0

y

=1 x15

=0 x14

=0 x13

=1 x12

=0 x11

=0 x10

=0 x9

=0 x8

=1 x7

=0 x6

=1 x5

=0 x4

=0 x3

=1 x2

=0 x1

=0

4 examples

x

₄

x

₁₀

y=1 x

₁

x

₄

x

₁₂

y=1 x

₁

x

₁₀

x

₁₂

y=1 x

₄

x

₃

x

₉

y=0 x

₂

x

₃

x

₁₄

y=0 x

₂

x

₉

x

₁₄

y=0 x

₃

x

₆

x

₈

y=1 x

₃

x

₆

x

₁₃

y=1 x

₃

x

₈

x

₁₃

y=1 x

₆

1 2 3

1

2

3

y

=1 x15

=1 x14

=0 x13

=0 x12

=0 x11

=1 x10

=0 x9

=0 x8

=1 x7

=0 x6

=0 x5

=0 x4

=0 x3

=0 x2

=0 x1

4

=0

x

₁₁

x

₁₅

y=0 x

₈

4

Round 1 Round 2 Round 3

Benchmark Problem: Digit Images Benchmark Problem: Digit Images

• 8x8=64 bit images (made from 64x64 scanned gray images)

• Training set: 3823 images

• Test set: 1797 images

(24)

47

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Pattern Information Processing Pattern Information Processing

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1 71319 25313743495561 677379859197 epoch

Classification rate

Order 1 Order 2

Effect of Memory

Effect of Memory- -Chunk Size Chunk Size

5 6 7 8 9

(25)

49

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Effect of the Error Tolerance Level Effect of the Error Tolerance Level

0 2 4 6 8 10

0 2 4 6 8 10 12 14

Corruption intensity

Average error (# of mismatches) tau 0

tau 1 tau 2 tau 3 tau 4

0 2 4 6 8 10

0 2 4 6 8 10 12 14

0 2 4 6 8 10

0 2 4 6 8 10 12 14

0 2 4 6 8 10

0 2 4 6 8 10 12 14

Biological Application Biological Application

120 samples from 60 leukemia patients

Diagnosis Gene expression data

Training with 6-fold validation

Class: ALL/AML

&

(26)

51

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Da Da Vinci’ Vinci ’s Dream of Flying Machines s Dream of Flying Machines

Engines of Flight

(27)

53

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Turing

Turing’ ’s Dream of Intelligent Machines s Dream of Intelligent Machines

Alan Turing (1912-1954)

Computing Machinery and Intelligence (1950)

Computers and Intelligence

(28)

55

(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/

Future Technology Enablers Future Technology Enablers

Source: Motorola, Inc, 2000

Now +2 +4 +6 +8 +10 +12

Full motion mobile video/office

Metal gates, Hi-k/metal oxides, Lo-k with Cu, SOI

Pervasive voice recognition, “smart”

transportation Vertical/3D

CMOS, Micro- wireless nets, Integrated optics

Smart lab-on-chip, plastic/printed ICs, self-assembly

Quantum computer, molecular electronics

Bio-electric computers

Wearable communications, wireless remote medicine,

‘hardware over internet’ ! 1e6-1e7 x lower power

for lifetime batteries

True neural computing

Artificial Intelligence (AI) Artificial Intelligence (AI)

Symbolic AI

Rule-Based Systems Connectionist AI

Neural Networks