Decision Making Methode

Decision Making Under Uncertainties

Outline

Decision theory

Basic decision analysis

Value of information analysis

Risk profile

Decision theory

Expected value criterion

• Suppose you face a situation where you must choose between alternatives A and B as follows:

• Alternative A: $10,000 for sure.

• Alternative B: 70% chance of receiving $18,000 and 30% chance of loosing $4,000.

What is your personal choice?

• Compare now Alternative B with:

• Alternative C: 70% chance of winning $24,600 and 30% chance of loosing $19,400

• Note that EMV(B) = EMV(C), but are they “equivalent”?

• Alternative C seems to be “more risky” than Alternative B even thought they have the same EMV.

• Conclusion: EMV does not take Risk into account

The Petersburg Paradox

• In 1713 Nicolas Bernoulli suggested playing the following games:

• An unbiased coin is tossed until it lands with Tails

• The player is paid $2 if tails comes up the opening toss, $4 if tails first appears on the second toss, $8 if tails appears on third toss, $16 if tails appears on the forth toss, and so forth

• What is the maximum you would pay to play the above game?

• If we follow the EMV criterion:

• This means that you should be willing to pay up to an infinite amount of money to play the game, but why people are unwilling to pay more than a few dollars?



=

 +



 

 +



 

 +



 



= 



 



=



^ 

=

...

) 8 8 ($

) 1 4 4 ($

) 1 2 2 ($

) 1 2 2 ($

1

1 k

k k

EMV

The Petersburg Paradox

• 25 years later, Nicolas’s cousin, Daniel Bernoulli, arrived at a solution that contained the first seeds of contemporary decision theory

• Daniel reasoned that the marginal increase the value or “utility” of money declines with the amount already possessed.

• A gain of $1,000 is more significant to a poor person than to a rich man through both gain same amount

• Specifically, Daniel Bernoulli argued that the value or utility of money should exhibit some form of diminishing marginal return with increase in wealth:

The measure to use to value the game is then the “expected utility”



^

( )

=



 



= 

1

2 2 1

k

k k

u EU

→ u is an increasing concave function, converge to a finite number

The rules of actional thought

• How a person should acts or decides rationally under uncertainty?

• Answer: by following the following rules or axioms:

• The ordering rule

• The equivalence or continuity rule

• The substitution or independence rule

• Decomposition rule

• The choice rule

• The above five rules form the axioms for Decision Theory

The ordering rule

• The decision maker must be able to state his preference among the prospects, outcomes, or prizes of any deal

• Furthermore, the transitivity property must be satisfied: that is, if he prefers X to Y, and Y to Z, then he must prefer X to Z

• Mathematically,

• The ordering rule implies that the decision maker can provide a complete preference ordering of all the outcomes from the best to the worst

• Suppose a person does not follow the transitivity property: the money pump argument

The equivalence or continuity rule

• Given a prospect A, B, and C such that , then there exists p where 0 < p < 1 such that the decision maker will be indifferent between receiving the prospect B for sure and receiving a deal with a probability p for prospect A and a probability of 1 – p for prospect C

• Given that

• B: certain equivalent of the uncertain deal on the right

• p: preference probability of prospect B with respect to prospects A and C

C B A 

C B

A  

The substitution rule

• We can always substitute a deal with its certainty equivalent without affecting preference

• For example, suppose the decision maker is indifferent between B and the A – C deal below

• Then he must be indifferent between the two deals below where B is substituted for the A – C deal

The decomposition rule

• We can reduce compound deals to simple ones using the rules of

probabilities

• For example, a decision maker should be indifferent between the following two deals:

The choice or monotonicity rule

• Suppose that a decision maker can choose between two deals L₁ and L₂ as follows:

• If the decision maker prefers A to B, then he must prefer L1 to L2 if and only if p1 > p2. That is, if

• In other words, the decision maker must prefer the deal that offers the greater chance of receiving the better outcome

B A 

Maximum expected utility principle

• Let a decision maker faces the choice between two uncertain deals or lotteries L1 and L2 with outcomes A1, A2, …, An as follows:

• There is no loss of generality in assuming that L₁ and L₂ have the same set of outcomes A₁, A₂, …, A_n because we can always assign zero probability to those outcomes that do not exist in either L₁ and L₂.

• It’s not clear whether L₁ or L₂ is preferred

• By ordering rule, let

A

₁

 A

₂

 ...  A

_n

Maximum expected utility principle

• Again, there is no loss of generality as we can always renumber the subscripts according to the preference ordering

• We note that A1 is the most preferred outcome, while An is the least preferred outcome

• By equivalent rule, for each outcome A_i (i =1, …, n) there is a number u_i such that 0 ≤ u_i ≤ 1 and

• Note that u₁ = 1 and u_n = 0. Why?

Maximum expected utility principle

• By the substitution rule, we replace each Ai (i=1,…,n) in L1 and L2 with the above constructed equivalent lotteries

Maximum expected utility principle

• By the decomposition rule, L

₁

and L

₂

may be reduced to equivalent deals with only two outcomes (A

₁

and A

_n

) each having different probabilities

• Finally, by the choice rule, since , the decision maker should prefer lottery L

₁

to lottery L

₂

if and only if

ⁿ

A A₁ 





= =



ⁿ

i

i i n

i

i

i

p u q

u

1 1

Utilities and utility functions

• We define the quantity u_i (i=1,…,n) as the utility of outcome A_i and the function that returns the values u_i given A_i as a utility function, i.e. u(A_i) = u_i

• The quantities

are known as the expected utilities for lotteries L₁ and L₂ respectively

• Hence the decision maker must prefer the lottery with a higher expected utility





= = n

i

i i

n

i

i

iu A q u A

p

1 1

) ( and

) (

Case for more than 2 alternatives

• The previous may be generalized to the case when a decision maker is faced with more than two uncertain alternatives. He should choose the one with maximum expected utility

• Hence

where is the probability for the outcome A_i in the alternative j



=

=

ⁿ

i

i j

j

p

i

u A

Max

1

) ( arg

e alternativ best

j

p

i

Comparing expected utility criterion with expected monetary value criterion

• The expected utility criterion takes into account both return and risk whereas expected monetary value criterion does not consider risk

• The alternative with the maximum expected utility is the best taking into account the trade off between return and risk

• The best preference trade-off depends on a person’s risk attitude

• Different types of utility function represent different attitudes and degree

of aversion to risk taking

Basic Decision analysis

The party problem

• Kim ingin mengadakan pesta ulang tahun. Dia

mempertimbangkan 3 lokasi tempat: outdoor, indoor, teras (porch)

Decision node

Chance node

Nodes

•

Decision node:

•

Berbentuk persegi

•

Melambangkan titik dalam tree yang menyatakan titik pengambilan

keputusan, decision maker mempunyai kebebasan penuh untuk mengambil keputusan

•

Chance node:

•

Berbentuk bulat

•

Melambangkan uncertain variable, decision maker tidak mempunyai kontrol

terhadap outcome variable ini

Solving party problem

Dengan menggunakan 5 rules:

• Ordering rule

• Equivalence rule

• Substitution rule

• Decomposition rule

• Choice rule

Ordering rule

• Possible outcome:

• Outdoor – sunny

• Outdoor – rainy

• Porch – sunny

• Porch – rainy

• Indoor – sunny

• Indoor – rainy

• Objective: memaksimumkan kepuasan

• Best outcome: outdoor -- sunny

• Worst outcome: outdoor -- rainy

Equivalence rule

• Semua intermediate outcome

ditentukan equivalensinya terhadap best outcome dan worst outcome.

Membuat decision tree

Misalkan:

• Probability cuaca besok cerah = 0.4

• Probability cuaca besok hujan = 0.6

Substitution rule – decomposition rule

Choice rule

• Jadi dipilih lokasi pesta indoor dengan probability untuk mendapatkan best outcome terbesar

Expected utility untuk setiap alternatif

Equivalent Monetary or Dollars Values

• Cara lain selain menggunakan utility value adalah menggunakan

equivalent monetary value untuk setiap outcome.

Dollar value vs utility value

Utility function u(x)

Certain equivalent

•

The certainty equivalent (CE) is the amount in which a person is just indifferent between receiving it for sure and an uncertain or risky prospect that might either pays more or less than this amount.

•

The Certainty Equivalent of a deal is the Personal Indifferent Selling Price

(PISP)

•

To find the CE of an alternative, we first compute its expected utility and then take

its inverse to convert it back into equivalent dollar value.

Contoh

_Outdoors:

Expected utility = 0.4.

Hence certainty

equivalent = u-1( 0.4 ) =

$26 Porch:

Expected utility = 0.57.

Hence certainty

equivalent = u-1( 0.57 )

= $40 Indoors:

Expected utility = 0.63.

Hence equivalent

= u-1( 0.63 ) = $46 Note that the best decision is also the one with the highest certainty equivalent.

Mengunakan utility function untuk kasus lain

• Misalkan Kim menghadapi deal sebagai berikut:

• EU = 0,5

• PISP = u

^-1

(0,5) = $32 (dari grafik utility vs dollar value)

Value of information

analysis

The Value of Clairvoyance (perfect information) on Weather

• Let’s use our previous party problem

• Suppose that a clairvoyant offers to tell Kim whether the weather will sunny or rainy tomorrow. However, he charges a fee of $15.

• Should Kim “buy” the information about the weather tomorrow from the

clairvoyance for a fee of $15?

Decision tree

• Conclusion: At a cost of $15, Kim should buy the information.

• Suppose that the cost of the

information is $16 instead? Should Kim buy it or not?

• What if the fee is $20 or $25?

Impact of Changing the Cost of Clairvoyance

• We determine, by resolving the

decision tree repeatedly, the expected utilities for “No clairvoyance” and “Buy Clairvoyance for $x) for values of x in the range $0 to $50.

Certainty equivalent

Kim is indifferent between “Buy clairvoyance” and “No

clairvoyance” when the cost of clairvoyance is about $20.

The Value of Clairvoyance or Expected Value of Perfect Information on an

uncertain variable is the cost of clairvoyance at which the decision maker is just indifferent

between buying and not buying the information.

Interpretation of Value of Information

•

The value of clairvoyance represents the maximum amount one should be willing to pay for the perfect information

•

The value of clairvoyance provides a benchmark against which to compare any

information gathering scheme that may be proposed

•

If the cost of the scheme exceeds the value of clairvoyance, then there is no need to examine the scheme in any further detail.

•

The expected value of perfect information or clairvoyance for Kim is $20. Thus, no

other sources of information about the weather could be worth more than $20 to

her.

Expected Value of Imperfect Information

•

Suppose, instead of clairvoyance, Kim was offered the service of an Acme Rain Detector which will indicate either “Rainy” or “Sunny” with an accuracy of only 80%.

•

That is, if the actual weather is going to be sunny, it will read “sunny” with

probability 0.8, and if the actual weather is going to be rainy, it will read “rainy”

with probability 0.8.

•

The fee for using the Acme detector is $12, a 20% discount on the $15 asking

price of the clairvoyant. Should Kim pay $12 to use the Acme detector which is

80%?

Probability tree

Rain detector performance

Flip the tree

Decision tree

Conclusion: It is not worth

paying $12 for the use of the

Acme rain detector.

Impact of Changing the Cost of Detector (Imperfect

Information)

• Kim is indifferent between “Use detector” and “No detector” when the cost of detector is about $8.80 → Expected Value of Imperfect Information is $8.80

• The Expected Value of (Imperfect) Information (EVI) of an uncertain event is

• the amount payable for which a decision maker is just indifferent between having and not having information on the event.

Risk Neutral Decision Maker

• Utility value vs dollar value

The optimal decision for Jane is to hold the party in the porch

Notice that this is different from Kim’s optimal choice which is indoors. This is due the difference in preferences between the two.

Comparing utility curves

• When the decision maker is risk

neutral, there is no need to use a utility function. Just work on the dollar values on the decision tree. This is equivalent to using the function u(x) = x

Selling Price for the Coin Tossing Game

• Jane: Expected utility = 0.5. Certainty equivalent = u-1(0.5) = $50.

• Kim: Expected utility = 0.5. Certainty equivalent = u-1(0.5) = $34.

• Kim is more averse to risk than Jane.

She is willing to sell off the deal at a lower price than Jane.

Value of Clairvoyance for Risk Neutral Case

• Expected Value Indifference Method

• When Jane is indifferent between no information and $x-clairvoyance:

• 70 – x = 48 ⇒ x = 70 - 48 = $22

• Hence Value of Clairvoyance = $22.00

Difference of Certainty

Equivalents Method

• We have thus found that Kim and Jane have different value of clairvoyance for weather.

• These differences arise solely from differences in taste (preference), not from differences in

• structure or information.

Expected Value of Imperfect

Information for Risk Neutral case

Jane’s expected value of information for rain detector is $59.20 - $48.00 =

$11.20

• Again, we have found that Kim and Jane have different expected value of imperfect information for the same rain detector (accuracy=80%).

• These differences arise solely from differences in preference, not from differences in structure nor information

Sensitivity Analysis

• Sensitivity of Kim’s Expected Utility to Probability of Sunshine

• Let p be the probability of sunshine

The expected utilities for the three alternatives as a function of p are:

EU(outdoors) = p

EU(porch) = 0.950p + 0.323(1-p) = 0.323 + 0.627p

EU(indoors) = 0.586p + 0.667(1-p) = 0.667 – 0.099p

Sensitivity of expected utility to probability of sunshine

• Probability of sunshine, p Best decision

• 0 ≤ p ≤ 0.47 Indoors

• 0.47 ≤ p ≤ 0.87 Porch

• 0.87 ≤ p ≤ 1 Outdoors

Sensitivity of Kim’s Expected Utility for Free Clairvoyance to Probability of Sunshine

• If the probability of sunshine is p, the expected utility of free clairvoyance is p (1) + (1- p)(0.667) = 0.667 + 0.333 p

Sensitivity of Kim’s Certainty Equivalent to Probability of

Sunshine

Sensitivity Analysis for Jane

Comparing Kim’s and Jane’s decision thresholds

• Probability of sunshine, p

• Kim

• Indoors 0 ≤ p ≤ 0.47

• Porch 0.47 ≤ p ≤ 0.87

• Outdoors 0.87 ≤ p ≤ 1

• Jane

• Indoors 0 ≤ p ≤ 0.375

• Porch 0.375 ≤ p ≤ 0.667

• Outdoors 0.667 ≤ p ≤ 1

Risk profile

Risk profile

• The risk associated with an alternative or decision can be described by its Risk Profile.

• A Risk Profile of an alternative is a plot of the probabilities associated with all the possible outcomes of that alternative.

• A Risk Profile indicates both probabilities of the various possible outcomes as well as how spread-out or “volatile” are these outcomes.

• A risk profile may be represented in any of the following forms:

• Probability density (mass) function (PDF) of the value of outcomes.

• Cumulative distribution function (CDF) of the value of outcomes.

• Excess probability function (EPF) of the value of outcomes (1 – CDF)

Example:

party

problem

Risk Profiles for Porch Alternative

Risk Profiles for Indoors Alternative

Stochastic Dominance Analysis

• The principle of maximum expected utility allows us the chose the best alternative that provides the best trade off between the value of outcomes and the risk associated with it, given the decision maker risk preference

• However, this requires the decision maker to provide his/her utility function

• In some situation, the required utility function may not be available, or the top management is reluctant to commit to using one.

• Under such situations, sometimes it may still be possible to make a

rational choice using the concept of stochastic dominance.

First-order Stochastic Dominance

• Consider a decision problem with two alternatives, and let

random variables X and Y denote the values of their outcomes, respectively

• In the absence of the utility function, we can generate from the decision tree, the probability distributions for X and Y, i.e., their risk profiles.

No conclusion!

Use CDF or EPF

• Note that CDF gives the probability of achieving up to a certain value, whereas EPF gives the probability of achieving at least a certain value

• Outcome X first order stochastically dominates outcome Y if Gx(W) ≥ Gy(W) for all W, and Gx(Wi) > Gy(Wi) for some Wi.

• In other words, the EPF for X is always either above or touches the EPF

for Y, and the EPF for X must be strictly above that of Y at least one point

Stochastic Dominance based on EPF

Stochastic Dominance based on

CDF

Rational Choice under First Order Stochastic Dominance

• Stochastic dominance may allows us to make rational choice even if the exact utility function of the decision maker is unknown

• Preference ordering under first order stochastic dominance

• If the outcomes of alternative X first order stochastically dominates that of alternative Y, and if the utility function u(W) is non-decreasing in wealth, then E[u(X)] > E[u(Y)] and alternative X is preferred to alternative Y

• Since all real-people’s utility functions are non-decreasing, it follows that first order stochastic dominance is practically applicable under all classes of risk attitude and is independent of the actual utility function (so long as it is non-decreasing)

• Hence this criterion can always be applied to eliminate alternatives without actually knowing the decision maker’s utility function

Example: Kim party problem

• As all three excess probability

distributions overlap, there is no first order stochastic dominance

• among the three alternatives.

• Hence no alternative for Kim can be eliminated based on first order

stochastic dominance

Second-Order Stochastic Dominance

• In first order stochastic dominance, the EPFs and CDFs must not cross each other. If they cross each other, we can check for second order stochastic

dominance.

• Definition (Second-Order Stochastic Dominance) Outcome X second order stochastically dominates outcome Y if

• This means that the accumulated area under the EPF for X must not be less the accumulated area under the EPF for Y, and the two curves must not be identical

 

i i

y i

x

i W

y x

W some for

W G W

G

W all for dW

W G W

G

i

) ( )

(

0 )

( )

(







−



−

Second-Order

Stochastic Dominance

• Note that first order stochastic dominance implies second order stochastic dominance, but the converse is not true

Rational Choice under Second-Order Stochastic Dominance

• If the outcomes of alternative X second order stochastically dominates that of alternative Y, and the utility function u(W) is risk averse, then

E[u(X)] > E[u(Y)]

• and alternative X should be preferred to alternative Y

Kim partyproblem

• Applying the criterion for second order stochastic dominance, we may conclude:

• Porch second order stochastically dominates Outdoors

• Indoors second order stochastically dominates Outdoors

• No second order stochastic dominance between Indoors and Porch

• Hence Kim will never select the Outdoors alternative (at p=0.4) as long as she is risk averse.

• The resolution between Indoors or Porch would requires Kim’s actual utility function

Acknowledgment

Assoc. Prof. Poh Kim Leng

Dept Industrial and Systems Engineering

NUS