Chapter 7 Consistency And Replication (2)

DISTRIBUTED SYSTEMS

Principles and Paradigms

Second Edition

ANDREW S. TANENBAUM

MAARTEN VAN STEEN

Chapter 7

Plan

•

Motivation

•

Consistency models

–

Data-centric consistency

–

Client-centric consistency

•

Replica management

Consistency Protocols

•

Describe algorithm for

achieving a given

consistency model

–

Data-centric

Continuous Consistency

•

Bounding deviations

so as to avoid

–

Numerical errors

•

E.g., that deviation of

replicas’ stock quote is

more than 1.00$

–

Staleness errors

•

E.g., that last stock

quote is more than 1

hour old

–

Order errors

Bounding Numerical Deviation

•

Assume

–

N servers,

S

₁

… S

_N

–

Single data item

x

–

A write,

W(x),

has a weight,

weight(W)

, that is the amount that

x

changes

•

E.g.,

x = 1; x := x * 3

<- weight is …

•

2

–

weight(w) > 0

•

We have a limit on numerical deviation for the value of

x

,

v

_i

, at

server

S

_i

W(x)

x1

x2

x3

origin(W)

W(x)

W(x)

L1

_L2

_L3

Log of writes

performed

i

δ

Bounding Numerical Deviation

•

Define

•

I.e., sum of writes executed by

S

_i

originating from

S

_j

–

Note,

T[i,i]

are all values submitted by

S

_i

W(x)

x1

x2

x3

origin(W)

W(x)

W(x)

L1

_L2

_L3

Log of writes

performed

∈

∧

=

=

_{

₍

₎

_|

₍

₎

_}

]

,

Bounding Numerical Deviation

•

Define

•

Goal

Bounding Numerical Deviation

•

Assume

–

S

_i

also sends

TW[i,k]

to

S

_k

–

S

_k

maintains this as a view,

TW

_k

[i,k]

, of what

S

_i

has

seen from

k

•

If then

S

_k

forwards writes

from its log,

L

_k

–

S

_k

waits to commit new writes until these are

commited at

S

_i

•

Guarantees that deviation is bounded as

Bounding Staleness Deviations

•

Goal

–

Ensure that there are no

W(x)

older than

TO

_i

that

S

_i

has not seen

•

”Older than”?

–

Vector clocks?

–

If we know drift rate of clocks and communication

delay, we can synchronize within a bound

W(x)

x1

x2

x3

origin(W)

W(x)

W(x)

L1

_L2

_L3

Bounding Staleness Deviations

•

In any case

–

Keep a real-time vector clock

•

VT

_k

[i]

= Local time of last write from

S

_i

–

If

|VT

_k

[i] – T(k)| > TO

_i

pull updates from

S

_i

W(x)

x1

x2

x3

origin(W)

W(x)

W(x)

L1

_L2

_L3

Bounding Ordering Deviations

•

Check size of local queue of tentative updates

–

Updates may not have been committed due to

ordering constraints

•

If too large, try to agree on ordering of updates

before proceeding…

–

Ensuring consistent ordering

W(x)

x1

x2

x3

origin(W)

W(x)

W(x)

L1

_L2

_L3

Ensuring Consistent Ordering

•

Make a specific process responsible for

specific data item

–

Primary-Based Protocols

•

Enable multiple processes to carry out

write operations

Remote-Write Protocols

Local-Write Protocols

Replicated-Write: Active Replication

•

Multicast updates to

all replicas

–

Need ordering of

multicast

•

May be inefficient

–

Combine

primary-based (”token-site”)

and replicated-write

(”symmetric”)protocols

Use

primary-based

Use

replicated-write

Largest inter-message

Replicated-Write: Quorum

•

Multicast W(x), read x

–

Has the write been committed?

–

Is the value read up-to-date?

•

Static process group

–

Basic idea

•

Write (version(x), value(x))

•

When writing new version, get more than half of the

processes to approve

•

When reading, get response with latest version from more

than half of the processes

–

Avoid write/write conflicts

–

Avoid write/read conflicts

Replicated-Write: Quorum

•

Process group of size

N

•

Define

–

write quorum

size,

N

_W

•

At least

N

_W

processes need to agree on value and

version for us to read

–

read quorum

size,

N

_R

•

At least

N

_R

processes need to agree on value and

version for us to read

•

To make sure to read latest version

–

N

_W

+ N

_R

> N

–

N

_W

+ N

_W

> N

At least half of the processes

will have latest version

At least one process

Quorum-Based Protocols

•

Figure 7-22. Three examples of the voting algorithm.

(a) A correct choice of read and write set. (b) A choice

that may lead to write-write conflicts. (c) A correct

Client-Centric Consistency

•

Each client keeps track of

–

Read set:

identifiers of the writes

that are relevant to the read

operations performed by the

client

–

Write set

: identifiers of writes

performed by the client

•

Monotonic-read consistency

–

Read performed

•

Client hands in

read set

to

server

•

Server may need to

communicate with other server

to become up-to-date

•

Server returns updated

read set

–

Write performed

•

(Nothing particular)

Client-Centric Consistency

•

Monotonic-write consistency

–

Write

•

Client hands in

write set

to server

•

Server may need to communicate with

other server to become up-to-date

•

Server returns updated

write set

•

Read-your-writes consistency

–

Read

•

Client hands in

write set

to server

•

Server may need to communicate with

other server to become up-to-date

•

Server returns updated

write set

•

Writes-follow-reads consistency

•

Client hands in

write set

and

read set

to

server

•

Server may need to communicate with

other server to become up-to-date with

read set

•

Add identifiers from the

read set

to the

write set

Client-Centric Consistency

•

Efficiency?

•

Bound size of writes and

reads by grouping operations

into

sessions

–

E.g., clear sets when a client

exits a specific application

•

Use a vector timestamp…

–

WVC

_i

[j]: the timestamp of the

latest write operation from S

_j

that

S

_i

has processed

–

SVC

_A

[j]: the timestamp of the

latest write operation in the

session that originates from S

_j

Summary

•

Various protocols for ensuring consistency

•

Data-centric consistency

–

Primary-based protocols

–

Replicated-write

•

Primary-based protocols most widely used due

to simplicity

–

Exact selection of protocol type depends on rate at

which request are processed and on message delay

•

Client-centric consistency may be efficiently