Chapter 7 Consistency and Replication (1)

DISTRIBUTED SYSTEMS

Principles and Paradigms

Second Edition

ANDREW S. TANENBAUM

MAARTEN VAN STEEN

Chapter 7

Plan

•

Motivation

•

Consistency models

–

Data-centric consistency

–

Client-centric consistency

•

Replica management

Replication

•

Examples

–

Many web servers with

same contents

Reasons for Replication

•

Data are replicated to

increase the reliability

of a system.

•

Replication for

performance

–

Scaling in numbers

–

Scaling in

Caveats

•

Cost of increased

bandwidth for maintaining

replication

–

Viability depends on ratio

between reads and writes

•

Possible

consistency

problems

–

What if Client writes to

Server 1…

–

Client reads from Server 2

before write is propagated?

–

May not always be a

problem…

Consistency Models

•

Data store

provides

–

Read and write operations

–

Write operations are propagated to other copies

•

Consistency model

–

Contract between processes and data store

Continuous Consistency

•

Defining inconsistencies

–

Deviation in numerical values between

replicas –

numerical error

•

E.g., $0.02 difference between to replicas of stock

price

–

Deviation in

staleness

between replicas

•

E.g., difference in when stock prize was last

updated

–

Deviation in the number of outstanding writes

–

order error

Continuous Consistency

•

Figure 7-2. An example of keeping track of consistency

deviations [adapted from (Yu and Vahdat, 2002)].

Continuous Consistency

•

May choose between

different levels of

acceptable errors for conit

•

E.g., bulletin board

–

Unseen messages:

numerical errors

–

Message delay: staleness

–

Message ordering per

newsgroup: order error

Continuous Consistency

Continuous Consistency

•

Figure 7-3. Choosing the appropriate granularity for a

conit.

Consistent Ordering

•

Augments continuous consistency

–

Need to reach agreement when tentative

updates need to be committed

Sequential Consistency (1)

•

Figure 7-4. Behavior of two processes operating

on the same data item. The horizontal axis is

time.

Sequential Consistency (2)

•

Concurrent operations by multiple processes

•

Sequential consistency:

–

All processes should see the same interleaving of

operations

–

The result of any execution is the same as if the (read

and write) operations by all processes on the data

store were executed in some sequential order

Sequential Consistency (3)

Sequential Consistency (4)

•

Figure 7-6. Three concurrently-executing

processes.

Write x

Sequential Consistency (5)

Causal Consistency (1)

•

Again concurrent operations by multiple

processes

•

Causal consistency:

–

P writes x, Q reads x, writes Q

•

The three events are potentially causally related

•

Must be seen by all processes in same order

–

Writes that are potentially causally related must be

seen by all processes in the same order

Causal Consistency (2)

Causal Consistency (3)

Causal Consistency (4)

Entry Consistency

•

Guard shared data with locks

•

Updates are performed with respect to

shared data when process acquires lock

Grouping Operations

•

Necessary criteria for correct synchronization:

–

An acquire access of a synchronization variable, not

allowed to perform until all updates to guarded shared

data have been performed with respect to that

process.

–

Before exclusive mode access to synchronization

variable by process is allowed to perform with respect

to that process, no other process may hold

synchronization variable, not even in nonexclusive

mode.

–

After exclusive mode access to synchronization

Client-Centric Consistency

•

Eventual consistency

–

No write-write conflicts, only read-write

conflicts

–

Propagate writes lazily

–

E.g., DNS, Web servers

•

What if client is mobile

Eventual Consistency

Monotonic Reads (1)

•

A data store is said to provide

monotonic-read consistency if the

following condition holds:

•

If a process reads the value of a data

item x …

Monotonic Reads (2)

•

Figure 7-12. The read operations performed by

a single process P at two different local copies of

the same data store.

(a) A monotonic-read consistent data store.

Set of write operations

Monotonic Reads (3)

•

Figure 7-12. The read operations performed by a single

Monotonic Writes (1)

•

In a monotonic-write consistent store, the

following condition holds:

•

A write operation by a process on a data

item x …

Monotonic Writes (2)

Monotonic Writes (3)

Read Your Writes (1)

•

A data store is said to provide

read-your-writes consistency, if the following

condition holds:

•

The effect of a write operation by a

process on data item x …

Read Your Writes (2)

Read Your Writes (3)

Writes Follow Reads (1)

•

A data store is said to provide

writes-follow-reads consistency, if the following

holds:

•

A write operation by a process on a data

item x following a previous read operation

on x by the same process …

Writes Follow Reads (2)

Writes Follow Reads (3)

•

Figure 7-15. (b) A data store that does

Replica Management

•

Questions to ask

–

Which servers should be replica servers?

Replica-Server Placement

•

Both a commercial and a technical (optimization)

problem

•

Minimum K-Median Problem

–

Input

–

NP-hard, a polynomial-time approximations exists

•

O(K N

2

₎

_{: Problem for online recalculations – Slashdot effect…}

Replica-Server Placement

•

Latency-driven replica placement

–

Select network regions

•

Groups of nodes with low inter-latency,

N

nodes in total

•

HotZone

algorithm

–

Place

K

replicas in most active cell

•

E.g., node with minimal distance to all other nodes in cell

–

??

–

Turns out that one can calculate this reasonably in constant time…

Content Replication and Placement

Permanent Replicas

•

Initial set of replicas

that constitute

distributed data store

–

E.g., web server

•

Replicated at a limited

number of nodes

–

Within ownership

domain

•

Mirrored at limited

number of nodes

Server-Initiated Replicas

•

Given a set of replica

servers, place contents

–

E.g.,

•

Maintain access threshold

for migrating file

•

Monitor possible access

threshold

Client-Initiated Replicas

•

(Client)

caches

Content Distribution

•

State versus Operations

–

Possibilities for what is to be propagated:

1.

Propagate only a notification of an

update.

2.

Transfer data from one copy to another.

3.

Propagate the update operation to other

copies.

Content Distribution

•

Pull versus Push Protocols

Summary

•

Consistency is essential in many

applications

–

Performance

–

Reliability

•

Consistency models

–

Data-centric consistency

–

Client-centric consistency

•

Replica management