CAS CS 351 - Distributed Systems

Detailed Lecture Notes

  • Lecture 1: Threads and Processes
  • Lecture 2: Communication Primitives
  • Lecture 3: Concurrency and Parallelism
  • Lecture 4: Sharding and Consistent Hashing
  • Lecture 5: MapReduce
  • Lecture 6: Time, Causality, Lamport Clocks
  • Lecture 7: Failure, Detection, Fault Tolerance, Recovery
  • Lecture 8: Failure Recovery and Snapshotting
  • Lecture 9: Replication
  • Lecture 10: Distributed Consensus I
  • Lecture 11: Distributed Consensus II
  • Lecture 12: Distributed Consensus III
  • Lecture 13: Consistency Models
  • Lecture 14: GFS and Weak Consistency
  • Lecture 15: Transactions and Serializability
  • Lecture 16: Dynamo
  • Lecture 17: Transactions and 2PC
  • Lecture 18 + 19: Spanner
  • Lecture 4: Sharding and Consistent Hashing

    A common feature of modern distributed systems is that they store massive amounts of data. For example, we may have a dataset of such a size (in the magnitude of terabytes or even petabytes) such that no single machine has the capacity to store it alone.

    To circumvent this storage limitation, we can shard the dataset. Sharding is the process of splitting a dataset into partitions. Thus, a shard is a subset of the data objects in our dataset. Now, instead of a server storing the entire database, it can store this smaller shard instead.

    Notice, sharding lends itself to data parallelism. Clients can perform tasks in parallel if they are requesting different shards. By sharding our dataset, we load-balance our requests across several storage servers instead of one.

    Each client can perform a task on a shard in parallel.
    Each client request is handled by different servers.

    Assigning Data to Shards

    So how does sharding work in practice? In order to shard our dataset we must consider two questions:

    1. How should we assign data to shards (what is our method of partitioning the dataset)
    2. How do clients know which shard some target data belongs to?

    Let’s inspect several approaches to how to partition our data.

    Say we have a key-value store. A key-value store is a type of database where alphanumeric-keys are associated with data objects.

    For example we may have a key-value store where users have an associated integer high-score.

    key value
    “alice” 24102
    “bob” 12312
    “john” 3212

    Partition by Alphanumeric-Ranges

    Let’s say we have three servers to host our sharded key-value store. We decide to partition the key space into three equal-size bins.

    Server 1: A-I
    Server 2: J-R
    Server 3: S-Z

    This simple scheme has a nice property: clients can easily contact the correct server for a piece of data they are looking for. If we want to perform put(joe, 100) we know to contact server 2.

    However, a harsh downside also exists. It is extremely likely that with this plan we will end up with unbalanced partitions. It is likely that our keyspace is not uniformly distributed.

    We may have peaks at certain letters of the alphabet:


    https://blogs.sas.com/content/iml/2011/01/14/two-letter-initials-which-are-the-most-common.html

    Thus, certain servers may have many more entries than others. Not only is this a problem in terms of size of the data stored, but also servers which have more entries will be more highly requested by clients. Thus, it is in our best interest to create a balanced partition of the dataset (i.e. a partitioning that is agnostic to the distribution of our keyspace).

    What is one such approach– random assignment!

    Partition by Randomly

    Say now, every time a client submits a request to our system, if this is the first time we have seen this key, we generate a random number for assigning that key to one of our three servers.

    However, a problem arises: when a client wishes to lookup that key later, how will they know which server to contact/which shard their data belongs to?

    One scheme could resemble the following:

    Unfortunately, having a single assignment server creates some issues: now clients need to perform double the amount of network communication for every operation. Furthermore, this assignment database has a hefty task– it will be heavily trafficked by requests and store a large mapping dataset.

    Thus, our question becomes, is there some way we could generate balanced partitions (i.e random assignment of keys to partitions), that requires no external lookups from the client? A key primitive in order to achieve these nice properties is a hash function.

    Hash Functions

    A hash function takes in input of an arbitrary size and maps it to a fixed size hash output space. Hash functions are deterministic: for example, given some arbitrary hash function H, always H(“apple”)=43112.

    Hash functions can be constructed to have desirable properties. Uniform hash functions map inputs uniformly across the hash output space.

    Informally, we can say that our hash function output “looks random”, such that even if there is some biased distribution in the keyspace, the hash output space is still uniform.

    We can use a hash function in order to shard a key-value store in the following manner:
    H(key) % number_of_partitions

    Benefits of this approach include:

    1. H(x) is known to clients, thus locally clients can perform H(key) % number_of_partitions and immediately know which server to contact without any external authority or mapping data structure.
    2. H(key) outputs are uniformly and “randomly” distributed across the output space, thus on expectation, the size of our partitions will be roughly equal.

    However, one downside emerges. What happens when a server fails (and we need to decrease the number of partitions) or what happens when we want to add a server to our system?

    As we have not yet discussed replicas of a dataset, we acknowledge that we will lose data if a server fails. Our main concern is: how can we continue to serve requests with the remaining live server.

    Note the following illustration of how our partitioned data results in a lot of unnecessary data migration when a server fails.

    Ideally, when a server is added or deleted we want:

    1. Minimal data shuffling
    2. To maintain balanced shard sizes

    However, by using H(key) % number_of_partitions as our partitioning function, we do not get this first guarantee. In fact, almost all of our data will have to be moved (as H(key) % 4 != H(key) % 3 for example in the above illustration).

    Consistent Hashing

    Consistent Hashing is a partitioning scheme that allows keys to map to the same server even as server membership changes (i.e. servers are added/deleted from our system).

    Consistent hashing works as follows:

    1. The hash output space is viewed as a “ring”
    2. Servers are allocated to some location on the hash output space ring
    3. The server is responsible for a piece of data is the next server clockwise from H(key)

    Consistent hashing has a nice property that when a server crashes or is added, no unnecessary data movement occurs.

    Also notice, to perform this operation, clients only must have access to H(x)->y function and a list of the servers and their location on the hash output space ring. Once the clients have this information (perhaps after a one-time connection step), they can run a simple algorithm to calculate which server they must contact for any piece of data. Consistent hashing does not require clients to contact an external lookup authority.

    However, notice that it is possible for shards to become unbalanced. If a server fails, it’s workload is left entirely to the server directly clockwise on the ring (effectively doubling it’s load)!

    Considering that we have N remaining healthy servers, is there a method we can use to ensure better load balancing in the face of failure?

    One solution to this flaw in consistent hashing is to decrease the granularity of the segments of the hash output space. One implementation of consistent hashing involves having each physical server represented multiple times on the hash output ring as a token or virtual servers.

    Now, an assignment of a server to a location on the ring is not one-to-one, but instead one server will have N tokens on the ring.

    Notice, now when the red server fails, where is red’s workload redirected? You can notice, that the workload is split more evenly between the remaining live servers, instead of all falling to one server.

    Dynamo (https://dl.acm.org/doi/10.1145/1323293.1294281) is one system that uses consistent hashing with tokens.