Detailed Lecture Notes
A common feature of modern distributed systems is that they store massive amounts of data. For example, we may have a dataset of such a size (in the magnitude of terabytes or even petabytes) such that no single machine has the capacity to store it alone.
To circumvent this storage limitation, we can shard the dataset. Sharding is the process of splitting a dataset into partitions. Thus, a shard is a subset of the data objects in our dataset. Now, instead of a server storing the entire database, it can store this smaller shard instead.
Notice, sharding lends itself to data parallelism. Clients can perform tasks in parallel if they are requesting different shards. By sharding our dataset, we load-balance our requests across several storage servers instead of one.
Each client can perform a task on a shard in parallel.
Each client request is handled by different servers.
So how does sharding work in practice? In order to shard our dataset we must consider two questions:
Let’s inspect several approaches to how to partition our data.
Say we have a key-value store. A key-value store is a type of database where alphanumeric-keys are associated with data objects.
For example we may have a key-value store where users have an associated integer high-score.
| key | value |
|---|---|
| “alice” | 24102 |
| “bob” | 12312 |
| “john” | 3212 |
Let’s say we have three servers to host our sharded key-value store. We decide to partition the key space into three equal-size bins.
Server 1: A-I
Server 2: J-R
Server 3: S-Z
This simple scheme has a nice property: clients can easily contact the correct server for a piece of data they are looking for. If we want to perform put(joe, 100) we know to contact server 2.
However, a harsh downside also exists. It is extremely likely that with this plan we will end up with unbalanced partitions. It is likely that our keyspace is not uniformly distributed.
We may have peaks at certain letters of the alphabet:
https://blogs.sas.com/content/iml/2011/01/14/two-letter-initials-which-are-the-most-common.html
Thus, certain servers may have many more entries than others. Not only is this a problem in terms of size of the data stored, but also servers which have more entries will be more highly requested by clients. Thus, it is in our best interest to create a balanced partition of the dataset (i.e. a partitioning that is agnostic to the distribution of our keyspace).
What is one such approach– random assignment!
Say now, every time a client submits a request to our system, if this is the first time we have seen this key, we generate a random number for assigning that key to one of our three servers.
However, a problem arises: when a client wishes to lookup that key later, how will they know which server to contact/which shard their data belongs to?
One scheme could resemble the following:
Unfortunately, having a single assignment server creates some issues: now clients need to perform double the amount of network communication for every operation. Furthermore, this assignment database has a hefty task– it will be heavily trafficked by requests and store a large mapping dataset.
Thus, our question becomes, is there some way we could generate balanced partitions (i.e random assignment of keys to partitions), that requires no external lookups from the client? A key primitive in order to achieve these nice properties is a hash function.
A hash function takes in input of an arbitrary size and maps it to a fixed size hash output space. Hash functions are deterministic: for example, given some arbitrary hash function H, always H(“apple”)=43112.
Hash functions can be constructed to have desirable properties. Uniform hash functions map inputs uniformly across the hash output space.
Informally, we can say that our hash function output “looks random”, such that even if there is some biased distribution in the keyspace, the hash output space is still uniform.
We can use a hash function in order to shard a key-value store in the following manner:
H(key) % number_of_partitions
Benefits of this approach include:
H(x) is known to clients, thus locally clients can perform H(key) % number_of_partitions and immediately know which server to contact without any external authority or mapping data structure.H(key) outputs are uniformly and “randomly” distributed across the output space, thus on expectation, the size of our partitions will be roughly equal.However, one downside emerges. What happens when a server fails (and we need to decrease the number of partitions) or what happens when we want to add a server to our system?
As we have not yet discussed replicas of a dataset, we acknowledge that we will lose data if a server fails. Our main concern is: how can we continue to serve requests with the remaining live server.
Note the following illustration of how our partitioned data results in a lot of unnecessary data migration when a server fails.
Ideally, when a server is added or deleted we want:
However, by using H(key) % number_of_partitions as our partitioning function, we do not get this first guarantee. In fact, almost all of our data will have to be moved (as H(key) % 4 != H(key) % 3 for example in the above illustration).
Consistent Hashing is a partitioning scheme that allows keys to map to the same server even as server membership changes (i.e. servers are added/deleted from our system).
Consistent hashing works as follows:
H(key)Consistent hashing has a nice property that when a server crashes or is added, no unnecessary data movement occurs.
Also notice, to perform this operation, clients only must have access to H(x)->y function and a list of the servers and their location on the hash output space ring. Once the clients have this information (perhaps after a one-time connection step), they can run a simple algorithm to calculate which server they must contact for any piece of data. Consistent hashing does not require clients to contact an external lookup authority.
However, notice that it is possible for shards to become unbalanced. If a server fails, it’s workload is left entirely to the server directly clockwise on the ring (effectively doubling it’s load)!
Considering that we have N remaining healthy servers, is there a method we can use to ensure better load balancing in the face of failure?
One solution to this flaw in consistent hashing is to decrease the granularity of the segments of the hash output space. One implementation of consistent hashing involves having each physical server represented multiple times on the hash output ring as a token or virtual servers.
Now, an assignment of a server to a location on the ring is not one-to-one, but instead one server will have N tokens on the ring.
Notice, now when the red server fails, where is red’s workload redirected? You can notice, that the workload is split more evenly between the remaining live servers, instead of all falling to one server.
Dynamo (https://dl.acm.org/doi/10.1145/1323293.1294281) is one system that uses consistent hashing with tokens.