The ‘Three Stooges’ Problem of Microservices
At The New Stack, we are constantly on the lookout for lessons in scalability. Something that works once or twice may still have issues completing successfully 1,000 or million times.
Take service requests for instance.
Weekend correspondent David Cassel recently found a fascinating recounting about how social news aggregation site Reddit handles sudden spikes in requests, a problem Reddit engineer Rajiv Shah likened to “The Three Stooges,” a famous Hollywood comedy trio of the mid-20th century.
“They often attempted to collaborate on simple daily tasks but invariably ended up getting in each other’s way and injuring each other,” Shah wrote of the funny men. For example, walking through a door, they all collide, “and ultimately, no one could get through.”
This comical constipation can also happen in microservice-driven architectures, Cassel explained. Like other social media service, Reddit can get a tsunami of requests within a very short time frame, which can quickly overwhelm its servers. Sure, the company’s engineers can add caches, but as the “time-to-live” for these caches quickly elapses, the servers are back to being overwhelmed again (“the cache stampede”).
The answer? De-duplicate identical requests, and store the answers on hash tables. Cassel writes:
Reddit’s API gateway collates all the incoming requests from different platforms into a standard form for easier processing (while throwing out any superfluous variables that just aren’t relevant). But when they reach the microservices level, deduplication ends up getting handled using a simple programming construct known as a hash table — where a value gets paired with a unique identifier that can retrieve it later (a key). This creates an easy way to spot the duplicate values, since they’ve already been assigned an identifier.
Concurrent programming requires clever thinking about what truly can be difficult problems. And so we appreciate that Shah took the time to explain his thinking. What clever solutions have you found for difficult problems in scalability? Let us know so we can share with our readers.