You open Zomato at 8 PM on a Friday. The app loads instantly. You scroll through hundreds of restaurants, place an order, and track your delivery in real-time — all while thousands of other people in your city are doing the exact same thing. Nobody waits. Nobody gets an error. It just works.
Have you ever stopped and wondered — how?
This isn't magic. It's engineering. Specifically, it's system design — one of the most underappreciated disciplines in software development, and the backbone of every company that operates at internet scale.
Let's pull back the curtain.
The Scale Problem Nobody Talks About
Before we dive into solutions, let's understand the actual numbers we're talking about.
- Netflix serves over 270 million subscribers across 190+ countries. At peak hours, it accounts for nearly 15% of global internet bandwidth.
- Flipkart during its Big Billion Days sale handles millions of concurrent users, with orders spiking to hundreds per second.
- Blinkit promises 10-minute delivery — meaning their backend has to process your order, assign a picker, dispatch a rider, and track everything in real-time, all within seconds of you clicking "Place Order."
- Zomato processes millions of orders daily, coordinating restaurants, delivery partners, and customers simultaneously across hundreds of cities.
If any of these companies ran on a single server — the way your college project does — it would collapse under this load in milliseconds. So what do they actually do?
Pillar 1: Don't Rely on One Machine — Horizontal Scaling
The first and most fundamental principle is simple: don't put all your eggs in one basket.
A traditional application runs on one server. One server has limits — a fixed amount of CPU, RAM, and network capacity. When too many users hit it simultaneously, it buckles.
The solution? Run thousands of servers in parallel and distribute the load across them. This is called horizontal scaling (adding more machines, rather than vertical scaling which means buying a bigger machine).
But you can't just throw servers at the problem. You need a Load Balancer — a system that sits in front of all your servers and intelligently routes incoming requests. When you open Netflix, you're not hitting "a server." You're hitting a load balancer that decides which of their thousands of servers is best positioned to handle your request right now.
Netflix, for instance, runs almost entirely on Amazon Web Services (AWS), using their Auto Scaling feature. During peak hours (say, 9 PM on a Saturday), AWS automatically spins up more servers. At 3 AM when traffic drops, it scales back down. You pay only for what you use, and the system never runs out of capacity.
Pillar 2: Never Ask the Database Twice — Caching
Databases are slow. Not slow in human terms, but slow in computer terms. A typical database query might take 50–200 milliseconds. That doesn't sound like much, but when a million users are hitting your system every second, each one waiting 200ms for a database read will kill your system.
The solution is caching — storing frequently accessed data in memory (RAM), which is orders of magnitude faster than a database read.
Think about Zomato's restaurant listing page. The list of restaurants available in Koramangala, Bengaluru doesn't change every second. So why query the database every time someone opens the app? Instead, Zomato fetches that data once, stores it in a cache (typically Redis or Memcached), and serves all subsequent requests from memory — in under a millisecond.
Netflix takes this further with Content Delivery Networks (CDNs). When you stream a show, the video isn't streaming from a central server in the US. Netflix has distributed servers called Open Connect Appliances installed directly inside ISPs (like Airtel, Jio) around the world. The video data is physically close to you — often just a few kilometers away — which is why Netflix streams in 4K without buffering even on a congested network.
Blinkit uses caching aggressively for their dark store inventory. Product availability data is cached and updated in near real-time, so your app always shows accurate stock without hammering the database millions of times per second.
Pillar 3: Break the Monolith — Microservices Architecture
In the early days, most applications were monolithic — one giant codebase doing everything. Your user authentication, order management, payments, notifications, and analytics all lived in one application.
This is a nightmare at scale. If your payment module has a bug and crashes, it takes down your entire application — including the parts that were working perfectly.
The modern solution is Microservices Architecture: break your application into dozens (or hundreds) of small, independent services, each responsible for one thing.
Flipkart's architecture, for instance, has separate services for:
- User authentication
- Product catalog
- Inventory management
- Cart management
- Payment processing
- Order fulfillment
- Notifications
- Reviews & ratings
- Search & recommendations
Each of these runs independently. If the Reviews service has an issue, your ability to place orders is completely unaffected. Teams can deploy updates to one service without touching others. Services can be scaled independently — during a sale, only the Order and Payment services need extra capacity, not the Reviews service.
Netflix famously decomposed their monolith into over 700 microservices. This is why Netflix's user experience is bulletproof — even when individual internal services fail, you still see content.
Pillar 4: Assume Failure — Fault Tolerance and Redundancy
Here's a truth that separates senior engineers from juniors: everything fails. Servers crash. Network cables get cut. Data centers flood. Hard drives fail. Cloud providers have outages.
Companies that "never go down" don't never experience failures. They just design their systems so that individual failures don't cause the whole thing to collapse. This is called fault tolerance.
Redundancy is the core technique: run multiple copies of everything. Netflix doesn't run in one AWS data center. They run simultaneously in multiple AWS regions across different continents. If the Mumbai AWS region has an issue, traffic automatically reroutes to Singapore or the US — in seconds, transparently, while you're watching your show.
Circuit Breakers are another critical pattern. Imagine Zomato's payment service is having trouble. Rather than letting failed requests pile up and slow down the entire system, a circuit breaker "trips" — like an electrical circuit breaker — and immediately returns a fallback response ("Sorry, try again in a moment") instead of waiting for a timeout. This protects the rest of the system from being dragged down by one failing component.
Netflix even built a tool called Chaos Monkey — a program that randomly kills servers in their production environment. The philosophy: if failures are inevitable, get good at handling them. Practice failure before it happens to you.
Pillar 5: Handle the Spike — Message Queues and Async Processing
What happens when Flipkart launches Big Billion Days and order volume spikes 100x in 30 seconds?
If every order directly hits the database simultaneously, the database collapses. This is where Message Queues come in — systems like Apache Kafka, RabbitMQ, or Amazon SQS.
Instead of processing everything instantly and synchronously, you put orders into a queue. The queue absorbs the spike — it can accept millions of messages per second. Backend workers then process orders from the queue at a controlled, sustainable pace.
This is why, when you place an order on Flipkart during a sale, you sometimes see "Order Confirmed" but your confirmation email arrives 2 minutes later. The order was queued immediately (confirming your purchase), but the email notification was processed asynchronously in the background.
Zomato uses Kafka extensively — every event (order placed, rider assigned, delivery completed) flows through Kafka topics, enabling real-time tracking without overwhelming any single system.
Pillar 6: Find It Fast — Distributed Search and Databases
When you search for "biryani" on Zomato or "iPhone" on Flipkart, results appear in milliseconds across millions of items. How?
Regular SQL databases are not built for full-text search at scale. These companies use Elasticsearch or Apache Solr — specialized distributed search engines designed for exactly this use case. Indexes are built in advance, distributed across many machines, and queried in parallel.
For databases themselves, many of these companies use sharding — splitting one massive database horizontally across multiple database servers. Flipkart might store users whose IDs start with 1–1 million on one database server, 1–2 million on another, and so on. No single database server holds all the data, so none becomes a bottleneck.
Some workloads don't need relational databases at all. Netflix uses Cassandra — a distributed NoSQL database — for storing viewing history. It's optimized for the specific access pattern: write once (you watched an episode), read many times (what did this user watch?), and scale linearly across hundreds of nodes.
Pillar 7: Know Everything in Real Time — Observability
You can't fix what you can't see. Every company at this scale has massive observability infrastructure — systems to monitor, log, and trace everything happening in their stack in real time.
Zomato's engineers know, at any given moment:
- Average API response time across all endpoints
- Number of active orders per city
- Error rates per service
- Database query performance
- Rider availability vs. demand, by zone
Tools like Prometheus (metrics), Grafana (dashboards), Jaeger (distributed tracing), and ELK Stack (logs) are industry standards here. When something goes wrong — a spike in error rates, a slow database query — alerts fire automatically and engineers are paged before customers even notice the issue.
This is why these companies can often fix problems before they become outages. They see the smoke before the fire.
Putting It All Together
When you order a burger on Zomato tonight, here's roughly what happens in the background:
- Your request hits a load balancer, which routes it to an available API server
- The server checks Redis cache for your location's restaurant list — serves it in under 1ms
- You place an order — this event goes into a Kafka queue
- An order service microservice picks it from the queue and writes to the database
- A notification service (separate microservice) sends you a confirmation
- A rider matching service runs an algorithm and assigns the nearest available rider
- A tracking service streams real-time GPS updates to your app
- Every single step is logged and monitored — engineers have full visibility
All of this happens while millions of other users are doing the exact same thing, simultaneously. And if any one step fails, the system has fallbacks, retries, and circuit breakers to handle it gracefully.
The Takeaway
The reason these companies "never go down" isn't because they have better luck or more money (though money helps). It's because they've deeply internalized a set of engineering principles:
- Distribute everything — no single point of failure
- Cache aggressively — don't recompute what you can store
- Design for failure — assume things will break and plan accordingly
- Decouple components — let each piece fail independently
- Observe relentlessly — know your system better than your system knows itself
This is system design. It's not glamorous. It doesn't trend on social media. But it's the reason you can order biryani at midnight and it arrives in 30 minutes without a single error on screen.
The next time an app "just works," know that hundreds of engineers worked very hard to make that look effortless.
If you found this interesting, the rabbit hole goes much deeper — topics like consistent hashing, CAP theorem, database replication, and API gateways are all worth exploring. Drop a comment if you'd like a deep dive into any specific concept.
Comments
Post a Comment