Vertical vs Horizontal Scaling: How Instagram Scaled With a Tiny Team
By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple
In April 2012, Facebook agreed to buy Instagram for about $1 billion. At that time Instagram had around 30 million users and only 13 employees. A few months earlier, in December 2011, Instagram's engineering blog said the app was serving 14 million+ users with just 3 engineers.
How does a team that small keep an app that big running? A large part of the answer is one of the most basic ideas in system design: scaling out instead of only scaling up, and doing it in the right places.
In this post I will explain vertical and horizontal scaling in simple English, show how Instagram used them, and give you a clean answer to the interview question: "How do you scale a stateful service versus a stateless one?" I have spent 14+ years in cloud, DevOps and SRE, and this question still comes up in almost every system design round I see.
1. What does "scaling" mean?
Scaling means handling more work (more users, more requests, more data) without the system getting slow or falling over.
There are two basic ways to do it:
Vertical scaling (scale up): make one machine bigger.
Horizontal scaling (scale out): add more machines.
Let's look at each.
2. Vertical scaling: a bigger machine
Vertical scaling means giving your server more CPU, more memory, faster disks or a faster network. On AWS, that is changing an instance from a small type to a larger one.
Why people like it:
Simple. No code changes. The app does not need to know anything changed.
Good for stateful systems. A single database on a bigger machine is still a single database: one place for data, easy transactions.
Quick fix. When a server is overloaded today, moving to a bigger size can buy you time.
The problems:
There is a ceiling. The biggest machine you can buy is still one machine. At some point there is no bigger size.
Cost grows fast. The largest machines are expensive, and you pay for that size even at quiet hours.
Single point of failure. If that one big machine fails, everything on it fails.
Downtime to resize. Changing instance size often means a restart.
Vertical scaling is not bad. It is often the right first step, especially for databases. It just cannot be your only plan.
3. Horizontal scaling: more machines
Horizontal scaling means running many copies of your service on many machines, and putting a load balancer in front to spread requests across them.
Why it is powerful:
No hard ceiling. Need more capacity? Add more machines.
Better availability. If one machine dies, the others keep serving. The load balancer stops sending traffic to the dead one.
Pay for what you use. With auto scaling (for example AWS Auto Scaling groups or Kubernetes Horizontal Pod Autoscaler), you add machines at peak and remove them at night.
The catch: horizontal scaling only works smoothly if any copy can handle any request. That leads to the most important word in this post: stateless.
4. Stateless vs stateful: the key idea
A stateless server keeps no important data in its own memory or disk between requests. Everything it needs comes from the request itself, or from shared systems like a database or a cache.
A stateful server keeps important data locally. Examples: a database server, or an app server that keeps your login session in its own memory.
Why does this matter?
If app servers are stateless, request 1 can go to server A and request 2 to server C, and nothing breaks. You can add or remove servers at any time.
If app servers are stateful (for example sessions stored in local memory), then a user must keep going to the same server ("sticky sessions"). If that server dies, the user loses their session. Scaling becomes painful.
The usual fix: move the state out. Store sessions in Redis or a database, files in object storage like S3, and keep the app servers stateless.
Then you have two different scaling problems:
Stateless tier (app servers): easy. Scale out horizontally behind a load balancer.
Stateful tier (databases): hard. Scale up first, add read replicas, and when that is not enough, shard.
5. How Instagram did it
Instagram's December 2011 engineering post ("What Powers Instagram") described their stack. Their core rules were to keep things very simple, not reinvent the wheel, and use proven technologies. Here is the shape of it, simplified:
The front door: load balancing
Every request first hit load balancers. Instagram used to run two NGINX machines with DNS round robin between them. The problem was that DNS changes take time to spread when a machine had to be removed. So they moved to Amazon's Elastic Load Balancer (ELB), with three NGINX instances behind it. NGINX machines that failed a health check were taken out of rotation automatically. They also terminated SSL at the ELB to save CPU on NGINX, and used Route 53 for DNS.
The app tier: stateless Django servers
Behind that ran Django application servers on AWS EC2 (High-CPU Extra-Large instances at the time). As usage grew, they went from a few app servers to more than 25. The post says this tier was easy to scale horizontally because the servers were stateless. That is the exact pattern from section 4.
The data tier: PostgreSQL, then sharding
Most of Instagram's data (users, photo metadata, tags and so on) lived in PostgreSQL. One database server could not hold it all, so they sharded it.
Their "Sharding & IDs at Instagram" post explains how:
They created several thousand "logical" shards, mapped in code to far fewer physical database servers.
Each logical shard was a PostgreSQL schema (a named group of tables inside one database).
To grow, they could move logical shards from one physical server to another, without re-bucketing their data.
They generated unique 64-bit IDs inside PostgreSQL itself: time-based bits, the shard ID, and a sequence number. That way IDs were unique across all shards and roughly sorted by time.
They also used streaming replication (replicas in another availability zone) and PgBouncer to pool database connections.
So Instagram used both kinds of scaling:
Horizontal for the stateless app servers: just add more.
Vertical, then sharded for the stateful database: big memory-heavy instances, and data split across many of them.
Lesson for us: Instagram did not invent new technology. It used boring, proven parts (load balancers, stateless app servers, PostgreSQL) and scaled each tier in the way that fits that tier.
6. What is sharding, simply?
Sharding means splitting one big dataset into smaller pieces called shards, and putting them on different database servers. Each shard holds a subset of the data, for example all data for users whose ID falls in a certain bucket.
user_id % number_of_logical_shards → logical shard (schema)
logical shard → physical database server
Benefits: each server handles less data and fewer queries, so you can grow almost without limit.
Costs:
Queries across shards are harder (for example "top photos across all users").
Transactions across shards are hard.
You must choose a good shard key. A bad key creates "hot" shards that get most of the traffic.
Moving data between shards needs planning. Instagram's logical-shard trick exists exactly to make this easier.
That is why the interview line is "shard stateful parts carefully". Sharding is powerful, but it is a one-way door for many designs.
7. The example visuals
The "4 CPU vs 64 CPU" servers and the "App 1–4" and "Shard 1–4" boxes in the Reel are examples to show the idea. They are not Instagram's real instance sizes or counts. The real numbers from the 2011 post were 25+ Django app servers and a main shard cluster of 12 large PostgreSQL instances (plus 12 replicas in another zone).
8. A quick comparison table
| Vertical (scale up) | Horizontal (scale out) | |
|---|---|---|
| How | Bigger machine | More machines + load balancer |
| Code changes | Usually none | App must be stateless |
| Limit | Biggest machine available | Very high |
| Failure | Single point of failure | One machine can fail safely |
| Best for | Databases (first step), legacy apps | Web/app servers, workers |
| Cloud tools | Change instance type | Auto Scaling groups, Kubernetes HPA |
9. How to answer this in an interview
Question: "How do you scale a stateful vs a stateless service?"
Strong answer:
"For stateless services, I scale horizontally: run many copies behind a load balancer with health checks, and use auto scaling. To make that work I keep state out of the app servers, with sessions in Redis or a database and files in S3. For stateful services like databases, I scale up first, add read replicas for read traffic, add caching, and when one primary isn't enough, I shard by a good key. Instagram is a classic example: stateless Django app servers that were easy to scale out, behind an ELB, and PostgreSQL sharded into thousands of logical shards on fewer physical servers. That let a very small team run a very large app."
Follow-ups to expect:
What are sticky sessions and why avoid them? They tie a user to one server, which breaks even load spreading and loses sessions when that server dies.
Read replicas vs sharding? Replicas copy all the data to scale reads. Shards split the data to scale reads and writes.
How do you pick a shard key? High cardinality, even spread, and matching your main access pattern (for Instagram, mostly by user).
When is vertical scaling the right answer? When it's cheap enough, the ceiling is far away, and simplicity matters more than unlimited growth.
Sources
Los Angeles Times, "Facebook to snap up Instagram photo-sharing app for $1 billion" (10 Apr 2012): 13 employees, about 30 million users: https://www.latimes.com/business/la-xpm-2012-apr-10-la-fi-facebook-instagram-20120410-story.html
The Atlantic, "Instagram Is Now Worth $77 Million Per Employee" (Apr 2012): 13 employees, 30M+ users: https://www.theatlantic.com/business/archive/2012/04/instagram-is-now-worth-77-million-per-employee/255640/
Instagram Engineering, "What Powers Instagram: Hundreds of Instances, Dozens of Technologies" (Dec 2011): 14M+ users, 3 engineers, ELB + NGINX, stateless Django app servers, PostgreSQL: https://instagram-engineering.com/what-powers-instagram-hundreds-of-instances-dozens-of-technologies-adf2e22da2ad
Instagram Engineering, "Sharding & IDs at Instagram": thousands of logical shards as PostgreSQL schemas, 64-bit IDs: https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c
High Scalability summary of the Instagram architecture (2011): https://highscalability.com/instagram-architecture-14-million-users-terabytes-of-photos/
Preparing for a Cloud, DevOps or SRE interview? Book a Free 15-min Cloud & DevOps Interview Guidance Call with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 https://topmate.io/cloudtechbydevang/2358399
About the author
- I help engineers learn cloud, DevOps, SRE and AI in simple English, crack interviews and follow clear roadmaps. 14+ years building and running platforms on AWS, GCP, Kubernetes and Terraform. One topic a day.
Comments
Join the discussionNo comments yet. Be the first to comment.
