<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Cloud Tech by Devang]]></title><description><![CDATA[Cloud, DevOps and AI made simple — system design, real incidents and interview prep from 14+ years of experience.]]></description><link>https://cloudtechbydevang.com</link><generator>RSS for Node</generator><lastBuildDate>Wed, 18 Nov 2026 11:13:32 GMT</lastBuildDate><atom:link href="https://cloudtechbydevang.com/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[We Had Backups. The Restore Still Failed: How to Make Sure Your Backups Actually Work]]></title><description><![CDATA[It started with a small mistake. A developer ran a cleanup script that was meant for a test database. The connection string pointed to production. A few…]]></description><link>https://cloudtechbydevang.com/backups-that-never-restored-test-your-restore</link><guid isPermaLink="true">https://cloudtechbydevang.com/backups-that-never-restored-test-your-restore</guid><category><![CDATA[Devops]]></category><category><![CDATA[AWS]]></category><category><![CDATA[Backup]]></category><category><![CDATA[SRE]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Wed, 18 Nov 2026 03:54:26 GMT</pubDate><content:encoded><![CDATA[<p>It started with a small mistake. A developer ran a cleanup script that was meant for a test database. The connection string pointed to production. A few important tables were gone in seconds.</p>
<p>Nobody panicked at first. "We have nightly backups," someone said in the incident channel. "Let's just restore."</p>
<p>Then we opened the backup bucket. The newest backup file was 0 bytes. So was the one before it. The backup job had been reporting "success" every night, because it only checked that the dump command finished. It never checked what the command produced.</p>
<p>In 14+ years of working with infrastructure, this is the lesson I've seen teams learn the hard way more than almost any other: <strong>a backup you have never restored is a hope, not a plan.</strong></p>
<p>It's also one of my favourite interview questions: <em>"How do you know your backups actually work?"</em></p>
<h2>Why backups fail silently</h2>
<p>Backups usually don't fail loudly. They fail quietly, for boring reasons:</p>
<ul>
<li><strong>Credentials expired</strong> or a password was rotated, and the dump command wrote an error instead of data.</li>
<li><strong>The disk filled up</strong> halfway through, leaving a truncated file.</li>
<li><strong>The schema changed</strong>, and a new database or table was never added to the backup list.</li>
<li><strong>The job "succeeds" on exit code alone.</strong> A wrapper script returns 0 even when the real command failed.</li>
<li><strong>Retention deleted the only good copy.</strong> The last good backup was older than the 7-day retention window.</li>
<li><strong>Backups were in the same account</strong> as production, so the same mistake, bad script or attacker could delete both.</li>
</ul>
<p>Every one of these looks green on a dashboard until the day you need it.</p>
<h2>Step 1: Restore on a schedule, not only in an emergency</h2>
<p>The only real proof a backup works is a successful restore. Make it a routine job, not a fire drill.</p>
<p>A simple weekly restore test for a PostgreSQL database might look like this:</p>
<pre><code class="language-bash"># 1. Get the latest backup
aws s3 cp s3://my-backups/prod/latest.dump /tmp/latest.dump

# 2. Restore into a throwaway database
createdb restore_test
pg_restore --no-owner -d restore_test /tmp/latest.dump

# 3. Run real checks, not just "did it run"
psql -d restore_test -c "SELECT count(*) FROM orders;"
psql -d restore_test -c "SELECT max(created_at) FROM orders;"

# 4. Clean up
dropdb restore_test
</code></pre>
<p>The checks in step 3 matter most. Row counts tell you the data is there. The latest timestamp tells you how fresh the backup really is.</p>
<p>On AWS RDS you can do the same idea by restoring a snapshot or a point-in-time copy into a new instance, running your checks, and deleting it:</p>
<pre><code class="language-bash">aws rds restore-db-instance-to-point-in-time \
  --source-db-instance-identifier prod-db \
  --target-db-instance-identifier restore-test \
  --use-latest-restorable-time
</code></pre>
<p>Time the restore too. That number is your real recovery time, not the one in the design document.</p>
<h2>Step 2: Monitor the backup itself, not the exit code</h2>
<p>Add alerts on what the backup produced:</p>
<ul>
<li><strong>Size:</strong> alert if today's backup is much smaller than yesterday's, or close to zero.</li>
<li><strong>Age:</strong> alert if the newest backup is older than expected (for example, more than 26 hours for a nightly job).</li>
<li><strong>Restore test result:</strong> alert if the weekly restore test fails or takes much longer than usual.</li>
</ul>
<p>A backup job that fails loudly is fine. A backup job that fails quietly for three weeks is the real danger.</p>
<h2>Step 3: Know your RPO and RTO</h2>
<p>Two terms every cloud engineer should be able to explain simply:</p>
<ul>
<li><strong>RPO (Recovery Point Objective):</strong> how much data can we afford to lose? If you back up once a night, you could lose up to 24 hours of data.</li>
<li><strong>RTO (Recovery Time Objective):</strong> how long can we be down? If a full restore takes 6 hours, that is your best case.</li>
</ul>
<p>Ask the business for these numbers, then check whether your setup can actually meet them. If they need minutes, not hours, nightly dumps aren't enough. You need things like point-in-time recovery, a standby replica, or a multi-AZ / multi-region design.</p>
<h2>Step 4: Keep one copy out of reach</h2>
<p>Follow a simple version of the 3-2-1 idea: more than one copy, on more than one type of storage, with at least one copy somewhere separate.</p>
<p>In the cloud, that usually means:</p>
<ul>
<li>Copy backups to a <strong>separate AWS account</strong> (or a separate project in GCP/Azure) with limited access.</li>
<li>Turn on <strong>versioning and Object Lock</strong> or deletion protection, so backups can't be deleted early, even by an admin with a bad script.</li>
<li>Consider a <strong>second region</strong> for critical data.</li>
</ul>
<p>Ransomware and "oops" scripts both love backups that sit right next to production.</p>
<h2>Step 5: Write the runbook and practise it</h2>
<p>At 2 AM during an incident, nobody wants to work out the restore steps from scratch. Write a short runbook:</p>
<ul>
<li>Where the backups are and who has access</li>
<li>The exact restore commands</li>
<li>How to point the application at the restored database</li>
<li>How to check the data is correct before you announce "we're back"</li>
</ul>
<p>Then practise it as a game day at least once a quarter, with someone who didn't write the runbook. If they get stuck, the runbook needs fixing.</p>
<h2>The interview angle</h2>
<p>When I ask <em>"How do you know your backups work?"</em>, a strong answer covers:</p>
<ul>
<li><strong>Regular restore tests</strong> into a separate environment, with real data checks.</li>
<li><strong>Monitoring backup size and age</strong>, not just the job's exit code.</li>
<li>Clear <strong>RPO and RTO</strong>, and a design that actually meets them.</li>
<li><strong>Isolated, protected copies</strong> in another account or region.</li>
<li>A <strong>written, practised runbook</strong>.</li>
</ul>
<p>The weak answer is "we take snapshots every night." That tells me you take backups. It doesn't tell me you can restore.</p>
<h2>The lesson</h2>
<p>That day, we got lucky. A read replica was still a few minutes behind and hadn't replayed the bad script yet, so we recovered most of the data from it. Luck is not a strategy. Afterwards we added a weekly automated restore test, size and age alerts, and cross-account copies. The restore test caught a broken backup within its first month.</p>
<p>Nobody cares that you take backups. They care that you can restore. Test the restore before the day you need it.</p>
<p>When did your team last do a real restore test, and did it work the first time?</p>
<hr />
<blockquote>
<p><strong>Let's connect: Free 15-min Cloud &amp; DevOps Interview Guidance Call 👆 link in bio</strong>
(<a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a>)</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[You Deleted the Docker Containers, but the Disk Is Still Full: Why, and How to Fix It]]></title><description><![CDATA[It was a Friday evening. Every pipeline on our shared CI runner turned red at the same time, all with the same message: The team had already done the…]]></description><link>https://cloudtechbydevang.com/docker-disk-full-after-deleting-containers</link><guid isPermaLink="true">https://cloudtechbydevang.com/docker-disk-full-after-deleting-containers</guid><category><![CDATA[Docker]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Linux]]></category><category><![CDATA[ci-cd]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Tue, 17 Nov 2026 04:00:46 GMT</pubDate><content:encoded><![CDATA[<p>It was a Friday evening. Every pipeline on our shared CI runner turned red at the same time, all with the same message:</p>
<pre><code>no space left on device
</code></pre>
<p>The team had already done the "obvious" thing. "We deleted all the old containers," someone said in the channel. I logged in and ran <code>df -h</code>. The disk was still 98% full.</p>
<p>This is one of the most common Docker problems I see on build servers, and in 14+ years of working with infrastructure it still catches good engineers. It is also a question I like to ask in DevOps interviews, because the answer shows whether someone understands how Docker actually stores things.</p>
<h2>Why deleting containers doesn't free much space</h2>
<p>A container is a thin, writable layer on top of an image. When you delete a container, you delete that thin layer, and usually not much else.</p>
<p>The space on a Docker host is mostly used by four other things:</p>
<ol>
<li><strong>Images.</strong> Every build creates a new image. Old tags and untagged ("dangling") images stay on the disk until you remove them. Deleting a container does not delete the image it came from.</li>
<li><strong>Build cache.</strong> BuildKit keeps intermediate layers so the next build is fast. On a busy CI runner that builds many projects, this cache can grow to tens of gigabytes.</li>
<li><strong>Volumes.</strong> Named and anonymous volumes survive after the container is gone. Databases and test containers often leave them behind.</li>
<li><strong>Container logs.</strong> With the default <code>json-file</code> logging driver and no size limit, one noisy container can write logs until the disk is full.</li>
</ol>
<p>So "I deleted the containers" is like taking the cups off the table and leaving the kitchen full of dirty dishes.</p>
<h2>Step 1: Find out what is using the space</h2>
<p>Don't start deleting blindly. First, ask Docker:</p>
<pre><code class="language-bash">docker system df
</code></pre>
<p>You'll see something like this:</p>
<pre><code>TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          187       6         61.2GB    58.9GB (96%)
Containers      6         4         310MB     120MB (38%)
Local Volumes   42        3         9.8GB     9.1GB (92%)
Build Cache     1240      0         38.4GB    38.4GB
</code></pre>
<p>(These numbers are an illustration of what a busy runner typically looks like.) The <code>RECLAIMABLE</code> column tells you where the easy wins are.</p>
<p>For more detail:</p>
<pre><code class="language-bash">docker system df -v          # per image, volume and cache entry
du -sh /var/lib/docker/*     # what is on disk, by Docker's own folders
</code></pre>
<p>If <code>docker system df</code> looks small but the disk is still full, check the logs:</p>
<pre><code class="language-bash">sudo du -sh /var/lib/docker/containers/*/*-json.log | sort -h | tail
</code></pre>
<h2>Step 2: Clean up on purpose</h2>
<p>Each cleanup command has a clear scope. Use the one that matches what you found.</p>
<pre><code class="language-bash"># Remove dangling (untagged) images
docker image prune

# Remove all images not used by a container (bigger cleanup)
docker image prune -a

# Remove build cache older than 3 days
docker builder prune --filter until=72h

# Remove volumes not used by any container
docker volume prune

# Everything unused at once - build servers only
docker system prune -a --volumes
</code></pre>
<p>A word of warning: <code>docker system prune -a --volumes</code> is fine on a CI runner where everything can be rebuilt. On a server that runs real workloads, it can delete a volume that holds data you care about. Always read the confirmation prompt.</p>
<p>For a huge log file, don't just <code>rm</code> it while the container is running; the process keeps the file open and the space is not freed. Truncate it instead:</p>
<pre><code class="language-bash">sudo truncate -s 0 /var/lib/docker/containers/&lt;id&gt;/&lt;id&gt;-json.log
</code></pre>
<h2>Step 3: Stop it from happening again</h2>
<p>Cleaning up once is not a fix. These are the changes I make on every build host.</p>
<p><strong>1. Limit log size</strong> in <code>/etc/docker/daemon.json</code>:</p>
<pre><code class="language-json">{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "50m",
    "max-file": "3"
  }
}
</code></pre>
<p>Restart Docker after the change. New containers pick it up; old ones need to be recreated.</p>
<p><strong>2. Cap the build cache.</strong> With BuildKit you can set a garbage-collection limit so the cache cleans itself:</p>
<pre><code class="language-json">{
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "20GB"
    }
  }
}
</code></pre>
<p><strong>3. Schedule a prune job</strong> on CI runners, for example a nightly cron or systemd timer:</p>
<pre><code class="language-bash">docker image prune -a -f --filter until=168h
docker builder prune -f --filter until=72h
</code></pre>
<p><strong>4. Alert early.</strong> Set a disk alert at 80%, not 95%. At 95% you are already firefighting on a Friday evening.</p>
<p><strong>5. Use ephemeral runners where you can.</strong> Auto-scaling or container-based runners that are thrown away after each job never collect old images in the first place.</p>
<h2>The interview angle</h2>
<p>When I ask "You removed the containers, but the disk is still full. Why?", here's what a strong answer covers:</p>
<ul>
<li>Containers are thin layers; <strong>images, build cache, volumes and logs</strong> are where the space really goes.</li>
<li><strong>Measure first</strong> with <code>docker system df</code> and <code>du</code>, then clean the right thing.</li>
<li>Knows the <strong>difference between the prune commands</strong>, and that <code>-a --volumes</code> is risky outside a build box.</li>
<li>Talks about <strong>prevention</strong>: log rotation, BuildKit GC, scheduled pruning, disk alerts, ephemeral runners.</li>
</ul>
<p>The weak answer is "restart the server" or "add a bigger disk". A bigger disk only gives you more time before the same outage.</p>
<h2>The lesson</h2>
<p>That Friday, the build cache alone was holding more space than every container we had deleted put together. We cleaned it in five minutes, but the real fix was the log limits, the BuildKit GC setting and a nightly prune job. That runner has not filled up since.</p>
<p>Before you delete anything, find out what is actually using the space. Then fix the reason, not just the symptom.</p>
<p>What filled up your build server the last time: images, build cache, volumes or logs?</p>
<hr />
<blockquote>
<p><strong>Let's connect: Free 15-min Cloud &amp; DevOps Interview Guidance Call 👆 link in bio</strong>
(<a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a>)</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[Terraform Plan Says Destroy and Recreate. What Now?]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple "Terraform plan shows a resource will be destroyed and recreated. What now?"…]]></description><link>https://cloudtechbydevang.com/terraform-plan-says-destroy-and-recreate-what-now</link><guid isPermaLink="true">https://cloudtechbydevang.com/terraform-plan-says-destroy-and-recreate-what-now</guid><category><![CDATA[#IaC]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Terraform]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Mon, 09 Nov 2026 05:11:06 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>"Terraform plan shows a resource will be destroyed and recreated. What now?" This is one of my favourite Terraform interview questions. A weak answer is "run apply, it'll recreate it." That's fine for a security group rule. On a database, a load balancer or a DNS-facing resource, it can mean an outage or data loss. Here's how a strong engineer handles it.</p>
<hr />
<h2>1. Find the trigger</h2>
<p>In the plan output, a replacement shows as <code>-/+</code> (or <code>+/-</code> with create_before_destroy), and the attribute causing it is marked <strong><code># forces replacement</code></strong>. That line is the whole story. Common triggers:</p>
<ul>
<li>Changing a resource name or identifier that the cloud API can't update in place</li>
<li>Moving to a different subnet, VPC or availability zone</li>
<li>A new AMI or image ID</li>
<li>Changing an immutable property such as encryption settings or engine type on some resources</li>
</ul>
<p>Some attributes can be updated in place and some can't. The provider decides, so check the provider documentation.</p>
<h2>2. Was it intended?</h2>
<p>Many surprise replacements aren't caused by the change you made:</p>
<ul>
<li><strong>Provider version upgrades</strong> can change defaults or how attributes are read. Pin provider versions and read the changelog before upgrading.</li>
<li><strong>Data sources</strong> such as "latest AMI" can return a new value on any run and quietly trigger replacement.</li>
<li><strong>Drift</strong>: someone changed the resource in the console, and Terraform now wants to put it back.</li>
</ul>
<p>Use <code>terraform plan -refresh-only</code> to see drift separately from your own changes.</p>
<h2>3. Renamed or moved? Tell Terraform</h2>
<p>If you renamed a resource, moved it into a module or changed a <code>count</code> index to <code>for_each</code>, Terraform sees a new address and plans destroy plus create. The real resource is the same, so update the state, not the infrastructure:</p>
<ul>
<li>A <strong><code>moved</code> block</strong> (Terraform 1.1+) records the refactor in code and is reviewable in the PR.</li>
<li><strong><code>terraform state mv</code></strong> does the same thing imperatively.</li>
</ul>
<p>Afterwards, the plan should show no changes.</p>
<h2>4. Protect what matters</h2>
<p>Use lifecycle rules on purpose:</p>
<ul>
<li><strong><code>prevent_destroy = true</code></strong> on stateful resources such as databases, buckets with data and KMS keys. Any plan that would destroy them fails.</li>
<li><strong><code>create_before_destroy = true</code></strong> where replacement is acceptable but downtime isn't. Watch for name collisions: unique names often need a prefix or random suffix.</li>
<li>Cloud-side protection as a second layer, such as deletion protection on databases and load balancers.</li>
</ul>
<h2>5. Use ignore_changes carefully</h2>
<p>If another system legitimately manages an attribute (autoscaling changing desired capacity, or tags added by a policy engine), <strong><code>ignore_changes</code></strong> on that specific attribute is right. Using it to silence a plan you don't understand is how config and reality drift apart for good.</p>
<h2>6. Make it a process</h2>
<ul>
<li>Save the plan with <code>-out</code>, review it in the PR, and apply exactly that plan.</li>
<li>Add a policy check (OPA/Conftest, Sentinel or a simple script on <code>terraform show -json</code>) that blocks or flags any delete action on critical resource types.</li>
<li>For replacements you can't avoid on stateful resources, plan it like a migration: backup or snapshot, a maintenance window, a DNS/endpoint change plan and a rollback path.</li>
</ul>
<hr />
<h2>The senior angle</h2>
<p>A plan is a <strong>contract</strong>, not a formality. "Destroy and recreate" on a database or load balancer can mean data loss, a new endpoint, broken connection strings or DNS churn that clients cache for hours. The seniority isn't in knowing the commands. It's in reading the plan line by line and refusing to apply anything you can't explain.</p>
<p>In my 14+ years in Cloud and DevOps, the most dangerous Terraform command has been an apply nobody read.</p>
<hr />
<blockquote>
<p><strong>Let's connect: Free 15-min Cloud &amp; DevOps Interview Guidance Call 👆 link in bio</strong>
(<a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a>)</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[Your Kubernetes Pod Keeps Getting OOMKilled. Why?]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple "Your pod keeps getting OOMKilled. Why?" is one of the questions I ask most in…]]></description><link>https://cloudtechbydevang.com/your-kubernetes-pod-keeps-getting-oomkilled-why</link><guid isPermaLink="true">https://cloudtechbydevang.com/your-kubernetes-pod-keeps-getting-oomkilled-why</guid><category><![CDATA[SRE]]></category><category><![CDATA[Kubernetes]]></category><category><![CDATA[Devops]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Thu, 05 Nov 2026 09:53:05 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>"Your pod keeps getting OOMKilled. Why?" is one of the questions I ask most in Kubernetes interviews. The usual answer is "increase the memory limit". Sometimes that's right, but on its own it's a junior answer. A senior engineer treats OOMKilled as a symptom and goes looking for the cause. Here's the full breakdown.</p>
<hr />
<h2>1. Read the evidence first</h2>
<p>Start with <code>kubectl describe pod &lt;pod&gt;</code>. Under the container's <strong>Last State</strong> you'll see <code>Terminated</code>, <code>Reason: OOMKilled</code> and <strong>exit code 137</strong> (128 + SIGKILL 9). Also check the restart count and <code>kubectl get events</code>.</p>
<p>Separate two cases:</p>
<ul>
<li><strong>Container OOMKilled</strong>: the container went over its own memory <strong>limit</strong>, and the kernel's cgroup OOM killer ended it.</li>
<li><strong>Pod evicted</strong>: the <strong>node</strong> ran low on memory and the kubelet evicted pods to protect itself. The reason here is <code>Evicted</code>, and the message mentions memory pressure.</li>
</ul>
<p>They have different fixes, so don't mix them up.</p>
<h2>2. Requests vs limits</h2>
<ul>
<li><strong>Requests</strong> are what the scheduler uses to place the pod. They're a reservation.</li>
<li><strong>Limits</strong> are the hard ceiling the kernel enforces.</li>
</ul>
<p>If you set a small request and a large limit, the scheduler packs many pods onto a node assuming they're small. When they all grow at once, the node runs out of memory. Memory isn't compressible like CPU. CPU gets throttled, but memory gets killed. For most production services, setting the memory request equal to the limit (or close to it) gives predictable behaviour.</p>
<h2>3. Make the runtime aware of the container</h2>
<p>Many runtimes size their heap from what they <em>think</em> the machine has:</p>
<ul>
<li><strong>JVM</strong>: modern JDKs are container-aware, but older ones weren't. Set <code>-XX:MaxRAMPercentage</code> (for example 70–75) so the heap leaves room for metaspace, threads and native memory.</li>
<li><strong>Node.js</strong>: set <code>--max-old-space-size</code> below the container limit. Leave headroom, because the V8 heap isn't the process's only memory.</li>
<li><strong>Python, Gunicorn, Celery</strong>: memory scales with the number of workers. Four workers × per-worker memory has to fit inside the limit.</li>
</ul>
<p>A common pattern: the heap limit is set equal to the container limit, so off-heap memory pushes the process over and the kernel kills it.</p>
<h2>4. Leak or real growth?</h2>
<p>Graph <strong>working-set memory</strong> (<code>container_memory_working_set_bytes</code>), which is what the kubelet and OOM decisions care about, over hours or days.</p>
<ul>
<li>A <strong>sawtooth</strong> that plateaus is normal garbage-collection behaviour.</li>
<li>A <strong>steady climb</strong> that only a restart resets is a leak. Raising the limit only stretches the time between crashes.</li>
<li><strong>Spikes</strong> that line up with specific requests (large uploads, unbounded queries, batch jobs) point to a code path that needs streaming or pagination.</li>
</ul>
<p>Heap dumps, profilers and per-endpoint memory metrics will find the cause faster than trial-and-error limit changes.</p>
<h2>5. Know your QoS class</h2>
<p>Kubernetes assigns a QoS class to each pod:</p>
<ul>
<li><strong>Guaranteed</strong>: requests equal limits for every container. These are the last to be evicted under node pressure.</li>
<li><strong>Burstable</strong>: requests are set below limits.</li>
<li><strong>BestEffort</strong>: nothing set. These are evicted first.</li>
</ul>
<p>Your critical workloads should be Guaranteed. Batch and dev workloads can be Burstable.</p>
<h2>6. Right-size with data</h2>
<p>Use real usage data. Check percentiles in your metrics, or run Vertical Pod Autoscaler in recommendation mode, and set limits with sensible headroom above the observed peak. Review them after major releases.</p>
<hr />
<h2>The senior angle</h2>
<p>OOMKilled isn't a sizing problem. It's information. Raising the limit without knowing <em>why</em> memory grows just moves the crash to 3 AM on a bigger, more expensive node. A senior answer explains the evidence, tells the container kill apart from node eviction, ties the runtime to the cgroup, and fixes the cause.</p>
<p>In my 14+ years in Cloud and DevOps, the engineers I trust most ask "what is using the memory?" before "how much more can we give it?"</p>
<hr />
<blockquote>
<p><strong>Let's connect: Free 15-min Cloud &amp; DevOps Interview Guidance Call 👆 link in bio</strong>
(<a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a>)</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[Sticky Sessions Explained: The Story of Two Waiters (Load Balancer Session Affinity)]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple This post is the full breakdown for the Two Waiters Reel. If you came here…]]></description><link>https://cloudtechbydevang.com/sticky-sessions-explained-the-story-of-two-waiters-load-balancer-session-affinity</link><guid isPermaLink="true">https://cloudtechbydevang.com/sticky-sessions-explained-the-story-of-two-waiters-load-balancer-session-affinity</guid><category><![CDATA[System Design]]></category><category><![CDATA[system design interview]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Load Balancing]]></category><category><![CDATA[Redis]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Fri, 30 Oct 2026 13:49:46 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<blockquote>
<p>This post is the full breakdown for the <strong>Two Waiters Reel</strong>. If you came here after commenting <strong>STICKY</strong>, welcome!</p>
</blockquote>
<p><strong>The story in the Reel:</strong> Sharma ji runs a busy dhaba with two waiters. <strong>Raju</strong> is running between six tables. <strong>Pappu</strong> is standing free with zero tables. Sharma ji asks, <em>"Why?"</em></p>
<p>Raju says, <em>"Sir, everyone's orders are in my notebook!"</em> And sure enough, Neha at table 4 calls out, <em>"Raju bhaiya! Ek aur butter naan!"</em> Pappu is right next to her, but she won't ask him, because only Raju knows her order.</p>
<p>Then Sharma ji has a worse thought: <em>"What if Raju goes home... and takes the notebook with him?"</em> Every order is gone, and every customer has to order again.</p>
<p>That is exactly what <strong>sticky sessions</strong> do in a load-balanced system. Below is the real tech, the problems it causes, the better design and how to answer it in an interview.</p>
<hr />
<h2>1. What is a sticky session (session affinity)?</h2>
<p>A load balancer normally spreads requests across a pool of servers. With <strong>sticky sessions</strong> (also called <strong>session affinity</strong>), the load balancer makes sure that <strong>every request from the same user goes to the same server</strong> for the life of the session.</p>
<p>Why would anyone want that? Because the app keeps the user's <strong>session state in the memory of one server</strong>:</p>
<ul>
<li>login state (who you are)</li>
<li>shopping cart</li>
<li>multi-step form or checkout progress</li>
<li>in-memory caches tied to that user</li>
</ul>
<p>If the next request went to a different server, that server wouldn't have the session, and the user would look logged out or see an empty cart. So the load balancer "sticks" the user to the server that has their data.</p>
<table>
<thead>
<tr>
<th>Dhaba</th>
<th>System</th>
</tr>
</thead>
<tbody><tr>
<td>Raju and Pappu</td>
<td>Server A and Server B</td>
</tr>
<tr>
<td>Customer at a table</td>
<td>User / client</td>
</tr>
<tr>
<td>Raju's notebook</td>
<td>Session data in Server A's memory</td>
</tr>
<tr>
<td>"Only Raju knows my order"</td>
<td>Session affinity</td>
</tr>
<tr>
<td>Raju goes home with the notebook</td>
<td>Server crash: sessions lost</td>
</tr>
<tr>
<td>Kitchen board</td>
<td>Shared session store (Redis / DB)</td>
</tr>
</tbody></table>
<h2>2. How the load balancer remembers: cookie vs IP hash</h2>
<p>There are two common ways a load balancer decides that "this user belongs to Server A".</p>
<h3>Cookie-based affinity</h3>
<p>On the first response, the load balancer (or the app) sets a cookie that identifies the server. On every later request, the browser sends the cookie back, and the load balancer routes to that server.</p>
<pre><code>1st request:   User ──&gt; LB ──&gt; Server A
               User &lt;── LB &lt;── Server A   Set-Cookie: srv=A
next requests: User ──(Cookie: srv=A)──&gt; LB ──&gt; Server A  (always)
</code></pre>
<ul>
<li><strong>Load-balancer-generated cookies:</strong> for example AWS ALB's <code>AWSALB</code> cookie, or NGINX <code>sticky cookie</code>.</li>
<li><strong>Application cookies:</strong> the LB follows a cookie the app already sets (such as a session ID).</li>
<li>A cookie usually has a <strong>duration / TTL</strong>, after which the user can be balanced again.</li>
</ul>
<p><strong>Pros:</strong> precise per-user stickiness, works even when many users share one IP.
<strong>Cons:</strong> only works for clients that keep cookies (browsers, not always APIs or mobile SDKs).</p>
<h3>IP hash (source-IP affinity)</h3>
<p>The load balancer hashes the client's IP address and maps the hash to a server, for example NGINX <code>ip_hash</code> or a Layer 4 "source IP" affinity.</p>
<p><strong>Pros:</strong> no cookies, works for any TCP traffic.
<strong>Cons:</strong> thousands of users behind one corporate NAT or mobile carrier IP all land on the <strong>same server</strong>, and a user whose IP changes (Wi-Fi to 4G) loses their server and their session.</p>
<h2>3. The problems with sticky sessions</h2>
<h3>Problem 1: uneven load</h3>
<p>Stickiness overrides the balancing algorithm. Heavy users, long sessions or one big NAT pool stay pinned to one server. That's Raju with six tables and Pappu with zero: <strong>one server at 99% CPU while another sits at 2%</strong>.</p>
<p>Autoscaling makes it worse. When you add new servers, existing users stay stuck to the old ones, so the new capacity barely helps until sessions expire.</p>
<h3>Problem 2: failover loses the session</h3>
<p>If the session only lives in Server A's memory and Server A crashes, gets redeployed or is scaled in, <strong>every session on it is gone</strong>. Users are logged out, carts are empty, and half-filled forms disappear. That's Raju going home with the notebook.</p>
<h3>Problem 3: harder deployments and scaling</h3>
<ul>
<li>Rolling deployments must <strong>drain</strong> each server and wait for sticky users to leave.</li>
<li>Scale-in can't simply kill an instance without hurting users on it.</li>
<li>Debugging gets harder because behaviour depends on which server a user is pinned to.</li>
</ul>
<h2>4. The better approach: stateless servers + a shared session store</h2>
<p>Sharma ji's fix is simple: <em>"From today, every order goes on the kitchen board. Any waiter can serve any table."</em></p>
<p>In system terms: <strong>keep the application servers stateless and move session state out of server memory</strong> into a shared store that every server can read.</p>
<pre><code>            ┌──────────── Load Balancer (round robin / least connections) ────────────┐
            ▼                              ▼                                          ▼
       App server A                  App server B                               App server C
       (stateless)                   (stateless)                                (stateless)
            └──────────────────────────────┬──────────────────────────────────────────┘
                                           ▼
                           Shared session store: Redis / DB
</code></pre>
<p>Common options:</p>
<ul>
<li><strong>Redis / Memcached</strong> (for example Amazon ElastiCache): fast, in-memory, with TTL per session key. The most common choice.</li>
<li><strong>A database</strong> (DynamoDB, PostgreSQL): durable, a bit slower, good when sessions must survive anything.</li>
<li><strong>Stateless tokens (JWT):</strong> the session data travels in a signed token, so the server stores nothing. Remember to plan for revocation and token size.</li>
</ul>
<p>What you gain:</p>
<ul>
<li><strong>Even load:</strong> any server can take any request, so the load balancer can truly balance.</li>
<li><strong>Safe failover:</strong> if a server crashes, the next request goes to another server, which reads the session from Redis. Nothing is lost.</li>
<li><strong>Easy scaling and deploys:</strong> add or remove servers freely, with no draining of sticky users.</li>
</ul>
<p><strong>Trade-offs to mention:</strong> one extra network hop per request (usually under a millisecond with Redis in the same region), and the session store itself must be highly available (Redis replication / cluster mode, Multi-AZ).</p>
<h2>5. When sticky sessions are still OK</h2>
<p>Sticky sessions aren't "wrong", they're a tool with a cost. Use them when:</p>
<ul>
<li><strong>Legacy apps</strong> keep session state in memory and can't be changed quickly.</li>
<li><strong>WebSockets or other long-lived connections</strong>, where the connection itself lives on one server (and some protocols, like Socket.IO's polling fallback, need affinity during the handshake).</li>
<li><strong>Local caches</strong> where hitting the same server gives a much better cache hit rate.</li>
</ul>
<p>Even then, keep a <strong>short TTL</strong> on the affinity cookie, enable <strong>connection draining</strong>, and don't rely on stickiness for correctness: plan for the server going away.</p>
<h2>6. How to answer in an interview</h2>
<p><strong>Q: What are sticky sessions, and would you use them?</strong></p>
<blockquote>
<p>"Sticky sessions, or session affinity, mean the load balancer sends all requests from one user to the same server, usually with a cookie or an IP hash, because the session lives in that server's memory. It's simple, but it causes uneven load, and when that server crashes or is scaled in, those users lose their sessions. My default is stateless app servers with session state in a shared store like Redis or a database, so any server can handle any request, and failover and autoscaling just work. I'd only use sticky sessions for legacy apps or WebSockets, with a short TTL and connection draining."</p>
</blockquote>
<p>Follow-up questions to be ready for:</p>
<ul>
<li><strong>Cookie vs IP hash?</strong> Cookie is precise per user; IP hash breaks behind NAT and when IPs change.</li>
<li><strong>How does AWS ALB do it?</strong> Target-group stickiness with a duration-based LB cookie (<code>AWSALB</code>) or an application-based cookie.</li>
<li><strong>What if Redis goes down?</strong> Replication and automatic failover (ElastiCache Multi-AZ), sensible timeouts, and a fallback such as re-authentication.</li>
<li><strong>JWT vs server-side sessions?</strong> JWT removes the store but makes revocation harder; server-side sessions in Redis are easy to revoke.</li>
</ul>
<p><strong>One-line summary:</strong> <em>sticky sessions pin a user to one server; the better default is stateless servers plus a shared session store.</em></p>
<hr />
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> With 14+ years in Cloud &amp; DevOps, I help engineers get interview-ready. Let's connect on a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong>: what to study, how to answer system-design questions like this one, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
<p>Follow <strong>@cloudtechbydevang</strong> on Instagram for more everyday tech, simply explained.</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[Polling vs Webhooks vs WebSockets: Explained with a Karva Chauth Story]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple This post is the full breakdown for the Karva Chauth Reel. If you came here…]]></description><link>https://cloudtechbydevang.com/polling-vs-webhooks-vs-websockets-explained-with-a-karva-chauth-story</link><guid isPermaLink="true">https://cloudtechbydevang.com/polling-vs-webhooks-vs-websockets-explained-with-a-karva-chauth-story</guid><category><![CDATA[Devops]]></category><category><![CDATA[System Design]]></category><category><![CDATA[Cloud]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Thu, 29 Oct 2026 13:40:36 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<blockquote>
<p>This post is the full breakdown for the <strong>Karva Chauth Reel</strong>. If you came here after commenting <strong>CHAAND</strong>, welcome, and Happy Karva Chauth!</p>
</blockquote>
<p><strong>The story in the Reel:</strong> it is Karva Chauth night, and the whole house has one question: <em>"Chaand nikla kya?"</em> (Has the moon come out yet?)</p>
<ul>
<li><strong>Mummy</strong> climbs to the roof every two minutes to check. Nothing has changed, but she keeps checking. That is <strong>polling</strong>.</li>
<li><strong>Papa</strong> says, <em>"I have told Sharma ji. When he sees the moon, he will call us."</em> Nobody checks. The news comes when the event happens. That is a <strong>webhook</strong>.</li>
<li><strong>Didi</strong> is on a video call from Australia, and the call stays connected all evening. Either side can talk at any moment. That is a <strong>WebSocket</strong>.</li>
</ul>
<p>Below is the real tech behind each one, with examples, a comparison table and interview answers.</p>
<hr />
<h2>1. Polling: "Has anything changed?"</h2>
<p>With polling, the <strong>client asks the server on a timer</strong>. Most of the time the answer is "no change".</p>
<pre><code>Client (Mummy)                 Server (the sky)
   | -- GET /moon --------------&gt;  |
   | &lt;-- 200 {"visible": false} -- |
   |        ... 2 minutes ...      |
   | -- GET /moon --------------&gt;  |
   | &lt;-- 200 {"visible": false} -- |
   |        ... 2 minutes ...      |
   | -- GET /moon --------------&gt;  |
   | &lt;-- 200 {"visible": true} --- |   finally!
</code></pre>
<p><strong>Where you see it in real life:</strong></p>
<ul>
<li><strong>Job status checks.</strong> You start a long video export or report. The API returns a <code>job_id</code> and the UI calls <code>GET /jobs/{id}</code> every few seconds until the status is <code>completed</code>.</li>
<li><strong>CI/CD dashboards</strong> and scripts that wait for a deployment to become healthy.</li>
<li><strong>Fallback for missed events.</strong> Razorpay's own docs suggest polling their payment API as a backup for business-critical flows, because webhooks are asynchronous.</li>
</ul>
<p><strong>Long polling</strong> is a smarter version. The client sends a request, and the server <strong>holds it open</strong> until there is news (or a timeout), then the client immediately asks again. Fewer empty answers, but each waiting client still holds a connection.</p>
<p><strong>The trade-off:</strong> polling is the simplest thing that works, and it works through any firewall or proxy. But you pay for every empty request, and your data is only as fresh as your interval. Poll every 2 minutes and news can be up to 2 minutes late. Poll every second and you multiply the load on the server.</p>
<hr />
<h2>2. Webhooks: "Call me when it happens"</h2>
<p>A webhook flips the direction. You give the provider a URL. When an <strong>event</strong> happens, the <strong>provider sends an HTTP POST to your server</strong>.</p>
<pre><code>Your app (Papa)               Provider (Sharma ji)
   | -- register: POST /webhooks {url} --&gt; |
   |                                       |  ... waiting, no traffic ...
   |                                       |  EVENT: moon visible!
   | &lt;-- POST /hooks/moon {event} -------- |
   | -- 200 OK -------------------------&gt;  |
</code></pre>
<p><strong>Real examples:</strong></p>
<ul>
<li><strong>Payments:</strong> Stripe and Razorpay send events like "payment captured" or "refund processed", so you mark the order paid even if the customer closed the browser tab.</li>
<li><strong>GitHub:</strong> a <code>push</code> or <code>pull_request</code> event triggers your CI pipeline or a Slack bot.</li>
</ul>
<p>Webhooks look simple, but production webhooks need four things:</p>
<p><strong>1. Verify the signature.</strong> Your webhook URL is public, so anyone can POST to it. Providers sign each payload with a shared secret using HMAC-SHA256: Stripe in the <code>Stripe-Signature</code> header, GitHub in <code>X-Hub-Signature-256</code>, Razorpay in <code>X-Razorpay-Signature</code>. Compute the HMAC over the <strong>raw request body</strong> (do not parse and re-serialise the JSON first) and compare it in constant time.</p>
<p><strong>2. Respond fast, process later.</strong> GitHub expects a 2xx within <strong>10 seconds</strong>, and Razorpay within <strong>5 seconds</strong>, or the delivery counts as failed. Validate, push the event onto a queue (SQS, RabbitMQ, Redis), return 200, and let a worker do the real work.</p>
<p><strong>3. Expect retries.</strong> If you fail, providers retry. Stripe retries for up to <strong>three days</strong> with exponential backoff in live mode. Razorpay retries with exponential backoff for <strong>24 hours</strong>, then disables the webhook. GitHub does not retry automatically. You redeliver failed deliveries yourself from the UI or API.</p>
<p><strong>4. Be idempotent.</strong> Retries mean <strong>at-least-once delivery</strong>: you will see the same event more than once. Store the event ID (<code>x-razorpay-event-id</code>, Stripe's <code>event.id</code>, GitHub's <code>X-GitHub-Delivery</code>) and skip events you have already processed. Otherwise you may ship the same order twice. Also do not assume order: an "updated" event can arrive before a "created" one, so fetch the current state when in doubt.</p>
<hr />
<h2>3. WebSockets: "Keep the line open"</h2>
<p>A WebSocket is <strong>one long-lived, two-way connection</strong>. It starts as a normal HTTP request with an <code>Upgrade: websocket</code> header. The server replies <code>101 Switching Protocols</code>, and from then on both sides can send messages at any time over the same TCP connection (RFC 6455).</p>
<pre><code class="language-mermaid">sequenceDiagram
    participant Ghar as Browser (Ghar, India)
    participant Didi as Server (Didi, Australia)
    Ghar-&gt;&gt;Didi: GET /call  (Upgrade: websocket)
    Didi--&gt;&gt;Ghar: 101 Switching Protocols
    Note over Ghar,Didi: connection stays open
    Didi--&gt;&gt;Ghar: "Mummy, yahan chaand dikh gaya!"
    Ghar-&gt;&gt;Didi: "Show us on camera!"
    Didi--&gt;&gt;Ghar: photo frame
</code></pre>
<p><strong>Real examples:</strong> chat apps, live cricket scores, trading and crypto dashboards with ticking prices, multiplayer games, collaborative editors (two people typing in one doc), and live delivery tracking.</p>
<p><strong>The trade-off:</strong> messages arrive in milliseconds with almost no per-message overhead. But each client holds an open connection, so the server is <strong>stateful</strong>. You need to think about connection limits, load balancers that support long-lived connections (sticky sessions or a pub/sub layer like Redis so any server can reach any user), heartbeats (ping/pong) to detect dead clients, and reconnect logic when mobile networks drop.</p>
<h3>The middle option: Server-Sent Events (SSE)</h3>
<p>What if only the server needs to talk? Use <strong>SSE</strong>. The browser opens one HTTP connection with <code>EventSource</code>, and the server streams <code>text/event-stream</code> messages down it. It is one-way (server to client), runs over plain HTTP, and the browser <strong>reconnects automatically</strong>. It is great for notifications, live news feeds, build logs and streaming AI responses. If the client also needs to send a lot of data back in real time, move up to WebSockets.</p>
<hr />
<h2>4. Comparison table</h2>
<table>
<thead>
<tr>
<th></th>
<th>Polling</th>
<th>Webhook</th>
<th>WebSocket</th>
</tr>
</thead>
<tbody><tr>
<td><strong>Who starts</strong></td>
<td>Client asks on a timer</td>
<td>Provider calls you on an event</td>
<td>Either side, any time</td>
</tr>
<tr>
<td><strong>Direction</strong></td>
<td>Client → server (request/response)</td>
<td>Server → server (one-way push)</td>
<td>Two-way</td>
</tr>
<tr>
<td><strong>Latency</strong></td>
<td>Up to one poll interval</td>
<td>Near real-time (seconds)</td>
<td>Real-time (milliseconds)</td>
</tr>
<tr>
<td><strong>Server load</strong></td>
<td>High, many empty requests</td>
<td>Low, traffic only on events</td>
<td>Steady, open connection per client</td>
</tr>
<tr>
<td><strong>Complexity</strong></td>
<td>Lowest</td>
<td>Medium (public endpoint, signatures, retries, idempotency)</td>
<td>Highest (stateful connections, scaling, reconnects)</td>
</tr>
<tr>
<td><strong>Works for browsers?</strong></td>
<td>Yes</td>
<td>No, needs a public server</td>
<td>Yes</td>
</tr>
<tr>
<td><strong>Best for</strong></td>
<td>Job status, simple dashboards, fallback checks</td>
<td>Payments, Git events, server-to-server integrations</td>
<td>Chat, live scores, trading, games, collaboration</td>
</tr>
</tbody></table>
<p>A quick rule from the Reel: <strong>checking again and again = polling. News when an event happens = webhook. Live, two-way, always open = WebSocket.</strong></p>
<hr />
<h2>5. Common mistakes</h2>
<ul>
<li><strong>Polling too fast.</strong> A 1-second interval across 100,000 clients is 100,000 requests per second, most of them returning nothing. Use backoff, or switch to webhooks or SSE.</li>
<li><strong>Trusting webhook payloads without verifying signatures.</strong> This is how fake "payment successful" events get your orders shipped for free.</li>
<li><strong>Doing heavy work inside the webhook handler.</strong> It times out, the provider retries, and now you have duplicates. Queue first, respond, then process.</li>
<li><strong>No idempotency.</strong> Duplicate events are normal behaviour, not a bug.</li>
<li><strong>Using WebSockets for everything.</strong> If data changes once an hour, an open connection per user is wasted cost. SSE or polling is enough.</li>
<li><strong>No reconnect or heartbeat logic.</strong> Mobile networks drop silently. Without ping/pong and retry with backoff, users stare at a "live" screen that stopped updating.</li>
<li><strong>Webhooks with no fallback.</strong> If your endpoint was down for a day, you need a reconciliation job that polls the provider's API to catch up.</li>
</ul>
<hr />
<h2>6. Interview questions and crisp answers</h2>
<p><strong>Q1. Polling vs webhooks: when would you pick each?</strong>
Polling when I control only the client, updates are rare or I need a simple fallback. Webhooks when the provider supports them and I want near real-time events without wasting requests, for example payment confirmations.</p>
<p><strong>Q2. How do you secure a webhook endpoint?</strong>
HTTPS only, verify the HMAC signature on the raw body with constant-time comparison, reject old timestamps to stop replays, optionally allowlist the provider's IPs, and keep the secret in a secrets manager.</p>
<p><strong>Q3. The same webhook event arrives twice. What do you do?</strong>
Expect it, because delivery is at-least-once. I store each event ID with a unique constraint, process inside a transaction, and if the ID already exists I skip the work and still return 200 so the provider stops retrying.</p>
<p><strong>Q4. How would you build a live cricket score feed for millions of users?</strong>
The score is mostly server to client, so SSE or WebSockets fan out from a pub/sub layer, with a CDN or edge layer if possible. I would use WebSockets if users also send things in real time, like chat or reactions. Clients reconnect with backoff and fetch the latest score on reconnect.</p>
<p><strong>Q5. How do you scale WebSockets horizontally?</strong>
Load balancers that support long-lived connections, a shared pub/sub (Redis, Kafka or a managed service) so a message can reach a user on any server, heartbeats to clean dead connections, and connection limits per node with autoscaling on connection count.</p>
<p><strong>Q6. What is long polling and why is it not enough?</strong>
The server holds the request until there is data, which cuts empty responses. But every waiting client still ties up a connection, and each message needs a new request, so for high-frequency two-way traffic WebSockets are more efficient.</p>
<hr />
<h3>Sources</h3>
<ul>
<li>Stripe Docs, "Receive Stripe events in your webhook endpoint" (Stripe-Signature, raw body, retries up to three days in live mode): <a href="https://docs.stripe.com/webhooks">https://docs.stripe.com/webhooks</a></li>
<li>GitHub Docs, "Best practices for using webhooks" (respond within 10 seconds, X-GitHub-Delivery): <a href="https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks">https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks</a></li>
<li>GitHub Docs, "Validating webhook deliveries" (X-Hub-Signature-256, constant-time compare): <a href="https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries">https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries</a></li>
<li>Razorpay Docs, "Webhooks best practices" and "Validate and test webhooks" (24-hour retries, at-least-once, x-razorpay-event-id, X-Razorpay-Signature): <a href="https://razorpay.com/docs/webhooks/best-practices">https://razorpay.com/docs/webhooks/best-practices</a></li>
<li>RFC 6455, "The WebSocket Protocol": <a href="https://www.rfc-editor.org/rfc/rfc6455">https://www.rfc-editor.org/rfc/rfc6455</a></li>
<li>MDN, "Using server-sent events": <a href="https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events">https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events</a></li>
</ul>
<hr />
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> With 14+ years in Cloud &amp; DevOps, I help engineers get interview-ready. Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design questions like this one, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
<p>Follow <strong>@cloudtechbydevang</strong> on Instagram for more Cloud &amp; DevOps interview prep, one story at a time. 🌕 Happy Karva Chauth!</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[How 10-minute delivery apps really work (system design)]]></title><description><![CDATA[It's 7:02 AM. You place a grocery order, and the app says 8 minutes. The big supermarket is 3 km away, yet the doorbell rings at 7:10. The rider didn't…]]></description><link>https://cloudtechbydevang.com/how-10-minute-delivery-apps-really-work-system-design</link><guid isPermaLink="true">https://cloudtechbydevang.com/how-10-minute-delivery-apps-really-work-system-design</guid><category><![CDATA[Devops]]></category><category><![CDATA[Devops articles]]></category><category><![CDATA[System Design]]></category><category><![CDATA[Cloud]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Wed, 28 Oct 2026 15:58:13 GMT</pubDate><content:encoded><![CDATA[<p>It's 7:02 AM. You place a grocery order, and the app says <strong>8 minutes</strong>. The big supermarket is 3 km away, yet the doorbell rings at 7:10.</p>
<p>The rider didn't break any speed limits. The speed was <strong>designed</strong> into the system before you even opened the app. Here is how, and how to explain it in a system design interview.</p>
<blockquote>
<p><strong>The core insight:</strong> delivery speed comes from <em>distance</em>, not from how fast the rider drives. Every design choice below exists to keep the distance and the waiting short.</p>
</blockquote>
<h2>1. Dark stores: shrink the distance</h2>
<p>A <strong>dark store</strong> is a small warehouse, usually 1,500-4,000 sq ft, that serves <strong>online orders only</strong>. No walk-in customers, no fancy aisles. Each one covers a radius of roughly <strong>1-2 km</strong>.</p>
<ul>
<li>Short radius → ride time of 3-5 minutes</li>
<li>Layout optimised for pickers, not shoppers: fast movers near the packing desk</li>
<li>A limited range (a few thousand SKUs) keeps the store small and the stock accurate</li>
</ul>
<p><strong>Trade-off:</strong> you need <em>many</em> small stores rather than a few big ones. That means higher real-estate and operations cost, and every store has to be stocked correctly.</p>
<h2>2. Demand forecasting: stock before you ask</h2>
<p>A small store can't hold everything, so it has to hold <strong>the right things</strong>. Overnight, forecasting models predict demand per store, per SKU, per hour, using:</p>
<ul>
<li>historical orders for that area and day of week</li>
<li>weather, holidays, festivals, local events</li>
<li>recent trends and promotions</li>
</ul>
<p>The output drives replenishment: trucks restock each dark store before the morning rush. Bread, milk and eggs are on the shelf by 6 AM because the system expected your order.</p>
<p><strong>Interview angle:</strong> this is a batch ML pipeline (data lake → feature store → model → replenishment orders), separate from the real-time ordering path.</p>
<h2>3. Geo-sharding and nearest-store routing</h2>
<p>The city map is divided into <strong>grid cells</strong> (for example geohash or H3 hexagons). Every cell is assigned to one serviceable dark store.</p>
<p>When you open the app:</p>
<ol>
<li>Your lat/long → cell ID (a cheap computation)</li>
<li>Cell ID → store ID (a key-value lookup)</li>
<li>Every later request is routed with that store ID</li>
</ol>
<p>This is <strong>geo-sharding</strong>: data and traffic are partitioned by location. Orders, inventory and rider assignment for one store mostly stay inside that shard, so the system scales horizontally by adding stores and shards. Cells can be reassigned dynamically when a store is overloaded or closed.</p>
<h2>4. Inventory sync: show only what's really there</h2>
<p>The app shows <strong>only the live stock of your assigned store</strong>. If the store has 0 units of something, you shouldn't be able to add it to your cart.</p>
<ul>
<li>Every pick, restock and damage write-off updates inventory in near real time (event stream such as Kafka → inventory service)</li>
<li>At checkout, items are <strong>reserved</strong> atomically (decrement with a check, or a short-TTL reservation) to avoid overselling</li>
<li>Unpaid or abandoned reservations expire and the stock is released</li>
</ul>
<p><strong>Trade-off:</strong> browsing can tolerate slightly stale counts (eventual consistency), but <strong>checkout must be strongly consistent</strong> for each SKU and store.</p>
<h2>5. Caching: make the app open instantly</h2>
<p>Catalog data (names, images, categories) changes rarely; prices change a few times a day. So:</p>
<ul>
<li><strong>CDN</strong> for images and static assets</li>
<li><strong>Redis / in-memory cache</strong> for catalog and price data per store</li>
<li>Cache invalidation driven by events when a price or listing changes</li>
<li>Only fast-changing data (stock, ETA) is fetched live</li>
</ul>
<p>The result: the home screen renders in milliseconds, and the database isn't hit by millions of identical reads.</p>
<h2>6. The last 2 minutes: pick, pack, dispatch</h2>
<ul>
<li>The order is pushed to the store's picker app with an optimised pick path</li>
<li>Pick and pack takes <strong>about 2 minutes</strong></li>
<li>A rider is <strong>already waiting at the store</strong>, assigned by a dispatch service that knows rider locations and the order queue</li>
<li>ETA = pick time + ride time, predicted per order and shown to the user</li>
</ul>
<p><strong>Timeline:</strong> about 1 min to confirm, 2 min to pick and pack, 4-5 min to ride = <strong>about 8 minutes</strong>.</p>
<h2>How to answer "Design a 10-minute delivery system" in an interview</h2>
<ol>
<li><strong>Clarify:</strong> city scale, number of SKUs, peak orders per minute, the SLA (10 minutes at p90?)</li>
<li><strong>Core idea:</strong> speed = short distance + no waiting, so dark stores within 1-2 km</li>
<li><strong>Components:</strong><ul>
<li>Location service: geo-sharding (geohash/H3) → nearest-store routing</li>
<li>Catalog service + cache (CDN, Redis)</li>
<li>Inventory service: event-driven sync, atomic reservation at checkout</li>
<li>Order service → picker app → dispatch/rider service</li>
<li>Forecasting + replenishment pipeline (batch, overnight)</li>
</ul>
</li>
<li><strong>Data and consistency:</strong> eventual for browsing, strong for checkout per SKU and store</li>
<li><strong>Scale and failure:</strong> shard by store/region, reassign cells on overload, graceful degradation (hide the 10-minute promise and show a longer ETA)</li>
<li><strong>Metrics:</strong> order-to-door p50/p90, pick time, stock-out rate, forecast accuracy</li>
</ol>
<p><strong>One-line summary:</strong> <em>dark stores, demand forecasting, nearest-store routing, inventory sync and caching.</em></p>
<hr />
<h3>Preparing for Cloud / DevOps / SRE interviews?</h3>
<p>I've spent 14+ years building and running cloud systems. Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong>: <a href="https://topmate.io/cloudtechbydevang">topmate.io/cloudtechbydevang</a></p>
<p>Watch the short story version on Instagram: <a href="https://instagram.com/cloudtechbydevang">@cloudtechbydevang</a></p>
]]></content:encoded></item><item><title><![CDATA[Caching 101: Why Facebook Keeps So Much in Memcached (and How to Cache Without Breaking Things)]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple This post is the full breakdown for the Bapu ki Class Reel on caching. If you…]]></description><link>https://cloudtechbydevang.com/caching-101-why-facebook-keeps-so-much-in-memcached-and-how-to-cache-without-breaking-things</link><guid isPermaLink="true">https://cloudtechbydevang.com/caching-101-why-facebook-keeps-so-much-in-memcached-and-how-to-cache-without-breaking-things</guid><category><![CDATA[caching]]></category><category><![CDATA[Redis]]></category><category><![CDATA[memcached]]></category><category><![CDATA[System Design]]></category><category><![CDATA[interview-prep]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Mon, 26 Oct 2026 16:31:42 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>This post is the full breakdown for the Bapu ki Class Reel on caching. If you came here after commenting CACHE, welcome. Bapu explained it with his chashma on the table; here is the full engineering version.</p>
<p>Your database is at 100% CPU. Pages are slow. You look at the queries and notice something: most of them are <strong>reading the same data again and again</strong>. The same user profiles, the same product details, the same home page rows.</p>
<p>This is the moment for a <strong>cache</strong>.</p>
<p>Caching is one of the most powerful tools in system design, and one of the easiest to get wrong. In interviews it often comes as a scenario like the one Beta brings to Bapu in the Reel:</p>
<blockquote>
<p><em>"Your database hits 100% CPU on repeated reads. Fix it."</em></p>
</blockquote>
<p>In this post I will explain, in simple English, what a cache is, how Facebook used memcache at enormous scale, the <strong>cache-aside</strong> pattern, why you should <strong>delete</strong> cache keys on write instead of updating them, how TTLs work, and how to stop a <strong>cache stampede</strong>. These are the exact points that separate a junior answer from a senior one. In 14+ years of running production systems, I have seen caches save databases, and I have seen badly designed caches take them down. Let's do it right.</p>
<hr />
<h2>1. What a cache is</h2>
<p>A cache is a <strong>fast, temporary copy</strong> of data, kept close to where it is needed. Usually that means <strong>in memory</strong> (RAM), which is far faster than reading from a database that has to parse a query, find rows and maybe read disk.</p>
<p>Popular in-memory caches are <strong>Memcached</strong> and <strong>Redis</strong>. On AWS, Amazon ElastiCache runs both as a managed service.</p>
<p>The idea is simple:</p>
<ul>
<li>Most reads hit the cache and return in well under a millisecond inside the data centre.</li>
<li>Only <strong>cache misses</strong> go to the database.</li>
<li>The database now handles a small share of the reads, so its CPU drops.</li>
</ul>
<p>The share of reads served from the cache is called the <strong>hit ratio</strong>. A high hit ratio is what makes caching worth it.</p>
<hr />
<h2>2. Facebook's memcache: caching at extreme scale</h2>
<p>In 2013, Facebook engineers published a well-known paper, <em>"Scaling Memcache at Facebook"</em>, at the USENIX NSDI conference. It describes how they used memcached as a huge caching layer in front of their databases.</p>
<p>Some facts from the paper:</p>
<ul>
<li>The system handled <strong>billions of requests per second</strong> and stored <strong>trillions of items</strong>.</li>
<li>It served <strong>over a billion users</strong> around the world.</li>
<li>Memcache was used as a <strong>demand-filled look-aside cache</strong>: web servers check the cache first and fill it from the database on a miss.</li>
<li>On writes, web servers updated the database and then sent a <strong>delete</strong> to memcache.</li>
<li>They used <strong>leases</strong> to solve stale data and thundering herd problems.</li>
<li>They had a <strong>Gutter</strong> pool of servers to take over when a few memcached servers failed, so the databases would not get flooded.</li>
<li>They used an invalidation daemon called <strong>mcsqueal</strong> that reads committed database changes and sends deletes to the caches.</li>
</ul>
<p>You do not need Facebook's scale to use these ideas. They work the same for a startup with one Redis node.</p>
<hr />
<h2>3. The cache-aside pattern (look-aside)</h2>
<p>This is the most common caching pattern, and it is what Facebook's paper calls a demand-filled look-aside cache.</p>
<h3>On a read</h3>
<ol>
<li>The app asks the cache for the key, for example <code>user:42</code>.</li>
<li><strong>Hit:</strong> the cache has it. Return it. Done.</li>
<li><strong>Miss:</strong> the cache does not have it. The app reads from the database.</li>
<li>The app <strong>puts the result in the cache</strong>, so the next read is a hit.</li>
<li>Return the data.</li>
</ol>
<pre><code class="language-python">def get_user(user_id):
    key = f"user:{user_id}"
    user = cache.get(key)
    if user is None:                 # cache miss
        user = db.query_user(user_id)
        cache.set(key, user, ttl=300) # keep for 5 minutes
    return user
</code></pre>
<h3>Why it is popular</h3>
<ul>
<li><strong>Simple.</strong> The app is in control. The cache is "just" a fast copy.</li>
<li><strong>Lazy.</strong> Only data that is actually read gets cached.</li>
<li><strong>Safe on failure.</strong> If the cache goes down, the app can still read from the database (slower, but correct). You must make sure the database can survive that extra load, though.</li>
</ul>
<p>Microsoft's Azure Architecture Center documents this as the <strong>Cache-Aside pattern</strong>, and it applies equally to Redis, Memcached, ElastiCache or any other cache.</p>
<hr />
<h2>4. On a write: delete, don't update</h2>
<p>When data changes, the cache copy becomes <strong>stale</strong>. What should you do?</p>
<p>The Facebook pattern is:</p>
<ol>
<li><strong>Update the database</strong> (the source of truth).</li>
<li><strong>Delete the cache key.</strong></li>
<li>The <strong>next read</strong> misses, reads the fresh value from the database and fills the cache again.</li>
</ol>
<p>Why delete instead of writing the new value into the cache? The Facebook paper explains that they chose to delete because <strong>deletes are idempotent</strong>: doing them twice is harmless. Updating the cache directly opens the door to race conditions. Two writers can update the database in one order and the cache in the other order, and you end up with an old value stuck in the cache.</p>
<p>Remember too that the cache is not the source of truth. It is allowed to lose data. The database is the authority.</p>
<hr />
<h2>5. TTL: let old data expire</h2>
<p>A <strong>TTL</strong> (time to live) tells the cache to remove a key after a set time, for example 5 minutes.</p>
<p>TTLs protect you from mistakes:</p>
<ul>
<li>If an invalidation is missed, the stale value still disappears after the TTL.</li>
<li>Data that nobody reads anymore does not stay in memory forever.</li>
</ul>
<p>How long should a TTL be? It depends on <strong>how stale the data is allowed to be</strong>:</p>
<ul>
<li>A product description: maybe minutes or hours.</li>
<li>A stock price or account balance: very short, or do not cache it at all.</li>
<li>A user's own profile right after they edit it: delete on write so they see their change immediately.</li>
</ul>
<p>A useful trick is to add a little <strong>random jitter</strong> to TTLs, so thousands of keys do not all expire at the same second.</p>
<hr />
<h2>6. The cache stampede (thundering herd)</h2>
<p>Here is the trap that takes down databases.</p>
<p>Imagine a very popular key, like the home page data. Its TTL expires, or it gets deleted because of an update. In that instant, <strong>thousands of requests</strong> miss the cache at the same time. All of them go to the database for the same data. The database, which was relaxed a second ago, suddenly gets a flood. This is called a <strong>cache stampede</strong> or <strong>thundering herd</strong>.</p>
<h3>How Facebook stopped it: leases</h3>
<p>The paper describes <strong>leases</strong>. When a client misses the cache, memcached gives it a <strong>lease token</strong>, a permission to fill that key. Memcached hands out a token for a given key only <strong>once every 10 seconds</strong> by default. Other clients that miss during that time are told to <strong>wait a short moment and retry</strong>. Usually the lease holder fills the cache within a few milliseconds, so the retries become hits.</p>
<p>The paper reports a striking result: for a set of keys prone to thundering herds, the peak database query rate dropped from <strong>17,000 per second without leases to 1,300 per second with leases</strong>.</p>
<h3>How you can stop it</h3>
<ul>
<li><strong>Locking / single flight:</strong> only one request rebuilds a missing key. Others wait or get the old value.</li>
<li><strong>Serve stale while revalidating:</strong> keep serving the slightly old value while one request refreshes it in the background.</li>
<li><strong>Early refresh:</strong> refresh hot keys a little before their TTL ends.</li>
<li><strong>TTL jitter:</strong> avoid many keys expiring together.</li>
</ul>
<hr />
<h2>7. Other things to plan for</h2>
<ul>
<li><strong>What to cache:</strong> data that is read often and changes rarely. Do not cache everything.</li>
<li><strong>Cache sizing and eviction:</strong> when memory is full, the cache removes keys, usually the least recently used (LRU).</li>
<li><strong>Cold start:</strong> a new or restarted cache is empty, and every request misses. Warm it up gradually.</li>
<li><strong>Cache failure:</strong> if a cache node dies, its keys go to the database. Facebook's Gutter pool is one answer; replicas and gradual failover are others.</li>
<li><strong>Hot keys:</strong> a single extremely popular key can overload one cache node. Replicate it or use a small local in-process cache in front.</li>
</ul>
<hr />
<h2>8. The example visuals</h2>
<p>The "database at 100% CPU on repeated reads" line is an <strong>example scenario</strong>, the way interviewers phrase it. The App, Cache and Database boxes in the Reel are simplified. The Facebook facts (billions of requests per second, trillions of items, look-aside caching, delete on write because deletes are idempotent, leases for stale sets and thundering herds, the 10-second token rate, 17K/s to 1.3K/s, Gutter and mcsqueal) are verified from the NSDI 2013 paper. Facebook's current infrastructure may differ from what the 2013 paper describes.</p>
<hr />
<h2>9. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"Your database hits 100% CPU on repeated reads. Fix it."</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"Since most queries read the same data, I would add an in-memory cache like Redis or Memcached in front of the database, using the cache-aside pattern. On a read, check the cache first. On a miss, read the database and fill the cache. On a write, update the database and then delete the cache key, rather than updating it, because deletes are idempotent and avoid race conditions. That is exactly what Facebook described in its memcache paper, where the system handled billions of requests per second. I would set TTLs based on how stale each type of data can be, with some jitter. And I would protect against cache stampedes, when a hot key expires and many requests hit the database at once, using locks or leases, like Facebook did. I would measure the hit ratio and make sure the database can survive if the cache fails."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><em>Cache-aside vs write-through?</em> Cache-aside fills on read; write-through writes to the cache and database together on every write.</li>
<li><em>Why delete instead of update on write?</em> Deletes are idempotent and avoid stale values from reordered updates.</li>
<li><em>What is a cache stampede?</em> Many requests miss a hot key at once and flood the database. Use locks, leases or stale-while-revalidate.</li>
<li><em>Redis or Memcached?</em> Memcached is a simple, fast key-value cache. Redis adds data structures, persistence options and replication.</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li>Nishtala et al., "Scaling Memcache at Facebook", USENIX NSDI 2013 (billions of requests per second, trillions of items; demand-filled look-aside cache; delete on write because deletes are idempotent; leases for stale sets and thundering herds, one token per key every 10 seconds, 17K/s to 1.3K/s; Gutter; mcsqueal): <a href="https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf">https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf</a></li>
<li>Microsoft Azure Architecture Center, "Cache-Aside pattern": <a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside">https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside</a></li>
<li>AWS, "Amazon ElastiCache" (managed Redis/Valkey and Memcached): <a href="https://aws.amazon.com/elasticache/">https://aws.amazon.com/elasticache/</a></li>
</ul>
<hr />
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[How Hotstar Streamed Cricket to 25 Million+ People at Once: Scaling for a Known Traffic Spike]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple This post is the full breakdown for the "cricket match night at home" story…]]></description><link>https://cloudtechbydevang.com/how-hotstar-streamed-cricket-to-25-million-people-at-once-scaling-for-a-known-traffic-spike</link><guid isPermaLink="true">https://cloudtechbydevang.com/how-hotstar-streamed-cricket-to-25-million-people-at-once-scaling-for-a-known-traffic-spike</guid><category><![CDATA[System Design]]></category><category><![CDATA[scalability]]></category><category><![CDATA[AWS]]></category><category><![CDATA[Kubernetes]]></category><category><![CDATA[Devops]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Sun, 25 Oct 2026 14:13:00 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<blockquote>
<p>This post is the full breakdown for the <strong>"cricket match night at home" story Reel</strong>. If you came here after commenting <strong>HOTSTAR</strong>, welcome!</p>
</blockquote>
<p><strong>The story in the Reel:</strong> on match night, the whole family arrives at once.</p>
<ul>
<li>If Mummy starts making chai only after the guests arrive, it is too late. That is <strong>reactive autoscaling</strong>: new servers take minutes to arrive.</li>
<li>Instead, she makes 20 cups <strong>before the toss, plus 5 extra</strong>. That is <strong>pre-warming with a buffer</strong>.</li>
<li>Hotstar did the same for 25.3 million concurrent viewers, and scaled on <strong>live user count, not CPU</strong>.</li>
<li>Interview line: for a known spike, pre-scale, keep a buffer, scale on business metrics and load test early.</li>
</ul>
<p>Below is the detailed version.</p>
<hr />
<p>On 10 July 2019, India played New Zealand in the semi-final of the ICC Cricket World Cup. As the match reached its tense final overs, <strong>25.3 million</strong> people were watching live on Hotstar at the same moment. Hotstar's own India Watch Report says that peak came just before MS Dhoni was run out. It was a world record for concurrent viewers on a streaming platform, beating Hotstar's earlier record of 18.6 million from the IPL 2019 final.</p>
<p>Think about what that means for the engineering team. It is not just "a lot of users". It is millions of people <strong>arriving within minutes</strong>, all at once, and all expecting the video to start instantly. And it happens on a known date and time.</p>
<p>This gives us a great interview question, a classic interview question:</p>
<blockquote>
<p><em>"A live match adds millions of users in minutes. Fix it."</em></p>
</blockquote>
<p>In this post I will explain, in simple English, why normal autoscaling is too slow for this kind of spike, what Hotstar's engineers have said publicly about how they prepare, and how you can use the same ideas in your own systems and interviews. I have spent 14+ years in cloud and SRE, and "scale for a known event" is one of the most practical skills you can show.</p>
<hr />
<h2>1. Why a cricket match is a special kind of traffic</h2>
<p>Most websites have a smooth daily pattern: traffic rises in the morning, peaks in the evening, falls at night. Autoscaling handles that well.</p>
<p>Live sport is different:</p>
<ul>
<li><strong>Huge and sudden.</strong> Millions of people can join within minutes, for example when a star batter walks in, or in the last overs of a close game.</li>
<li><strong>Very spiky inside the match.</strong> Breaks, wickets and key moments cause sudden jumps and drops.</li>
<li><strong>Zero tolerance.</strong> If the stream fails during the final over, users do not come back later. The moment is gone.</li>
<li><strong>Known in advance.</strong> The match schedule is public. You know roughly when the peak will come.</li>
</ul>
<p>That last point is the key. A known spike lets you <strong>prepare</strong> instead of <strong>react</strong>.</p>
<hr />
<h2>2. Why normal autoscaling is too slow</h2>
<p>A typical autoscaler watches a metric like CPU. When CPU goes above a threshold, it adds servers. That works for gradual growth, but it has a built-in delay.</p>
<p>Hotstar's engineering blog post "Scaling for Tsunami traffic" (April 2019) breaks this delay down. Adding new infrastructure capacity took around <strong>90 seconds</strong>, and then containers and the application took around <strong>75 seconds</strong> more to start and become ready. Their post uses these as typical numbers ("let's say"): that is about 3 minutes lost, and once you add the time to gather metrics and make the scaling decision, the total delay can be around <strong>4 minutes</strong>. The post says that in those 4 minutes they would have missed about <strong>two million people</strong> joining the platform.</p>
<p>During those minutes, millions of new users can arrive. Existing servers get overloaded, response times jump, and errors spread. By the time the new servers are ready, the damage is done.</p>
<p>So the lesson is: <strong>for a big, known spike, you cannot wait for the spike to tell you to scale.</strong></p>
<hr />
<h2>3. Fix 1: pre-warm before the match</h2>
<p>In their AWS re:Invent 2019 talk on scaling Hotstar for 25 million concurrent viewers, Hotstar's engineers explained that they did not rely on traditional autoscaling for these big events. Instead, they <strong>pre-warmed</strong> their infrastructure: they scaled up ahead of the expected peak.</p>
<p>The "Scaling for Tsunami traffic" post adds an important detail: they kept a <strong>buffer</strong>. During live events they planned capacity for about <strong>2 million more concurrent users</strong> than the current level, so a sudden jump would land on capacity that was already running.</p>
<p>In simple terms:</p>
<ol>
<li>Forecast the peak for the match (based on past matches, teams playing, stage of the tournament).</li>
<li>Scale up <strong>before</strong> the match starts, not when traffic arrives.</li>
<li>Keep a <strong>headroom buffer</strong> above current traffic at all times during the event.</li>
<li>Scale down carefully after the match, once traffic really drops.</li>
</ol>
<p>Pre-warming also applies to things beyond your servers: load balancers, databases, caches and CDNs. A cold cache in front of millions of users is almost as bad as no servers at all.</p>
<hr />
<h2>4. Fix 2: scale on business metrics, not just CPU</h2>
<p>CPU is a lagging signal. By the time CPU is high, users are already waiting.</p>
<p>Hotstar's engineers described using their own scaling logic based on <strong>concurrency</strong> (how many users are on the platform) and <strong>request rate</strong>, not only CPU. They planned capacity in <strong>ladders</strong>: for each level of concurrent users (one million, two million, and so on), they knew how much capacity each service needed.</p>
<p>Why this is smart:</p>
<ul>
<li><strong>Concurrency is a leading signal.</strong> When users join, they show up in the concurrency number immediately, before the backend gets hot.</li>
<li><strong>Ladders make scaling predictable.</strong> Instead of guessing, the team has a tested table: "at X million users, service A needs N servers".</li>
<li><strong>Different services scale differently.</strong> The login service, the video manifest service and the comments service each have their own ladder.</li>
</ul>
<p>For your own systems, good business metrics might be active users, requests per second per service, queue depth, or orders per minute.</p>
<hr />
<h2>5. Fix 3: load test before fans arrive</h2>
<p>You cannot discover your limits during the final. Hotstar's talks describe <strong>game days</strong> and large-scale <strong>load testing</strong>, using their own load generation tooling to simulate traffic beyond what they expected.</p>
<p>A good load-testing practice for big events includes:</p>
<ul>
<li><strong>Test beyond the forecast peak</strong>, so you know where things break.</li>
<li><strong>Test the whole path</strong>: DNS, CDN, load balancers, APIs, databases, caches, third-party services.</li>
<li><strong>Test the spike shape</strong>, not just the steady state. A jump from 5 million to 15 million in a few minutes is a different test from a steady 15 million.</li>
<li><strong>Run game days</strong>: practise the event with the on-call team, dashboards and runbooks, so people know what to do.</li>
</ul>
<hr />
<h2>6. Fix 4: have a plan for when it still gets too hot</h2>
<p>Even with all of this, you plan for the worst. Common techniques for big live events:</p>
<ul>
<li><strong>Graceful degradation:</strong> turn off non-critical features (for example, heavy personalisation or some social features) to protect the main video stream.</li>
<li><strong>Rate limiting and queuing</strong> for expensive operations like login spikes.</li>
<li><strong>Caching aggressively</strong> at the edge, so repeated requests never reach the origin.</li>
<li><strong>Panic buttons:</strong> pre-built switches that the on-call team can flip in seconds.</li>
</ul>
<p>The principle is simple: the video must keep playing. Everything else can wait.</p>
<hr />
<h2>7. Known spike vs surprise spike</h2>
<p>This is a great point to raise in interviews:</p>
<table>
<thead>
<tr>
<th></th>
<th>Known spike (a match, a sale)</th>
<th>Surprise spike (viral post, news)</th>
</tr>
</thead>
<tbody><tr>
<td>Can you forecast it?</td>
<td>Yes, roughly</td>
<td>No</td>
</tr>
<tr>
<td>Main tool</td>
<td>Pre-scaling with a buffer</td>
<td>Fast autoscaling, caching, degradation</td>
</tr>
<tr>
<td>Metric to watch</td>
<td>Concurrency, requests</td>
<td>Requests, latency, errors</td>
</tr>
<tr>
<td>Preparation</td>
<td>Load tests, game days, runbooks</td>
<td>Headroom, fast scaling, rate limits</td>
</tr>
</tbody></table>
<p>Most real systems need both: pre-scaling for events you know about, and fast reactive scaling plus graceful degradation for everything else.</p>
<hr />
<h2>8. The example visuals</h2>
<p>The "a live match adds millions of users in minutes" line is an <strong>example scenario</strong> in interview style. The ladder and buffer graphics in the Reel are simplified illustrations. The 25.3 million record (India vs New Zealand, ICC CWC 2019 semi-final) and the 18.6 million IPL 2019 final record are verified from the ICC release and Hotstar's report. The roughly 90 s and 75 s delays, the 2 million buffer, the concurrency and request-rate scaling, the ladders, pre-warming and game days come from Hotstar's engineering blog and their AWS re:Invent 2019 talk. Exact numbers may have changed since; JioHotstar today runs at a different scale.</p>
<hr />
<h2>9. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"A live match adds millions of users in minutes. Fix it."</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"Normal autoscaling reacts too late for this. It waits for CPU to rise, then it takes time to add servers and start the apps. Hotstar's engineers said adding infrastructure took about 90 seconds and starting containers about 75 more, while millions of users were joining. Because a match is a known event, I would pre-scale before it starts, based on a forecast, and keep a buffer above current traffic. I would scale on business metrics like concurrent users and request rate, not just CPU, with tested capacity ladders for each service. Before the event I would run load tests beyond the expected peak and a game day with the on-call team. And I would have a degradation plan to switch off non-critical features so the video keeps playing."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><em>What about the database and cache?</em> Pre-warm caches, use read replicas, and protect the database from stampedes.</li>
<li><em>How do you forecast the peak?</em> Past matches, the teams playing, the stage of the tournament and marketing data.</li>
<li><em>What if the forecast is wrong?</em> Keep the buffer, use reactive scaling as a backup, and degrade gracefully.</li>
<li><em>How do you control cost?</em> Scale down after the event, and use the ladders to avoid over-provisioning.</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li>ICC media release, "ICC Men's Cricket World Cup 2019 shatters audience records" (12 July 2019; 25.3 million concurrent views on Hotstar for India v New Zealand semi-final; previous record 18.6 million): <a href="https://www.icc-cricket.com/media-releases/icc-mens-cricket-world-cup-2019-shatters-audience-records">https://www.icc-cricket.com/media-releases/icc-mens-cricket-world-cup-2019-shatters-audience-records</a></li>
<li>Hotstar, "India Watch Report 2019" (25.3M peak concurrency, just before MS Dhoni was run out; 18.6M IPL 2019 final): <a href="https://secure-media.hotstar.com/media-website/indiawatchreport2019/India_Watch_Report_2019.pdf">https://secure-media.hotstar.com/media-website/indiawatchreport2019/India_Watch_Report_2019.pdf</a></li>
<li>Hotstar Tech blog, "Scaling for Tsunami traffic" (April 2019; about 90 s to add infrastructure, about 75 s for containers and apps, around 4 minutes total delay; buffer of 2 million concurrent users; ladders; custom scaling): <a href="https://blog.hotstar.com/scaling-for-tsunami-traffic-2ec290c37504">https://blog.hotstar.com/scaling-for-tsunami-traffic-2ec290c37504</a></li>
<li>AWS re:Invent 2019, "Scaling Hotstar.com for 25 million concurrent viewers" (CMY302) (pre-warming instead of traditional autoscaling; scaling on request rate and concurrency; game days and load testing): <a href="https://www.youtube.com/watch?v=mFpqrVxxwKc">https://www.youtube.com/watch?v=mFpqrVxxwKc</a></li>
</ul>
<hr />
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[How Banks and Netflix Stop Bots Trying Millions of Stolen Passwords: Credential Stuffing, Made Simple]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple This post is the full breakdown for the Bapu ki Class Reel on credential…]]></description><link>https://cloudtechbydevang.com/how-banks-and-netflix-stop-bots-trying-millions-of-stolen-passwords-credential-stuffing-made-simple</link><guid isPermaLink="true">https://cloudtechbydevang.com/how-banks-and-netflix-stop-bots-trying-millions-of-stolen-passwords-credential-stuffing-made-simple</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Credential Stuffing]]></category><category><![CDATA[MFA]]></category><category><![CDATA[owasp]]></category><category><![CDATA[Web Security]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Sat, 24 Oct 2026 14:13:43 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<blockquote>
<p>This post is the full breakdown for the <strong>Bapu ki Class Reel on credential stuffing</strong>. If you came here after commenting <strong>BOTS</strong>, welcome!</p>
</blockquote>
<p><strong>The story in the Reel:</strong> Beta panics: <em>"Bapu, 50,000 wrong passwords a minute on our login page!"</em> Bapu calmly explains:</p>
<ul>
<li>This is <strong>credential stuffing</strong>: bots take passwords leaked from other sites and try them on every door.</li>
<li>Blocking IPs alone fails. Block one and a hundred proxy IPs appear.</li>
<li>Use layers: <strong>limit attempts per account</strong>, show a <strong>CAPTCHA only when a login looks risky</strong>, and <strong>reject passwords found in known breaches</strong>.</li>
<li>The biggest lock is <strong>MFA</strong>. Per Microsoft's analysis (cited by OWASP), it would have stopped 99.9% of account compromises.</li>
</ul>
<p>Below is the detailed version.</p>
<hr />
<p>Every big login page on the internet is attacked by bots, all day. Not by a hacker typing passwords by hand, but by automated tools trying <strong>millions of username and password pairs</strong> that leaked from other websites. This attack is called <strong>credential stuffing</strong>, and it is one of the most common ways accounts get taken over.</p>
<p>If you work in cloud, DevOps, SRE or backend, you will meet this in production, and you will meet it in interviews. Here is how the question often sounds:</p>
<blockquote>
<p><em>"Your login API gets 50,000 failed logins a minute, from thousands of IPs. Stop it."</em></p>
</blockquote>
<p>In this post I will explain, in simple English, what credential stuffing is, why simply blocking IP addresses does not work, the layered defences that do work, and how to answer this in an interview. After 14+ years in cloud and operations, my strongest lesson here is simple: <strong>there is no single fix. You need layers.</strong></p>
<p>A quick note: banks and Netflix do not publish the full details of their login defences, and they should not. I use them as familiar <strong>examples</strong> of services that are constantly targeted. The defences below come from the OWASP Credential Stuffing Prevention Cheat Sheet, which is the standard reference.</p>
<hr />
<h2>1. What is credential stuffing?</h2>
<p>People reuse passwords. When one website is breached, the leaked email and password pairs get collected into huge lists. Attackers then use bots to <strong>replay</strong> those pairs against other websites: banks, streaming services, shopping sites, email providers. Because many people reuse the same password, a small percentage of attempts succeed. A small percentage of millions is still a lot of hijacked accounts.</p>
<h3>Three attacks people mix up</h3>
<p>The OWASP cheat sheet separates these clearly:</p>
<ul>
<li><strong>Brute force:</strong> trying many passwords against <strong>one</strong> account.</li>
<li><strong>Credential stuffing:</strong> trying <strong>known username and password pairs</strong> from other breaches.</li>
<li><strong>Password spraying:</strong> trying <strong>a few common passwords</strong> (like <code>Password123</code>) against <strong>many</strong> accounts.</li>
</ul>
<p>They look different in your logs, so the defences need to cover all three.</p>
<hr />
<h2>2. What the attack looks like in your system</h2>
<p>In the interview scenario, you see:</p>
<ul>
<li>A big jump in <strong>failed logins</strong> (50,000 a minute in our example).</li>
<li>Requests from <strong>thousands of different IP addresses</strong>.</li>
<li>Each IP sends only a <strong>few attempts</strong>, so per-IP limits barely trigger.</li>
<li>Many <strong>different usernames</strong>, often ones that do not even exist on your site.</li>
<li>Some logins <strong>succeed</strong>, and those accounts may then be abused: changed emails, stolen gift cards, fraud.</li>
</ul>
<p>The goal is to make the attack <strong>slow, expensive and noisy</strong> for the attacker, while keeping login easy for real users.</p>
<hr />
<h2>3. Why blocking IPs alone fails</h2>
<p>The first idea most people have is "block the bad IPs". It helps a little, but OWASP warns that you should not rely on IP blocking alone. Attackers rotate through large <strong>proxy networks</strong>, including residential proxies, so traffic comes from IPs that look like normal home users. If you block too aggressively, you also block real customers who share those networks.</p>
<p>IP information is still useful as <strong>one signal</strong>: known hosting providers, known bad proxy ranges, unusual countries for that user. But it must be combined with others.</p>
<hr />
<h2>4. The layered defence</h2>
<h3>Layer 1: Multi-factor authentication (MFA)</h3>
<p>The OWASP cheat sheet calls MFA <strong>by far the best defence</strong> against password-based attacks. It cites Microsoft's analysis that MFA would have stopped <strong>99.9%</strong> of account compromises. Even if the attacker has the right password, they do not have the second factor.</p>
<p>In practice, many services use <strong>risk-based or step-up MFA</strong>: you are not asked for a code every time, but you are asked when the login looks unusual (new device, new country, many recent failures).</p>
<h3>Layer 2: Rate limiting on more than IP</h3>
<p>Limit login attempts on several keys:</p>
<ul>
<li><strong>Per account:</strong> slow down repeated failures for one username (but be careful, an attacker can use this to lock real users out, so prefer delays and step-up checks over hard lockouts).</li>
<li><strong>Per IP and per network range.</strong></li>
<li><strong>Globally:</strong> alert when the overall failure rate jumps far above normal.</li>
</ul>
<h3>Layer 3: Device and bot signals</h3>
<p>OWASP describes <strong>device fingerprinting</strong> and <strong>connection fingerprinting</strong> (for example, the TLS client fingerprint known as JA3). Real browsers and phones look different from simple bot scripts. Combine signals like:</p>
<ul>
<li>Is the device known for this user?</li>
<li>Does the client behave like a real browser?</li>
<li>Does the connection fingerprint match a known automation tool?</li>
</ul>
<p>Many companies use a managed <strong>bot management</strong> service or a web application firewall (WAF) with bot rules for this.</p>
<h3>Layer 4: CAPTCHA, but only when risky</h3>
<p>CAPTCHAs annoy real users and can be solved by paid services, so OWASP recommends using them <strong>only when a request looks suspicious</strong>, not on every login. Think of a CAPTCHA as a speed bump for risky traffic, not a wall for everyone.</p>
<h3>Layer 5: Block breached passwords</h3>
<p>Stop the problem at the source: do not let users choose passwords that are already in breach lists. The <strong>Pwned Passwords</strong> service from Have I Been Pwned lets you check a password safely using <strong>k-anonymity</strong>: you hash the password and send only the <strong>first five characters</strong> of the hash. The service returns all matching hash suffixes, and you check locally. The full password, and even the full hash, never leaves your system.</p>
<h3>Layer 6: Tell the user</h3>
<p>Notify users about logins from new devices or locations, and about password changes. If an attacker does get in, the real user can react quickly.</p>
<hr />
<h2>5. Putting it together: a login request, step by step</h2>
<p>Here is a simple design you can draw on a whiteboard:</p>
<ol>
<li><strong>Edge / WAF:</strong> basic rate limits per IP and range, known-bad IP lists, bot rules.</li>
<li><strong>Login service:</strong> checks per-account and global failure rates.</li>
<li><strong>Risk engine:</strong> scores the attempt using device, connection fingerprint, location and history.</li>
<li><strong>Decision:</strong><ul>
<li>Low risk: allow normal login.</li>
<li>Medium risk: show a CAPTCHA or ask for MFA.</li>
<li>High risk: block, delay, or require MFA.</li>
</ul>
</li>
<li><strong>After login:</strong> notify on new devices, and protect sensitive actions (email change, payout) with step-up MFA.</li>
<li><strong>Monitoring:</strong> dashboards for failed-login rate, success rate on suspicious traffic, and accounts that changed details right after login.</li>
</ol>
<hr />
<h2>6. Common mistakes</h2>
<ul>
<li><strong>Only IP blocking.</strong> Proxy networks make it weak on its own.</li>
<li><strong>Hard account lockouts after a few failures.</strong> Attackers can lock out real users on purpose. Prefer increasing delays and step-up checks.</li>
<li><strong>Different error messages</strong> for "user does not exist" vs "wrong password". This tells attackers which accounts are real. Use one generic message.</li>
<li><strong>CAPTCHA for everyone.</strong> It hurts conversion and attackers solve it anyway.</li>
<li><strong>No monitoring.</strong> Many teams only notice the attack when customers complain.</li>
</ul>
<hr />
<h2>7. The example visuals</h2>
<p>The "50,000 failed logins a minute from thousands of IPs" line is an <strong>example scenario</strong>, the way interviewers phrase it. The proxy boxes in the Reel are illustrative. Banks and Netflix are used as familiar examples of targeted services; their exact defences are not public. The definitions, the MFA guidance and the 99.9% figure (Microsoft, cited by OWASP), the CAPTCHA and IP guidance, fingerprinting, and the Pwned Passwords k-anonymity model are verified from the sources below.</p>
<hr />
<h2>8. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"Your login API gets 50,000 failed logins a minute, from thousands of IPs. Stop it."</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"This looks like credential stuffing: bots replaying username and password pairs leaked from other sites. Blocking IPs alone won't work because attackers spread requests across large proxy networks. So I would add layers. Rate limits per account, per IP and globally. Device and connection fingerprinting to spot bots. A CAPTCHA only for risky attempts, not for everyone. A breached-password check so users can't pick leaked passwords. And the strongest control is MFA, ideally risk-based, because Microsoft's analysis says MFA would stop 99.9% of account compromises. I would also notify users about new-device logins and monitor the failed-login rate."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><em>Credential stuffing vs brute force vs password spraying?</em> Known pairs vs many passwords on one account vs few passwords on many accounts.</li>
<li><em>Why not lock the account after 5 failures?</em> Attackers can lock out real users. Use delays and step-up checks.</li>
<li><em>How do you check breached passwords without sending them?</em> k-anonymity: send only a hash prefix.</li>
<li><em>How do you know it is working?</em> Failed-login rate drops, success rate on suspicious traffic drops, fewer account takeover reports.</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li>OWASP, "Credential Stuffing Prevention Cheat Sheet" (definitions; MFA as the best defence and Microsoft's 99.9% figure; CAPTCHA only when suspicious; IP blocking alone is not enough; device and connection fingerprinting including JA3; breached password checks; notifying users): <a href="https://cheatsheetseries.owasp.org/cheatsheets/Credential_Stuffing_Prevention_Cheat_Sheet.html">https://cheatsheetseries.owasp.org/cheatsheets/Credential_Stuffing_Prevention_Cheat_Sheet.html</a></li>
<li>Have I Been Pwned, "Pwned Passwords" and API v3 docs (k-anonymity range API, first five characters of the SHA-1 hash): <a href="https://haveibeenpwned.com/Passwords">https://haveibeenpwned.com/Passwords</a> ; <a href="https://haveibeenpwned.com/API/v3">https://haveibeenpwned.com/API/v3</a></li>
<li>OWASP, "Authentication Cheat Sheet" (generic error messages, protecting against automated attacks): <a href="https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html">https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html</a></li>
</ul>
<hr />
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[How Google Keeps You Logged In Across Gmail, YouTube and Drive: Sessions vs JWT vs SSO, Made Simple]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple This post is the full breakdown for the "first day at a new office" story Reel…]]></description><link>https://cloudtechbydevang.com/how-google-keeps-you-logged-in-across-gmail-youtube-and-drive-sessions-vs-jwt-vs-sso-made-simple</link><guid isPermaLink="true">https://cloudtechbydevang.com/how-google-keeps-you-logged-in-across-gmail-youtube-and-drive-sessions-vs-jwt-vs-sso-made-simple</guid><category><![CDATA[JWT]]></category><category><![CDATA[oauth]]></category><category><![CDATA[SSO]]></category><category><![CDATA[Web Security]]></category><category><![CDATA[System Design]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Fri, 23 Oct 2026 14:10:20 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<blockquote>
<p>This post is the full breakdown for the <strong>"first day at a new office" story Reel</strong>. If you came here after commenting <strong>JWT</strong>, welcome!</p>
</blockquote>
<p><strong>The story in the Reel:</strong> on his first day, Rohan gets stopped on every floor.</p>
<ul>
<li>The guard checks his name in the <strong>visitor register</strong> every time. That is a <strong>session</strong>: the server keeps the login record, and you only carry an ID number.</li>
<li>Then HR gives him a <strong>signed ID card</strong>. Any guard can check the seal and let him in, without opening the register. That is a <strong>JWT</strong>.</li>
<li>The catch: a lost card works until it expires. So keep expiry short and get a fresh card with a <strong>refresh token</strong>.</li>
<li>One card that opens the cafeteria, gym and parking too? That is <strong>SSO</strong>, like "Sign in with Google".</li>
</ul>
<p>Below is the detailed version.</p>
<hr />
<p>You sign in to Gmail in the morning. Later you open YouTube and Google Drive, and you are already signed in. You did not type your password again. And on hundreds of other websites you can click <strong>"Sign in with Google"</strong> and skip creating a new password at all.</p>
<p>How does that work? It comes down to three ideas every backend and cloud engineer must know: <strong>sessions</strong>, <strong>tokens (JWT)</strong> and <strong>single sign-on (SSO)</strong>.</p>
<p>These come up in almost every system design and security interview. The question is often phrased like the one interviewers love:</p>
<blockquote>
<p><em>"Users log in to one of our apps, but our other apps ask them to log in again. Fix it."</em></p>
</blockquote>
<p>In this post I will explain, in simple English, how sessions work, what a JWT is, the big catch with JWTs, how SSO with OpenID Connect ties many apps together, and how to answer this in an interview. I have spent 14+ years building and running cloud systems, and login is one area where "simple and correct" beats "clever" every time.</p>
<p>A quick note: Google does not publish every internal detail of how its own services share your sign-in. What is public is that you sign in with one Google Account, and that "Sign in with Google" for other websites is built on <strong>OpenID Connect</strong>. I use Google as a familiar <strong>example</strong> of single sign-on, not as a description of its private internals.</p>
<hr />
<h2>1. The basic problem: HTTP forgets you</h2>
<p>HTTP is <strong>stateless</strong>. Each request is separate. After you log in, the server needs a way to recognise you on the next request without asking for your password again. There are two main ways to do that:</p>
<ol>
<li><strong>Server-side sessions:</strong> the server remembers you.</li>
<li><strong>Tokens (like JWT):</strong> you carry a signed proof with you.</li>
</ol>
<hr />
<h2>2. Option one: server-side sessions</h2>
<p>Here is the flow:</p>
<ol>
<li>You log in with your username and password.</li>
<li>The server creates a <strong>session</strong>: a record like "session abc123 belongs to user 42, logged in at 9:00 AM". It stores this in memory, a database, or a shared store like Redis.</li>
<li>The server sends your browser a <strong>cookie</strong> that holds only the session ID (<code>abc123</code>), not your data.</li>
<li>On every request, the browser sends the cookie. The server looks up <code>abc123</code>, finds user 42, and serves the page.</li>
</ol>
<h3>Why sessions are good</h3>
<ul>
<li><strong>Easy logout and revocation.</strong> To log someone out, delete the session on the server. It stops working immediately. To "log out everywhere", delete all of that user's sessions.</li>
<li><strong>Small cookie.</strong> The cookie is just an ID. User data stays on the server.</li>
<li><strong>Simple to reason about.</strong> It is the model most web frameworks give you by default.</li>
</ul>
<h3>The cost</h3>
<ul>
<li>Every request needs a <strong>lookup</strong> in the session store.</li>
<li>If you run many servers, they all need to reach the <strong>same</strong> session store (or use sticky sessions, which I do not recommend as the main design).</li>
</ul>
<h3>Lock down the cookie</h3>
<p>The OWASP Session Management Cheat Sheet recommends setting these cookie attributes:</p>
<ul>
<li><strong>Secure:</strong> the cookie is sent only over HTTPS.</li>
<li><strong>HttpOnly:</strong> JavaScript in the page cannot read it, which limits damage from XSS.</li>
<li><strong>SameSite:</strong> limits when the cookie is sent on cross-site requests, which helps against CSRF.</li>
</ul>
<p>Also use a long, random session ID, create a <strong>new</strong> session ID after login, and set idle and absolute timeouts.</p>
<hr />
<h2>3. Option two: JWT (JSON Web Token)</h2>
<p>RFC 7519 defines a JSON Web Token as a <strong>compact, URL-safe</strong> way to represent <strong>claims</strong> between two parties. In practice, a JWT is a small piece of text with three parts:</p>
<ul>
<li><strong>Header:</strong> which signing algorithm is used.</li>
<li><strong>Payload:</strong> the claims, for example <code>sub</code> (the user ID), <code>exp</code> (expiry time) and maybe roles.</li>
<li><strong>Signature:</strong> made with a secret key or a private key.</li>
</ul>
<p>When you log in, the auth server gives you a signed JWT. You send it with each request, usually in the <code>Authorization: Bearer &lt;token&gt;</code> header. Any service that has the right key can <strong>check the signature</strong> and read the claims, <strong>without calling a database</strong>.</p>
<h3>Why JWTs are popular</h3>
<ul>
<li><strong>Stateless verification.</strong> Each service can verify the token itself. This is handy with many microservices or APIs in different places.</li>
<li><strong>Works well for APIs and mobile apps</strong>, where cookies are less natural.</li>
<li><strong>Standard format</strong> with libraries in every language.</li>
</ul>
<h3>The big catch: you cannot easily cancel a JWT</h3>
<p>Because services trust the signature and do not check a central store, a JWT stays valid <strong>until it expires</strong>. If a token is stolen, or the user logs out, or you ban the user, that token can keep working until its <code>exp</code> time.</p>
<p>You can build a deny-list to block specific tokens, but then every service must check that list on every request, and you are back to a central lookup, like a session.</p>
<p>The common, practical answer is:</p>
<ul>
<li>Keep <strong>access tokens short-lived</strong> (often minutes).</li>
<li>Use a <strong>refresh token</strong> to get a new access token when it expires. RFC 6749 (OAuth 2.0) describes refresh tokens as credentials used to obtain new access tokens when the current one expires.</li>
<li>Store refresh tokens on the server side so you <strong>can revoke them</strong>. When a user logs out or you detect abuse, revoke the refresh token. The user is fully out within one access-token lifetime.</li>
</ul>
<h3>Two more JWT rules</h3>
<ul>
<li><strong>A JWT payload is not secret.</strong> It is usually only signed, not encrypted. Anyone holding the token can read the claims. Do not put passwords or sensitive data in it.</li>
<li><strong>Always validate properly.</strong> Check the signature, the allowed algorithm, <code>exp</code>, the issuer and the audience. Do not accept unsigned tokens.</li>
</ul>
<hr />
<h2>4. Sessions vs JWT side by side</h2>
<table>
<thead>
<tr>
<th></th>
<th>Server-side session</th>
<th>JWT access token</th>
</tr>
</thead>
<tbody><tr>
<td>Where is the login state?</td>
<td>On the server (session store)</td>
<td>Inside the signed token</td>
</tr>
<tr>
<td>Check on each request</td>
<td>Look up the session ID</td>
<td>Verify the signature</td>
</tr>
<tr>
<td>Logout / revoke</td>
<td>Easy: delete the session</td>
<td>Hard: valid until expiry</td>
</tr>
<tr>
<td>Best fit</td>
<td>One web app, server-rendered</td>
<td>APIs, many services, mobile</td>
</tr>
<tr>
<td>Main risk</td>
<td>Session store becomes a bottleneck</td>
<td>Stolen tokens until expiry</td>
</tr>
</tbody></table>
<p>Neither is "better". They solve different problems. Many real systems use <strong>both</strong>: a session cookie for the browser, and short-lived tokens between services.</p>
<hr />
<h2>5. Many apps, one login: single sign-on (SSO)</h2>
<p>Now back to the interview question. Users log in to one app, but the other apps ask again. Sessions and JWTs alone do not fix this, because each app has its own login.</p>
<p>The fix is <strong>single sign-on</strong>:</p>
<ol>
<li>You run <strong>one login server</strong> (an identity provider, or IdP).</li>
<li>When a user opens App B and is not logged in, App B <strong>redirects</strong> them to the login server.</li>
<li>If the user already has a session at the login server (because they logged in to App A earlier), the login server does not ask for a password again. It sends the user back to App B with proof of who they are.</li>
<li>App B checks that proof and creates its own local session or token.</li>
</ol>
<p>To the user, it feels like "log in once, every app knows me".</p>
<h3>OpenID Connect and OAuth 2.0</h3>
<p>The standard way to do this today is <strong>OpenID Connect (OIDC)</strong>. The OpenID Connect Core spec describes it as a simple <strong>identity layer on top of OAuth 2.0</strong>.</p>
<ul>
<li><strong>OAuth 2.0</strong> is about <strong>authorisation</strong>: giving an app permission to access something (an access token).</li>
<li><strong>OpenID Connect</strong> adds <strong>authentication</strong>: telling the app <em>who the user is</em>, using an <strong>ID token</strong>, which is a JWT.</li>
</ul>
<p>Google's developer documentation says its OAuth 2.0 APIs can be used for both authentication and authorisation, and that its implementation conforms to the OpenID Connect specification. That is what powers "Sign in with Google" on other websites.</p>
<p>Popular identity providers include Google, Microsoft Entra ID, Okta, Auth0, Keycloak and Amazon Cognito. In interviews, you do not need to name a product. You need to explain the flow.</p>
<hr />
<h2>6. A clean design for "10 apps, one login"</h2>
<p>Here is what I would propose for a company with many apps:</p>
<ol>
<li><strong>One identity provider</strong> using OpenID Connect. All apps trust it.</li>
<li><strong>Authorization Code flow with PKCE</strong> for web and mobile apps (the recommended flow for most apps today).</li>
<li>Each app receives an <strong>ID token</strong> (who the user is) and an <strong>access token</strong> (what they can call).</li>
<li><strong>Short-lived access tokens</strong> for APIs, plus <strong>refresh tokens</strong> that can be revoked.</li>
<li><strong>Secure, HttpOnly, SameSite cookies</strong> for browser sessions.</li>
<li><strong>Central logout:</strong> revoke refresh tokens and the IdP session, and use the IdP's logout features to sign out of other apps.</li>
<li><strong>MFA at the identity provider</strong>, so every app gets stronger login for free.</li>
</ol>
<hr />
<h2>7. The example visuals</h2>
<p>The "users log in to one app, but our other apps ask again" line is an <strong>example scenario</strong>, the way interviewers phrase it. The Mail, Video, Docs and Drive boxes in the Reel are example apps. Google is used as a familiar example of single sign-on. The definitions of JWT, refresh tokens, OpenID Connect and cookie attributes are verified from the sources below.</p>
<hr />
<h2>8. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"Users log in to one app, but our other apps ask them to log in again. Fix it."</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"I would add single sign-on with one identity provider using OpenID Connect, which is an identity layer on top of OAuth 2.0. When a user opens another app, it redirects to the identity provider. If the user already has a session there, they are sent back with an ID token, without typing the password again. Inside each app, I would use a server-side session for a simple web app, because logout is easy. Between services I would use short-lived JWT access tokens, because any service can verify the signature without a database call. Since a JWT is hard to cancel before it expires, I keep access tokens short and use refresh tokens that I can revoke. Cookies are Secure, HttpOnly and SameSite."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><em>How do you log a user out everywhere?</em> Delete their sessions, revoke refresh tokens, end the IdP session, and let short access tokens expire.</li>
<li><em>Where do you store tokens in the browser?</em> Prefer HttpOnly cookies over localStorage, to limit XSS damage.</li>
<li><em>OAuth vs OpenID Connect?</em> OAuth is for access (authorisation). OIDC adds identity (authentication) with an ID token.</li>
<li><em>Is a JWT encrypted?</em> Usually not, only signed. Do not put secrets in it.</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li>RFC 7519, "JSON Web Token (JWT)" (compact, URL-safe means of representing claims; <code>exp</code> claim): <a href="https://www.rfc-editor.org/rfc/rfc7519">https://www.rfc-editor.org/rfc/rfc7519</a></li>
<li>RFC 6749, "The OAuth 2.0 Authorization Framework", section 1.5 (refresh tokens obtain new access tokens): <a href="https://www.rfc-editor.org/rfc/rfc6749">https://www.rfc-editor.org/rfc/rfc6749</a></li>
<li>OpenID Connect Core 1.0 (identity layer on top of OAuth 2.0; ID token): <a href="https://openid.net/specs/openid-connect-core-1_0.html">https://openid.net/specs/openid-connect-core-1_0.html</a></li>
<li>Google Identity, "OpenID Connect" (Google's OAuth 2.0 APIs for authentication; conforms to OpenID Connect): <a href="https://developers.google.com/identity/openid-connect/openid-connect">https://developers.google.com/identity/openid-connect/openid-connect</a></li>
<li>RFC 9700, "Best Current Practice for OAuth 2.0 Security" (use PKCE with the authorization code flow): <a href="https://www.rfc-editor.org/rfc/rfc9700">https://www.rfc-editor.org/rfc/rfc9700</a></li>
<li>OWASP Session Management Cheat Sheet (Secure, HttpOnly, SameSite, session ID renewal, timeouts): <a href="https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html">https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html</a></li>
</ul>
<hr />
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[Bapu Class: How Amazon Finds Your Order in One Second (Database Indexes, Made Simple)]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple This post is the full breakdown for the "Bapu Class" Reel on database indexes…]]></description><link>https://cloudtechbydevang.com/bapu-class-how-amazon-finds-your-order-in-one-second-database-indexes-made-simple</link><guid isPermaLink="true">https://cloudtechbydevang.com/bapu-class-how-amazon-finds-your-order-in-one-second-database-indexes-made-simple</guid><category><![CDATA[System Design]]></category><category><![CDATA[Devops]]></category><category><![CDATA[database]]></category><category><![CDATA[SQL]]></category><category><![CDATA[interview]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Wed, 21 Oct 2026 21:38:57 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<blockquote>
<p>This post is the full breakdown for the <strong>"Bapu Class" Reel on database indexes</strong>. If you came here from the Reel after commenting <strong>INDEX</strong>, welcome!</p>
</blockquote>
<p><strong>The story in the Reel:</strong> a young engineer asks his grandfather, <em>"Bapu, Amazon has crores of orders. How does my order show up in one second?"</em> Bapu answers in the rain:</p>
<ul>
<li>Without an index, you have to check every shelf. That is a <strong>full table scan</strong>.</li>
<li>With an index, like the index at the back of a book, you jump straight to the right page.</li>
<li>A database index is a <strong>sorted B-tree</strong>. Even with crores of rows, the database reaches your row in about <strong>4–5 steps</strong>.</li>
<li>But do not index every column. <strong>Every index slows down writes and takes space.</strong> Index only the columns you actually search or filter on.</li>
</ul>
<p>Below is the detailed version, with SQL you can try yourself.</p>
<hr />
<p>You open "Your Orders" on a shopping app and your list shows up almost instantly. Behind that screen there could be a table with billions of order rows. The database does not read all of them. It uses an <strong>index</strong>.</p>
<p>Indexes are one of the most asked topics in backend and system design interviews, and one of the most common causes of real production slowdowns. In 14+ years of working with cloud systems, I have seen more "the site is slow" incidents fixed by one correct index than by any amount of extra hardware.</p>
<p>In this post I will explain, in simple English, what an index is, why a B-tree is so fast, how to design a composite index, why you should not index every column, and how to answer the classic interview question:</p>
<blockquote>
<p><em>"Your orders query on 1 crore rows takes 4 seconds. Fix it."</em></p>
</blockquote>
<p>A quick note before we start: Amazon does not publish the exact database design behind its "Your Orders" page. I use an "orders table" as an <strong>example</strong> because everyone understands it. The index ideas here are general and work the same way in PostgreSQL, MySQL and most relational databases.</p>
<hr />
<h2>1. The problem: a full table scan</h2>
<p>Imagine this table:</p>
<pre><code class="language-sql">CREATE TABLE orders (
  order_id    BIGINT PRIMARY KEY,
  user_id     BIGINT NOT NULL,
  order_date  TIMESTAMP NOT NULL,
  status      TEXT NOT NULL,
  total       NUMERIC(10,2)
);
</code></pre>
<p>And this query, which runs every time a user opens their orders page:</p>
<pre><code class="language-sql">SELECT order_id, order_date, total
FROM orders
WHERE user_id = 42
ORDER BY order_date DESC
LIMIT 20;
</code></pre>
<p>Without a useful index, the database has only one option: read <strong>every row</strong> in the table, keep the ones where <code>user_id = 42</code>, sort them by date and return the top 20. This is called a <strong>sequential scan</strong> (or full table scan). With 1 crore (10 million) rows, that is a lot of disk and memory work for 20 rows of output. With more users opening the page at the same time, the database CPU climbs and every query gets slower.</p>
<hr />
<h2>2. Step one in any slow-query problem: EXPLAIN</h2>
<p>Before you add anything, look at what the database is actually doing. In PostgreSQL:</p>
<pre><code class="language-sql">EXPLAIN ANALYZE
SELECT order_id, order_date, total
FROM orders
WHERE user_id = 42
ORDER BY order_date DESC
LIMIT 20;
</code></pre>
<p><code>EXPLAIN</code> shows the <strong>query plan</strong>: the steps the database chose. <code>EXPLAIN ANALYZE</code> also runs the query and shows real timings. If you see <code>Seq Scan on orders</code>, the database is reading the whole table. That is your first clue.</p>
<p>This is also the first thing to say in an interview. Interviewers like candidates who measure before they change things. "I would run EXPLAIN ANALYZE and check whether it is doing a sequential scan" is a much better start than "I would add Redis".</p>
<hr />
<h2>3. What an index actually is</h2>
<p>Think about the index at the back of a textbook. If you want every page that mentions "TCP", you do not read the whole book. You go to the index, find "TCP", and it tells you the page numbers. You jump straight there.</p>
<p>A database index is the same idea. It is a separate data structure that keeps the values of one or more columns <strong>in sorted order</strong>, along with a pointer to where each full row lives. The most common type is the <strong>B-tree</strong>. PostgreSQL, MySQL (InnoDB), SQL Server and Oracle all use B-tree style indexes as the default.</p>
<h3>How a B-tree is shaped</h3>
<p>A B-tree has three kinds of nodes:</p>
<ul>
<li><strong>Root node:</strong> the top. It holds a few keys that split the whole range into big chunks.</li>
<li><strong>Branch nodes:</strong> the middle levels. Each splits its chunk into smaller chunks.</li>
<li><strong>Leaf nodes:</strong> the bottom. They hold the sorted index entries and point to the actual rows. Leaf nodes are linked to each other, so the database can also walk through them in order.</li>
</ul>
<p>To find <code>user_id = 42</code>, the database starts at the root, picks the right branch, goes down one level at a time, and lands on the leaf that holds 42. Then it follows the pointer to the row.</p>
<h3>Why it is so fast: the tree stays shallow</h3>
<p>Each node holds many entries (often hundreds), so the tree is very wide and very short. Markus Winand, in <em>Use The Index, Luke!</em>, explains that the tree depth grows <strong>logarithmically</strong> with the number of rows, and that real-world indexes with millions of rows usually have a depth of only <strong>four or five levels</strong>. That means the database needs just a handful of page reads to find any value, instead of reading millions of rows.</p>
<p>You may have seen a hook online that says an index lets the database "read about 30 rows". That is not quite right. The useful mental model is: a few levels of tree lookups to find the starting point, then it reads only the matching entries.</p>
<hr />
<h2>4. Designing the right index for the query</h2>
<p>For our query, the database needs to:</p>
<ol>
<li>Find rows where <code>user_id = 42</code>.</li>
<li>Return them sorted by <code>order_date</code>, newest first.</li>
<li>Stop after 20.</li>
</ol>
<p>A single index on <code>user_id</code> helps with step 1, but the database still has to sort all of that user's orders. A better choice is a <strong>composite (multicolumn) index</strong>:</p>
<pre><code class="language-sql">CREATE INDEX idx_orders_user_date
  ON orders (user_id, order_date DESC);
</code></pre>
<p>Now the index entries are sorted first by <code>user_id</code>, and within each user by <code>order_date</code> (newest first). The database jumps to user 42, reads the first 20 entries in order, and stops. No full scan, no big sort.</p>
<h3>Column order matters</h3>
<p>The PostgreSQL documentation explains that a multicolumn B-tree index is most efficient when there are constraints on the <strong>leading (leftmost) columns</strong>. Think of a phone book sorted by last name, then first name. It is great for "Sharma, Rohit" and good for "all Sharmas", but useless for "everyone named Rohit".</p>
<p>A simple rule that works for most queries:</p>
<ul>
<li>Put <strong>equality</strong> columns first (<code>user_id = 42</code>).</li>
<li>Then the <strong>sort</strong> or <strong>range</strong> column (<code>ORDER BY order_date</code>, <code>order_date &gt; '2026-01-01'</code>).</li>
</ul>
<p>If you flip the order to <code>(order_date, user_id)</code>, the index is sorted by date first, and user 42's orders are spread all over it. The database cannot use it well for this query.</p>
<h3>Covering indexes (a bonus)</h3>
<p>If the index also contains every column the query needs, the database may answer the query from the index alone, without visiting the table. PostgreSQL calls this an <strong>index-only scan</strong>, and you can add extra columns with <code>INCLUDE</code>:</p>
<pre><code class="language-sql">CREATE INDEX idx_orders_user_date_cover
  ON orders (user_id, order_date DESC) INCLUDE (total);
</code></pre>
<p>Use this carefully. Wider indexes take more space and more write work.</p>
<hr />
<h2>5. Why not index every column?</h2>
<p>This is the follow-up question the interviewer in the Reel asks, and it is a great one.</p>
<p>Indexes are <strong>not free</strong>:</p>
<ul>
<li><strong>Every write gets slower.</strong> When you <code>INSERT</code> a row, the database must also add an entry to every index on that table. When you <code>UPDATE</code> an indexed column or <code>DELETE</code> a row, every affected index must change too. Ten indexes means roughly ten extra pieces of work on each write.</li>
<li><strong>More storage.</strong> Each index is a full extra structure on disk and in memory. Big indexes push useful data out of the cache.</li>
<li><strong>Not every index helps.</strong> A column with very few distinct values (like <code>status</code> with only "placed", "shipped", "delivered") rarely helps alone, because the database would still read a huge part of the table.</li>
<li><strong>More choices for the planner.</strong> Many overlapping indexes make plans harder to predict and maintain.</li>
</ul>
<p>The PostgreSQL docs say it directly: indexes with more than three columns are unlikely to be helpful unless the table usage is very stylised, and indexes should be used sparingly.</p>
<p>So the rule is: <strong>index for your real query patterns</strong>, not "just in case".</p>
<hr />
<h2>6. How I approach a slow query in real life</h2>
<p>Here is the checklist I use with teams:</p>
<ol>
<li><strong>Find the real slow queries.</strong> Use your database's slow query log or tools like <code>pg_stat_statements</code> in PostgreSQL. Do not guess.</li>
<li><strong>Run EXPLAIN ANALYZE.</strong> Look for <code>Seq Scan</code> on big tables, big sorts, and row estimates that are far from actual rows.</li>
<li><strong>Match the index to the WHERE and ORDER BY.</strong> Equality columns first, then range or sort columns.</li>
<li><strong>Create the index safely.</strong> On a busy PostgreSQL table, use <code>CREATE INDEX CONCURRENTLY</code> so you do not block writes for a long time.</li>
<li><strong>Run EXPLAIN ANALYZE again.</strong> Confirm it now uses an <code>Index Scan</code> or <code>Index Only Scan</code>, and compare timings.</li>
<li><strong>Watch write latency and storage.</strong> Make sure the new index did not hurt inserts.</li>
<li><strong>Remove unused indexes.</strong> Many databases track index usage. Dropping dead indexes speeds up writes.</li>
</ol>
<hr />
<h2>7. Where indexes fit in a bigger design</h2>
<p>Indexes are the first tool, not the only one. As data and traffic grow, you may also use:</p>
<ul>
<li><strong>Read replicas</strong> to spread read load.</li>
<li><strong>Caching</strong> for very hot data (a cache in front of the database).</li>
<li><strong>Partitioning</strong> big tables by date, so old data does not slow new queries.</li>
<li><strong>Sharding</strong> across many database servers when one server cannot hold the data or the write load.</li>
<li><strong>Purpose-built stores</strong> like key-value databases for simple lookups by key at massive scale.</li>
</ul>
<p>But in interviews and in real life, the order matters. Measure first, fix the query and indexes next, and only then add more infrastructure.</p>
<hr />
<h2>8. The example visuals</h2>
<p>The "1 crore rows" and "4 seconds" numbers in the Reel are an <strong>example scenario</strong>, the way interviewers phrase it. The "Amazon orders" framing is also an example, because Amazon does not publish its exact order database design. The B-tree depth of four to five levels, the leftmost-column rule and the write cost of indexes are verified from the sources below.</p>
<hr />
<h2>9. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"Your orders query on 1 crore rows takes 4 seconds. Fix it."</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"First I would run EXPLAIN ANALYZE to see the plan. If it is doing a sequential scan, the database is reading every row. The query filters by user_id and sorts by order_date, so I would add a composite B-tree index on (user_id, order_date DESC). Equality column first, then the sort column. A B-tree stays only a few levels deep even for millions of rows, so the database can jump to that user's newest orders and stop after 20. I would create it concurrently to avoid blocking writes, then run EXPLAIN again to confirm an index scan. I would not index every column, because every insert and update has to update every index, which slows writes and uses storage."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><em>Why not index every column?</em> Write cost, storage, and low-selectivity columns do not help.</li>
<li><em>What is a covering index?</em> An index that includes every column the query needs, so the table is not read.</li>
<li><em>Why does column order matter in a composite index?</em> The index is sorted by the first column first. It is most efficient for conditions on the leftmost columns.</li>
<li><em>The index exists but the query is still slow. Why?</em> Wrong column order, a function on the column (<code>WHERE LOWER(email) = ...</code>), outdated statistics, or the query returns a large share of the table anyway.</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li>Markus Winand, <em>Use The Index, Luke!</em>, "The Search Tree (B-Tree) Makes the Index Fast" (logarithmic growth, real-world depth of four or five for millions of rows): <a href="https://use-the-index-luke.com/sql/anatomy/the-tree">https://use-the-index-luke.com/sql/anatomy/the-tree</a></li>
<li>PostgreSQL documentation, 11.3 "Multicolumn Indexes" (leading columns, more than three columns rarely helpful, use sparingly): <a href="https://www.postgresql.org/docs/current/indexes-multicolumn.html">https://www.postgresql.org/docs/current/indexes-multicolumn.html</a></li>
<li>PostgreSQL documentation, "Using EXPLAIN": <a href="https://www.postgresql.org/docs/current/using-explain.html">https://www.postgresql.org/docs/current/using-explain.html</a></li>
<li>PostgreSQL documentation, "Index-Only Scans and Covering Indexes": <a href="https://www.postgresql.org/docs/current/indexes-index-only-scans.html">https://www.postgresql.org/docs/current/indexes-index-only-scans.html</a></li>
<li>PostgreSQL documentation, "CREATE INDEX" (CONCURRENTLY): <a href="https://www.postgresql.org/docs/current/sql-createindex.html">https://www.postgresql.org/docs/current/sql-createindex.html</a></li>
</ul>
<hr />
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[How Google Serves the Whole World From One IP Address]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple Type 8.8.8.8 into your network settings, and you are using Google Public DNS…]]></description><link>https://cloudtechbydevang.com/how-google-serves-the-whole-world-from-one-ip-address</link><guid isPermaLink="true">https://cloudtechbydevang.com/how-google-serves-the-whole-world-from-one-ip-address</guid><category><![CDATA[anycast]]></category><category><![CDATA[bgp]]></category><category><![CDATA[System Design]]></category><category><![CDATA[dns]]></category><category><![CDATA[networking]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Tue, 20 Oct 2026 14:05:17 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>Type <code>8.8.8.8</code> into your network settings, and you are using Google Public DNS. Someone in Mumbai uses the same address. So does someone in São Paulo, in Frankfurt and in Sydney. One IP address, used by the whole world.</p>
<p>Yet these people are not all talking to one server in one building. Each of them is answered by a Google location near them. How can <strong>one IP address</strong> live in many places at once?</p>
<p>The answer is <strong>anycast</strong>. In this post I will explain anycast in simple English, how internet routing (BGP) makes it work, what happens when a location fails, and how it compares with <strong>DNS-based global load balancing</strong> like AWS Route 53 latency and geolocation routing. At the end you get an interview answer for: <em>"How would you route users to the nearest region?"</em> I have spent 14+ years in cloud, DevOps and SRE, and multi-region routing is where many teams make expensive mistakes, so this one is worth knowing well.</p>
<hr />
<h2>1. Unicast: the normal way</h2>
<p>Most of the internet uses <strong>unicast</strong>: one IP address belongs to <strong>one</strong> machine (or one network endpoint). If you send a packet to that address, it goes to that one place, wherever it is in the world.</p>
<p>That is simple, but it has problems for global services:</p>
<ul>
<li>Users far away get <strong>high latency</strong>, because every packet travels the long way.</li>
<li>One location takes <strong>all the traffic</strong>, including attack traffic.</li>
<li>If that location goes down, the address goes down with it.</li>
</ul>
<hr />
<h2>2. Anycast: one address, many locations</h2>
<p>With <strong>anycast</strong>, the <strong>same IP address is announced from many locations</strong> at the same time. Each location says to the internet: "I can deliver traffic for <code>8.8.8.8</code>."</p>
<p>Google's Public DNS FAQ explains it directly: anycast routing sends your queries to the closest Google Public DNS server. The locations announcing these anycast addresses (<code>8.8.8.8</code>, <code>8.8.4.4</code> and the IPv6 addresses) include nearly all of the Core data centres and Edge Points of Presence in the Google Edge Network. Google also notes that the exact set of locations changes with network conditions and traffic load.</p>
<p>So when you send a DNS query to <code>8.8.8.8</code>, it doesn't travel to one global server. It goes to a nearby Google site.</p>
<p>Cloudflare's learning centre describes anycast the same way: requests to a single IP address can be routed to many different locations, typically the nearest data centre that can handle them.</p>
<hr />
<h2>3. How the internet picks "nearest": BGP</h2>
<p>The internet is made of thousands of networks (ISPs, cloud providers, universities, companies). Each one is called an <strong>autonomous system</strong>. They tell each other which IP addresses they can reach using a routing protocol called <strong>BGP</strong> (Border Gateway Protocol).</p>
<p>When many Google locations announce the same address range, your ISP's routers hear several routes to it. Each router picks its <strong>best route</strong>, usually the shortest or most preferred path through the networks. Your packets follow that path to whichever Google location is at the end of it.</p>
<p>Two important details:</p>
<ul>
<li><strong>"Nearest" means network-nearest, not map-nearest.</strong> BGP chooses by routing policy and path, not by kilometres. Usually the result is geographically close, but not always. Someone in Pune might be served from Mumbai on one ISP and from a different site on another ISP.</li>
<li><strong>Each packet makes its own choice.</strong> For short exchanges like a DNS query (one small question, one small answer), this works perfectly. For long connections, operators must make sure routes are stable so a connection doesn't "flap" between sites in the middle. Cloudflare notes that building a true anycast network needs your own network hardware, direct relationships with carriers, and careful route tuning to avoid exactly this.</li>
</ul>
<hr />
<h2>4. What happens when a location fails?</h2>
<p>This is one of anycast's best features.</p>
<p>If one anycast location goes down, or is taken out for maintenance, it <strong>stops announcing</strong> the route. Routers around the internet notice that path is gone and pick the <strong>next best route</strong>, to another location still announcing the same IP. Traffic shifts automatically. Users keep using the same IP address. They never change any setting.</p>
<p>Cloudflare puts it simply: because traffic finds the best path, an entire data centre can be taken offline and traffic will flow to a nearby data centre.</p>
<p>Anycast also helps against <strong>DDoS attacks</strong>. Attack traffic from bots all over the world is spread across many locations, instead of hitting one site, so each location only has to absorb its share.</p>
<p><strong>Lesson for us:</strong> anycast gives you failover and attack spreading <strong>at the network layer</strong>, with no client-side changes and no DNS TTLs to wait for.</p>
<hr />
<h2>5. Who uses anycast?</h2>
<ul>
<li><strong>Public DNS resolvers:</strong> Google (<code>8.8.8.8</code>), Cloudflare (<code>1.1.1.1</code>), Quad9 (<code>9.9.9.9</code>).</li>
<li><strong>DNS root servers:</strong> many root server letters are run as anycast clouds with many instances worldwide.</li>
<li><strong>CDNs and edge networks:</strong> many CDNs use anycast to bring users to the nearest edge.</li>
<li><strong>Cloud global front doors:</strong> for example, <strong>AWS Global Accelerator</strong> gives you static IP addresses. Users enter the AWS global network at a nearby Global Accelerator edge location, and you get fast failover across Regions without depending on DNS caches. Google Cloud's global external load balancers also give one IP address for users worldwide.</li>
</ul>
<p>You usually don't build anycast yourself. You <strong>rent</strong> it from a cloud provider or CDN.</p>
<hr />
<h2>6. The other way: DNS-based global load balancing</h2>
<p>Not every system uses anycast. The other common way to send users to a nearby region is <strong>DNS-based routing</strong>, sometimes called <strong>GSLB</strong> (global server load balancing) or <strong>GeoDNS</strong>.</p>
<p>The idea: your service has <strong>different IP addresses in different regions</strong> (for example a load balancer in Mumbai and one in Virginia). When a user's resolver asks for <code>app.example.com</code>, the DNS service <strong>chooses which answer to give</strong> based on where the user is, which region is fastest, or which region is healthy.</p>
<h3>Route 53 routing policies</h3>
<p>AWS Route 53 offers several <strong>routing policies</strong>. The ones that matter most for global routing are:</p>
<ul>
<li><strong>Latency routing:</strong> for resources in multiple AWS Regions. It routes users to the Region that gives the <strong>best latency</strong>.</li>
<li><strong>Geolocation routing:</strong> routes by <strong>where the user is</strong> (continent, country, or US state). This is useful for data residency or language-specific content.</li>
<li><strong>Geoproximity routing:</strong> routes by the location of your resources, and can <strong>shift traffic</strong> between locations using a bias.</li>
<li><strong>Failover routing:</strong> <strong>active-passive</strong> setup. Traffic goes to the primary, and to the secondary if health checks say the primary is down.</li>
<li><strong>Weighted routing:</strong> split traffic by percentages (for example a 10% canary in a new region).</li>
<li>Also <strong>simple</strong>, <strong>multivalue answer</strong> (up to eight healthy records at random), and <strong>IP-based</strong> routing.</li>
</ul>
<h3>Anycast vs DNS routing</h3>
<table>
<thead>
<tr>
<th></th>
<th>Anycast</th>
<th>DNS-based routing (GeoDNS / Route 53)</th>
</tr>
</thead>
<tbody><tr>
<td>What changes per user</td>
<td>The <strong>network path</strong> to the same IP</td>
<td>The <strong>IP address</strong> returned by DNS</td>
</tr>
<tr>
<td>Who decides</td>
<td>BGP routers on the internet</td>
<td>Your DNS service</td>
</tr>
<tr>
<td>Failover speed</td>
<td>Fast (routing reconverges)</td>
<td>Depends on DNS <strong>TTL</strong> and client caching</td>
</tr>
<tr>
<td>Control</td>
<td>Less fine-grained (network decides)</td>
<td>Very fine-grained (latency, geo, weights, health)</td>
</tr>
<tr>
<td>Effort</td>
<td>Needs a big network, so usually rented</td>
<td>Easy to set up yourself</td>
</tr>
<tr>
<td>Typical use</td>
<td>DNS resolvers, CDNs, global LB front doors</td>
<td>Choosing a region for your own app</td>
</tr>
</tbody></table>
<p>In practice, many large systems use <strong>both</strong>: an anycast front door (a CDN or global load balancer) for the first hop, and DNS or load balancer rules to pick the right backend region.</p>
<p><strong>Watch out for DNS caching.</strong> When Route 53 changes an answer during a failover, clients and resolvers that cached the old answer keep using it until the TTL expires. Use <strong>short TTLs</strong> for records you might fail over, and <strong>health checks</strong> so Route 53 stops returning unhealthy endpoints.</p>
<hr />
<h2>7. The example visuals</h2>
<p>The city names in the Reel (Mumbai, Frankfurt, Iowa, Sydney, Singapore) and "You in Pune" are <strong>examples</strong> to explain the idea. They are not a list of the exact sites that answer <code>8.8.8.8</code>. Google says the set of locations changes with network conditions and load, and publishes location data in its Public DNS FAQ. "ap-south-1" is shown as an example AWS region (Mumbai).</p>
<hr />
<h2>8. Try it yourself</h2>
<p>You can see which Google Public DNS location answers you:</p>
<pre><code class="language-bash">dig +short TXT o-o.myaddr.l.google.com @8.8.8.8
</code></pre>
<p>This returns an IP address that identifies the Google site handling your query. (Google's FAQ explains how to map these to locations.) Try it from your laptop, then from a cloud VM in another region. Same <code>8.8.8.8</code>, different site.</p>
<hr />
<h2>9. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"How would you route users to the nearest region?"</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"There are two main tools. The first is anycast: the same IP is announced from many locations, and BGP sends each user to the network-nearest one. If a location fails it stops announcing, and traffic moves to the next closest automatically. Google Public DNS on 8.8.8.8 works this way, and on AWS you get a similar single-IP front door with Global Accelerator, or with a CDN. The second is DNS-based routing: Route 53 latency routing sends users to the region with the lowest latency, geolocation routes by user location, and failover routing with health checks moves traffic to a backup region. DNS gives more control, but failover depends on TTLs. So: anycast for one global IP and fast network-level failover, DNS routing to pick a region."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><em>Why is anycast great for DNS?</em> DNS queries are short, usually one small UDP request and response, so a route change between packets doesn't break anything.</li>
<li><em>What's the risk with anycast for long TCP connections?</em> Route changes can move a connection to a different site mid-stream. Providers manage this with stable routing and connection handling at the edge.</li>
<li><em>How do you fail over a whole region?</em> Health checks plus Route 53 failover (or Global Accelerator endpoint weights), short TTLs, and enough spare capacity in the other region.</li>
<li><em>Latency vs geolocation routing?</em> Latency picks the fastest region. Geolocation picks by user location, which is useful for compliance or local content.</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li>Google Public DNS FAQ (anycast routing to the closest server; locations include nearly all Core data centres and Edge PoPs and change with network conditions and load; o-o.myaddr lookup): <a href="https://developers.google.com/speed/public-dns/faq">https://developers.google.com/speed/public-dns/faq</a></li>
<li>Cloudflare Learning Center, "What is Anycast? How does Anycast work?" (nearest data centre; a data centre can go offline and traffic flows to a nearby one; DDoS spreading; route tuning to avoid flapping): <a href="https://www.cloudflare.com/learning/cdn/glossary/anycast-network/">https://www.cloudflare.com/learning/cdn/glossary/anycast-network/</a></li>
<li>AWS, "Choosing a routing policy" (Route 53: simple, failover, geolocation, geoproximity, latency, IP-based, multivalue answer, weighted): <a href="https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html">https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html</a></li>
<li>AWS Global Accelerator (static IPs, onboarding at edge locations, fast multi-Region failover independent of DNS): <a href="https://aws.amazon.com/global-accelerator/">https://aws.amazon.com/global-accelerator/</a></li>
</ul>
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[How Load Balancers Split Traffic Between Thousands of Servers]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple When you open a big website, your request almost never goes straight to "the…]]></description><link>https://cloudtechbydevang.com/how-load-balancers-split-traffic-between-thousands-of-servers</link><guid isPermaLink="true">https://cloudtechbydevang.com/how-load-balancers-split-traffic-between-thousands-of-servers</guid><category><![CDATA[networking]]></category><category><![CDATA[System Design]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Load Balancing]]></category><category><![CDATA[AWS]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Mon, 19 Oct 2026 13:55:42 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>When you open a big website, your request almost never goes straight to "the server". It goes to a <strong>load balancer</strong> first. The load balancer decides which of many servers should handle your request. It does this for every request, millions of times a second, and you never notice it. That's the point.</p>
<p>Load balancers are one of the most important building blocks in cloud architecture. They give you scale (many servers), availability (broken servers are skipped) and flexibility (you can add, remove or update servers without users noticing).</p>
<p>In this post I will explain, in simple English, what a load balancer does, the main ways it picks a server, how health checks work, and the difference between <strong>Layer 4</strong> and <strong>Layer 7</strong> load balancers, including AWS's NLB and ALB. At the end you will get a clean answer for the classic interview question: <em>"L4 vs L7 load balancer: when do you use which?"</em> I have spent 14+ years in cloud, DevOps and SRE, and I have seen many outages that came down to one misconfigured health check, so we will spend time on that too.</p>
<hr />
<h2>1. What a load balancer does</h2>
<p>A <strong>load balancer</strong> sits between clients (browsers, mobile apps, other services) and a group of servers. In AWS language the servers are <strong>targets</strong>, grouped into a <strong>target group</strong>. In NGINX language they are an <strong>upstream</strong>.</p>
<p>Its main jobs:</p>
<ol>
<li><strong>Single front door.</strong> Clients talk to one name or IP address. They don't need to know how many servers there are.</li>
<li><strong>Spread the load.</strong> It sends each request (or connection) to one of the healthy servers.</li>
<li><strong>Skip broken servers.</strong> It checks server health and stops sending traffic to servers that fail.</li>
<li><strong>Enable scaling.</strong> You can add servers when traffic grows and remove them when it drops. AWS Auto Scaling groups can register new instances with a target group automatically.</li>
<li><strong>Often more:</strong> TLS termination, routing rules, sticky sessions, and protection from some attacks.</li>
</ol>
<p>The AWS documentation sums it up well: the load balancer is a single point of contact for clients, and it distributes incoming traffic across its <strong>healthy</strong> registered targets.</p>
<hr />
<h2>2. How does it pick a server?</h2>
<p>This is called the <strong>load balancing algorithm</strong>. The most common ones are below.</p>
<h3>Round robin</h3>
<p>Send requests to servers <strong>in turn</strong>: 1, 2, 3, 4, then back to 1. It is simple and fair when all servers are similar and requests cost about the same. Round robin is the <strong>default</strong> for AWS Application Load Balancer target groups and for NGINX.</p>
<h3>Least connections</h3>
<p>Send the next request to the server with the <strong>fewest active connections</strong>, the least busy one. It works better when some requests take much longer than others (for example file uploads or reports). NGINX supports this as <code>least_conn</code>.</p>
<p>On <strong>AWS ALB</strong>, the similar option is called <strong>least outstanding requests</strong>. It sends the request to the target with the fewest requests still in progress.</p>
<h3>Other methods you may meet</h3>
<ul>
<li><strong>Weighted:</strong> give stronger servers a bigger share (for example 70/30 during a canary release). ALB also offers <strong>weighted random</strong>.</li>
<li><strong>IP hash / consistent hashing:</strong> the same client (or the same key) always goes to the same server. This is useful for caches.</li>
<li><strong>Least time:</strong> pick the server with the fastest recent responses (NGINX Plus).</li>
</ul>
<p><strong>Lesson for us:</strong> start with round robin. Switch to least connections or least outstanding requests when request times vary a lot. Use hashing only when you really need the same client to keep hitting the same server.</p>
<hr />
<h2>3. Health checks: never send users to a broken server</h2>
<p>A load balancer is only as good as its <strong>health checks</strong>.</p>
<p>A health check is a small, regular test. For example: "Every 10 seconds, call <code>GET /health</code> on each server. If it fails a few times in a row, mark it unhealthy. If it passes a few times in a row, mark it healthy again."</p>
<p>The load balancer <strong>only routes to healthy targets</strong>. When a server crashes, hangs or is being replaced, traffic moves to the others automatically.</p>
<p>Some useful related features on AWS:</p>
<ul>
<li><strong>Deregistration delay (connection draining):</strong> when you remove a target, the load balancer stops sending it <em>new</em> requests but lets in-flight requests finish. The ALB default is <strong>300 seconds</strong>.</li>
<li><strong>Slow start:</strong> a new target gets a slowly increasing share of traffic instead of a full share at once, so a cold app or cache isn't flooded.</li>
<li><strong>Target group health thresholds:</strong> you can say "if fewer than N targets are healthy, fail over" instead of relying on just one healthy server.</li>
</ul>
<p><strong>Common mistakes I see in real systems:</strong></p>
<ul>
<li><strong>Health check too shallow.</strong> It returns 200 even when the app cannot reach its database, so users get errors from "healthy" servers.</li>
<li><strong>Health check too deep.</strong> It fails when an optional dependency is slow, so <em>all</em> servers go unhealthy at once and the whole site goes down.</li>
<li><strong>Thresholds too slow or too twitchy.</strong> Too slow and users hit dead servers. Too twitchy and servers flap in and out.</li>
</ul>
<p>A good health check answers one question: "Can this server do useful work right now?"</p>
<hr />
<h2>4. Layer 4 vs Layer 7</h2>
<p>"Layer" refers to the <strong>OSI model</strong> of networking. You only need two layers here.</p>
<h3>Layer 4 (transport layer)</h3>
<p>A Layer 4 load balancer works with <strong>TCP and UDP</strong>. It sees IP addresses and ports, but it does <strong>not</strong> read the content of your HTTP request. It forwards connections.</p>
<ul>
<li><strong>Very fast</strong> and handles huge volumes.</li>
<li>Works for <strong>any TCP/UDP protocol</strong>: databases, MQTT, gaming, custom protocols, not just HTTP.</li>
<li>Cannot route by URL path or headers, because it doesn't look at them.</li>
</ul>
<p>On AWS this is the <strong>Network Load Balancer (NLB)</strong>. AWS describes it as working at the fourth layer of the OSI model and able to handle <strong>millions of requests per second</strong>.</p>
<h3>Layer 7 (application layer)</h3>
<p>A Layer 7 load balancer <strong>understands HTTP/HTTPS</strong> (and often gRPC). It reads the request: path, host name, headers, query strings, methods.</p>
<ul>
<li><strong>Smart routing:</strong> <code>/api/*</code> to the API service, <code>/images/*</code> to the image service, <code>admin.example.com</code> to the admin app.</li>
<li>Can do <strong>TLS termination</strong>, redirects, fixed responses, authentication integration, and sticky sessions with cookies.</li>
<li>A bit more processing per request than Layer 4.</li>
</ul>
<p>On AWS this is the <strong>Application Load Balancer (ALB)</strong>. AWS describes it as working at the application layer, the seventh layer of the OSI model. It evaluates <strong>listener rules</strong> in priority order to decide which target group gets each request.</p>
<h3>Side by side</h3>
<table>
<thead>
<tr>
<th></th>
<th>Layer 4 (e.g. AWS NLB)</th>
<th>Layer 7 (e.g. AWS ALB)</th>
</tr>
</thead>
<tbody><tr>
<td>Sees</td>
<td>IP, port, TCP/UDP</td>
<td>Full HTTP request: path, host, headers</td>
</tr>
<tr>
<td>Routing</td>
<td>By connection</td>
<td>By content (path, host, header, method)</td>
</tr>
<tr>
<td>Protocols</td>
<td>Any TCP/UDP</td>
<td>HTTP, HTTPS, gRPC</td>
</tr>
<tr>
<td>Strength</td>
<td>Raw speed, any protocol</td>
<td>Smart routing, HTTP features</td>
</tr>
<tr>
<td>Typical use</td>
<td>Non-HTTP services, very high throughput</td>
<td>Web apps, APIs, microservices</td>
</tr>
</tbody></table>
<p>In many real systems you use <strong>both</strong>: for example an NLB at the edge for raw performance and a fixed entry point, and an ALB or an ingress controller (like NGINX) behind it for HTTP routing.</p>
<hr />
<h2>5. Sticky sessions: useful, but be careful</h2>
<p><strong>Sticky sessions</strong> (session affinity) send the same user to the same server, usually using a cookie. ALB supports this (it is off by default).</p>
<p>It can help with old apps that keep sessions in server memory. But it fights against good load spreading, and if that server dies the user's session is lost. The better long-term fix is to make app servers <strong>stateless</strong> and store sessions in Redis or a database. (I cover this in topic 004 of this series, on how Instagram scaled.)</p>
<hr />
<h2>6. Where load balancers fit in a bigger design</h2>
<p>A typical web architecture on AWS:</p>
<pre><code>Users
  │
  ▼
DNS (Route 53)
  │
  ▼
CloudFront (CDN, optional)
  │
  ▼
Application Load Balancer  ── health checks ──►  unhealthy targets skipped
  │        │         │
  ▼        ▼         ▼
 App      App       App      (Auto Scaling group, stateless)
  │
  ▼
Database / cache
</code></pre>
<p>And inside Kubernetes, the same ideas show up as <strong>Services</strong> (Layer 4 spreading across pods) and <strong>Ingress</strong> or <strong>Gateway API</strong> (Layer 7 routing).</p>
<hr />
<h2>7. The example visuals</h2>
<p>The "srv-1 to srv-4" boxes, the broken server and the "1, 2, 3, 4, repeat" labels in the Reel are <strong>examples</strong> to show the idea. Real fleets can have a few servers or thousands. The algorithms and NLB/ALB layers are accurate. Note that the Reel uses the general term "least connections". On AWS ALB the closest option is called "least outstanding requests".</p>
<hr />
<h2>8. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"L4 vs L7 load balancer: when do you use which?"</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"A Layer 4 load balancer works at the TCP/UDP level. It routes connections by IP and port without reading the request, so it's very fast and works for any protocol. On AWS that's the Network Load Balancer. A Layer 7 load balancer understands HTTP, so it can route by path, host or header, terminate TLS and do things like redirects and sticky sessions. On AWS that's the Application Load Balancer. I use Layer 4 for raw speed or non-HTTP traffic, and Layer 7 for web apps and microservices that need smart routing. Either way, health checks are critical: the load balancer must only send traffic to targets that can really do work."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><em>Round robin vs least connections?</em> Round robin for similar, short requests. Least connections (or least outstanding requests on ALB) when request times vary.</li>
<li><em>How do you deploy without dropping requests?</em> Use connection draining (deregistration delay), roll servers one batch at a time, and use slow start for new ones.</li>
<li><em>What happens if all targets are unhealthy?</em> Depending on settings, the load balancer may fail open (send to all) or fail over via DNS. Know your configuration before an incident.</li>
<li><em>Is the load balancer a single point of failure?</em> Managed cloud load balancers run across multiple Availability Zones. For self-managed ones you run at least two, with failover.</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li>AWS, "What is a Network Load Balancer?" (Layer 4, millions of requests per second): <a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html">https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html</a></li>
<li>AWS, "What is an Application Load Balancer?" (Layer 7, listener rules): <a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html">https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html</a></li>
<li>AWS, "Target groups for your Application Load Balancers" (healthy targets only, round_robin default, least_outstanding_requests, weighted_random, deregistration delay default 300s, slow start, stickiness, target group health): <a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html">https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html</a></li>
<li>NGINX, "HTTP Load Balancing" (Round Robin, Least Connections, Least Time, IP Hash, Generic Hash): <a href="https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/">https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/</a></li>
</ul>
<blockquote>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">topmate.io/cloudtechbydevang</a></p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[Vertical vs Horizontal Scaling: How Instagram Scaled With a Tiny Team]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple In April 2012, Facebook agreed to buy Instagram for about $1 billion. At that…]]></description><link>https://cloudtechbydevang.com/vertical-vs-horizontal-scaling-how-instagram-scaled-with-a-tiny-team</link><guid isPermaLink="true">https://cloudtechbydevang.com/vertical-vs-horizontal-scaling-how-instagram-scaled-with-a-tiny-team</guid><category><![CDATA[System Design]]></category><category><![CDATA[PostgreSQL]]></category><category><![CDATA[sharding]]></category><category><![CDATA[Django]]></category><category><![CDATA[scalability]]></category><category><![CDATA[AWS]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Sun, 18 Oct 2026 14:03:15 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>In April 2012, Facebook agreed to buy Instagram for about $1 billion. At that time Instagram had around <strong>30 million users</strong> and only <strong>13 employees</strong>. A few months earlier, in December 2011, Instagram's engineering blog said the app was serving <strong>14 million+ users</strong> with just <strong>3 engineers</strong>.</p>
<p>How does a team that small keep an app that big running? A large part of the answer is one of the most basic ideas in system design: <strong>scaling out instead of only scaling up</strong>, and doing it in the right places.</p>
<p>In this post I will explain vertical and horizontal scaling in simple English, show how Instagram used them, and give you a clean answer to the interview question: <em>"How do you scale a stateful service versus a stateless one?"</em> I have spent 14+ years in cloud, DevOps and SRE, and this question still comes up in almost every system design round I see.</p>
<hr />
<h2>1. What does "scaling" mean?</h2>
<p><strong>Scaling</strong> means handling more work (more users, more requests, more data) without the system getting slow or falling over.</p>
<p>There are two basic ways to do it:</p>
<ul>
<li><p><strong>Vertical scaling (scale up):</strong> make one machine <strong>bigger</strong>.</p>
</li>
<li><p><strong>Horizontal scaling (scale out):</strong> add <strong>more machines</strong>.</p>
</li>
</ul>
<p>Let's look at each.</p>
<hr />
<h2>2. Vertical scaling: a bigger machine</h2>
<p>Vertical scaling means giving your server more CPU, more memory, faster disks or a faster network. On AWS, that is changing an instance from a small type to a larger one.</p>
<p><strong>Why people like it:</strong></p>
<ul>
<li><p><strong>Simple.</strong> No code changes. The app does not need to know anything changed.</p>
</li>
<li><p><strong>Good for stateful systems.</strong> A single database on a bigger machine is still a single database: one place for data, easy transactions.</p>
</li>
<li><p><strong>Quick fix.</strong> When a server is overloaded today, moving to a bigger size can buy you time.</p>
</li>
</ul>
<p><strong>The problems:</strong></p>
<ul>
<li><p><strong>There is a ceiling.</strong> The biggest machine you can buy is still one machine. At some point there is no bigger size.</p>
</li>
<li><p><strong>Cost grows fast.</strong> The largest machines are expensive, and you pay for that size even at quiet hours.</p>
</li>
<li><p><strong>Single point of failure.</strong> If that one big machine fails, everything on it fails.</p>
</li>
<li><p><strong>Downtime to resize.</strong> Changing instance size often means a restart.</p>
</li>
</ul>
<p>Vertical scaling is not bad. It is often the <strong>right first step</strong>, especially for databases. It just cannot be your only plan.</p>
<hr />
<h2>3. Horizontal scaling: more machines</h2>
<p>Horizontal scaling means running <strong>many copies</strong> of your service on many machines, and putting a <strong>load balancer</strong> in front to spread requests across them.</p>
<p><strong>Why it is powerful:</strong></p>
<ul>
<li><p><strong>No hard ceiling.</strong> Need more capacity? Add more machines.</p>
</li>
<li><p><strong>Better availability.</strong> If one machine dies, the others keep serving. The load balancer stops sending traffic to the dead one.</p>
</li>
<li><p><strong>Pay for what you use.</strong> With <strong>auto scaling</strong> (for example AWS Auto Scaling groups or Kubernetes Horizontal Pod Autoscaler), you add machines at peak and remove them at night.</p>
</li>
</ul>
<p><strong>The catch:</strong> horizontal scaling only works smoothly if any copy can handle any request. That leads to the most important word in this post: <strong>stateless</strong>.</p>
<hr />
<h2>4. Stateless vs stateful: the key idea</h2>
<p>A <strong>stateless</strong> server keeps <strong>no important data in its own memory or disk</strong> between requests. Everything it needs comes from the request itself, or from shared systems like a database or a cache.</p>
<p>A <strong>stateful</strong> server keeps important data locally. Examples: a database server, or an app server that keeps your login session in its own memory.</p>
<p>Why does this matter?</p>
<ul>
<li><p>If app servers are <strong>stateless</strong>, request 1 can go to server A and request 2 to server C, and nothing breaks. You can add or remove servers at any time.</p>
</li>
<li><p>If app servers are <strong>stateful</strong> (for example sessions stored in local memory), then a user must keep going to the same server ("sticky sessions"). If that server dies, the user loses their session. Scaling becomes painful.</p>
</li>
</ul>
<p>The usual fix: <strong>move the state out</strong>. Store sessions in Redis or a database, files in object storage like S3, and keep the app servers stateless.</p>
<p>Then you have two different scaling problems:</p>
<ol>
<li><p><strong>Stateless tier (app servers):</strong> easy. Scale out horizontally behind a load balancer.</p>
</li>
<li><p><strong>Stateful tier (databases):</strong> hard. Scale up first, add read replicas, and when that is not enough, <strong>shard</strong>.</p>
</li>
</ol>
<hr />
<h2>5. How Instagram did it</h2>
<p>Instagram's December 2011 engineering post ("What Powers Instagram") described their stack. Their core rules were to keep things very simple, not reinvent the wheel, and use proven technologies. Here is the shape of it, simplified:</p>
<h3>The front door: load balancing</h3>
<p>Every request first hit <strong>load balancers</strong>. Instagram used to run two NGINX machines with <strong>DNS round robin</strong> between them. The problem was that DNS changes take time to spread when a machine had to be removed. So they moved to <strong>Amazon's Elastic Load Balancer (ELB)</strong>, with three NGINX instances behind it. NGINX machines that failed a <strong>health check</strong> were taken out of rotation automatically. They also <strong>terminated SSL at the ELB</strong> to save CPU on NGINX, and used <strong>Route 53</strong> for DNS.</p>
<h3>The app tier: stateless Django servers</h3>
<p>Behind that ran <strong>Django</strong> application servers on AWS EC2 (High-CPU Extra-Large instances at the time). As usage grew, they went from a few app servers to <strong>more than 25</strong>. The post says this tier was easy to scale horizontally <strong>because the servers were stateless</strong>. That is the exact pattern from section 4.</p>
<h3>The data tier: PostgreSQL, then sharding</h3>
<p>Most of Instagram's data (users, photo metadata, tags and so on) lived in <strong>PostgreSQL</strong>. One database server could not hold it all, so they <strong>sharded</strong> it.</p>
<p>Their "Sharding &amp; IDs at Instagram" post explains how:</p>
<ul>
<li><p>They created <strong>several thousand "logical" shards</strong>, mapped in code to <strong>far fewer physical</strong> database servers.</p>
</li>
<li><p>Each logical shard was a <strong>PostgreSQL schema</strong> (a named group of tables inside one database).</p>
</li>
<li><p>To grow, they could <strong>move logical shards</strong> from one physical server to another, without re-bucketing their data.</p>
</li>
<li><p>They generated unique <strong>64-bit IDs</strong> inside PostgreSQL itself: time-based bits, the shard ID, and a sequence number. That way IDs were unique across all shards and roughly sorted by time.</p>
</li>
</ul>
<p>They also used <strong>streaming replication</strong> (replicas in another availability zone) and <strong>PgBouncer</strong> to pool database connections.</p>
<p>So Instagram used <strong>both</strong> kinds of scaling:</p>
<ul>
<li><p><strong>Horizontal</strong> for the stateless app servers: just add more.</p>
</li>
<li><p><strong>Vertical, then sharded</strong> for the stateful database: big memory-heavy instances, and data split across many of them.</p>
</li>
</ul>
<p><strong>Lesson for us:</strong> Instagram did not invent new technology. It used boring, proven parts (load balancers, stateless app servers, PostgreSQL) and scaled each tier in the way that fits that tier.</p>
<hr />
<h2>6. What is sharding, simply?</h2>
<p><strong>Sharding</strong> means splitting one big dataset into smaller pieces called <strong>shards</strong>, and putting them on different database servers. Each shard holds a subset of the data, for example all data for users whose ID falls in a certain bucket.</p>
<pre><code class="language-plaintext">user_id % number_of_logical_shards  → logical shard (schema)
logical shard                       → physical database server
</code></pre>
<p><strong>Benefits:</strong> each server handles less data and fewer queries, so you can grow almost without limit.</p>
<p><strong>Costs:</strong></p>
<ul>
<li><p>Queries across shards are harder (for example "top photos across all users").</p>
</li>
<li><p>Transactions across shards are hard.</p>
</li>
<li><p>You must choose a good <strong>shard key</strong>. A bad key creates "hot" shards that get most of the traffic.</p>
</li>
<li><p>Moving data between shards needs planning. Instagram's logical-shard trick exists exactly to make this easier.</p>
</li>
</ul>
<p>That is why the interview line is "<strong>shard stateful parts carefully</strong>". Sharding is powerful, but it is a one-way door for many designs.</p>
<hr />
<h2>7. The example visuals</h2>
<p>The "4 CPU vs 64 CPU" servers and the "App 1–4" and "Shard 1–4" boxes in the Reel are <strong>examples</strong> to show the idea. They are not Instagram's real instance sizes or counts. The real numbers from the 2011 post were 25+ Django app servers and a main shard cluster of 12 large PostgreSQL instances (plus 12 replicas in another zone).</p>
<hr />
<h2>8. A quick comparison table</h2>
<table>
<thead>
<tr>
<th></th>
<th>Vertical (scale up)</th>
<th>Horizontal (scale out)</th>
</tr>
</thead>
<tbody><tr>
<td>How</td>
<td>Bigger machine</td>
<td>More machines + load balancer</td>
</tr>
<tr>
<td>Code changes</td>
<td>Usually none</td>
<td>App must be stateless</td>
</tr>
<tr>
<td>Limit</td>
<td>Biggest machine available</td>
<td>Very high</td>
</tr>
<tr>
<td>Failure</td>
<td>Single point of failure</td>
<td>One machine can fail safely</td>
</tr>
<tr>
<td>Best for</td>
<td>Databases (first step), legacy apps</td>
<td>Web/app servers, workers</td>
</tr>
<tr>
<td>Cloud tools</td>
<td>Change instance type</td>
<td>Auto Scaling groups, Kubernetes HPA</td>
</tr>
</tbody></table>
<hr />
<h2>9. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"How do you scale a stateful vs a stateless service?"</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"For stateless services, I scale horizontally: run many copies behind a load balancer with health checks, and use auto scaling. To make that work I keep state out of the app servers, with sessions in Redis or a database and files in S3. For stateful services like databases, I scale up first, add read replicas for read traffic, add caching, and when one primary isn't enough, I shard by a good key. Instagram is a classic example: stateless Django app servers that were easy to scale out, behind an ELB, and PostgreSQL sharded into thousands of logical shards on fewer physical servers. That let a very small team run a very large app."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><p><em>What are sticky sessions and why avoid them?</em> They tie a user to one server, which breaks even load spreading and loses sessions when that server dies.</p>
</li>
<li><p><em>Read replicas vs sharding?</em> Replicas copy all the data to scale reads. Shards split the data to scale reads and writes.</p>
</li>
<li><p><em>How do you pick a shard key?</em> High cardinality, even spread, and matching your main access pattern (for Instagram, mostly by user).</p>
</li>
<li><p><em>When is vertical scaling the right answer?</em> When it's cheap enough, the ceiling is far away, and simplicity matters more than unlimited growth.</p>
</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li><p>Los Angeles Times, "Facebook to snap up Instagram photo-sharing app for $1 billion" (10 Apr 2012): 13 employees, about 30 million users: <a href="https://www.latimes.com/business/la-xpm-2012-apr-10-la-fi-facebook-instagram-20120410-story.html">https://www.latimes.com/business/la-xpm-2012-apr-10-la-fi-facebook-instagram-20120410-story.html</a></p>
</li>
<li><p>The Atlantic, "Instagram Is Now Worth $77 Million Per Employee" (Apr 2012): 13 employees, 30M+ users: <a href="https://www.theatlantic.com/business/archive/2012/04/instagram-is-now-worth-77-million-per-employee/255640/">https://www.theatlantic.com/business/archive/2012/04/instagram-is-now-worth-77-million-per-employee/255640/</a></p>
</li>
<li><p>Instagram Engineering, "What Powers Instagram: Hundreds of Instances, Dozens of Technologies" (Dec 2011): 14M+ users, 3 engineers, ELB + NGINX, stateless Django app servers, PostgreSQL: <a href="https://instagram-engineering.com/what-powers-instagram-hundreds-of-instances-dozens-of-technologies-adf2e22da2ad">https://instagram-engineering.com/what-powers-instagram-hundreds-of-instances-dozens-of-technologies-adf2e22da2ad</a></p>
</li>
<li><p>Instagram Engineering, "Sharding &amp; IDs at Instagram": thousands of logical shards as PostgreSQL schemas, 64-bit IDs: <a href="https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c">https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c</a></p>
</li>
<li><p>High Scalability summary of the Instagram architecture (2011): <a href="https://highscalability.com/instagram-architecture-14-million-users-terabytes-of-photos/">https://highscalability.com/instagram-architecture-14-million-users-terabytes-of-photos/</a></p>
</li>
</ul>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">https://topmate.io/cloudtechbydevang/2358399</a></p>
]]></content:encoded></item><item><title><![CDATA[Monolith vs Microservices: Why Amazon Split (and Prime Video Merged Back)]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple For about ten years, the loudest advice in software was: "Break your monolith…]]></description><link>https://cloudtechbydevang.com/monolith-vs-microservices-why-amazon-split-and-prime-video-merged-back</link><guid isPermaLink="true">https://cloudtechbydevang.com/monolith-vs-microservices-why-amazon-split-and-prime-video-merged-back</guid><category><![CDATA[Microservices]]></category><category><![CDATA[monolith]]></category><category><![CDATA[System Design]]></category><category><![CDATA[AWS]]></category><category><![CDATA[serverless]]></category><category><![CDATA[software architecture]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Sat, 17 Oct 2026 13:59:13 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>For about ten years, the loudest advice in software was: "Break your monolith into microservices." Then, in March 2023, a team inside Amazon, the company most famous for microservices, published a blog post saying they had moved one service <em>back</em> into a single process and <strong>cut its infrastructure cost by over 90%</strong>.</p>
<p>The internet went wild. "Microservices are dead!" "Monoliths are back!" Both reactions were wrong.</p>
<p>In this post I will explain, in simple English, what a monolith is, what microservices are, why Amazon split its monolith in the early 2000s, and what the Prime Video team actually did in 2023. Then I'll give you a clear answer for the interview question: <em>"When would you choose a monolith over microservices?"</em> I have spent 14+ years in cloud, DevOps and SRE, and I have seen teams win and lose with both styles. The style is never the point. Fit is.</p>
<hr />
<h2>1. First, the basics</h2>
<h3>What is a monolith?</h3>
<p>A <strong>monolith</strong> is one application, built and deployed as <strong>one unit</strong>. All the features live in one codebase: login, product pages, cart, payments, emails. You build it once and deploy it once. Usually it talks to one main database.</p>
<p>Good things about a monolith:</p>
<ul>
<li><p><strong>Simple to start.</strong> One repo, one build, one deploy.</p>
</li>
<li><p><strong>Fast calls.</strong> One part of the code calls another as a normal function call, in memory. No network.</p>
</li>
<li><p><strong>Easy to debug.</strong> One process, one log stream, one stack trace.</p>
</li>
<li><p><strong>Easy transactions.</strong> One database means one place to keep data consistent.</p>
</li>
</ul>
<p>Problems as it grows:</p>
<ul>
<li><p><strong>Everyone shares everything.</strong> A small change by one team needs a full redeploy of the whole app.</p>
</li>
<li><p><strong>Scaling is all-or-nothing.</strong> If only search is busy, you still scale the whole app.</p>
</li>
<li><p><strong>One bug can take everything down.</strong> A memory leak in one feature can crash the whole process.</p>
</li>
<li><p><strong>Unclear ownership.</strong> When 200 engineers touch one codebase, nobody owns anything fully.</p>
</li>
</ul>
<h3>What are microservices?</h3>
<p><strong>Microservices</strong> split the application into <strong>small, separate services</strong>. Each service does one job (for example orders, search, payments), owns its own data, and talks to others over the network through an API. Each service is owned by one team and can be <strong>deployed on its own</strong>.</p>
<p>Good things:</p>
<ul>
<li><p><strong>Independent deploys.</strong> The payments team can ship without waiting for the search team.</p>
</li>
<li><p><strong>Independent scaling.</strong> Scale only the busy service.</p>
</li>
<li><p><strong>Smaller blast radius.</strong> If recommendations fail, checkout can still work.</p>
</li>
<li><p><strong>Clear ownership.</strong> One team, one service, end to end.</p>
</li>
</ul>
<p>Costs:</p>
<ul>
<li><p><strong>Network calls everywhere.</strong> Calls are slower and can fail. You need timeouts, retries and circuit breakers.</p>
</li>
<li><p><strong>Harder debugging.</strong> One user request may touch ten services. You need distributed tracing.</p>
</li>
<li><p><strong>Data is split.</strong> No single database transaction across services.</p>
</li>
<li><p><strong>More infrastructure.</strong> More pipelines, more monitoring, more things to secure and pay for.</p>
</li>
</ul>
<p>Microservices are a trade. You pay in complexity to buy team independence and independent scaling.</p>
<hr />
<h2>2. Why Amazon split its monolith</h2>
<p>In a 2006 interview with ACM Queue, Amazon's CTO Werner Vogels described how Amazon.com started as a <strong>monolithic application</strong>: a web server talking to a database on the back end. That application held all the business logic and display logic.</p>
<p>He explained that by <strong>2001</strong> it was clear the front-end application could not scale any more. The problem was not only performance. Many complex pieces were combined into one system, so it could not evolve. Parts that needed to scale on their own were tied to shared resources, and there was <strong>no isolation and no clear ownership</strong>. The shared databases were also hard to scale.</p>
<p>So Amazon moved to a <strong>service-oriented architecture</strong>. In Amazon's version, each service owns its data together with the business logic, and the only way in is through a published service interface. No other team may read the service's database directly. Over time this grew into <strong>hundreds of services</strong>. In that interview Vogels said that a single Amazon.com page could call more than 100 services to build it.</p>
<p>The team side mattered just as much as the technology. AWS describes Amazon's <strong>two-pizza teams</strong>: a team small enough to be fed by two pizzas, ideally fewer than 10 people, with single-threaded ownership of one service. Vogels' famous line from the same interview sums up the culture: <strong>"You build it, you run it."</strong> The team that writes the service also operates it.</p>
<p><strong>Lesson for us:</strong> Amazon did not split its monolith because microservices were fashionable. (Vogels even said they did it before "service-oriented" was a buzzword.) They split because the monolith was blocking <strong>team speed</strong> and <strong>scaling</strong>, and they had the scale to justify the cost.</p>
<hr />
<h2>3. Prime Video: when distributed was the wrong choice</h2>
<p>Now the 2023 story. It is about one specific tool, not all of Prime Video.</p>
<p>Prime Video's <strong>Video Quality Analysis (VQA)</strong> team owns a tool that checks video streams for quality problems, for example corrupted frames or audio and video going out of sync. They needed it to monitor <strong>thousands of concurrent streams</strong>, and to grow beyond that.</p>
<h3>The original design</h3>
<p>The first version was a <strong>distributed, serverless design</strong>:</p>
<ul>
<li><p><strong>AWS Step Functions</strong> orchestrated the workflow.</p>
</li>
<li><p><strong>AWS Lambda</strong> functions ran the detectors.</p>
</li>
<li><p><strong>Amazon S3</strong> stored video frames between the steps, so each component could pick them up.</p>
</li>
</ul>
<p>On paper this looks perfect: serverless, independently scalable pieces.</p>
<h3>What went wrong</h3>
<p>According to the Prime Video Tech blog (Marcin Kolny, 22 March 2023):</p>
<ul>
<li><p>The design <strong>hit a hard scaling limit at around 5% of the expected load.</strong></p>
</li>
<li><p><strong>Orchestration was the main bottleneck.</strong> The service did several state transitions for every second of every stream, so it quickly hit account limits. And Step Functions charges per state transition.</p>
</li>
<li><p><strong>Passing data between components was expensive.</strong> Every frame went into S3 and was read back out, which cost a lot at this volume.</p>
</li>
</ul>
<h3>The fix</h3>
<p>The team moved all the components into <strong>a single process</strong>. Frames now moved <strong>in memory</strong> instead of through S3, and the orchestration logic ran inside the same instance. They deployed it on <strong>Amazon EC2 and Amazon ECS</strong>, and scaled by running more copies of the whole process.</p>
<p>The result, in their words: moving to a monolith <strong>cut infrastructure cost by over 90%</strong> and <em>increased</em> their ability to scale. They could handle thousands of streams with room to grow, and could also use EC2 compute savings plans to lower costs further.</p>
<h3>What it does NOT mean</h3>
<ul>
<li><p>It does <strong>not</strong> mean Amazon or Prime Video dropped microservices. This was one service.</p>
</li>
<li><p>Some engineers argued the new design is really "one bigger microservice". It still runs as a separate service, just not split into tiny serverless steps. That is a fair point. The real lesson is about <strong>where you draw the boundaries</strong>.</p>
</li>
<li><p>It does <strong>not</strong> mean serverless is bad. Serverless is great for spiky, event-driven work. It was a poor fit here because the workload moved <strong>huge amounts of data</strong> between steps at <strong>high, steady volume</strong>.</p>
</li>
</ul>
<p><strong>Lesson for us:</strong> if two parts of your system exchange large amounts of data all the time, putting a network (or S3) between them is expensive. Keep tightly connected, data-heavy steps together.</p>
<hr />
<h2>4. Putting the two stories side by side</h2>
<table>
<thead>
<tr>
<th></th>
<th>Amazon.com (early 2000s)</th>
<th>Prime Video VQA (2023)</th>
</tr>
</thead>
<tbody><tr>
<td>Starting point</td>
<td>Monolith</td>
<td>Distributed serverless (Step Functions + Lambda + S3)</td>
</tr>
<tr>
<td>Main pain</td>
<td>No isolation, no ownership, shared databases, can't scale</td>
<td>Scaling limit at ~5% of load, high orchestration and data-transfer cost</td>
</tr>
<tr>
<td>Move</td>
<td>Split into services, two-pizza teams</td>
<td>Merged components into one process on EC2/ECS</td>
</tr>
<tr>
<td>Why it worked</td>
<td>Many teams needed to move independently</td>
<td>Components were tightly coupled and moved lots of data</td>
</tr>
</tbody></table>
<p>Both teams made the <strong>right call for their problem</strong>. That's the whole point.</p>
<hr />
<h2>5. A simple decision guide</h2>
<p>These are rules of thumb from my experience, not laws.</p>
<p><strong>Start with a monolith (or a "modular monolith") when:</strong></p>
<ul>
<li><p>You are a small team (one or two teams).</p>
</li>
<li><p>The product is new and the boundaries are not clear yet.</p>
</li>
<li><p>Parts of the system share a lot of data and change together.</p>
</li>
<li><p>You want to move fast with few operational costs.</p>
</li>
</ul>
<p>A <strong>modular monolith</strong> is a good middle path: one deployable app, but with clean internal modules and clear interfaces between them. If a module later needs to scale or move on its own, you can pull it out into a service.</p>
<p><strong>Move to microservices when:</strong></p>
<ul>
<li><p>Many teams are blocked by each other's releases.</p>
</li>
<li><p>Different parts have very different scaling needs (for example search vs. account settings).</p>
</li>
<li><p>You need a smaller blast radius for critical paths like checkout.</p>
</li>
<li><p>You have the platform maturity: CI/CD, monitoring, tracing, on-call.</p>
</li>
</ul>
<p><strong>Warning signs you split too much:</strong></p>
<ul>
<li><p>One user action needs a long chain of synchronous calls.</p>
</li>
<li><p>Services always deploy together ("distributed monolith").</p>
</li>
<li><p>Huge data moves between services on every request.</p>
</li>
<li><p>Your cloud bill is dominated by data transfer and orchestration, not real work.</p>
</li>
</ul>
<hr />
<h2>6. The example visuals</h2>
<p>The service names in the Reel (Orders, Search, Payments, Catalog) are <strong>examples</strong> to show the idea of splitting an app. They are not an official list of Amazon's services. The Prime Video before/after diagram is a simplified version of the architecture the team described.</p>
<hr />
<h2>7. How to answer this in an interview</h2>
<p><strong>Question:</strong> <em>"When would you choose a monolith over microservices?"</em></p>
<p><strong>Strong answer:</strong></p>
<blockquote>
<p>"I'd start with a well-structured monolith when the team is small, the domain isn't clear yet, or the parts share a lot of data, because it's simpler, cheaper and faster to change. I'd split out services when teams are blocking each other or parts need to scale differently. Amazon split its monolith in the early 2000s for exactly those reasons, with two-pizza teams owning services. But the split has costs: network calls, data transfer, and operational overhead. Prime Video's video quality team showed that in 2023. Their Step Functions + Lambda + S3 design hit a limit at about 5% of the expected load, and putting it into one process on ECS cut infrastructure cost by over 90%. So I split when teams and scale need it, not by default."</p>
</blockquote>
<p><strong>Follow-ups to expect:</strong></p>
<ul>
<li><p><em>What is a distributed monolith?</em> Services that are split physically but still must deploy together. You get the costs of both styles and the benefits of neither.</p>
</li>
<li><p><em>How do you split a monolith safely?</em> The strangler fig pattern: put a router in front, move one feature at a time to a new service, and retire the old code path.</p>
</li>
<li><p><em>How do services share data?</em> Through APIs or events, not by reading each other's databases.</p>
</li>
<li><p><em>How do you handle a transaction across services?</em> Sagas: a series of local transactions with compensating actions if something fails.</p>
</li>
</ul>
<hr />
<h3>Sources</h3>
<ul>
<li><p>Marcin Kolny, Prime Video Tech, "Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%" (22 Mar 2023). Archived copy: <a href="https://archive.is/h788m">https://archive.is/h788m</a></p>
</li>
<li><p>The New Stack, "Return of the Monolith: Amazon Dumps Microservices for Video Monitoring" (4 May 2023): <a href="https://thenewstack.io/return-of-the-monolith-amazon-dumps-microservices-for-video-monitoring/">https://thenewstack.io/return-of-the-monolith-amazon-dumps-microservices-for-video-monitoring/</a></p>
</li>
<li><p>The Stack, "Prime Video service dumps microservices, cuts AWS bill 90%" (May 2023): <a href="https://www.thestack.technology/amazon-prime-video-microservices-monolith/">https://www.thestack.technology/amazon-prime-video-microservices-monolith/</a></p>
</li>
<li><p>ACM Queue, "A Conversation with Werner Vogels" (2006): monolith until 2001, service-oriented architecture, 100+ services per page, "you build it, you run it": <a href="https://queue.acm.org/detail.cfm?id=1142065">https://queue.acm.org/detail.cfm?id=1142065</a></p>
</li>
<li><p>AWS Executive Insights, "Powering Innovation and Speed with Amazon's Two-Pizza Teams": <a href="https://aws.amazon.com/executive-insights/content/amazon-two-pizza-team/">https://aws.amazon.com/executive-insights/content/amazon-two-pizza-team/</a></p>
</li>
</ul>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">https://topmate.io/cloudtechbydevang/2358399</a></p>
]]></content:encoded></item><item><title><![CDATA[What Actually Happens When You Type google.com and Press Enter]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple You type google.com into your browser and press Enter. Less than a second…]]></description><link>https://cloudtechbydevang.com/what-actually-happens-when-you-type-google-com-and-press-enter</link><guid isPermaLink="true">https://cloudtechbydevang.com/what-actually-happens-when-you-type-google-com-and-press-enter</guid><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Fri, 16 Oct 2026 14:07:19 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>You type <code>google.com</code> into your browser and press Enter. Less than a second later, the page is on your screen. It feels like nothing happened. In fact, a small relay race just ran across the internet: your laptop, your router, your internet provider, a DNS resolver, several name servers, and a Google server near you all took part.</p>
<p>"What happens when you type a URL and press Enter?" is one of the most common warm-up questions in cloud, DevOps and backend interviews. It looks simple. But it tests whether you really understand networking, security and caching, the three things that sit under every system you will ever build.</p>
<p>In this post I will walk through the whole journey in simple English, step by step. At the end I will give you a short answer you can say in an interview. I have spent 14+ years in cloud, DevOps and SRE, and almost every production incident I have debugged touched one of these steps: a stale DNS record, an expired certificate, a load balancer health check, or a missing cache.</p>
<hr />
<h2>The big picture first</h2>
<p>Here is the whole flow in one line:</p>
<p><strong>DNS → TCP → TLS → HTTP → render</strong>, with <strong>caching at every step</strong>.</p>
<ol>
<li><p><strong>DNS</strong> turns the name <code>google.com</code> into an IP address.</p>
</li>
<li><p><strong>TCP</strong> opens a reliable connection to that IP address.</p>
</li>
<li><p><strong>TLS</strong> makes the connection private and proves you are talking to the real Google.</p>
</li>
<li><p><strong>HTTP</strong> sends your request and brings back the page.</p>
</li>
<li><p>Your <strong>browser</strong> draws the page, and fetches more files it needs.</p>
</li>
</ol>
<p>Modern browsers can also use <strong>HTTP/3</strong>, which runs over <strong>QUIC</strong> on UDP instead of TCP. QUIC combines the connection and encryption steps. I will cover that near the end. First, let's take the classic path, step by step.</p>
<hr />
<h2>1. DNS: finding the address</h2>
<p>Computers do not talk to names. They talk to <strong>IP addresses</strong>, such as <code>142.250.x.x</code> (IPv4) or a longer IPv6 address. <strong>DNS</strong> (Domain Name System) is the internet's phone book. It turns a name into an address.</p>
<h3>Step 1a: check the caches close to you</h3>
<p>Before asking anyone on the internet, your computer checks what it already knows:</p>
<ul>
<li><p><strong>Browser cache.</strong> Browsers keep recent DNS answers for a short time. (In Chrome you can see this at <code>chrome://net-internals/#dns</code>.)</p>
</li>
<li><p><strong>Operating system cache.</strong> If the browser does not have it, it asks the OS. The part of the OS that does this is called the <strong>stub resolver</strong>.</p>
</li>
</ul>
<p>Every DNS answer comes with a <strong>TTL</strong> (time to live). The TTL says how many seconds you may keep that answer before you must ask again. If a cached answer is still within its TTL, the lookup ends right here. That is why the second visit to a site is often faster than the first.</p>
<h3>Step 1b: ask the recursive resolver</h3>
<p>If there is no cached answer, your OS sends the question to a <strong>recursive resolver</strong>. This is usually run by your internet provider, or a public resolver like Google Public DNS (<code>8.8.8.8</code>) or Cloudflare (<code>1.1.1.1</code>).</p>
<p>The resolver's job is to find the answer for you. It also has a big cache. Many people use the same resolver, so popular names like <code>google.com</code> are usually already cached there.</p>
<h3>Step 1c: the full walk (when nothing is cached)</h3>
<p>If the resolver has nothing cached, it walks down the DNS tree:</p>
<ol>
<li><p>It asks a <strong>root name server</strong>: "Who handles <code>.com</code>?" The root replies with the addresses of the <code>.com</code> servers.</p>
</li>
<li><p>It asks a <code>.com</code> <strong>TLD (top-level domain) name server</strong>: "Who handles <code>google.com</code>?" The TLD server replies with Google's own name servers.</p>
</li>
<li><p>It asks <strong>Google's authoritative name server</strong>: "What is the IP address of <code>google.com</code>?" This server holds the real record and gives the final answer.</p>
</li>
<li><p>The resolver sends the IP address back to your computer, and caches it for next time.</p>
</li>
</ol>
<p>In real life, most lookups skip some of these steps because of caching. Resolvers keep the <code>.com</code> server list and Google's name server list in cache too, so the full walk is rare for popular sites.</p>
<p>One more thing worth knowing: big companies like Google may return <strong>different IP addresses to different users</strong>, depending on where the user is. This is one way they send you to a nearby server. (I cover this, and anycast, in topic 006 of this series.)</p>
<p><strong>Lesson for us:</strong> DNS is a cache-heavy system. When you change a DNS record in production, the old answer can live on in caches until its TTL runs out. Lower the TTL <em>before</em> a planned migration, not during it.</p>
<hr />
<h2>2. TCP: opening a reliable connection</h2>
<p>Now your browser knows the IP address. Next, it needs a reliable connection to it. The classic way is <strong>TCP</strong> (Transmission Control Protocol), on port <strong>443</strong> for HTTPS.</p>
<p>TCP gives you three promises: data arrives <strong>in order</strong>, <strong>without gaps</strong>, and <strong>without duplicates</strong>. To set this up, both sides first agree on starting sequence numbers. They do this with the famous <strong>three-way handshake</strong>:</p>
<ol>
<li><p><strong>SYN.</strong> Your computer says: "I want to talk. Here is my starting number."</p>
</li>
<li><p><strong>SYN-ACK.</strong> The server says: "OK. I got your number. Here is mine."</p>
</li>
<li><p><strong>ACK.</strong> Your computer says: "Got yours. We are connected."</p>
</li>
</ol>
<p>This costs <strong>one round trip</strong> before any useful data can flow. A round trip (RTT) is the time for a message to go to the server and back. If the server is 100 ms away, the handshake alone costs about 100 ms. That is why physical distance matters so much, and why big companies put servers close to users.</p>
<p><strong>Lesson for us:</strong> reuse connections. Keep-alive connections, connection pooling in your apps, and HTTP/2 multiplexing all exist so you don't pay the handshake cost again and again.</p>
<hr />
<h2>3. TLS: making it private and proving who you are talking to</h2>
<p>The <code>S</code> in <code>HTTPS</code> means the connection is protected by <strong>TLS</strong> (Transport Layer Security). TLS does two jobs:</p>
<ol>
<li><p><strong>Encryption.</strong> Nobody in the middle (your Wi-Fi, your ISP, anyone else) can read or change the data.</p>
</li>
<li><p><strong>Authentication.</strong> The server shows a <strong>certificate</strong>, signed by a trusted certificate authority. Your browser checks that the certificate is valid and really belongs to <code>google.com</code>.</p>
</li>
</ol>
<p>With <strong>TLS 1.3</strong> (RFC 8446, published in 2018), a full handshake takes just <strong>one round trip</strong> (often called <strong>1-RTT</strong>). Older TLS 1.2 handshakes needed two round trips. So TLS 1.3 made every new HTTPS connection faster.</p>
<p>In simple terms, the TLS 1.3 handshake goes like this:</p>
<ul>
<li><p>Your browser sends a "hello" with the encryption options it supports and its share of a key exchange.</p>
</li>
<li><p>The server replies with its choices, its key share, its certificate and a proof that it owns the certificate.</p>
</li>
<li><p>Both sides now compute the same secret keys, and encrypted data can flow.</p>
</li>
</ul>
<p>TLS 1.3 also has a <strong>0-RTT</strong> mode for repeat visits, where the browser can send some data in its very first message. It is faster, but that early data can be replayed by an attacker, so servers only allow it for safe requests.</p>
<p><strong>Lesson for us:</strong> a huge number of real outages are simply <strong>expired certificates</strong>. Automate renewal (for example with ACM on AWS, or cert-manager on Kubernetes) and alert well before expiry.</p>
<hr />
<h2>4. The request reaches Google: the Google Front End</h2>
<p>Now your encrypted connection is open. Who is on the other end? Not the server that builds your search page. You are talking to Google's edge.</p>
<p>Google's own security design paper explains that when a service needs to be on the internet, it registers with an infrastructure service called the <strong>Google Front End (GFE)</strong>. The GFE <strong>terminates TLS</strong> with the correct certificates and best practices, and it adds protection against denial-of-service attacks. Then it forwards your request to the right <strong>backend service</strong> inside Google's network.</p>
<p>This pattern is not special to Google. Almost every large website works the same way:</p>
<ul>
<li><p>A <strong>front end / edge layer</strong> (load balancer, reverse proxy, CDN) that faces the internet.</p>
</li>
<li><p><strong>Backend services</strong> that do the real work, hidden behind it.</p>
</li>
</ul>
<p>On AWS, you would build the same shape with <strong>CloudFront</strong> and an <strong>Application Load Balancer</strong> in front of your services. On Google Cloud, the external Application Load Balancer runs on the same front-end idea.</p>
<p><strong>Lesson for us:</strong> never expose your app servers directly to the internet. Put a load balancer or reverse proxy in front. It terminates TLS, absorbs attacks, and lets you add or remove servers without users noticing.</p>
<hr />
<h2>5. HTTP: the actual request and response</h2>
<p>Inside the encrypted connection, your browser sends an <strong>HTTP request</strong>:</p>
<pre><code class="language-plaintext">GET / HTTP/2
Host: www.google.com
User-Agent: ...
Accept: text/html
Cookie: ...
</code></pre>
<p>The server sends back an <strong>HTTP response</strong>: a status code (for example <code>200 OK</code>, or a <code>301</code> redirect from <code>google.com</code> to <code>www.google.com</code>), some headers, and the HTML of the page.</p>
<p>There are three main versions of HTTP in use today:</p>
<table>
<thead>
<tr>
<th>Version</th>
<th>Runs on</th>
<th>Key idea</th>
</tr>
</thead>
<tbody><tr>
<td>HTTP/1.1</td>
<td>TCP</td>
<td>One request at a time per connection</td>
</tr>
<tr>
<td>HTTP/2</td>
<td>TCP</td>
<td>Many requests at once on one connection (multiplexing)</td>
</tr>
<tr>
<td>HTTP/3</td>
<td><strong>QUIC</strong> (on UDP)</td>
<td>Like HTTP/2, but without TCP's "one lost packet blocks everything" problem</td>
</tr>
</tbody></table>
<p>HTTP/3 is defined in RFC 9114. It uses the <strong>QUIC</strong> transport, which handles reliability, streams and flow control itself, and has TLS 1.3 built in. That is why with QUIC the connection setup and the encryption setup happen together, saving a round trip compared with TCP + TLS.</p>
<p>Google created the early version of QUIC, and Google's services widely support HTTP/3. Your browser and the server agree on which version to use. Older networks or browsers fall back to HTTP/2 or HTTP/1.1 over TCP.</p>
<hr />
<h2>6. The browser draws the page</h2>
<p>The HTML is only the start. The browser reads it and finds more things it needs: CSS, JavaScript, images, fonts. It fetches them too, often from the same connection (thanks to HTTP/2 or HTTP/3) or from CDN servers. Then it builds the page structure (the DOM), applies styles, runs scripts, and paints pixels on your screen.</p>
<p>This part is mostly front-end engineering, but it matters for cloud people too: a slow page is often not a slow server, but too many files, no compression, or no caching headers.</p>
<hr />
<h2>7. Caching at every step</h2>
<p>If you remember one thing from this post, remember this. <strong>Every step has a cache, and caching is why the web feels fast.</strong></p>
<table>
<thead>
<tr>
<th>Step</th>
<th>Where the cache lives</th>
<th>What it saves</th>
</tr>
</thead>
<tbody><tr>
<td>DNS</td>
<td>Browser, OS, recursive resolver</td>
<td>The whole DNS walk</td>
</tr>
<tr>
<td>TCP/TLS</td>
<td>Connection reuse, TLS session resumption</td>
<td>New handshakes</td>
</tr>
<tr>
<td>HTTP</td>
<td>Browser cache, CDN edge, reverse proxy</td>
<td>The trip to the origin server</td>
</tr>
<tr>
<td>Backend</td>
<td>In-memory caches (Redis, Memcached)</td>
<td>Database queries</td>
</tr>
</tbody></table>
<p>The first visit pays the full price. The second visit is cheaper at almost every layer.</p>
<hr />
<h2>8. A simple diagram</h2>
<p>The diagrams in the Reel and here are simplified <strong>examples</strong> for teaching. Real flows have more steps (for example CNAME records, redirects and CDN layers).</p>
<pre><code class="language-plaintext">You type google.com + Enter
   │
   ▼
[Browser cache] → [OS cache] → [Recursive resolver]
                                   │ (if not cached)
                                   ├─► Root server  ("ask .com")
                                   ├─► .com server  ("ask Google's name servers")
                                   └─► Google name server ("here is the IP")
   │
   ▼
TCP handshake (SYN, SYN-ACK, ACK)  ── 1 round trip
   │
   ▼
TLS 1.3 handshake                   ── 1 round trip
   │
   ▼
Google Front End (near you) → backend service → HTML
   │
   ▼
Browser renders the page, fetches CSS/JS/images
</code></pre>
<hr />
<h2>9. How to answer this in an interview</h2>
<p><strong>Short answer (30 seconds):</strong></p>
<blockquote>
<p>"First, DNS turns google.com into an IP address. The browser and OS check their caches, then a recursive resolver asks the root, the .com servers and Google's name servers. Then the browser opens a TCP connection with a three-way handshake, and TLS 1.3 sets up encryption in one round trip. The request reaches a Google Front End near me, which terminates TLS and forwards it to a backend. The response comes back over HTTP/2 or HTTP/3, which runs on QUIC, and the browser renders the page. And there's caching at every step: DNS, connections, CDN and the backend."</p>
</blockquote>
<p><strong>Follow-up questions to expect:</strong></p>
<ul>
<li><p><em>What is a TTL and why does it matter?</em> How long a DNS answer can be cached. It controls how fast DNS changes spread.</p>
</li>
<li><p><em>Why is TLS 1.3 faster than TLS 1.2?</em> One round trip for a full handshake instead of two.</p>
</li>
<li><p><em>Why does HTTP/3 use UDP?</em> QUIC builds its own reliability on UDP, so one lost packet only blocks its own stream, and it combines the transport and TLS handshakes.</p>
</li>
<li><p><em>Where would you put a load balancer?</em> At the edge, terminating TLS, in front of stateless app servers.</p>
</li>
<li><p><em>How would you debug a slow page?</em> Check DNS time, connect time, TLS time and time to first byte separately (for example with <code>curl -w</code> or browser dev tools).</p>
</li>
</ul>
<p><strong>A senior touch:</strong> mention failure points. DNS misconfiguration, an expired certificate, an unhealthy backend behind the load balancer, or a cold cache after a deploy. Interviewers love when you connect the theory to real incidents.</p>
<hr />
<h2>10. Try it yourself</h2>
<p>You can see each step on your own machine:</p>
<pre><code class="language-bash">dig google.com                 # DNS answer and TTL
dig +trace google.com          # the full root → .com → Google walk
curl -so /dev/null -w "dns:%{time_namelookup} connect:%{time_connect} tls:%{time_appconnect} ttfb:%{time_starttransfer}\n" https://www.google.com
</code></pre>
<p>Run the <code>curl</code> command twice and compare. The difference is caching at work.</p>
<hr />
<h3>Sources</h3>
<ul>
<li><p>Cloudflare Learning Center, "What is DNS?" (resolution steps, browser/OS caching, TTL): <a href="https://www.cloudflare.com/learning/dns/what-is-dns/">https://www.cloudflare.com/learning/dns/what-is-dns/</a></p>
</li>
<li><p>RFC 9293, Transmission Control Protocol (TCP), section 3.5 "Establishing a Connection" (three-way handshake): <a href="https://www.rfc-editor.org/rfc/rfc9293">https://www.rfc-editor.org/rfc/rfc9293</a></p>
</li>
<li><p>RFC 8446, The Transport Layer Security (TLS) Protocol Version 1.3 (1-RTT handshake, 0-RTT data and replay): <a href="https://www.rfc-editor.org/rfc/rfc8446">https://www.rfc-editor.org/rfc/rfc8446</a></p>
</li>
<li><p>Google Cloud, "Google infrastructure security design overview" (Google Front End terminates TLS and fronts internet-facing services): <a href="https://cloud.google.com/docs/security/infrastructure/design">https://cloud.google.com/docs/security/infrastructure/design</a></p>
</li>
<li><p>RFC 9114, HTTP/3 (HTTP over the QUIC transport): <a href="https://www.rfc-editor.org/rfc/rfc9114">https://www.rfc-editor.org/rfc/rfc9114</a></p>
</li>
<li><p>Google Public DNS FAQ (anycast resolver): <a href="https://developers.google.com/speed/public-dns/faq">https://developers.google.com/speed/public-dns/faq</a></p>
</li>
</ul>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">https://topmate.io/cloudtechbydevang/2358399</a></p>
]]></content:encoded></item><item><title><![CDATA[How Netflix Serves 325M+ Members Without Going Down
]]></title><description><![CDATA[By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple You open Netflix on a Friday night. You press Play. The video starts in a…]]></description><link>https://cloudtechbydevang.com/how-netflix-serves-325m-members-without-going-down</link><guid isPermaLink="true">https://cloudtechbydevang.com/how-netflix-serves-325m-members-without-going-down</guid><category><![CDATA[Cloud]]></category><category><![CDATA[System Design]]></category><category><![CDATA[Devops]]></category><category><![CDATA[netflix]]></category><dc:creator><![CDATA[Devang Pandya]]></dc:creator><pubDate>Thu, 15 Oct 2026 14:16:08 GMT</pubDate><content:encoded><![CDATA[<p><em>By Devang Pandya · Cloud Tech by Devang · Cloud • DevOps • AI — Made Simple</em></p>
<p>You open Netflix on a Friday night. You press Play. The video starts in a second or two, in good quality, with no buffering. Millions of other people around the world are doing the same thing at the same time.</p>
<p>Netflix crossed <strong>325 million paid memberships</strong> in late 2025. That is a huge number of people who expect the app to "just work". So how does Netflix stay up?</p>
<p>There is no single magic tool. It is a set of simple ideas, used with a lot of discipline. In this post I will explain these ideas in simple English. At the end, I will show you how to use them in a system design interview. I have spent 14+ years in cloud, DevOps and SRE, and I see the same patterns in every system that scales well, big or small.</p>
<hr />
<h2>1. It started with an outage</h2>
<p>In August 2008, Netflix had a major database corruption. For three days, it could not ship DVDs to its members.</p>
<p>For a DVD business, that was painful. For a streaming business, it would have been a disaster. If streaming had been down for three days, nobody could watch anything at all.</p>
<p>Netflix took one big lesson from this: <strong>a single, vertically scaled database is a single point of failure.</strong> "Vertically scaled" means you make one machine bigger and bigger. When that one machine breaks, everything breaks.</p>
<p>So Netflix decided to move to <strong>horizontally scalable, distributed systems in the cloud.</strong> "Horizontally scaled" means you add more machines instead of making one machine bigger. If one fails, the others keep working.</p>
<p>They chose <strong>AWS</strong> (Amazon Web Services). Most customer-facing systems were moved before 2015. In <strong>January 2016</strong>, after <strong>seven years</strong> of work, Netflix shut down the last data centre parts used by its streaming service.</p>
<p>The key point: Netflix did not just copy its old systems to the cloud ("lift and shift"). It rebuilt them. In their own words, they moved from a monolithic app to hundreds of microservices, and they changed their data model to use NoSQL databases.</p>
<p><strong>Lesson for us:</strong> moving to the cloud does not make you reliable by itself. Changing the design does.</p>
<hr />
<h2>2. Two paths: the "brain" and the "video"</h2>
<p>This is the most important idea in the whole post, so let's make it simple.</p>
<p>When you use Netflix, two very different kinds of work happen:</p>
<ol>
<li><p><strong>The brain (control plane).</strong> Logging in, choosing a profile, loading your home screen, showing recommendations, checking your plan and licence, and deciding <em>which server should send you the video</em>. This is lots of small requests. It runs on <strong>AWS</strong>.</p>
</li>
<li><p><strong>The video (data plane).</strong> The actual movie file: gigabytes of data flowing to your TV or phone. This comes from <strong>Open Connect</strong>, Netflix's own content delivery network (CDN).</p>
</li>
</ol>
<p>Here is what happens when you press Play:</p>
<ol>
<li><p>Your device sends a request to Netflix's front door on AWS. This front door is an <strong>API gateway</strong> called <strong>Zuul</strong>. It routes requests, helps with authentication, and can shed load when things go wrong.</p>
</li>
<li><p>Zuul sends the request to the right <strong>microservices</strong>: profiles, recommendations, licensing, playback and more.</p>
</li>
<li><p>The playback service looks at which video servers are healthy, which ones have your title, and which ones are closest to you on the network. It sends your device a <strong>ranked list of servers</strong> to stream from.</p>
</li>
<li><p>Your device downloads the video from an <strong>Open Connect server</strong>, often one that sits inside your own internet provider's network.</p>
</li>
</ol>
<p>Why split it like this? Because the two jobs need very different things. The brain needs flexibility, fast changes and lots of compute. The video needs huge bandwidth, close to the user. Putting each job on the right platform is cheaper and more reliable than forcing both into one place.</p>
<hr />
<h2>3. Open Connect: the video comes from near your home</h2>
<p>A <strong>CDN</strong> (content delivery network) is a group of servers spread around the world that keep copies of content close to users. Most companies rent a CDN. Netflix built its own, called <strong>Open Connect</strong>, starting around 2012.</p>
<p>As of December 2022, Netflix said it had <strong>18,000 servers in 6,000 locations across 175 countries.</strong></p>
<p>How it works, in simple steps:</p>
<ul>
<li><p><strong>Servers inside ISPs.</strong> Netflix gives its Open Connect Appliances (OCAs) <strong>free of charge</strong> to qualifying internet service providers. Netflix provides the hardware. The ISP provides rack space, power and network connection.</p>
</li>
<li><p><strong>Servers at internet exchanges.</strong> Netflix also places OCAs at internet exchange points, where many networks connect to each other, and peers with ISPs there.</p>
</li>
<li><p><strong>Filled at night.</strong> Netflix predicts what people will watch and copies those files onto the servers during off-peak hours, when the network is quiet. So when a big new show comes out, it is already sitting near you.</p>
</li>
<li><p><strong>Steered by the control plane.</strong> Each OCA reports its health, the routes it has learned (using BGP, the routing protocol of the internet) and the files it has to the Open Connect control plane, which runs in AWS. The control plane uses this data to point your device to the best server, and gives a list of options in case one fails.</p>
</li>
</ul>
<p>The result: the video travels a short distance. It starts faster, buffers less, and puts less load on the wider internet. Netflix estimated that in 2021 alone the program helped ISPs avoid about $1.25 billion in costs.</p>
<p><strong>Lesson for us:</strong> move heavy, read-only content as close to users as you can. You may not build your own CDN, but CloudFront, Cloud CDN, Akamai or Cloudflare give you the same idea.</p>
<hr />
<h2>4. Microservices: many small teams, many small services</h2>
<p>Netflix runs <strong>hundreds of microservices</strong>. Each one does one job well: one for profiles, one for viewing history, one for recommendations, and so on.</p>
<p>Why does this help reliability?</p>
<ul>
<li><p><strong>Small blast radius.</strong> If the recommendations service has a problem, you might see a simpler home screen, but you can still play a show. One broken part does not take down everything.</p>
</li>
<li><p><strong>Independent scaling.</strong> The playback service can scale up on a Friday night without scaling everything else.</p>
</li>
<li><p><strong>Independent deployments.</strong> Teams can release small changes often, instead of one big risky release.</p>
</li>
</ul>
<p>But microservices bring new problems. When one screen needs many service calls, a slow service can make everything slow. Netflix built (and open-sourced) tools to deal with this:</p>
<ul>
<li><p><strong>Eureka</strong> for service discovery (how services find each other).</p>
</li>
<li><p><strong>Ribbon</strong> for client-side load balancing.</p>
</li>
<li><p><strong>Hystrix</strong> for fault tolerance: it isolated calls between services, stopped one failure from spreading to others ("cascading failure"), and gave <strong>fallbacks</strong>, a simpler answer when a service could not respond.</p>
</li>
</ul>
<p>Netflix later moved to Spring Boot and Spring Cloud as its core Java framework (announced in 2018), and Hystrix is now in maintenance mode. But the ideas are still the base of every good microservice system: <strong>timeouts, retries with limits, circuit breakers, and fallbacks.</strong> Today you will see them in service meshes like Istio and Envoy, and in libraries like Resilience4j.</p>
<p>A key design rule at Netflix: <strong>many services are stateless.</strong> They do not keep important data in their own memory. They keep it in caches or databases. This makes it easy to add or remove servers, and even to serve any member from any AWS region.</p>
<hr />
<h2>5. Caching: don't ask the database every time</h2>
<p>Your home screen needs a lot of data, very fast: what you watched, what you rated, what rows to show. Hitting a database for every one of these would be slow and expensive.</p>
<p>So Netflix uses <strong>EVCache</strong>, a caching service built on <strong>Memcached</strong>. It keeps frequently read data in memory, so reads take milliseconds or less. Netflix's own blog says EVCache is used widely across its microservices and is open source.</p>
<p>Netflix also built <strong>global replication</strong> for EVCache. Its cloud runs in more than one AWS region (their blog named Northern Virginia, Oregon and Ireland). When data changes in one region, the change is sent to the other regions through a Kafka-based replication system. So if traffic moves to another region, the cache there is already warm.</p>
<p>There are really two layers of caching at Netflix:</p>
<ol>
<li><p><strong>Data cache</strong> (EVCache) for the brain.</p>
</li>
<li><p><strong>Content cache</strong> (Open Connect) for the video.</p>
</li>
</ol>
<p><strong>Lesson for us:</strong> find the data that is read often and changes rarely, and cache it. Think about what happens when the cache is empty ("cold"), and how you keep it fresh.</p>
<hr />
<h2>6. Chaos engineering: break things on purpose</h2>
<p>This is the part everyone remembers.</p>
<p>In 2011, Netflix wrote about <strong>Chaos Monkey</strong>, a tool that <strong>randomly shuts down production servers</strong>. Why would anyone do that?</p>
<p>Because in the cloud, servers <em>will</em> fail. Netflix's thinking was simple: if failure is going to happen anyway, make it happen often, in a controlled way, while engineers are awake and watching. Then you find weak spots and build automatic recovery. In their words, the goal was that when an instance fails at 3 AM on a Sunday, they won't even notice.</p>
<p>Chaos Monkey ran during business hours, in a carefully monitored environment, with engineers standing by. In July 2012, Netflix open-sourced it under the Apache License.</p>
<p>Chaos Monkey was the first member of the <strong>Simian Army</strong>:</p>
<ul>
<li><p><strong>Latency Monkey</strong> added artificial delays to test slow services.</p>
</li>
<li><p><strong>Conformity Monkey</strong> found servers that did not follow best practices.</p>
</li>
<li><p><strong>Chaos Gorilla</strong> simulated losing a whole AWS <strong>Availability Zone</strong>.</p>
</li>
<li><p><strong>Chaos Kong</strong> simulated losing a whole AWS <strong>region</strong>.</p>
</li>
</ul>
<p>Netflix practises <strong>region evacuation</strong>: moving all traffic out of one AWS region into the others. With a project called Nimble, which keeps extra capacity ready, Netflix said it cut evacuation time from about 50 minutes to about 8 minutes.</p>
<p><strong>Lesson for us:</strong> you don't need to start with production. Start with a game day in staging. Kill a pod. Block a dependency. Watch your alerts. Write down what surprised you. Make the blast radius small and grow it slowly. Today, tools like AWS Fault Injection Service, Chaos Mesh and LitmusChaos make this easier.</p>
<hr />
<h2>7. Putting it all together</h2>
<p>Here is the full picture in one place:</p>
<table>
<thead>
<tr>
<th>Problem</th>
<th>Netflix answer</th>
<th>What you can use</th>
</tr>
</thead>
<tbody><tr>
<td>One database = single point of failure</td>
<td>Distributed systems on AWS, NoSQL</td>
<td>Managed DBs with replicas, DynamoDB, Cassandra</td>
</tr>
<tr>
<td>Video is heavy and far away</td>
<td>Open Connect CDN inside ISPs</td>
<td>CloudFront, Cloud CDN, Cloudflare</td>
</tr>
<tr>
<td>One big app is hard to scale and change</td>
<td>Hundreds of microservices behind Zuul</td>
<td>API Gateway, Kubernetes, Ingress</td>
</tr>
<tr>
<td>One slow service slows everything</td>
<td>Timeouts, circuit breakers, fallbacks</td>
<td>Istio/Envoy, Resilience4j</td>
</tr>
<tr>
<td>Reads are slow and costly</td>
<td>EVCache, replicated across regions</td>
<td>ElastiCache, Memorystore, Redis</td>
</tr>
<tr>
<td>Failures surprise you at 3 AM</td>
<td>Chaos Monkey, Chaos Kong, region evacuation</td>
<td>AWS FIS, Chaos Mesh, game days</td>
</tr>
</tbody></table>
<p>None of these ideas is new on its own. What makes Netflix strong is that they use <strong>all of them together</strong>, and they practise failure as a normal part of work.</p>
<hr />
<h2>8. How to use this in a system design interview</h2>
<p>A very common interview question is: <strong>"Design a video streaming platform like Netflix."</strong></p>
<p>Here is a simple structure you can follow:</p>
<ol>
<li><p><strong>Clarify.</strong> How many users? Which regions? Live or on-demand? Which devices? What matters most: start time, quality, cost?</p>
</li>
<li><p><strong>Split control plane and data plane.</strong> APIs (login, catalogue, recommendations, playback decisions) on the cloud. Video delivery on a CDN. Say this early. It shows you understand the problem.</p>
</li>
<li><p><strong>Content pipeline.</strong> Upload, encode into many bitrates and formats, store in object storage (like S3), push popular content to the CDN before peak hours.</p>
</li>
<li><p><strong>Playback flow.</strong> Client asks the playback service, which returns a ranked list of CDN servers. The client uses adaptive bitrate streaming, so quality goes down instead of stopping when the network is slow.</p>
</li>
<li><p><strong>Scale and reliability.</strong> Stateless microservices with auto-scaling. Caching for hot data. Multi-AZ always, multi-region for critical services. Timeouts, retries and circuit breakers between services.</p>
</li>
<li><p><strong>Prove it.</strong> Monitoring on key metrics (stream starts per second is a great business metric), alerts, and chaos testing.</p>
</li>
<li><p><strong>Trade-offs.</strong> Own CDN vs rented CDN. Active-active multi-region (more cost, faster recovery) vs active-passive (cheaper, slower recovery). Strong vs eventual consistency for things like viewing history.</p>
</li>
</ol>
<p><strong>Senior tip:</strong> interviewers listen for trade-offs and for real experience. If you can, add one line from your own work, for example: "In my last project we put static content behind a CDN and cut origin load by X%." Numbers and your role make the answer believable.</p>
<hr />
<h2>9. Key takeaways</h2>
<ul>
<li><p>Remove single points of failure. Scale out, not just up.</p>
</li>
<li><p>Split the brain (control plane) from the heavy data (data plane).</p>
</li>
<li><p>Put content close to users with a CDN.</p>
</li>
<li><p>Keep services small and stateless, with timeouts and fallbacks.</p>
</li>
<li><p>Cache what is read often.</p>
</li>
<li><p>Practise failure on purpose, starting small.</p>
</li>
</ul>
<p>If this helped, save it, share it with a friend who is preparing for interviews, and follow <strong>@cloudtechbydevang</strong> for one cloud and DevOps topic a day.</p>
<hr />
<h3>Sources</h3>
<ul>
<li><p>Netflix Q4 2025 shareholder letter (Jan 2026): "crossed the 325M paid memberships milestone". <a href="https://ir.netflix.net/files/doc%5C_financials/2025/q4/FINAL-Q4-25-Shareholder-Letter.pdf">https://ir.netflix.net/files/doc\_financials/2025/q4/FINAL-Q4-25-Shareholder-Letter.pdf</a> · CNBC coverage: <a href="https://www.cnbc.com/2026/01/20/netflix-nflx-earnings-q4-2025.html">https://www.cnbc.com/2026/01/20/netflix-nflx-earnings-q4-2025.html</a></p>
</li>
<li><p>Variety (Apr 2024), Netflix stops quarterly subscriber reporting from Q1 2025: <a href="https://variety.com/2024/tv/news/netflix-stop-reporting-subscriber-numbers-starting-2025-1235975341/">https://variety.com/2024/tv/news/netflix-stop-reporting-subscriber-numbers-starting-2025-1235975341/</a></p>
</li>
<li><p>Netflix, "Completing the Netflix Cloud Migration" (Feb 2016): <a href="https://about.netflix.com/news/completing-the-netflix-cloud-migration">https://about.netflix.com/news/completing-the-netflix-cloud-migration</a></p>
</li>
<li><p>Netflix, "Open Connect: Celebrating a Decade" (Dec 2022): <a href="https://about.netflix.com/news/open-connect-celebrating-a-decade-of-smooth-and-efficient-streaming">https://about.netflix.com/news/open-connect-celebrating-a-decade-of-smooth-and-efficient-streaming</a></p>
</li>
<li><p>Open Connect overview and partner docs: <a href="https://openconnect.netflix.com/en">https://openconnect.netflix.com/en</a> · <a href="https://openconnect.netflix.com/Open-Connect-Overview.pdf">https://openconnect.netflix.com/Open-Connect-Overview.pdf</a></p>
</li>
<li><p>Netflix Tech Blog, "The Netflix Simian Army" (Jul 2011): <a href="https://netflixtechblog.com/the-netflix-simian-army-16e57fbab116">https://netflixtechblog.com/the-netflix-simian-army-16e57fbab116</a></p>
</li>
<li><p>Ars Technica, Chaos Monkey open-sourced (Jul 2012): <a href="https://arstechnica.com/information-technology/2012/07/netflix-attacks-own-network-with-chaos-monkey-and-now-you-can-too/">https://arstechnica.com/information-technology/2012/07/netflix-attacks-own-network-with-chaos-monkey-and-now-you-can-too/</a></p>
</li>
<li><p>Netflix Tech Blog, "Open Sourcing Zuul 2": <a href="https://netflixtechblog.com/open-sourcing-zuul-2-82ea476cb2b3">https://netflixtechblog.com/open-sourcing-zuul-2-82ea476cb2b3</a></p>
</li>
<li><p>Netflix Tech Blog, "Introducing Hystrix for Resilience Engineering": <a href="https://netflixtechblog.com/introducing-hystrix-for-resilience-engineering-13531c1ab362">https://netflixtechblog.com/introducing-hystrix-for-resilience-engineering-13531c1ab362</a></p>
</li>
<li><p>Netflix Tech Blog, "Netflix OSS and Spring Boot — Coming Full Circle": <a href="https://netflixtechblog.com/netflix-oss-and-spring-boot-coming-full-circle-4855947713a0">https://netflixtechblog.com/netflix-oss-and-spring-boot-coming-full-circle-4855947713a0</a></p>
</li>
<li><p>Netflix Tech Blog, "Caching for a Global Netflix" (EVCache): <a href="https://netflixtechblog.com/caching-for-a-global-netflix-7bcc457012f1">https://netflixtechblog.com/caching-for-a-global-netflix-7bcc457012f1</a></p>
</li>
<li><p>Netflix Tech Blog, "Project Nimble: Region Evacuation Reimagined": <a href="https://netflixtechblog.com/project-nimble-region-evacuation-reimagined-d0d0568254d4">https://netflixtechblog.com/project-nimble-region-evacuation-reimagined-d0d0568254d4</a></p>
</li>
</ul>
<p><strong>Preparing for a Cloud, DevOps or SRE interview?</strong> Book a <strong>Free 15-min Cloud &amp; DevOps Interview Guidance Call</strong> with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 <a href="https://topmate.io/cloudtechbydevang/2358399">https://topmate.io/cloudtechbydevang/2358399</a></p>
]]></content:encoded></item></channel></rss>