When a request reaches your backend, four questions decide what happens next: what machine runs the code, where the data is kept, what work can wait until later, and whether the server remembers anything between requests. This post covers the four core backend concepts that answer them. You will start with Servers — the machines and programs that run your application logic. Then you will learn about Storage — the difference between databases for structured data and object storage for files and media. Next comes Background Processing — doing slow work outside the request path so users never wait. Finally you will understand Stateless vs Stateful — the single most important property that decides how easily your system scales. Each topic is explained with real-world analogies, step-by-step examples, and clear diagrams — starting from zero.

Every backend begins with a simple physical fact: your code has to run somewhere. That "somewhere" is a server. But "server" means two things at once — a program that listens for requests, and the machine it runs on. In this section we will separate those meanings, learn the crucial difference between a web server and an application server, see how one server juggles thousands of requests at the same time, and understand how servers are packaged today — from bare metal to containers to serverless.

Imagine it's 7 PM and thousands of people open your online store shopeasy.com at once. Every tap — browse products, search, add to cart, check out — becomes a request that must land on a machine somewhere running your code. That machine, and the program on it that answers the request, is a server.

The word "server" is used two ways, and keeping them straight is the first step:

• Server (the program) — software that listens on a network port and responds to requests. Nginx, a Node.js app, and PostgreSQL are all "servers" in this sense.
• Server (the machine) — the physical or virtual computer the program runs on. One machine can run many server programs at once.

There are also two very different jobs a server can do. A web server hands back files that already exist — HTML, CSS, images. An application server runs your business logic — checking prices, processing the order, talking to the database. Most real systems use both, and telling them apart is the heart of this section.

A single server can only do so much. A well-tuned machine might handle a few thousand simple requests per second, but heavy work — image processing, complex queries — drops that number fast. That is why large systems run fleets of servers: Google operates millions of servers across its data centres, Netflix runs on tens of thousands of instances, and even a mid-sized startup at scale runs dozens. Almost every scaling decision in system design starts with "how many servers, of what kind, doing what?"

• If you don't know a server's capacity limit, you can't decide when to add more — the core of scalability.
• Separating web serving from application logic lets you scale each independently — cheap file servers for static content, powerful compute for logic.
• Choosing how servers are hosted (VM, container, serverless) decides your cost, speed of deployment, and operational burden.
• Understanding how a server handles concurrency explains why one crashed process can drop thousands of users at once.

Why system design cares: "Add more servers" is the answer to a huge share of scaling problems — but only if the server is stateless (covered in Sub-topic 4). Servers and statelessness together are what make horizontal scaling possible.

Think of a large hotel. At the front desk, the concierge hands you things that are already prepared — a city map, a printed menu, a brochure. No custom work; instant. That is the web server: it returns files that already exist. But when you want a tailored 3-day itinerary with restaurant bookings, the concierge sends your request to a specialist in the back office who does custom work for you. That specialist is the application server.

The key idea: pre-made things come back instantly from the front desk (web server), while custom work is routed to a specialist (application server). Real systems put a fast, simple front desk in front of the expensive specialists — exactly how backends are structured.

Let us break servers down into four pieces: what "running a server" actually means, how web and application servers split the work, how one server handles many requests at once, and how servers are packaged today.

A server is simply a program that starts up, binds to a port, and waits. It sits in a loop: accept a connection, read the request, produce a response, send it back, repeat. When people say "the server is listening on port 443," they mean exactly this loop is running and waiting for HTTPS connections. Kill that process and the machine is still on — but nothing answers.

These two do different jobs, and production systems usually chain them. The web server is the fast front door; the application server is where your code runs.

A typical flow: the web server (often acting as a reverse proxy) receives every request. If it's for a static file like /logo.png, it returns it directly. If it's for something dynamic like /checkout, it forwards the request to the application server, which runs the logic and returns the result.

A server almost never handles one request at a time — it serves thousands concurrently. There are three common models for how:

• Process-per-request / process pool — the server runs several copies (processes), each handling one request at a time. Robust but memory-heavy.
• Thread-per-request — one process spawns many lightweight threads, one per request. Cheaper than processes; used by Java/Spring.
• Event loop (async) — a single thread juggles thousands of requests by never waiting idle — while one request waits on the database, it works on others. Used by Node.js and Nginx.

Why this matters: a server's model sets its capacity. An event-loop server handles huge numbers of slow, waiting connections cheaply; a thread-per-request server can be simpler but runs out of memory sooner. When you hear "this server maxes out at 10,000 concurrent connections," this is why.

The same server program can be deployed in very different wrappers, trading control for convenience:

• Bare metal — your program runs directly on a physical machine. Maximum performance, maximum operational work.
• Virtual machine (VM) — a software computer sharing physical hardware with others. The classic cloud unit (EC2).
• Container — a lightweight, isolated package (Docker) that starts in seconds and packs many per machine.
• Serverless — you upload a function; the cloud runs it only when called and manages the servers entirely (AWS Lambda).

Servers vary along two axes: the role they play and how they're hosted.

"Serverless" still uses servers. The name means you don't manage them — the cloud provider does. Your code still runs on real machines; they're just invisible to you and scale automatically.

The diagram below shows the classic backend layout: a web server receives every request, serves static files itself, and forwards dynamic requests to the application server, which runs the logic and talks to the database.

Reading the diagram: the web server answers static requests instantly on its own, and only passes dynamic work to the application server. That split keeps cheap, fast file-serving separate from expensive logic — and lets you scale each layer on its own.

Two decisions come up constantly: whether to separate web and application servers, and how to host them. Here's a practical guide.

For how to host, match the wrapper to your needs:

Rule of thumb: Start with a VM (or a container) running your app, and put a web server / reverse proxy in front once you serve real static content or need TLS and routing. Reach for serverless when workloads are event-driven or bursty and you'd rather not manage capacity at all.

When a shopper opens a product page, a single page load fans out across several server types, each doing what it's best at:

New terms above? The CDN in step ③ gets its own dedicated post in Phase 2. For now, just notice that static images come from a separate, closer server so the application server never wastes effort serving files — it focuses only on logic.

Splitting your backend into specialised server tiers (web, application, database) is powerful, but it isn't free:

• "Server" = program + machine — the software that listens, and the host it runs on.
• Web server vs application server — one serves pre-made files fast; the other runs your logic and hits the database.
• One server serves many requests — via processes, threads, or an event loop; the model sets its capacity.
• Servers come in wrappers — bare metal, VM, container, or serverless, trading control for convenience.
• Design for many servers — never a single one; specialise the tiers and scale each independently.

📸 Challenge: Plan the servers for a photo-sharing app

You are designing a photo-sharing app like a small Instagram. Think through the following:

• Which server types would you use, and what does each one do?
• Where should the photo files be served from — and why not the application server?
• A photo upload needs thumbnail generation, which is slow. Which server handles the upload request, and is generating the thumbnail the right job for it?
• Traffic spikes 10× every evening. How do your server choices help you handle that?

👁️ Show Answer

Server types:

• 🌐 Web server / reverse proxy (Nginx) — terminates HTTPS, serves the app shell and static assets, routes API calls
• ⚙️ Application servers — handle login, feed building, likes, comments (run several behind a load balancer)
• 🗄️ Database server — stores users, posts, and metadata
• 🌍 Object storage + CDN — stores and delivers the actual photo files

Serving photos: photo files live in object storage and are delivered by a CDN, not the application server — serving large files from your compute tier wastes CPU and memory and slows down real logic. (Object storage is Sub-topic 2.)

Slow thumbnail generation: the application server accepts the upload and returns quickly, but it should not generate the thumbnail inline — that would block the request. It hands the job to a background worker (Sub-topic 3) and responds "upload received" immediately.

Evening 10× spike: because the application servers are stateless (Sub-topic 4), you run them behind a load balancer and simply add more instances during peak — containers or serverless make that scaling fast and automatic — then scale back down overnight.

Every cloud offers the same ladder of options for running server code, from a raw VM you control fully to a function the platform runs for you:

Simplest AWS picture: run your application server on an EC2 instance (or a container on ECS), put it behind a load balancer, and serve static files from S3 + CloudFront. For bursty, event-driven work, replace the always-on server with a Lambda function.

---

Servers run your logic, but the moment a server restarts, everything in its memory is gone. For data to survive, it has to be written somewhere permanent — that is storage. But not all data is alike: an order record and a product photo need completely different homes. In this section you will learn the difference between a database for structured records and object storage for files, meet the three fundamental storage types (block, file, object), and see the standard pattern real systems use to store big files without slowing down their database.

Back to our online store, ShopEasy. It needs to store two very different kinds of data. First, structured records: users, orders, prices, stock counts — neat rows of fields you constantly search and update ("show me all orders over $50 from last week"). Second, big files: product photos, invoice PDFs, review images — large blobs you save once and hand back whole.

These two needs pull in opposite directions, so they use different storage. Structured records live in a database — built to query, filter, and update fields fast. Files live in object storage — built to hold huge amounts of data cheaply and serve each file by its address. Putting a 5 MB photo inside a database row, or trying to run a "find all orders" query against a pile of files, are both mistakes this section will help you avoid.

Underneath both is one non-negotiable idea: persistence. Storage must keep data safely on disk so it survives restarts, crashes, and deployments — unlike a server's memory, which vanishes the moment the process stops.

Storage is where your users' data lives forever, so getting it right is critical. The scale is staggering: Amazon S3 holds well over 100 trillion objects; Netflix stores its entire media library as files in object storage; Dropbox manages exabytes of user files. Meanwhile a company's structured data — orders, accounts, transactions — is its lifeblood, and losing it can end a business.

• Choosing the wrong storage makes systems slow and expensive — a database choking on huge files, or files you can't search.
• Durability (data never lost) is a hard requirement; storage systems achieve it by keeping multiple copies. S3 advertises 99.999999999% ("eleven nines") durability.
• Cost differs by orders of magnitude — object storage is far cheaper per gigabyte than database storage, so files belong there.
• Nearly every system design question includes "where does the data go?" — and the answer is almost always "structured data in a database, files in object storage."

Why system design cares: the classic pattern — store the file in object storage, store a pointer (URL) plus metadata in the database — appears in the design of almost every media-heavy system (Instagram, YouTube, Dropbox). Recognising it instantly is a mark of solid fundamentals.

Think of a company's back office. It has a filing cabinet with labelled, indexed folders — you can instantly find "all invoices from March over $500" because everything is structured and cross-referenced. That is a database. It also has a warehouse full of numbered shelves holding big sealed boxes. You can't search inside a box, but if you know its shelf number you can fetch the whole box in seconds, and the warehouse holds far more, far more cheaply. That is object storage.

The everyday trick the office uses is the key one for system design: keep a small indexed note in the cabinet ("Contract for Acme → Box #4471") and the bulky document itself in the warehouse. Databases store the searchable metadata and a pointer; object storage holds the heavy file.

Let us break storage into four pieces: what makes data persistent and durable, how databases hold structured records, how object storage holds files, and the three fundamental storage types you should know.

A server's memory (RAM) is fast but volatile — cut the power and it's gone. Storage writes data to disk (SSD/HDD), which survives restarts. But a single disk can fail, so durable storage keeps multiple copies (replication) across machines or data centres. Persistence means "survives a restart"; durability means "survives a hardware failure." Production systems need both.

A database organises data into a defined shape — for example rows and columns — so you can query by any field, filter, sort, and update precisely. When ShopEasy asks "all pending orders for user 128, newest first," the database answers efficiently using indexes. That power is why structured, frequently-queried data belongs in a database — not in files.

🔗 SQL vs NoSQL: databases come in families — relational (SQL) and non-relational (NoSQL). The differences and when to choose each are a big topic covered fully in blog_23 (Database Basics). Here, just hold the idea: "structured, queryable data → a database."

Object storage keeps each file as an object: the raw bytes, a unique key, and some metadata, sitting inside a bucket. You PUT a file to store it and GET it by key to retrieve it — often via a plain HTTPS URL. You do not edit an object in place; you replace the whole thing. In return you get near-unlimited capacity, very low cost, and built-in durability. The database then just stores the object's URL plus searchable metadata:

The heavy image bytes live in object storage at that URL; the database holds only the pointer and the fields it needs to search on. Small and fast in the database, big and cheap in object storage.

At the infrastructure level there are three fundamental ways to store bytes. Knowing the difference is a common gap:

Storage splits along two lines: the infrastructure storage type (how bytes are stored) and the database family (how structured data is organised).

Coming in Phase 1 & 2: the database families above are previewed here only to place them in the storage picture. blog_23 (Database Basics) covers SQL vs NoSQL in depth, and Caching gets its own dedicated post. For now: structured data → database; files → object storage.

The diagram below shows the standard split: the application server writes small structured records to the database and big files to object storage, and the database keeps the file's URL so the two stay linked.

Reading the diagram: the app server saves the searchable record (plus the file's URL) to the database and pushes the actual bytes to object storage. Later the client fetches the file directly from object storage by its URL — the database and app server never handle the heavy bytes.

Picking the right home for each kind of data is one of the most common design decisions. This table maps the need to the storage:

Rule of thumb: structured data → database; big files → object storage; a fast disk for a database → block storage; a shared filesystem → file storage. When a file is large, never put the bytes in the database — store the file in object storage and keep just the URL in the database.

When a seller adds a new product with a photo, the request splits cleanly across the two storage systems — the exact pattern used by Instagram, YouTube, and every media-heavy app:

Notice the division of labour: the database stays small and fast because it never holds image bytes — only the URL. Object storage holds the heavy file cheaply and serves it directly to browsers. This "metadata in the DB, file in object storage" split is one of the most reused patterns in all of system design.

The database-plus-object-storage split is the default for good reasons, but it has costs worth knowing:

• Storage makes data survive — memory is volatile; storage persists to disk and stays durable via replicas.
• Database vs object storage — structured, queryable records go in a database; large files go in object storage.
• Three storage types — block (raw disk for one server), file (shared filesystem), object (files by key at scale).
• The reused pattern — store the file in object storage, store the URL + metadata in the database.
• Never put big files in the DB, on a local disk, or stream them through the app server — use object storage and a CDN.

🎥 Challenge: Design storage for a video platform

You are designing storage for a small video platform like a mini-YouTube. Think through the following:

• Where should the video files and thumbnails go, and where should video titles, view counts, and comments go?
• How does a video's database record stay linked to its file?
• When a viewer plays a video, should the bytes flow through your app server — why or why not?
• What happens to the stored file when a user deletes their video?

👁️ Show Answer

Where each thing goes:

• 🪣 Object storage — the video files and thumbnails (large blobs, cheap and durable)
• 🗄️ Database — title, description, uploader, view count, and comments (structured, queryable), plus the video's object-storage URL

Linking: the video's DB row stores the object key/URL (e.g. videos/9f3/master.mp4). The metadata is searchable in the database; the bytes live in object storage.

Playback: the bytes should not flow through the app server — that would waste enormous bandwidth and CPU. The player streams directly from object storage, fronted by a CDN so it's delivered from a location near the viewer. (CDN is a Phase 2 topic.)

Deletion: deleting the video must remove both the database record and the object in storage. If you only delete the row, the file becomes an orphan that silently costs money forever — a common real-world bug.

Each storage type maps to a well-known managed service on every cloud:

Simplest AWS picture: keep structured records in RDS (or DynamoDB), store files in S3, and put the DB's disk on EBS. When a user uploads a photo, the bytes go to S3 and the returned URL goes into RDS — the pattern behind almost every media app.

---

Not every piece of work needs to finish before you reply to the user. Sending a confirmation email, generating an invoice PDF, or encoding a video can take seconds or minutes — far too long to make someone stare at a spinner. Background processing is the art of doing slow or deferrable work outside the request path, so the user gets an instant response while the heavy lifting happens later. In this section you will learn the synchronous-vs-asynchronous split, the queue-and-worker pattern that powers almost all background work, the different kinds of background jobs, and how to make them reliable.

Picture a shopper on ShopEasy clicking Place Order. Behind that one click, a lot needs to happen: validate the order, reserve stock, charge the card, send a confirmation email, generate an invoice PDF, update sales analytics, and notify the warehouse. If the app server did all of that before responding, the shopper would watch a spinner for ten seconds — and if the email service were slow, the whole checkout would hang.

The fix is to split the work. The few things the user truly needs confirmed — order validated, stock reserved, payment taken — happen synchronously, and the server replies "Order confirmed!" in a fraction of a second. Everything else — email, PDF, analytics, warehouse notification — is handed off to run in the background, moments later, without the user waiting.

Background processing means running work outside the request-response cycle. The request finishes fast; the deferred work is dropped into a queue and picked up by separate worker processes that chew through it on their own time.

Background processing is what keeps large systems fast and resilient. YouTube accepts your video upload in seconds but transcodes it into every resolution in the background over minutes. Every major service queues its emails rather than sending them inline. When a task takes anywhere from a few hundred milliseconds (sending an email) to several minutes (encoding video), forcing the user to wait for it is simply not an option.

• Fast responses — users get an instant reply; slow work never blocks the request.
• Reliability — a queued job that fails can be retried automatically; an inline failure would just break the user's request.
• Absorbing spikes — a queue acts as a buffer. If 10,000 orders arrive at once, workers drain the queue steadily instead of the system collapsing.
• Independent scaling — when the backlog grows, add more workers without touching the web tier.

Why system design cares: "Do this in the background with a queue and workers" is one of the most common answers in system design — for emails, notifications, media processing, analytics, and search indexing. Recognising which work belongs in the background is a core skill.

Think of a busy restaurant kitchen. When you order, the waiter doesn't stand at your table cooking — they write a ticket, clip it to the ticket rail, and immediately move on to the next customer. The cooks pull tickets off the rail one by one and prepare dishes at their own pace. If the kitchen gets slammed, tickets simply queue up on the rail; add another cook and they clear faster.

The key move: the waiter never blocks. They take your order, hand it off, and stay free to serve others — exactly how an app server enqueues background work and immediately returns to handling new requests.

Let us break background processing into four pieces: the synchronous-vs-asynchronous decision, the queue-and-worker pattern, the kinds of background work, and what makes it reliable.

Synchronous work happens inside the request, and the user waits for the result — you need it to log in, to see a search result, to confirm payment. Asynchronous work is deferred: the request returns immediately and the work runs later. The question to ask for every task is simple: does the user need this finished before they get a response? If no, it belongs in the background.

Almost all background processing uses the same three-part shape. A producer (your app server) drops a job into a queue; one or more workers pull jobs off the queue and execute them. The queue decouples the two sides — producers and workers don't need to run at the same speed, or even at the same time. A job is just a small message describing the work:

The app server enqueues this in under a millisecond and returns "Order confirmed!" The email worker picks it up moments later, renders the email, and sends it — completely outside the user's request.

• Deferred tasks (on-demand) — triggered by a user action but run after the response: send email, generate PDF, resize an image.
• Scheduled jobs (cron) — run at fixed times: "send the daily digest at 8 AM," "expire old sessions at midnight."
• Recurring / polling workers — run continuously or on an interval: retry failed payments every 10 minutes.
• Batch processing — crunch large data sets periodically: nightly analytics roll-ups, report generation.

Because a worker runs away from the user, failures must be handled automatically. Three mechanisms do the heavy lifting:

• Retries — if a job fails (the email service was down), the worker tries again, usually with increasing delays (backoff).
• Idempotency — since a job might run more than once (retry after a partial success), it must be safe to repeat. Charging a card twice is the classic disaster; an idempotency key prevents it.
• Dead-letter queue — a job that keeps failing after N attempts is moved aside for humans to inspect, instead of blocking the queue forever.

🔗 Queues get their own post: the internals of message queues — delivery guarantees, ordering, at-least-once vs exactly-once, and reliability patterns — are covered in depth in blog_26 (Message Queues & Async). Here we focus on the idea: defer work via a queue and workers.

Background work is triggered in a few distinct ways. Knowing which pattern fits a task is the practical skill.

Simple rule: user-triggered slow work → task queue; time-triggered work → cron; reacting to something that happened → event-driven; heavy periodic number-crunching → batch.

The diagram below shows the fast path and the background path. The app server responds to the user immediately, drops a job on the queue, and separate workers process it afterwards — the user never waits for the slow work.

Reading the diagram: steps ① and ② are the fast synchronous path — the user gets a response right away. Steps ③–⑤ are the background path — the job sits in the queue and is processed by workers independently. Add more workers and the queue drains faster, with zero impact on response time.

The decision is almost always about one question: does the user need the result now? If not, push it to the background.

Rule of thumb: keep the request handler doing the minimum the user needs to see a correct response, and defer everything else. A checkout should reserve stock and take payment synchronously, then queue the email, invoice, analytics, and warehouse notification.

Here is how a single "Place Order" click splits into a fast synchronous part and several background jobs:

Notice: the shopper waited only for steps ② and ③ — the essentials — and got a response in a fraction of a second. Everything in steps ⑤–⑧ happened moments later, in parallel, and could each fail and retry independently without ever affecting the shopper's experience. This is exactly how real checkout systems are built.

Moving work to the background is powerful, but it trades immediacy and simplicity for speed and resilience:

• Background processing defers slow work — it runs outside the request so the user gets a fast response.
• Synchronous vs asynchronous — do only what the user needs now inline; queue the rest.
• Queue + workers — a producer enqueues jobs; workers pull and run them, scaling independently.
• Many trigger types — on-demand task queues, scheduled cron, recurring, batch, event-driven, and streaming.
• Reliability is essential — retries, idempotency, dead-letter queues, and backlog monitoring keep jobs safe.

📨 Challenge: Design background processing for a newsletter app

You run an app that lets creators send a newsletter to up to 1 million subscribers with one click. Think through the following:

• What should happen synchronously when the creator clicks "Send," and what should be deferred?
• How would you structure the 1 million emails as background work?
• The email provider fails on some sends. How do you make sure nobody is skipped and nobody gets the email twice?
• How do you handle a sudden burst of 50 creators all sending at once?

👁️ Show Answer

Synchronous vs deferred: when the creator clicks "Send," the app synchronously validates the newsletter, saves it, and responds "Your newsletter is being sent to 1,000,000 subscribers ✅." The actual sending is entirely background work.

Structuring the million emails: enqueue one job per subscriber (or per small batch, e.g. 500 recipients). A pool of email workers pulls jobs off the queue and sends them. Because it's a queue, the million jobs drain steadily rather than all at once.

No skips, no duplicates: use a durable queue so no job is lost on a crash, and retry failed sends with backoff. Make each job idempotent — record "sent" per (newsletter, subscriber) so a retried job that already succeeded doesn't send a second copy. Jobs that keep failing go to a dead-letter queue.

Burst of 50 creators: the queue absorbs the 50 million jobs as a buffer; you simply autoscale the worker pool based on queue depth to drain faster, then scale back down. The web tier stays fast because it only ever enqueues.

Every cloud provides managed queues, worker compute, and schedulers so you rarely build this plumbing yourself:

Simplest AWS picture: the app server drops a job into SQS → a Lambda (or ECS worker) is triggered to process it → failed messages retry and land in an SQS dead-letter queue. For scheduled work, EventBridge Scheduler fires the job on a cron.

---

This last concept ties the whole post together. Back in Sub-topic 1 we said the answer to most scaling problems is "add more servers" — but that only works if the servers are stateless. Whether a server remembers anything between requests is the single property that decides how easily your system scales. In this section you will learn exactly what "state" means, why stateless services scale effortlessly while stateful ones fight you, and the standard techniques for keeping servers stateless by pushing state somewhere else.

ShopEasy has grown, so it now runs three application servers behind a load balancer that spreads requests across them. A shopper logs in — that request happens to hit Server A, which stores "logged in as Sam" in its own memory. A moment later the shopper adds an item to the cart, but the load balancer sends that request to Server B — which has never heard of Sam and asks them to log in again. The shopper gets randomly logged out. Frustrating, and entirely self-inflicted.

The bug is that Server A kept state in its own memory. That is a stateful design. The fix is to make every server stateless — none of them remembers anything about the shopper between requests. Instead, the identity travels with each request (a token) or lives in a shared store both servers can read. Now it doesn't matter which server handles a request; they're interchangeable.

• Stateful server — keeps data about the client in its own memory between requests. The client must keep hitting the same server.
• Stateless server — keeps nothing between requests; each request carries or fetches everything it needs. Any server can handle any request.

Statelessness is the property that makes modern scaling possible. It is why a service can run on 3 servers today and 300 tomorrow: because any server can serve any request, you just add more behind the load balancer. The entire cloud autoscaling model — AWS Auto Scaling groups, Kubernetes pods scaling up and down — assumes your application tier is stateless. Companies like Netflix and Amazon run tens of thousands of interchangeable, stateless instances for exactly this reason.

• Horizontal scaling — add or remove identical servers freely, because none holds unique client state.
• Load balancing — the balancer can route any request to any server without worrying about "which server has this user's data."
• Fault tolerance — if a stateless server crashes, its requests just go to another one; no user data is lost with it.
• Zero-downtime deploys — replace servers one by one during a release; since they hold no state, nothing is lost.

Why system design cares: "Keep the application tier stateless so it scales horizontally, and push state to a shared store or a token" is one of the most important sentences in all of system design. It connects directly to the REST statelessness you saw in blog_21 — same principle, applied to servers.

Imagine a bank. In the stateful version, you start a transaction with one specific teller who keeps your details in their head — if that teller goes on break, you're stuck; you can only be served by them. In the stateless version, you carry a passbook with all your account information, and the bank's records live in a shared system every teller can access. Now any teller can serve you instantly, and if one leaves, the next continues seamlessly.

The lesson: don't lock the customer to one teller. Give them a passbook and keep records in a shared system, and every teller becomes interchangeable — exactly how stateless servers plus externalized state let you scale.

Let us break this down into four pieces: what "state" actually is, how the stateless approach works, the two ways to cope with state that must exist, and why stateless scales so well.

State is anything a server holds about a specific client that it would need again on a later request. Common culprits that quietly make a server stateful:

• Session data in memory — "user 128 is logged in," stored in a local variable.
• In-memory caches — per-server caches that other servers can't see.
• Uploaded files on local disk — a file saved to /tmp that only this server has.
• In-progress data — a multi-step wizard's partial data kept between requests.

A stateless server makes each request self-contained: everything needed to handle it either travels with the request or is fetched from a shared source. Identity comes from a token the client sends every time; session and cart data live in a shared store (like Redis) or the database. The server itself remembers nothing — so it can be replaced at any moment.

Because the token proves who the caller is on every request, Server A and Server B are equally able to handle it — the random-logout bug disappears.

When state must exist, there are two ways to cope — and one is much better than the other:

Sticky sessions are a crutch: they keep a stateful design limping along, but they undermine the benefits of load balancing. The clean solution is to externalize state to a shared store so servers stay truly stateless.

Once no server holds unique client state, servers become identical, disposable units. The load balancer sprays requests across all of them; you add servers when traffic rises and remove them when it falls; a crashed server is simply replaced. This is horizontal scaling, and it is the direct payoff of statelessness — the "add more servers" promise from Sub-topic 1 finally cashed in.

It helps to see which components are naturally stateless, which are inherently stateful, and where externalized state can live.

The key reframe: "stateless" never means your system has no state — every real system has state. It means the application servers don't hold it. State is pushed to the client (token), a shared cache (Redis), or the database — places built to hold it.

The diagram below shows the stateless design: a load balancer sends requests to any of several identical app servers, none of which stores session data locally — they all read and write it to a shared session store and the database.

Reading the diagram: the load balancer can route a request to any of the three app servers, because none of them holds the session — they all share the same store. Add a fourth server and it works instantly; kill one and the others carry on. That is statelessness enabling horizontal scaling.

The guiding principle: make the tier you need to scale stateless, and confine state to purpose-built stateful systems.

Rule of thumb: keep your application tier stateless and treat every app server as disposable. Store identity in a token, session/hot data in a shared cache (Redis), and durable data in the database. Let the databases and caches be the stateful specialists — never your web tier.

Here is the ShopEasy random-logout bug and its fix, side by side — the same two requests, one stateful design and one stateless:

Notice: the only change was where the state lives — moved out of the server's memory into a token plus a shared Redis store. That one change turns a fragile, unscalable system into one that scales to hundreds of servers and survives crashes. This is the payoff the whole post has been building toward.

Stateless design is the default for scalable systems, but it isn't free — the state has to go somewhere, and that somewhere has its own cost:

• Stateful remembers, stateless forgets — a stateful server keeps client data in its memory; a stateless one keeps nothing between requests.
• Statelessness enables horizontal scaling — interchangeable servers mean you add, remove, and replace them freely.
• Externalize the state — push it to a token (identity), a shared cache like Redis (sessions), or the database (durable data).
• Avoid sticky sessions — they prop up a stateful design and undermine load balancing and failover.
• Keep the app tier stateless — let databases, caches, and queues be the stateful specialists.

🎮 Challenge: Make a stateful app scale

A team built a multiplayer quiz app on a single server. It keeps each player's login, score, and current question in memory. It works for 100 players but falls over under load, and adding a second server logs everyone out. Think through the following:

• Which pieces of in-memory data are making the server stateful?
• Where should each piece move so the server becomes stateless?
• The game uses live WebSocket connections for real-time updates. Why is that harder to make stateless, and what would help?
• Once stateless, how do you handle a sudden jump from 100 to 100,000 players?

👁️ Show Answer

What makes it stateful: login/session, each player's score, and the current-question pointer are all held in the single server's memory — so only that server can serve a given player, and a restart wipes everything.

Where each piece moves:

• 🎟️ Login/identity → a signed token (JWT) the client sends each request
• ⚡ Scores & current question → a shared store like Redis, readable by any server
• 🗃️ Final results & history → the database (durable source of truth)

Now any app server can handle any player's request, so you run several behind a load balancer.

WebSockets are harder: a live connection is inherently pinned to one server (it holds that open socket), which is per-connection state. To scale, put a shared pub/sub layer (e.g. Redis pub/sub) behind the WebSocket servers so a message on one server reaches players connected to another. The connection stays local, but the game state and messaging are shared.

100 → 100,000 players: because the app tier is now stateless, autoscale it behind the load balancer — add servers as players surge, remove them afterwards — and scale Redis and the database (the stateful specialists) with their own techniques covered later in the series.

Staying stateless relies on a few cloud services: a place to keep externalized session state, a load balancer, and autoscaling for the stateless tier:

Simplest AWS picture: put stateless app servers in an Auto Scaling group behind an Elastic Load Balancer, store sessions in ElastiCache (Redis), and carry identity in a token. Traffic rises → the group adds servers; a server dies → the load balancer routes around it. Stateless servers plus externalized state — the foundation of scalable backends.

---