Prowid Photos got into a roundup on a popular blog. On Friday evening the load grew tenfold: instead of the usual 350 RPS at the peak — 3,500. The only server hit 100% CPU, pages take eight seconds to open, and then 502 errors begin. What now? There are two paths: make the server more powerful or add more servers. The choice affects not only the hosting bill but also how you must write the application. In this lesson we look at the limits of a single server, vertical and horizontal scaling, the main requirement for your code — stateless — and autoscaling.
One server and its limits
Any server runs into one of its resources: CPU, memory, disk, network or the number of database connections. A Node.js process runs JavaScript in a single thread, so one process uses one CPU core (how to use all cores is covered in the lesson "Scaling: The cluster Module and Stateless Apps").
The danger is that a server does not break gradually. A simplified queueing model says: response time ≈ processing time ÷ (1 − utilization). At 50% utilization a request takes twice as long as on an idle server, at 80% five times, at 90% ten times, at 95% twenty times. Real numbers differ, but the "hockey stick" is real — that is why servers are not loaded above 60–70%.
💡 Before scaling, measure and find the bottleneck (
top,docker stats, metrics). It is often cheaper to add a database index or a cache than to buy a server.
Vertical and horizontal scaling
Vertical scaling (scale up) means making one server more powerful: more cores, more memory, a faster disk. It is simple and you do not have to change the code. But the method has a ceiling: the biggest machine a cloud offers is limited (hundreds of cores and terabytes of memory), and the price grows faster than the power. On top of that there is still one server — if it dies, the whole site dies — and resizing a machine usually means restarting it.
Horizontal scaling (scale out) means putting more servers side by side and spreading requests between them (the next lesson is about the balancer). There is almost no ceiling, losing one server is not a disaster, and you can grow bit by bit. The price is complexity: you need a balancer, careful deployments, monitoring, and the application must be stateless.
| Vertical (scale up) | Horizontal (scale out) | |
|---|---|---|
| What we do | one more powerful server | more servers |
| Ceiling | yes: the biggest machine | almost none |
| A server fails | everything is down | the others keep working |
| Application code | does not change | must be stateless |
| Complexity | low | higher: balancer, deployment, state |
A sensible order is: measure and optimize, then grow vertically (the simplest step), but write the code so that it can be multiplied. How many instances do you need? Take the peak, divide it by the load you can safely give to one instance (≈ 70% of its measured maximum) and add one spare — instances are needed not only for power but also to survive the failure of one of them (remember: two servers at 99% give 99.99%):
