Fundamentals
The response codes for HTTP are divided into five categories:
- Informational (100-199)
- Success (200-299)
- Redirection (300-399)
- Client Error (400-499)
- Server Error (500-599)
These codes are defined in RFC 9110. To save you from reading the entire document (which is about 200 pages), here is a summary of the most common ones.
Top 5 common ways to improve API performance.
Result Pagination:
This method is used to optimize large result sets by streaming them back to the client, enhancing service responsiveness and user experience.
Asynchronous Logging:
This approach involves sending logs to a lock-free buffer and returning immediately, rather than dealing with the disk on every call. Logs are periodically flushed to the disk, significantly reducing I/O overhead.
Data Caching:
Frequently accessed data can be stored in a cache to speed up retrieval. Clients check the cache before querying the database, with data storage solutions like Redis offering faster access due to in-memory storage.
Payload Compression:
To reduce data transmission time, requests and responses can be compressed (e.g., using gzip), making the upload and download processes quicker.
Connection Pooling:
This technique involves using a pool of open connections to manage database interaction, which reduces the overhead associated with opening and closing connections each time data needs to be loaded. The pool manages the lifecycle of connections for efficient resource use.
Scaling a system is an iterative process. Iterating on what we have learned in this chapter could get us far. More fine-tuning and new strategies are needed to scale beyond millions of users. For example, you might need to optimize your system and decouple the system to even smaller services. All the techniques learned in this chapter should provide a good foundation to tackle new challenges. To conclude this chapter, we provide a summary of how we scale our system to support millions of users:
-
Keep web tier stateless
-
Build redundancy at every tier
-
Cache data as much as you can
-
Support multiple data centers
-
Host static assets in CDN
-
Scale your data tier by sharding
-
Split tiers into individual services
-
Monitor your system and use automation tools
The cache tier is a temporary data store layer, much faster than the database. The benefits of having a separate cache tier include better system performance, ability to reduce database workloads, and the ability to scale the cache tier independently.
- Cache Types (In-memory caching, Distributed caching, Client-side caching)
- Cache Strategies (Cache-Aside, Write-Through, Write-Behind, Read-Through)
- Measuring Cache Effectiveness (Calculate the cache hit rate, Analyse cache eviction rate, Monitor data consistency, Determine the right cache expiration time)
- Canary Deployments In software engineering, canary deployment is the practice of making staged releases. We roll out a software update to a small part of the users first, so they may test it and provide feedback. Once the change is accepted, the update is rolled out to the rest of the users.
Benefits of Canary Deployments Why go to the trouble of implementing a canary strategy? The benefits are many:
A/B testing: we can use the canary to do A/B testing. In other words, we present two alternatives to the users and see which gets better reception.
Capacity test: it’s impossible to test the capacity of a large production environment. With canary deployments, capacity tests are built-in. Any performance issues we have in our system will begin to crop up as we slowly migrate the users to the canary.
Feedback: we get invaluable input from real users.
No cold-starts: new systems can take a while to start up. Canary deployments slowly build up momentum to prevent cold-start slowness.
No downtime: like blue-green deployments, a canary deployment doesn’t generate downtime.
Easy rollback: if something goes wrong, we can easily roll back to the previous version.
