{"id":492996,"date":"2026-08-31T17:43:21","date_gmt":"2026-08-31T17:43:21","guid":{"rendered":"https:\/\/savepearlharbor.com\/?p=492996"},"modified":"-0001-11-30T00:00:00","modified_gmt":"2026-08-31T17:43:21","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=492996","title":{"rendered":"I Revisited My 7x Go API Benchmark: What ApacheBench Was Actually Measuring"},"content":{"rendered":"
\n

My Go API benchmark once showed an almost 7x throughput improvement after I moved a read\u2011heavy endpoint from PostgreSQL to a local in\u2011memory cache. The result looked obvious: PostgreSQL was the bottleneck, the cache removed it, and the API became much faster. Later I realized that this interpretation was too simple. ApacheBench had measured an entire request path consisting of HTTP handling, connection management, pgxpool waiting, SQL execution, decoding, serialization, and response writing. I decided to decompose that path and understand what the 7x result actually meant.<\/p>\n

<\/figure>\n

The number was correct, but my explanation was too simple<\/h3>\n

The original setup was deliberately small. A Go service exposed an endpoint that returned a user by ID. Without caching, every request reached PostgreSQL. With caching enabled, frequently requested objects were returned directly from process memory.<\/p>\n

The performance difference was large. ApacheBench showed enough separation between the two implementations that the result could reasonably be summarized as an approximately sevenfold throughput improvement.<\/p>\n

At first I treated that as evidence that PostgreSQL itself was the bottleneck.<\/p>\n

That sentence now bothers me.<\/p>\n

A PostgreSQL\u2011backed HTTP request is not a SQL query. It is a chain of operations, and the query is only one of them.<\/p>\n

The database version of the endpoint effectively followed this path:<\/p>\n

HTTP request    \u2193router    \u2193pgxpool acquisition    \u2193PostgreSQL protocol round trip    \u2193index lookup and SQL execution    \u2193row transfer    \u2193Scan into Go values    \u2193JSON serialization    \u2193HTTP response<\/code>
<\/a><\/div><\/pre>\n
The cache hit was much shorter:<\/p>\n
HTTP request    \u2193router    \u2193cache synchronization    \u2193map lookup    \u2193HTTP response<\/code>
<\/a><\/div><\/pre>\n
This was the first thing I had underestimated.<\/p>\n
I had not replaced PostgreSQL with a map.<\/p>\n
I had removed an entire section of the request pipeline.<\/p>\n
In my implementation, the cache could even contain the already serialized representation. In that case a cache hit avoided not only database access, but also scanning values into a struct and running JSON serialization again.<\/p>\n
Calling the complete difference PostgreSQL overhead therefore hides too much information.<\/p>\n
A more useful simplified model is:<\/p>\n
Tdb_request =    Thttp  + Tpool  + Troundtrip  + Tsql  + Tdecode  + Tserialize  + Twrite<\/code>
<\/a><\/div><\/pre>\n
For a cache hit:<\/p>\n
Tcache_request =    Thttp  + Tlock  + Tlookup  + Twrite<\/code>
<\/a><\/div><\/pre>\n
The benchmark compared these two sums.<\/p>\n
It never isolated one SQL operation and one memory lookup.<\/p>\n
That sounds obvious once written down. It was much less obvious while looking at a single requests\u2011per\u2011second number.<\/p>\n
ApacheBench is part of the experiment<\/h3>\n
The second problem was my mental model of the load generator.<\/p>\n
I had implicitly treated ApacheBench as a stopwatch sitting outside the system.<\/p>\n
It is not.<\/p>\n
A benchmark client creates a particular workload, and the workload affects the result.<\/p>\n
Take these two commands:<\/p>\n
ab -n 100000 -c 64 http:\/\/127.0.0.1:8080\/users\/42<\/code>
<\/a><\/div><\/pre>\n
and:<\/p>\n
ab -k -n 100000 -c 64 http:\/\/127.0.0.1:8080\/users\/42<\/code>
<\/a><\/div><\/pre>\n
The second enables persistent HTTP connections.<\/p>\n
From the perspective of the application code, nothing changed. The handler is identical. PostgreSQL is identical. The cache is identical.<\/p>\n
The transport behavior is not.<\/p>\n
Without connection reuse, the benchmark creates much more connection\u2011management work. Even on localhost<\/a>, creating and closing sockets is not free. The kernel maintains TCP state, the server accepts connections, file descriptors are used, and additional system calls appear around requests.<\/p>\n
This matters particularly when one endpoint becomes extremely cheap.<\/p>\n
Imagine that the PostgreSQL path requires several layers of work while the cached path is already close to the minimum cost of serving an HTTP response.<\/p>\n
Once the cache path approaches the limit of the HTTP stack, making the cache itself twice as fast does not make the API twice as fast.<\/p>\n
Amdahl’s law appears in a very practical form.<\/p>\n
If only a small fraction of the remaining request time belongs to the cache lookup, optimizing that lookup has very little effect on the complete request.<\/p>\n
This also means that connection handling can alter the ratio between the database and cache versions without changing either implementation.<\/p>\n
The speedup is not a constant property of the architecture.<\/p>\n
It is a property of the architecture under a particular workload.<\/p>\n
pgxpool can quietly become the real queue<\/h3>\n
There is another layer between Go and PostgreSQL that is easy to ignore when looking only at SQL execution time.<\/p>\n
The connection pool.<\/p>\n
Suppose the benchmark runs with a concurrency of 64\u00a0while pgxpool is limited to eight connections.<\/p>\n
Sixty\u2011four requests can be active at the HTTP layer.<\/p>\n
Only eight can use PostgreSQL at the same time.<\/p>\n
The remaining requests have to wait.<\/p>\n
From the outside, this waiting appears as API latency.<\/p>\n
PostgreSQL may still execute each query extremely quickly.<\/p>\n
This creates a confusing but perfectly valid pair of observations:<\/p>\n
SQL execution: fastHTTP request: slow<\/code>
<\/a><\/div><\/pre>\n
The missing time can exist before PostgreSQL even sees the query.<\/p>\n
pgxpool exposes enough information to detect this:<\/p>\n
stat := pool.Stat()log.Printf(    \"total=%d acquired=%d idle=%d acquire_count=%d acquire_duration=%s\",    stat.TotalConns(),    stat.AcquiredConns(),    stat.IdleConns(),    stat.AcquireCount(),    stat.AcquireDuration(),)<\/code>
<\/a><\/div><\/pre>\n
The field I care about most here is AcquireDuration<\/code>.<\/p>\n
If it grows sharply as concurrency increases, requests are spending noticeable time waiting for connections.<\/p>\n
At that point a benchmark is partly measuring queueing inside the application.<\/p>\n
The obvious response is to increase MaxConns<\/code>.<\/p>\n
That helps only until the bottleneck moves again.<\/p>\n
With a very small pool, requests wait inside Go.<\/p>\n
With a larger pool, PostgreSQL receives more parallel work.<\/p>\n
Increase it far enough and more concurrency can start producing contention rather than useful throughput. CPU scheduling, shared structures, memory pressure and database backend activity become increasingly important.<\/p>\n
The queue did not disappear.<\/p>\n
It moved.<\/p>\n
That idea changed how I interpret connection\u2011pool tuning.<\/p>\n
There is no universally correct pool size that makes PostgreSQL infinitely parallel.<\/p>\n
A pool is a concurrency control mechanism.<\/p>\n
Too small and the application queues excessively.<\/p>\n
Too large and the database may receive more simultaneous work than it can process efficiently.<\/p>\n
A benchmark that ignores this can easily attribute pool behavior to PostgreSQL performance.<\/p>\n
I should have measured the empty HTTP path first<\/h3>\n
A nearly empty handler is one of the simplest controls I can add:<\/p>\n
func baseline(w http.ResponseWriter, r *http.Request) {    w.WriteHeader(http.StatusNoContent)}<\/code>
<\/a><\/div><\/pre>\n
It does almost nothing.<\/p>\n
That is exactly why it is useful.<\/p>\n
It establishes a rough lower bound for the cost of serving requests through the current HTTP stack.<\/p>\n
Suppose a cached endpoint approaches the throughput of this empty handler.<\/p>\n
At that point, continuing to optimize the map implementation is unlikely to produce a proportional API improvement.<\/p>\n
The request has another bottleneck.<\/p>\n
It may be socket processing.<\/p>\n
It may be response writing.<\/p>\n
It may be the Go scheduler.<\/p>\n
It may even be the benchmark generator.<\/p>\n
Without a baseline, I can keep optimizing a component that has already stopped mattering.<\/p>\n
The same principle applies to PostgreSQL.<\/p>\n
Instead of measuring only the full HTTP endpoint, I want to think in three layers:<\/p>\n
PostgreSQL query onlyGo \u2192 PostgreSQL \u2192 GoHTTP \u2192 Go \u2192 PostgreSQL \u2192 Go \u2192 HTTP<\/code>
<\/a><\/div><\/pre>\n
The first isolates database execution as much as possible.<\/p>\n
The second adds the driver, pool, protocol transfer, decoding and runtime.<\/p>\n
The third adds the HTTP stack.<\/p>\n
If the SQL query is fast but the second layer is much slower, SQL optimization may not be where the largest gain exists.<\/p>\n
If the second layer is fast while the complete endpoint is much slower, the database may not be the interesting part at all.<\/p>\n
This decomposition is far more useful than knowing that one complete endpoint produced some impressive RPS value.<\/p>\n
Closed\u2011loop load changes when the server becomes slow<\/h3>\n
There was another property of the benchmark that I initially ignored.<\/p>\n
With a fixed concurrency, ApacheBench behaves essentially like a closed\u2011loop load generator.<\/p>\n
A certain number of requests are active.<\/p>\n
When one finishes, another can take its place.<\/p>\n
This creates an approximate relationship:<\/p>\n
throughput \u2248 concurrency \/ response_time<\/code>
<\/a><\/div><\/pre>\n
If requests become slower, the achieved request rate naturally decreases because existing requests occupy concurrency slots for longer.<\/p>\n
That is perfectly valid for one type of benchmark.<\/p>\n
It does not behave exactly like every production workload.<\/p>\n
A real public API can receive requests at an externally determined rate.<\/p>\n
If 20,000\u00a0requests per second arrive and the database suddenly becomes slower, clients do not necessarily reduce their arrival rate to protect the server.<\/p>\n
Requests continue to arrive.<\/p>\n
Queues grow.<\/p>\n
Tail latency grows.<\/p>\n
Timeouts begin.<\/p>\n
Retries can potentially create even more load.<\/p>\n
This difference matters particularly close to saturation.<\/p>\n
A closed\u2011loop benchmark partly reduces offered load as the server slows down.<\/p>\n
A constant\u2011rate workload does not necessarily do that.<\/p>\n
That is why maximum requests per second should not be the only result I care about.<\/p>\n
I also want to know what happens while approaching the saturation point.<\/p>\n
p99\u00a0tells a different story from average throughput<\/h3>\n
RPS is attractive because it creates a ranking.<\/p>\n
Implementation A produces more requests per second than implementation B.<\/p>\n
A wins.<\/p>\n
Real systems are less cooperative.<\/p>\n
Two implementations can reach similar throughput while having completely different latency distributions.<\/p>\n
Consider two endpoints.<\/p>\n
Both can sustain approximately the same request rate.<\/p>\n
The first has a narrow distribution: p50, p95\u00a0and p99\u00a0remain relatively close.<\/p>\n
The second has a reasonable median but a rapidly growing p99.<\/p>\n
They do not provide the same user experience.<\/p>\n
The second system may already be building queues even though its headline throughput still looks healthy.<\/p>\n
The PostgreSQL path contains several places where variability can enter:<\/p>\n
pool acquisitiondatabase schedulingquery executionprotocol transferrow decodingGo schedulingserializationresponse writing<\/code>
<\/a><\/div><\/pre>\n
Most requests may pass through all of them quickly.<\/p>\n
A few do not.<\/p>\n
Those few define the tail.<\/p>\n
The cache path removes several of these stages entirely.<\/p>\n
That can make its biggest practical advantage predictability rather than maximum throughput.<\/p>\n
For a latency\u2011sensitive API, reducing p99\u00a0from a problematic region can be more valuable than increasing maximum RPS by another impressive percentage.<\/p>\n
This is why I now consider a performance comparison incomplete if it contains only average latency and throughput.<\/p>\n
At minimum I want one coherent set of measurements:<\/p>\n