{"id":494298,"date":"2026-09-10T19:50:00","date_gmt":"2026-09-10T19:50:00","guid":{"rendered":"https:\/\/savepearlharbor.com\/?p=494298"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=494298","title":{"rendered":"sync.Map Was 2.8x Faster Than RWMutex. My Go API Barely Got Faster"},"content":{"rendered":"<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p>I have become suspicious of optimizations that look spectacular in microbenchmarks.<\/p>\n<p>The pattern is easy to recognize.<\/p>\n<p>You benchmark two implementations. One is two, three, maybe five times faster. The result looks convincing enough to justify changing the production code.<\/p>\n<p>Then you put the faster implementation back into the actual application.<\/p>\n<p>Almost nothing happens.<\/p>\n<p>I wanted to reproduce this effect with something much smaller than a database, so I tested three ways of building a simple in-memory cache in Go:<\/p>\n<ul>\n<li>\n<p>a regular map protected by sync.RWMutex<\/p>\n<\/li>\n<li>\n<p>a map split into 64 independently locked shards<\/p>\n<\/li>\n<li>\n<p>sync.Map<\/p>\n<\/li>\n<\/ul>\n<p>The isolated benchmark produced a very clear winner.<\/p>\n<p>sync.Map reached about 29.4 million reads per second.<\/p>\n<p>The simple RWMutex implementation managed about 10.6 million.<\/p>\n<p>That is roughly a 2.8x difference.<\/p>\n<p>Then I put exactly the same cache behind a small HTTP endpoint.<\/p>\n<p>The advantage almost disappeared.<\/p>\n<p>That result turned out to be much more interesting than the microbenchmark itself.<\/p>\n<figure class=\"full-width \"><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/255\/19e\/ddc\/25519eddcbb101f3605f2d9bd5cfe3f1.png\" width=\"1672\" height=\"941\" sizes=\"auto, (max-width: 780px) 100vw, 50vw\" srcset=\"https:\/\/habrastorage.org\/r\/w780\/getpro\/habr\/upload_files\/255\/19e\/ddc\/25519eddcbb101f3605f2d9bd5cfe3f1.png 780w,&#10;       https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/255\/19e\/ddc\/25519eddcbb101f3605f2d9bd5cfe3f1.png 781w\" loading=\"lazy\" decode=\"async\"\/><\/figure>\n<h3>Three deliberately boring caches<\/h3>\n<p>I did not try to build a production-ready cache.<\/p>\n<p>There was no TTL, LRU, eviction, refresh logic, database fallback or distributed invalidation.<\/p>\n<p>I wanted to isolate one question:<\/p>\n<p>How much does the synchronization strategy matter when many goroutines read from the same in-memory data structure?<\/p>\n<p>The first implementation used a normal Go map and one RWMutex.<\/p>\n<pre><code>type Cache struct {    mu sync.RWMutex    m  map[uint64]uint64}func (c *Cache) Get(k uint64) (uint64, bool) {    c.mu.RLock()    v, ok := c.m[k]    c.mu.RUnlock()    return v, ok}func (c *Cache) Set(k, v uint64) {    c.mu.Lock()    c.m[k] = v    c.mu.Unlock()}<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:87px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>This is probably the first implementation I would write for a small service.<\/p>\n<p>It is easy to understand.<\/p>\n<p>It is easy to debug.<\/p>\n<p>And until contention becomes significant, it may be completely sufficient.<\/p>\n<p>The second implementation used 64 shards.<\/p>\n<pre><code>type shard struct {    mu sync.RWMutex    m  map[uint64]uint64}type ShardedCache struct {    shards []shard    mask   uint64}func (c *ShardedCache) Get(k uint64) (uint64, bool) {    s := c.shardFor(k)    s.mu.RLock()    v, ok := s.m[k]    s.mu.RUnlock()    return v, ok}<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>Instead of every key competing for one lock, different keys can land in different shards.<\/p>\n<p>The third implementation used sync.Map.<\/p>\n<pre><code>type SyncCache struct {    m sync.Map}func (c *SyncCache) Get(k uint64) (uint64, bool) {    v, ok := c.m.Load(k)    if !ok {        return 0, false    }    return v.(uint64), true}func (c *SyncCache) Set(k, v uint64) {    c.m.Store(k, v)}<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>All caches were populated before the benchmark.<\/p>\n<p>The key space contained 65,536 entries.<\/p>\n<p>The workload was intentionally read-heavy because I first wanted to see how much the synchronization strategy alone could change lookup throughput.<\/p>\n<h3>The microbenchmark<\/h3>\n<p>For the first test I removed almost everything except cache access.<\/p>\n<p>No HTTP.<\/p>\n<p>No JSON.<\/p>\n<p>No TCP.<\/p>\n<p>No database.<\/p>\n<p>No request parsing.<\/p>\n<p>Multiple goroutines repeatedly selected keys and called Get.<\/p>\n<p>The benchmark was run several times, and I compared the median results from the same environment.<\/p>\n<p>The numbers were approximately:<\/p>\n<ul>\n<li>\n<p>RWMutex map: 10.55 million operations per second<\/p>\n<\/li>\n<li>\n<p>64-shard map: 21.04 million operations per second<\/p>\n<\/li>\n<li>\n<p>sync.Map: 29.37 million operations per second<\/p>\n<\/li>\n<\/ul>\n<p>The difference was large enough that it did not need creative interpretation.<\/p>\n<p>The sharded cache was roughly twice as fast as the single-lock version.<\/p>\n<p>sync.Map was roughly 2.8x faster than RWMutex.<\/p>\n<p>If this were the only benchmark I had run, the conclusion would have been tempting:<\/p>\n<p>Replace the mutex-protected map with sync.Map and get a huge performance improvement.<\/p>\n<p>But that conclusion contains a hidden assumption.<\/p>\n<p>It assumes that cache lookup is a large part of the work performed by the actual application.<\/p>\n<p>The benchmark never tested that assumption.<\/p>\n<h3>What the benchmark actually measured<\/h3>\n<p>The isolated test measured something close to:<\/p>\n<pre><code>operation = cache lookup<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>That is useful.<\/p>\n<p>It tells me something about the cache implementation.<\/p>\n<p>But an HTTP request looks more like this:<\/p>\n<pre><code>request =    read HTTP request    + route request    + parse parameters    + convert key    + cache lookup    + build response    + encode response    + write response    + network stack    + scheduler overhead<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>The cache can become dramatically faster while the rest of this path remains unchanged.<\/p>\n<p>That is exactly what happened.<\/p>\n<h3>Putting the cache behind HTTP<\/h3>\n<p>For the second experiment I created a small net\/http server.<\/p>\n<p>Each request performed roughly the following operations:<\/p>\n<ol>\n<li>\n<p>read a key from the request<\/p>\n<\/li>\n<li>\n<p>convert it to uint64<\/p>\n<\/li>\n<li>\n<p>call the cache<\/p>\n<\/li>\n<li>\n<p>build a tiny response<\/p>\n<\/li>\n<li>\n<p>return the response over HTTP<\/p>\n<\/li>\n<\/ol>\n<p>I intentionally kept PostgreSQL out of this experiment.<\/p>\n<p>Adding a database would have made the result more realistic for many APIs, but it would also have introduced another large variable.<\/p>\n<p>I wanted to see whether HTTP processing alone was enough to hide a large cache-level improvement.<\/p>\n<p>The answer was yes.<\/p>\n<p>The load test used:<\/p>\n<ul>\n<li>\n<p>persistent HTTP connections<\/p>\n<\/li>\n<li>\n<p>128 concurrent clients<\/p>\n<\/li>\n<li>\n<p>40,000 requests per run<\/p>\n<\/li>\n<li>\n<p>the same 65,536-key working set<\/p>\n<\/li>\n<\/ul>\n<p>The tests were performed using Go 1.23.2 in an environment with five available CPUs.<\/p>\n<p>The server and load generator were kept separate so the client benchmark would not execute inside the same Go process as the server.<\/p>\n<p>That distinction matters more than it initially seems.<\/p>\n<p>If the load generator and server live inside one process, both compete for the same scheduler, CPU budget and garbage collector. At that point I may accidentally benchmark my test harness almost as much as the server.<\/p>\n<h3>And then the 2.8x advantage disappeared<\/h3>\n<p>At the HTTP level, all three implementations ended up in approximately the same range:<\/p>\n<p>15,000 to 16,000 requests per second.<\/p>\n<p>There was run-to-run variation.<\/p>\n<p>The ordering between implementations was not stable enough for me to make a serious claim that one cache was consistently faster at the HTTP level.<\/p>\n<p>And that is the important result.<\/p>\n<p>The microbenchmark said:<\/p>\n<pre><code>RWMutex:   ~10.6M ops\/ssync.Map:  ~29.4M ops\/s<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>A huge difference.<\/p>\n<p>The HTTP benchmark said, effectively:<\/p>\n<pre><code>all implementations: ~15K\u201316K req\/s<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>A small difference.<\/p>\n<p>Changing the cache implementation did not turn a 15K req\/s server into a 30K, 40K or 45K req\/s server.<\/p>\n<p>The 2.8x component improvement was almost invisible at the application boundary.<\/p>\n<h3>This is not a contradiction<\/h3>\n<p>The result looks strange only if cache throughput and API throughput are treated as the same thing.<\/p>\n<p>They are not.<\/p>\n<p>Imagine that one request takes 10 microseconds.<\/p>\n<p>Suppose only one microsecond is spent inside the cache.<\/p>\n<p>The remaining nine microseconds are spent somewhere else.<\/p>\n<p>Now make the cache lookup three times faster.<\/p>\n<p>The cache portion falls from approximately 1 microsecond to 0.33 microseconds.<\/p>\n<p>The total request time changes from:<\/p>\n<pre><code>10 us<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>to roughly:<\/p>\n<pre><code>9.33 us<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>The cache improved by 3x.<\/p>\n<p>The request improved by only about 7 percent.<\/p>\n<p>And if the cache occupied an even smaller fraction of the request, the difference would be smaller again.<\/p>\n<p>This is basically Amdahl&#8217;s law appearing in a tiny Go HTTP server.<\/p>\n<p>If fraction P of execution time can be accelerated by factor S, the theoretical total speedup is:<\/p>\n<pre><code>speedup = 1 \/ ((1 - P) + P \/ S)<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>A large S does not help much when P is tiny.<\/p>\n<p>That is the part that microbenchmarks tend to hide.<\/p>\n<h3>The microbenchmark was still correct<\/h3>\n<p>I do not think the first benchmark was useless.<\/p>\n<p>This distinction matters.<\/p>\n<p>sync.Map really was substantially faster under that isolated read-heavy workload.<\/p>\n<p>The sharded map really did reduce the cost associated with one global lock.<\/p>\n<p>The benchmark answered a valid question:<\/p>\n<p>Which implementation can perform more concurrent cache lookups under these conditions?<\/p>\n<p>The mistake would be changing the question after seeing the result.<\/p>\n<p>It did not answer:<\/p>\n<p>Which implementation will make my HTTP API 2.8x faster?<\/p>\n<p>That would require an end-to-end benchmark.<\/p>\n<p>One of the easiest performance mistakes is getting a correct answer to the wrong question.<\/p>\n<h3>Why sharding helped so much in isolation<\/h3>\n<p>The RWMutex implementation has one obvious synchronization point.<\/p>\n<p>Every reader needs the same RLock.<\/p>\n<p>RWMutex allows multiple readers to proceed concurrently, so this does not mean all reads execute serially.<\/p>\n<p>But they still interact with the same synchronization structure.<\/p>\n<p>As concurrency increases, that shared point becomes more important.<\/p>\n<p>Sharding changes the geometry of the problem.<\/p>\n<p>Instead of:<\/p>\n<pre><code>all keys   |one lock<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>I get something closer to:<\/p>\n<pre><code>keys 0..N   |hash   |64 separate locks<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>Operations touching different shards no longer compete through one shared lock.<\/p>\n<p>That explains why the sharded version roughly doubled lookup throughput in my isolated test.<\/p>\n<p>But it also adds work.<\/p>\n<p>Every lookup needs to determine its shard.<\/p>\n<p>There is more code.<\/p>\n<p>There are more maps.<\/p>\n<p>The implementation is harder to maintain.<\/p>\n<p>In the microbenchmark, reducing contention was worth the extra complexity.<\/p>\n<p>In the HTTP test, the benefit mostly vanished into the rest of the request path.<\/p>\n<p>That changes the engineering decision.<\/p>\n<h3>Faster code is not always a faster system<\/h3>\n<p>This sounds obvious until a benchmark produces a large number.<\/p>\n<p>29.4 million ops\/s looks much better than 10.6 million ops\/s.<\/p>\n<p>It feels like a meaningful optimization.<\/p>\n<p>And at the data-structure level, it is.<\/p>\n<p>But production software rarely exists at the data-structure level.<\/p>\n<p>A real handler may also perform:<\/p>\n<ul>\n<li>\n<p>authentication<\/p>\n<\/li>\n<li>\n<p>authorization<\/p>\n<\/li>\n<li>\n<p>tracing<\/p>\n<\/li>\n<li>\n<p>logging<\/p>\n<\/li>\n<li>\n<p>JSON encoding<\/p>\n<\/li>\n<li>\n<p>decompression<\/p>\n<\/li>\n<li>\n<p>validation<\/p>\n<\/li>\n<li>\n<p>database access<\/p>\n<\/li>\n<li>\n<p>RPC calls<\/p>\n<\/li>\n<li>\n<p>metrics<\/p>\n<\/li>\n<li>\n<p>allocations<\/p>\n<\/li>\n<li>\n<p>filesystem operations<\/p>\n<\/li>\n<\/ul>\n<p>If the cache is responsible for 2 percent of request cost, spending two days making it three times faster is unlikely to change the service.<\/p>\n<p>If it is responsible for 60 percent, the same optimization may be extremely valuable.<\/p>\n<p>Without measuring that fraction, I do not know which situation I am in.<\/p>\n<h3>The easiest thing to benchmark is often the easiest thing to over-optimize<\/h3>\n<p>There is another reason these mistakes happen.<\/p>\n<p>Small functions are pleasant to benchmark.<\/p>\n<p>A cache lookup can be wrapped in a loop.<\/p>\n<p>A serialization function can be wrapped in a loop.<\/p>\n<p>A hash function can be wrapped in a loop.<\/p>\n<p>Then Go gives me clean numbers.<\/p>\n<pre><code>ns\/opallocs\/opB\/op<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>Very satisfying.<\/p>\n<p>The harder problems are usually less clean.<\/p>\n<p>How much time does the request spend waiting for a database connection?<\/p>\n<p>How often does the scheduler become relevant?<\/p>\n<p>What happens to p99 latency during bursts?<\/p>\n<p>Does GC become expensive only when traffic reaches a certain level?<\/p>\n<p>Is the application limited by CPU, sockets, downstream services or memory bandwidth?<\/p>\n<p>Those questions require more effort.<\/p>\n<p>So it is easy to optimize the part that gives the nicest benchmark output instead of the part limiting the application.<\/p>\n<p>I have started treating that as a warning sign.<\/p>\n<p>If an optimization begins with a microbenchmark, I now want an end-to-end measurement before calling it a success.<\/p>\n<h3>There is also no universal winner between these three caches<\/h3>\n<p>I would not use this experiment to argue that sync.Map is always best.<\/p>\n<p>That is another conclusion the benchmark does not support.<\/p>\n<p>The workload here was intentionally favorable to concurrent reads.<\/p>\n<p>Change the workload and the result can change.<\/p>\n<p>For example:<\/p>\n<ul>\n<li>\n<p>frequent writes change synchronization behavior<\/p>\n<\/li>\n<li>\n<p>continuously adding new keys changes the workload<\/p>\n<\/li>\n<li>\n<p>deleting keys changes it again<\/p>\n<\/li>\n<li>\n<p>a single hot key is different from uniformly distributed keys<\/p>\n<\/li>\n<li>\n<p>64 entries are different from 6 million entries<\/p>\n<\/li>\n<li>\n<p>one goroutine is different from hundreds<\/p>\n<\/li>\n<li>\n<p>large values change cache behavior<\/p>\n<\/li>\n<li>\n<p>pointer-heavy values may change GC costs<\/p>\n<\/li>\n<\/ul>\n<p>Even the sharding strategy itself matters.<\/p>\n<p>The number of shards matters.<\/p>\n<p>The hash function matters.<\/p>\n<p>The key distribution matters.<\/p>\n<p>The relationship between readers and writers matters.<\/p>\n<p>So I would not ask which Go concurrent map is fastest.<\/p>\n<p>I would ask which implementation is appropriate for this access pattern and whether the map is important enough to optimize at all.<\/p>\n<p>The second question is usually more valuable.<\/p>\n<h3>How I would test this in a real service<\/h3>\n<p>If I encountered a mutex-protected cache in production, I would not replace it merely because another implementation wins a synthetic benchmark.<\/p>\n<p>First I would look for evidence that the current cache is actually a problem.<\/p>\n<p>I would check:<\/p>\n<ul>\n<li>\n<p>CPU profiles<\/p>\n<\/li>\n<li>\n<p>mutex contention<\/p>\n<\/li>\n<li>\n<p>time spent inside cache operations<\/p>\n<\/li>\n<li>\n<p>cache hit rate<\/p>\n<\/li>\n<li>\n<p>read\/write ratio<\/p>\n<\/li>\n<li>\n<p>key distribution<\/p>\n<\/li>\n<li>\n<p>allocation rate<\/p>\n<\/li>\n<li>\n<p>GC CPU<\/p>\n<\/li>\n<li>\n<p>request throughput<\/p>\n<\/li>\n<li>\n<p>p95 and p99 latency<\/p>\n<\/li>\n<li>\n<p>the point where the service starts saturating<\/p>\n<\/li>\n<\/ul>\n<p>If the mutex is clearly visible in the profile, then optimizing it becomes interesting.<\/p>\n<p>If it barely appears, replacing the cache may only improve a benchmark.<\/p>\n<p>The application does not care which code looks faster in isolation.<\/p>\n<p>It cares about the critical path.<\/p>\n<h3>A useful hierarchy for performance tests<\/h3>\n<p>After this experiment, I find it useful to think about benchmarks in three levels.<\/p>\n<h4>Level 1: component<\/h4>\n<p>This is the microbenchmark.<\/p>\n<p>It answers questions like:<\/p>\n<ul>\n<li>\n<p>how fast is Get<\/p>\n<\/li>\n<li>\n<p>how expensive is hashing<\/p>\n<\/li>\n<li>\n<p>how much contention does this lock create<\/p>\n<\/li>\n<li>\n<p>how many allocations does this function perform<\/p>\n<\/li>\n<\/ul>\n<p>This is where I observed the 2.8x difference.<\/p>\n<h4>Level 2: subsystem<\/h4>\n<p>Now the component sits inside part of the real execution path.<\/p>\n<p>For example:<\/p>\n<pre><code>routing+validation+cache+serialization<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>This tells me whether the component remains important once neighboring work returns.<\/p>\n<h4>Level 3: end to end<\/h4>\n<p>Now I measure the system at the boundary that users or other services actually see.<\/p>\n<p>For an API this might include:<\/p>\n<pre><code>client\u2192 network\u2192 HTTP server\u2192 application\u2192 cache\/database\u2192 serialization\u2192 network\u2192 client<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>At this level, many impressive micro-optimizations become difficult to see.<\/p>\n<p>That does not invalidate them.<\/p>\n<p>It tells me their contribution to the whole system is small.<\/p>\n<h3>The number I care about changed<\/h3>\n<p>Before this experiment, I could easily look at:<\/p>\n<pre><code>10.55M ops\/s<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>versus:<\/p>\n<pre><code>29.37M ops\/s<\/code><div class=\"code-explainer\"><a href=\"https:\/\/sourcecraft.dev\/\" class=\"tm-button code-explainer__link\" style=\"visibility: hidden;\"><img style=\"width:14px;height:14px;object-fit:cover;object-position:left;\"\/><\/a><\/div><\/pre>\n<p>and think I had found something important.<\/p>\n<p>Now the number I want immediately afterward is:<\/p>\n<p>How much of the real request does this operation represent?<\/p>\n<p>Without that information, the speedup is incomplete.<\/p>\n<p>The cache benchmark told me sync.Map could process almost 2.8 times as many reads as my RWMutex version under the tested conditions.<\/p>\n<p>The HTTP benchmark told me something more useful:<\/p>\n<p>My HTTP server did not care very much.<\/p>\n<p>That distinction is exactly why I ran the second test.<\/p>\n<p>And, increasingly, it is the distinction I want to see whenever someone shows me a large microbenchmark speedup.<\/p>\n<\/div>\n<p>\u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/1081034\/\">https:\/\/habr.com\/ru\/articles\/1081034\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I have become suspicious of optimizations that look spectacular in microbenchmarks.The pattern is easy to recognize.You benchmark two implementations. One is two, three, maybe five times faster. The result looks convincing enough to justify changing the production code.Then you put the faster implementation back into the actual application.Almost nothing happens.I wanted to reproduce this effect with something much smaller than a database, so I tested three ways of building a simple in-memory cache in Go:a regular map protected by sync.RWMutexa map split into 64 independently locked shardssync.MapThe isolated benchmark produced a very clear winner.sync.Map reached about 29.4 million reads per second.The simple RWMutex implementation managed about 10.6 million.That is roughly a 2.8x difference.Then I put exactly the same cache behind a small HTTP endpoint.The advantage almost disappeared.That result turned out to be much more interesting than the microbenchmark itself.Three deliberately boring cachesI did not try to build a production-ready cache.There was no TTL, LRU, eviction, refresh logic, database fallback or distributed invalidation.I wanted to isolate one question:How much does the synchronization strategy matter when many goroutines read from the same in-memory data structure?The first implementation used a normal Go map and one RWMutex.type Cache struct {    mu sync.RWMutex    m  map[uint64]uint64}func (c *Cache) Get(k uint64) (uint64, bool) {    c.mu.RLock()    v, ok := c.m[k]    c.mu.RUnlock()    return v, ok}func (c *Cache) Set(k, v uint64) {    c.mu.Lock()    c.m[k] = v    c.mu.Unlock()}This is probably the first implementation I would write for a small service.It is easy to understand.It is easy to debug.And until contention becomes significant, it may be completely sufficient.The second implementation used 64 shards.type shard struct {    mu sync.RWMutex    m  map[uint64]uint64}type ShardedCache struct {    shards []shard    mask   uint64}func (c *ShardedCache) Get(k uint64) (uint64, bool) {    s := c.shardFor(k)    s.mu.RLock()    v, ok := s.m[k]    s.mu.RUnlock()    return v, ok}Instead of every key competing for one lock, different keys can land in different shards.The third implementation used sync.Map.type SyncCache struct {    m sync.Map}func (c *SyncCache) Get(k uint64) (uint64, bool) {    v, ok := c.m.Load(k)    if !ok {        return 0, false    }    return v.(uint64), true}func (c *SyncCache) Set(k, v uint64) {    c.m.Store(k, v)}All caches were populated before the benchmark.The key space contained 65,536 entries.The workload was intentionally read-heavy because I first wanted to see how much the synchronization strategy alone could change lookup throughput.The microbenchmarkFor the first test I removed almost everything except cache access.No HTTP.No JSON.No TCP.No database.No request parsing.Multiple goroutines repeatedly selected keys and called Get.The benchmark was run several times, and I compared the median results from the same environment.The numbers were approximately:RWMutex map: 10.55 million operations per second64-shard map: 21.04 million operations per secondsync.Map: 29.37 million operations per secondThe difference was large enough that it did not need creative interpretation.The sharded cache was roughly twice as fast as the single-lock version.sync.Map was roughly 2.8x faster than RWMutex.If this were the only benchmark I had run, the conclusion would have been tempting:Replace the mutex-protected map with sync.Map and get a huge performance improvement.But that conclusion contains a hidden assumption.It assumes that cache lookup is a large part of the work performed by the actual application.The benchmark never tested that assumption.What the benchmark actually measuredThe isolated test measured something close to:operation = cache lookupThat is useful.It tells me something about the cache implementation.But an HTTP request looks more like this:request =    read HTTP request    + route request    + parse parameters    + convert key    + cache lookup    + build response    + encode response    + write response    + network stack    + scheduler overheadThe cache can become dramatically faster while the rest of this path remains unchanged.That is exactly what happened.Putting the cache behind HTTPFor the second experiment I created a small net\/http server.Each request performed roughly the following operations:read a key from the requestconvert it to uint64call the cachebuild a tiny responsereturn the response over HTTPI intentionally kept PostgreSQL out of this experiment.Adding a database would have made the result more realistic for many APIs, but it would also have introduced another large variable.I wanted to see whether HTTP processing alone was enough to hide a large cache-level improvement.The answer was yes.The load test used:persistent HTTP connections128 concurrent clients40,000 requests per runthe same 65,536-key working setThe tests were performed using Go 1.23.2 in an environment with five available CPUs.The server and load generator were kept separate so the client benchmark would not execute inside the same Go process as the server.That distinction matters more than it initially seems.If the load generator and server live inside one process, both compete for the same scheduler, CPU budget and garbage collector. At that point I may accidentally benchmark my test harness almost as much as the server.And then the 2.8x advantage disappearedAt the HTTP level, all three implementations ended up in approximately the same range:15,000 to 16,000 requests per second.There was run-to-run variation.The ordering between implementations was not stable enough for me to make a serious claim that one cache was consistently faster at the HTTP level.And that is the important result.The microbenchmark said:RWMutex:   ~10.6M ops\/ssync.Map:  ~29.4M ops\/sA huge difference.The HTTP benchmark said, effectively:all implementations: ~15K\u201316K req\/sA small difference.Changing the cache implementation did not turn a 15K req\/s server into a 30K, 40K or 45K req\/s server.The 2.8x component improvement was almost invisible at the application boundary.This is not a contradictionThe result looks strange only if cache throughput and API throughput are treated as the same thing.They are not.Imagine that one request takes 10 microseconds.Suppose only one microsecond is spent inside the cache.The remaining nine microseconds are spent somewhere else.Now make the cache lookup three times faster.The cache portion falls from approximately 1 microsecond to 0.33 microseconds.The total request time changes from:10 usto roughly:9.33 usThe cache improved by 3x.The request improved by only about 7 percent.And if the cache occupied an even smaller fraction of the request, the difference would be smaller again.This is basically Amdahl&#8217;s law appearing in a tiny Go HTTP server.If fraction P of execution time can be accelerated by factor S, the theoretical total speedup is:speedup = 1 \/ ((1 &#8212; P) + P \/ S)A large S does not help much when P is tiny.That is the part that microbenchmarks tend to hide.The microbenchmark was still correctI do not think the first benchmark was useless.This distinction matters.sync.Map really was substantially faster under that isolated read-heavy workload.The sharded map really did reduce the cost associated with one global lock.The benchmark answered a valid question:Which implementation can perform more concurrent cache lookups under these conditions?The mistake would be changing the question after seeing the result.It did not answer:Which implementation will make my HTTP API 2.8x faster?That would require an end-to-end benchmark.One of the easiest performance mistakes is getting a correct answer to the wrong question.Why sharding helped so much in isolationThe RWMutex implementation has one obvious synchronization point.Every reader needs the same RLock.RWMutex allows multiple readers to proceed concurrently, so this does not mean all reads execute serially.But they still interact with the same synchronization structure.As concurrency increases, that shared point becomes more important.Sharding changes the geometry of the problem.Instead of:all keys   |one lockI get something closer to:keys 0..N   |hash   |64 separate locksOperations touching different shards no longer compete through one shared lock.That explains why the sharded version roughly doubled lookup throughput in my isolated test.But it also adds work.Every lookup needs to determine its shard.There is more code.There are more maps.The implementation is harder to maintain.In the microbenchmark, reducing contention was worth the extra complexity.In the HTTP test, the benefit mostly vanished into the rest of the request path.That changes the engineering decision.Faster code is not always a faster systemThis sounds obvious until a benchmark produces a large number.29.4 million ops\/s looks much better than 10.6 million ops\/s.It feels like a meaningful optimization.And at the data-structure level, it is.But production software rarely exists at the data-structure level.A real handler may also perform:authenticationauthorizationtracingloggingJSON encodingdecompressionvalidationdatabase accessRPC callsmetricsallocationsfilesystem operationsIf the cache is responsible for 2 percent of request cost, spending two days making it three times faster is unlikely to change the service.If it is responsible for 60 percent, the same optimization may be extremely valuable.Without measuring that fraction, I do not know which situation I am in.The easiest thing to benchmark is often the easiest thing to over-optimizeThere is another reason these mistakes happen.Small functions are pleasant to benchmark.A cache lookup can be wrapped in a loop.A serialization function can be wrapped in a loop.A hash function can be wrapped in a loop.Then Go gives me clean numbers.ns\/opallocs\/opB\/opVery satisfying.The harder problems are usually less&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-494298","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/494298","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=494298"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/494298\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=494298"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=494298"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=494298"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}