{"id":392067,"date":"2024-06-29T10:12:10","date_gmt":"2024-06-29T10:12:10","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=392067"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=392067","title":{"rendered":"<span>Run MongoDB Atlas locally for testing<\/span>"},"content":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-1\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p>What happens to your MongoDB replica set when it comes to failures like network partitioning, restarting, reconfiguration of the existing topology, etc.? This question is especially important these days because of <a href=\"https:\/\/www.mongodb.com\/blog\/post\/introducing-multicloud-clusters-on-mongodb-atlas?utm_campaign=INT_EM_Newsletter%252520Developer_01_21_WW&amp;utm_source=eloqua&amp;utm_medium=email&amp;utm_term=MongoDB%252520Developer%252520Newsletter%25253A%252520February%2525204%25252C%2525202021\" rel=\"nofollow noopener noreferrer\">the popularity gained by the multi-cloud model<\/a> where chances of these scenarios are quite realistic.<\/p>\n<p>  <\/p>\n<p>However, is there a solution, preferably a free one, for testing such cases that would obviate the need of writing manual scripts and poring over the official documentation? As software developers, we would be better off preparing our applications in advance to survive these failures.<\/p>\n<p><a name=\"habracut\"><\/a>  <\/p>\n<p>Having a look at MongoDB Atlas and their free <a href=\"https:\/\/docs.atlas.mongodb.com\/reference\/free-shared-limitations\/#std-label-atlas-free-tier\" rel=\"nofollow noopener noreferrer\">Shared Clusters<\/a>, we cannot configure a replica set given by default which is Primary with Two Secondary Members (hereinafter the P-S-S), not to mention testing some features like <code>restartPrimaries<\/code>. All of it is for Dedicated Clusters requiring payments at hourly rates.<\/p>\n<p>  <\/p>\n<p>While cloud solutions seem to be the de facto choice for production systems, some software developers are likely to be in need of free local alternatives, which is proven by projects like LocalStack, act, etc.<\/p>\n<p>  <\/p>\n<p>Given that, what about developing a solution for running locally some functionality that the paid version of MongoDB Atlas does? One possible way is to opt for the following:<\/p>\n<p>  <\/p>\n<ul>\n<li>Docker to run MongoDB containers;<\/li>\n<li>Testcontainers to handle the containers via a programming language like Java;<\/li>\n<li>Toxiproxy to simulate network conditions.<\/li>\n<\/ul>\n<p>  <\/p>\n<p>All good, but to build a cluster of more than one node MongoDB requires that <a href=\"https:\/\/docs.mongodb.com\/manual\/tutorial\/deploy-replica-set\/\" rel=\"nofollow noopener noreferrer\">&#171;each member of a replica set is accessible by way of resolvable DNS or hostnames&#187;<\/a>. If you run Docker version 18.03+ on hosts like Mac and Win or Docker 20.04+ on Linux, you are free to use the special DNS name\u00a0<code>host.docker.internal<\/code>. During installation Docker might add it to your OS host file. However, for those who cannot upgrade their Docker for some reason, we can employ a special container which redirects traffic to the host, for instance <a href=\"https:\/\/github.com\/qoomon\/docker-host\" rel=\"nofollow noopener noreferrer\">Qoomon docker-host<\/a>. Similarly, we can add <code>dockerhost<\/code> to the OS host file and also run the container with <code>NET_ADMIN<\/code> and <code>NET_RAW<\/code> kernel capabilities.<\/p>\n<p>  <\/p>\n<p>The good news is that we do not have to connect all the above-mentioned pieces together manually, but instead can take the free <a href=\"https:\/\/github.com\/silaev\/mongodb-replica-set\" rel=\"nofollow noopener noreferrer\">MongoDBReplicaSet<\/a>.<\/p>\n<p>  <\/p>\n<p>Let us get started by creating a new Gradle project\u00a0with\u00a0Java and adding some dependencies from Maven central:<\/p>\n<p>  <\/p>\n<pre><code class=\"plaintext\">testCompile(\"com.github.silaev:mongodb-replica-set:0.4.3\") testImplementation(\"org.mongodb:mongodb-driver-sync:4.2.1\")<\/code><\/pre>\n<p>  <\/p>\n<p>1) Note that it is up to you which MongoDB driver to use here, <code>mongodb-driver-sync<\/code> is an example.<\/p>\n<p>  <\/p>\n<p>Then we need to check and possibly change our OS host file. Regarding <code>host.docker.internal<\/code>, it might be already there, otherwise add <code>127.0.0.1 host.docker.internal<\/code>. As per <code>dockerhost<\/code>, add <code>127.0.0.1 dockerhost<\/code>. You have to pick only one of them and use it within your test execution.<\/p>\n<p>  <\/p>\n<p>Our journey begins and we are ready to write a test to simulate network partitioning in the P-S-S. Here is the description of it:<\/p>\n<p>  <\/p>\n<pre><code class=\"java\">try (   final MongoDbReplicaSet mongoReplicaSet = MongoDbReplicaSet.builder()     .mongoDockerImageName(\"mongo:4.4.4\")     .useHostDockerInternal(true)     .addToxiproxy(true)     .replicaSetNumber(3)     .commandLineOptions(Arrays.asList(\"--oplogSize\", \"50\"))     .build() ) {<\/code><\/pre>\n<p>  <\/p>\n<p>1) Use <code>mongo:4.4.4<\/code> as the latest MongoDB Docker image at the moment of writing;<br \/>  2) If <code>useHostDockerInternal<\/code> is true, use <code>host.docker.internal<\/code> of Docker, otherwise take <code>dockerhost<\/code> of Qoomon docker-host;<br \/>  3) Put a ToxiproxyContainer.ContainerProxy against each MongoDB node;<br \/>  4) Set 3 (possible up to 7) members to construct the P-S-S;<br \/>  5) Optionally, add some command line options, for example set 50MB as the replication operation log;<br \/>  6) Auto-close all the containers via try-with-resources in case of any exception or completion.<\/p>\n<p>  <\/p>\n<p>Now we can start the replica set by <code>mongoReplicaSet.start()<\/code>, get its URL and make some assertions:<\/p>\n<p>  <\/p>\n<pre><code class=\"java\">final String replicaSetUrl = mongoReplicaSet.getReplicaSetUrl(); assertThat(   mongoReplicaSet.nodeStates(mongoReplicaSet.getMongoRsStatus().getMembers()) ).containsExactlyInAnyOrder(PRIMARY, SECONDARY, SECONDARY);<\/code><\/pre>\n<p>  <\/p>\n<p>1) Internally, getMongoRsStatus() calls <code>rs.status()<\/code> in MongoDB shell.<\/p>\n<p>  <\/p>\n<p>Then we can, for instance, create a <code>MongoClient<\/code> to insert some data and subsequently assert it (see <a href=\"https:\/\/github.com\/silaev\/mongodb-replica-set-examples\/blob\/main\/src\/test\/java\/com\/github\/silaev\/mongodb\/replicaset\/examples\/FailureTest.java\" rel=\"nofollow noopener noreferrer\">the full example on Github<\/a>).<\/p>\n<p>  <\/p>\n<pre><code class=\"java\">try (   final MongoClient mongoSyncClient = MongoClients.create(new ConnectionString(replicaSetUrl)) ) {<\/code><\/pre>\n<p>  <\/p>\n<p>1) Note that we are also able to use the more convenient <code>MongoClientSettings<\/code> as the parameter of the <code>create<\/code> method to set timeouts, read\/write concerns, turn off retries at the connection level. <\/p>\n<p>  <\/p>\n<p>The first failure comes here so let our replica set survive the disconnection of the master node:<\/p>\n<p>  <\/p>\n<pre><code class=\"java\">\/\/ TODO: Insert a document here to assert total number at the end final MongoNode masterNodeBeforeFailure1 = mongoReplicaSet.getMasterMongoNode(   mongoReplicaSet.getMongoRsStatus().getMembers() ); mongoReplicaSet.disconnectNodeFromNetwork(masterNodeBeforeFailure1); mongoReplicaSet.waitForMasterReelection(masterNodeBeforeFailure1); assertThat(   mongoReplicaSet.nodeStates(mongoReplicaSet.getMongoRsStatus().getMembers()) ).containsExactlyInAnyOrder(PRIMARY, SECONDARY, DOWN);<\/code><\/pre>\n<p>  <\/p>\n<p>1) We need to wait for a new master node elected by providing the previous master node to the <code>waitForMasterReelection(...)<\/code> method.<\/p>\n<p>  <\/p>\n<p>Going further, the next accident leads to the newcomer master node getting cut off:<\/p>\n<p>  <\/p>\n<pre><code class=\"java\">\/\/ TODO: Insert a document here to assert total number at the end final MongoNode masterNodeBeforeFailure2 = mongoReplicaSet.getMasterMongoNode(   mongoReplicaSet.getMongoRsStatus().getMembers() ); mongoReplicaSet.disconnectNodeFromNetwork(masterNodeBeforeFailure2); mongoReplicaSet.waitForMongoNodesDown(2); assertThat(   mongoReplicaSet.nodeStates(mongoReplicaSet.getMongoRsStatus().getMembers()) ).containsExactlyInAnyOrder(SECONDARY, DOWN, DOWN);<\/code><\/pre>\n<p>  <\/p>\n<p>1) We wait for a moment when our single secondary detects the other 2 nodes being down.<\/p>\n<p>  <\/p>\n<p>Our journey is drawing to a close, so let us bring all the disconnected nodes back by way of a happy end:<\/p>\n<p>  <\/p>\n<pre><code class=\"java\">\/\/ TODO: Insert a document here to assert total number at the end mongoReplicaSet.connectNodeToNetwork(masterNodeBeforeFailure1); mongoReplicaSet.connectNodeToNetwork(masterNodeBeforeFailure2); mongoReplicaSet.waitForAllMongoNodesUp(); mongoReplicaSet.waitForMaster(); assertThat(   mongoReplicaSet.nodeStates(mongoReplicaSet.getMongoRsStatus().getMembers()) ).containsExactlyInAnyOrder(PRIMARY, SECONDARY, SECONDARY);<\/code><\/pre>\n<p>  <\/p>\n<p>1) The <code>waitForAllMongoNodesUp(...)<\/code> method waits for all the disconnected nodes to be up and running;<br \/>  2) Then the <code>waitForMaster()<\/code> method waits for the elections to complete.<\/p>\n<p>  <\/p>\n<h3 id=\"conclusion\">Conclusion<\/h3>\n<p>  <\/p>\n<p>Why writing such tests? To address this question, let us set <code>write concern<\/code> as <code>majority<\/code> with <code>journaling<\/code> enabled and <code>read concern<\/code> as <code>majority<\/code> as well. Then we can replace <code>\/\/ TODO:...<\/code> in the above-mentioned code examples with the <code>mongoSyncClient.insertOne(\u2026)<\/code> method handling possible exceptions to add a new document 3 times. Running this test 120 times and waiting for a while at the end, I found out that approximately half the time total number was 2 and the other half it was 3. Therefore, the idea behind these tests is to be ready for some corner cases beforehand.<\/p>\n<p>  <\/p>\n<h3 id=\"links\">Links:<\/h3>\n<p>  <\/p>\n<ol>\n<li>Find the example from this article <a href=\"https:\/\/github.com\/silaev\/mongodb-replica-set-examples\/blob\/main\/src\/test\/java\/com\/github\/silaev\/mongodb\/replicaset\/examples\/FailureTest.java\" rel=\"nofollow noopener noreferrer\">here<\/a>;<\/li>\n<li><a href=\"https:\/\/github.com\/silaev\/mongodb-replica-set\" rel=\"nofollow noopener noreferrer\">MongoDBReplicaSet on Github<\/a>.<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/544460\/\"> https:\/\/habr.com\/ru\/articles\/544460\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-1\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p>What happens to your MongoDB replica set when it comes to failures like network partitioning, restarting, reconfiguration of the existing topology, etc.? This question is especially important these days because of <a href=\"https:\/\/www.mongodb.com\/blog\/post\/introducing-multicloud-clusters-on-mongodb-atlas?utm_campaign=INT_EM_Newsletter%252520Developer_01_21_WW&amp;utm_source=eloqua&amp;utm_medium=email&amp;utm_term=MongoDB%252520Developer%252520Newsletter%25253A%252520February%2525204%25252C%2525202021\" rel=\"nofollow noopener noreferrer\">the popularity gained by the multi-cloud model<\/a> where chances of these scenarios are quite realistic.<\/p>\n<p>  <\/p>\n<p>However, is there a solution, preferably a free one, for testing such cases that would obviate the need of writing manual scripts and poring over the official documentation? As software developers, we would be better off preparing our applications in advance to survive these failures.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-392067","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/392067","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=392067"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/392067\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=392067"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=392067"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=392067"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}