{"id":405256,"date":"2024-06-29T18:12:45","date_gmt":"2024-06-29T18:12:45","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=405256"},"modified":"-0001-11-30T00:00:00","modified_gmt":"2024-06-29T18:12:45","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=405256","title":{"rendered":"<span>Even shell scripts require unit tests<\/span>"},"content":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-2\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p>Once a upon a time I moved to brand new project. And without much thought, I decided to take DevOps responsibilities (after a long period of Frontend). Huge mono-repository (Angular and Node.js) gives rise to many specific problems. And this project was no exception. At the very beginning CI\/CD duration was about 1.5h. And that was the biggest problem to take care of.<\/p>\n<p>But at first, I want to talk about &#171;<strong>B<\/strong>ourne<em> <\/em><strong>a<\/strong>gain<em> <\/em><strong>sh<\/strong>ell&#187;, cause CI\/CD automation almost entirely was implemented by means of shell scripts (Bash). In context of huge mono-repo even regular build becomes a piece of odd stuff. That`s why a huge amount of scripts with complicated logic inside (build, test, deploy, generation of release notes, collection of logs and metrics, &#8230;) was another significant problem.<\/p>\n<p>Regardless of lang, quality of complex logic should be under control. It is no secret that key aspect of code quality control &#8212; is tests. For example, to refactor safely major logic should be covered with tests. We decided to keep shell scripts untouched and cover the major ones with tests before any refactoring. Of course, it is possible to use Jest (or Mocha) with a bunch of awful utils to test shell scripts. This approach is a bit wordy and has no value if scripts under test are written in Bash. Also I have no idea how to mock external shell commands (such as <em>curl, ls, touch, npm, &#8230;<\/em>) with Jest or similar framework.<\/p>\n<p>In our particular case it was great to have the following to test CI\/CD scripts:<\/p>\n<ul>\n<li>\n<p>Tiny testing framework with no assertions API. Double square brackets statement covers majority of cases; check <a href=\"https:\/\/devhints.io\/bash#conditionals\" rel=\"noopener noreferrer nofollow\"><u>this<\/u><\/a> out for more detail.<\/p>\n<\/li>\n<li>\n<p>Tests isolation (for parallelization).<\/p>\n<\/li>\n<li>\n<p>Mock API (to mock external shell commands).<\/p>\n<\/li>\n<li>\n<p>JUnit reporting format (to integrate with CI).<\/p>\n<\/li>\n<\/ul>\n<p>It was not hard to implement appropriate testing framework from scratch <a href=\"https:\/\/github.com\/redneckz\/red-shell-unit\" rel=\"noopener noreferrer nofollow\"><u>https:\/\/github.com\/redneckz\/red-shell-unit<\/u><\/a><\/p>\n<h3>How to run tests<\/h3>\n<p>Lets start from the very basic example:<\/p>\n<p>add.sh<\/p>\n<pre><code class=\"bash\">#!\/usr\/bin\/env bash echo $(( $1 + $2 ))<\/code><\/pre>\n<p>This script just adds the first argument to the second one and outputs the result to <em>stdout.<\/em><\/p>\n<p>One of the approaches is to place tests along with modules. This approach simplifies things, cause technically there is no need to explicitly point out path to shell script under test. Path can be computed relative to spec.<\/p>\n<p>add.spec.sh<\/p>\n<pre><code class=\"bash\">#!\/usr\/bin\/env red-shu.sh  it 'should add two numbers'   # Act   result=$(run 2 2)   # Assert   [[ $result -eq 4 ]] ti<\/code><\/pre>\n<p>In order to make this test up and running shebang should reference <em>red-shu.sh.<\/em> It is a wrapper which provides core functionality (<em>it\/ti\/run\/mock<\/em>) and tests isolation<em>.<\/em><\/p>\n<p><em>it\/ti <\/em>commands<em> <\/em>declare isolated block to test the only requirement. For each block subshell is initiated. If block exits with non-zero code, appropriate requirement turns red. Framework provides <em>run <\/em>command to execute script\/command under test. Such decoupling makes refactoring less expensive. For example, you can move or rename scripts along with corresponding tests without difficulty.<\/p>\n<p>The best technical solution is a solution that does not require any new components or modules. That`s why exit codes are used to interpret test results. In this case, no dedicated assertion API is needed. Bash`s double square brackets statement is powerful enough to cover most needs and assert anything you ever wanted. Please check <a href=\"https:\/\/devhints.io\/bash#conditionals\" rel=\"noopener noreferrer nofollow\"><u>this<\/u><\/a> out.<\/p>\n<p>Also commands like <em>grep<\/em> or <em>diff <\/em>are suitable for more complicated kinds of assertions.<\/p>\n<h3>How to mock external commands<\/h3>\n<p>Mocks are just necessary to write unit tests. Most dependencies should be mocked to test module functionality in isolation and highlight external parts of contract.<\/p>\n<p>The implementation of mocks turned out to be surprisingly straightforward. To mock something just write regular function named appropriately:<\/p>\n<pre><code class=\"bash\">function node() {   mock::log node \"$@\"   if [[ \"$1\" == --version ]]; then     echo v16.4.0;   fi }<\/code><\/pre>\n<p><em>mock::log<\/em> command makes it possible to assert invocation of mocked commands as follows:<\/p>\n<pre><code class=\"bash\">node --version mock::called node --version<\/code><\/pre>\n<p>Under the hood, <em>mock::log <\/em>writes every and each invocation to temporary file; <em>mock::called &#8212;<\/em> just <em>greps<\/em> this file. Also regular expressions could be used to assert args:<\/p>\n<pre><code class=\"bash\">mock::called node '.*' [[ $(mock::called_times node '.*') -eq 999 ]]<\/code><\/pre>\n<p>Optionally, the following utility allows to declare mocks without implementation:<\/p>\n<pre><code class=\"bash\">mock node node --version<\/code><\/pre>\n<p>Lots of shell commands require <em>stdin<\/em> as well as args. That was one of the reasons this shell-based framework to take its place. For example, to pipe something to script\/command under test just use basic shell structures:<\/p>\n<pre><code class=\"bash\">echo \"Lorem ipsum...\" | command under test<\/code><\/pre>\n<p>Or:<\/p>\n<pre><code class=\"bash\">diff .\/file.txt &lt;(printf '%s\\n' 'first line' 'second line' 'third line')<\/code><\/pre>\n<p>To mock command which require <em>stdin<\/em> and assert such input:<\/p>\n<pre><code class=\"bash\"># Arrange mock::stdin wc # Act run # Assert mock::consumed wc \"some line\"<\/code><\/pre>\n<h3>How to report test results<\/h3>\n<p>Ofcourse, test result should be binary &#8212; either passed or failed. There should be no need to interpret test result manually. This framework offers a simple approach. As mentioned above, test is considered red if exit code is non-zero, otherwise it is green.<\/p>\n<p>On the other hand, parsable output is a mandatory feature for each testing framework to make it possible to integrate with IDE, CI\/CD and whatever else.<\/p>\n<p>The framework produces <em>TSV<\/em> . For example:<\/p>\n<pre><code class=\"bash\">$ add.spec.sh REDSHU CASE 2020-02-24T12:39:01+0300 add.sh 1 should add two numbers REDSHU PASS 2020-02-24T12:39:02+0300 add.sh 1<\/code><\/pre>\n<p>Columns:<\/p>\n<ol>\n<li>\n<p>REDSHU &#8212; constant prefix<\/p>\n<\/li>\n<li>\n<p>CASE\/PASS\/FAIL &#8212; test case status<\/p>\n<\/li>\n<li>\n<p>Timestamp<\/p>\n<\/li>\n<li>\n<p>Script\/command under test<\/p>\n<\/li>\n<li>\n<p>Test case ID<\/p>\n<\/li>\n<li>\n<p>Requirement description<\/p>\n<\/li>\n<\/ol>\n<p><em>Jenkins<\/em>, <em>Bamboo<\/em>, <em>GitLab CI\/CD<\/em> and all of them support <em>JUnit<\/em> reporting format. This framework comes with special utility to reformat test output to <em>JUint<\/em> format:<\/p>\n<pre><code class=\"bash\">$ add.spec.sh | red-shu-2-junit.sh &lt;testsuites name=\"CI Tests\">   &lt;testsuite name=\"add.sh\" tests=\"1\" timestamp=\"2020-02-24T12:39:43+0300\" time=\"0\">     &lt;testcase name=\"add.sh should add two numbers\">&lt;\/testcase>   &lt;\/testsuite> &lt;\/testsuites><\/code><\/pre>\n<p>&#8230;<\/p>\n<p>I would be grateful if you share your experience of testing of CI\/CD scripts. Please try <a href=\"https:\/\/github.com\/redneckz\/red-shell-unit\" rel=\"noopener noreferrer nofollow\"><u>Red Shell Unit<\/u><\/a>. I`m open to your proposals and new ideas.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/655297\/\"> https:\/\/habr.com\/ru\/articles\/655297\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-2\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p>Once a upon a time I moved to brand new project. And without much thought, I decided to take DevOps responsibilities (after a long period of Frontend). Huge mono-repository (Angular and Node.js) gives rise to many specific problems. And this project was no exception. At the very beginning CI\/CD duration was about 1.5h. And that was the biggest problem to take care of.<\/p>\n<p>But at first, I want to talk about &#171;<strong>B<\/strong>ourne<em> <\/em><strong>a<\/strong>gain<em> <\/em><strong>sh<\/strong>ell&#187;, cause CI\/CD automation almost entirely was implemented by means of shell scripts (Bash). In context of huge mono-repo even regular build becomes a piece of odd stuff. That`s why a huge amount of scripts with complicated logic inside (build, test, deploy, generation of release notes, collection of logs and metrics, &#8230;) was another significant problem.<\/p>\n<p>Regardless of lang, quality of complex logic should be under control. It is no secret that key aspect of code quality control &#8212; is tests. For example, to refactor safely major logic should be covered with tests. We decided to keep shell scripts untouched and cover the major ones with tests before any refactoring. Of course, it is possible to use Jest (or Mocha) with a bunch of awful utils to test shell scripts. This approach is a bit wordy and has no value if scripts under test are written in Bash. Also I have no idea how to mock external shell commands (such as <em>curl, ls, touch, npm, &#8230;<\/em>) with Jest or similar framework.<\/p>\n<p>In our particular case it was great to have the following to test CI\/CD scripts:<\/p>\n<ul>\n<li>\n<p>Tiny testing framework with no assertions API. Double square brackets statement covers majority of cases; check <a href=\"https:\/\/devhints.io\/bash#conditionals\" rel=\"noopener noreferrer nofollow\"><u>this<\/u><\/a> out for more detail.<\/p>\n<\/li>\n<li>\n<p>Tests isolation (for parallelization).<\/p>\n<\/li>\n<li>\n<p>Mock API (to mock external shell commands).<\/p>\n<\/li>\n<li>\n<p>JUnit reporting format (to integrate with CI).<\/p>\n<\/li>\n<\/ul>\n<p>It was not hard to implement appropriate testing framework from scratch <a href=\"https:\/\/github.com\/redneckz\/red-shell-unit\" rel=\"noopener noreferrer nofollow\"><u>https:\/\/github.com\/redneckz\/red-shell-unit<\/u><\/a><\/p>\n<h3>How to run tests<\/h3>\n<p>Lets start from the very basic example:<\/p>\n<p>add.sh<\/p>\n<pre><code class=\"bash\">#!\/usr\/bin\/env bash echo $(( $1 + $2 ))<\/code><\/pre>\n<p>This script just adds the first argument to the second one and outputs the result to <em>stdout.<\/em><\/p>\n<p>One of the approaches is to place tests along with modules. This approach simplifies things, cause technically there is no need to explicitly point out path to shell script under test. Path can be computed relative to spec.<\/p>\n<p>add.spec.sh<\/p>\n<pre><code class=\"bash\">#!\/usr\/bin\/env red-shu.sh  it 'should add two numbers'   # Act   result=$(run 2 2)   # Assert   [[ $result -eq 4 ]] ti<\/code><\/pre>\n<p>In order to make this test up and running shebang should reference <em>red-shu.sh.<\/em> It is a wrapper which provides core functionality (<em>it\/ti\/run\/mock<\/em>) and tests isolation<em>.<\/em><\/p>\n<p><em>it\/ti <\/em>commands<em> <\/em>declare isolated block to test the only requirement. For each block subshell is initiated. If block exits with non-zero code, appropriate requirement turns red. Framework provides <em>run <\/em>command to execute script\/command under test. Such decoupling makes refactoring less expensive. For example, you can move or rename scripts along with corresponding tests without difficulty.<\/p>\n<p>The best technical solution is a solution that does not require any new components or modules. That`s why exit codes are used to interpret test results. In this case, no dedicated assertion API is needed. Bash`s double square brackets statement is powerful enough to cover most needs and assert anything you ever wanted. Please check <a href=\"https:\/\/devhints.io\/bash#conditionals\" rel=\"noopener noreferrer nofollow\"><u>this<\/u><\/a> out.<\/p>\n<p>Also commands like <em>grep<\/em> or <em>diff <\/em>are suitable for more complicated kinds of assertions.<\/p>\n<h3>How to mock external commands<\/h3>\n<p>Mocks are just necessary to write unit tests. Most dependencies should be mocked to test module functionality in isolation and highlight external parts of contract.<\/p>\n<p>The implementation of mocks turned out to be surprisingly straightforward. To mock something just write regular function named appropriately:<\/p>\n<pre><code class=\"bash\">function node() {   mock::log node \"$@\"   if [[ \"$1\" == --version ]]; then     echo v16.4.0;   fi }<\/code><\/pre>\n<p><em>mock::log<\/em> command makes it possible to assert invocation of mocked commands as follows:<\/p>\n<pre><code class=\"bash\">node --version mock::called node --version<\/code><\/pre>\n<p>Under the hood, <em>mock::log <\/em>writes every and each invocation to temporary file; <em>mock::called &#8212;<\/em> just <em>greps<\/em> this file. Also regular expressions could be used to assert args:<\/p>\n<pre><code class=\"bash\">mock::called node '.*' [[ $(mock::called_times node '.*') -eq 999 ]]<\/code><\/pre>\n<p>Optionally, the following utility allows to declare mocks without implementation:<\/p>\n<pre><code class=\"bash\">mock node node --version<\/code><\/pre>\n<p>Lots of shell commands require <em>stdin<\/em> as well as args. That was one of the reasons this shell-based framework to take its place. For example, to pipe something to script\/command under test just use basic shell structures:<\/p>\n<pre><code class=\"bash\">echo \"Lorem ipsum...\" | command under test<\/code><\/pre>\n<p>Or:<\/p>\n<pre><code class=\"bash\">diff .\/file.txt &lt;(printf '%s\\n' 'first line' 'second line' 'third line')<\/code><\/pre>\n<p>To mock command which require <em>stdin<\/em> and assert such input:<\/p>\n<pre><code class=\"bash\"># Arrange mock::stdin wc # Act run # Assert mock::consumed wc \"some line\"<\/code><\/pre>\n<h3>How to report test results<\/h3>\n<p>Ofcourse, test result should be binary &#8212; either passed or failed. There should be no need to interpret test result manually. This framework offers a simple approach. As mentioned above, test is considered red if exit code is non-zero, otherwise it is green.<\/p>\n<p>On the other hand, parsable output is a mandatory feature for each testing framework to make it possible to integrate with IDE, CI\/CD and whatever else.<\/p>\n<p>The framework produces <em>TSV<\/em> . For example:<\/p>\n<pre><code class=\"bash\">$ add.spec.sh REDSHU CASE 2020-02-24T12:39:01+0300 add.sh 1 should add two numbers REDSHU PASS 2020-02-24T12:39:02+0300 add.sh 1<\/code><\/pre>\n<p>Columns:<\/p>\n<ol>\n<li>\n<p>REDSHU &#8212; constant prefix<\/p>\n<\/li>\n<li>\n<p>CASE\/PASS\/FAIL &#8212; test case status<\/p>\n<\/li>\n<li>\n<p>Timestamp<\/p>\n<\/li>\n<li>\n<p>Script\/command under test<\/p>\n<\/li>\n<li>\n<p>Test case ID<\/p>\n<\/li>\n<li>\n<p>Requirement description<\/p>\n<\/li>\n<\/ol>\n<p><em>Jenkins<\/em>, <em>Bamboo<\/em>, <em>GitLab CI\/CD<\/em> and all of them support <em>JUnit<\/em> reporting format. This framework comes with special utility to reformat test output to <em>JUint<\/em> format:<\/p>\n<pre><code class=\"bash\">$ add.spec.sh | red-shu-2-junit.sh &lt;testsuites name=\"CI Tests\">   &lt;testsuite name=\"add.sh\" tests=\"1\" timestamp=\"2020-02-24T12:39:43+0300\" time=\"0\">     &lt;testcase name=\"add.sh should add two numbers\">&lt;\/testcase>   &lt;\/testsuite> &lt;\/testsuites><\/code><\/pre>\n<p>&#8230;<\/p>\n<p>I would be grateful if you share your experience of testing of CI\/CD scripts. Please try <a href=\"https:\/\/github.com\/redneckz\/red-shell-unit\" rel=\"noopener noreferrer nofollow\"><u>Red Shell Unit<\/u><\/a>. I`m open to your proposals and new ideas.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/655297\/\"> https:\/\/habr.com\/ru\/articles\/655297\/<\/a><br \/><\/br><\/br><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-405256","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/405256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=405256"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/405256\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=405256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=405256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=405256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}