{"id":398810,"date":"2024-06-29T14:17:39","date_gmt":"2024-06-29T14:17:39","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=398810"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=398810","title":{"rendered":"ArGOtecture<\/span>"},"content":{"rendered":"
<\/div>\n
\n
\n
\n
\n
<\/figcaption><\/figure>\n

This is an article that describes my vision of building a system that actively uses Go as the main programming language and SOA\/microservices as a design paradigm.\u00a0<\/p>\n

Here I will try to cover 4 chapters that together allow us to build a solid and reliable system.<\/p>\n

1. System design<\/h4>\n

Let’s start from the global view of a system that can handle web requests. This schema shows which layers our system has.<\/p>\n

<\/figcaption><\/figure>\n

On the left side, we can see\u00a0Users<\/strong>. They can be real\u00a0persons<\/u>\u00a0or third-party\u00a0machines<\/u>\u00a0that use us as a provider of some service. They interact with\u00a0the Clients layer<\/strong>\u00a0and in some cases with\u00a0the API layer.\u00a0<\/strong><\/p>\n

The Clients layer\u00a0<\/strong>contains applications like FrontOffice, BackOffice, iosApp, etc. Those applications can be written in JS\/Kotlin\/Swift\/etc. Generally, they’re UI applications.\u00a0This is what is called FrontEnd. The purpose of this layer is to provide a good user experience.<\/p>\n

The API layer<\/strong>\u00a0receives requests from\u00a0the Clients layer\u00a0<\/strong>or directly from\u00a0the Users<\/strong>. This layer contains the API Gateway(GTW) and SingleSignOn(Auth) services. The API Gateway allows us to terminate SSL Requests, do caching, do authentication\/etc. The SSO(Auth) service encapsulates information about users, roles, and permissions granted to entities and operations. It can use RBAC\/ABAC approach. When we receive a request in GTW we retrieve information about the user, its roles, permissions from SSO and inject this information into the request and pass it to the next layer.<\/p>\n

The BFF(BackendForFrontend) layer\u00a0<\/strong>contains specific APIs that were built for specific clients cause sometimes we want to have different behavior in different clients. We can create a separate API for the ios application or for the integration with a big third-party client. If we don’t need to have some specific behavior for clients we can use Generic API for simplicity.\u00a0<\/p>\n

The Deep Backend layer.\u00a0<\/strong>This is the place where general magic happens. This block contains our services that handle business needs. They receive requests from\u00a0BFF<\/strong>\u00a0and go to\u00a0the External Service Gateways.\u00a0<\/strong>This layer cannot interact with the external world directly.\u00a0Deep Backend\u00a0<\/strong>contains some business services and some utility services.<\/p>\n

Lock Coordinator<\/u>\u00a0is a service using for distributed transactions(should be avoided as much as possible) and distributed locks(when you want to guarantee that some operation with the specific parameters will not be executed multiple at the same time).<\/p>\n

Feature Hub<\/u>\u00a0is a service using for enabling or disabling some functionality of our system. It’s a single point of this truth that can say is a specific feature is enabled or not. We can even build AB\/MV tests here. In some cases, this service can be moved to\u00a0the API layer.<\/strong><\/p>\n

Object Storage<\/u>\u00a0is a service using for file handling. This is the only place where our files are stored. It can be backed by S3\/EFS\/etc. If some service wants to store or retrieve a file it should interact with this service.<\/p>\n

Message Bus<\/u>\u00a0is a service(it can be just a Kafka cluster ) using for asynchronous communications. Every service can send events(something that happened) or just async request\/response to communicate with another service.<\/p>\n

The External Service Gateway<\/strong>\u00a0layer sends requests out of our system. When we have the integration with a third-party service we place a service here. For example, if you want to communicate with Google you place google-eg svc here. When your services want to communicate with Google they come to this service and it goes to the real world. This approach allows you to mock this part of request execution in some cases and switch to another provider in a short period of time if it’s needed.<\/p>\n

2. Service structure<\/h4>\n

Having looked at our system from a higher perspective, we can now dive into services and their structure.<\/p>\n

It is recommended to put the service source code into a separate directory and the API of this service into a directory with the\u00a0_sdk<\/strong>\u00a0suffix.<\/p>\n

<\/figcaption><\/figure>\n

The SDK repo contains information about how other services can communicate with ours.<\/p>\n

We put the interface of the service into the proto directory and generate clients for other languages by Makefile.<\/p>\n

The service is placed in another directory. This design allows clients to include only the part with the interface and not the whole service.<\/p>\n

3. Basis repo<\/h4>\n

Sometimes we have the functionality that can be shared between services. This is the basis repo. The basis is a set of utility libraries like a framework.<\/p>\n

<\/figcaption><\/figure>\n

It can contain common HTTP-middlewares, logger, RDBMS-connectors, etc.<\/p>\n

It can be mono-repo like\u00a0\/basis<\/strong>\u00a0or split into multiple repositories like\u00a0\/basis-core,<\/strong>\u00a0\/basis-network<\/strong>,\u00a0\/basis-storage, etc.<\/strong><\/p>\n

Creating this repository allows us to extract common modules and reuse them. At the same time, this block of code can be\u00a0transferred to the support of the infra team.<\/p>\n

4.\u00a0Useful patterns<\/h4>\n

1) When we have a system where every service goes to another service we have a distributed monolith. It means that our system is very unstable. When one service goes down it can affect multiple parts of the system.<\/p>\n

The better way is to use the\u00a0CQRS pattern<\/a>. When we update some data in some service that is the master of those data we produce an event. This event describes what was changed.<\/p>\n

Other services listen to this event and update their copy of the data. When they need to fetch some information they don’t go to the master-service they go to their own copy. It reduces network calls and improves the stability of the system.<\/p>\n

Obviously, we may have some delay between updating data in the main service and copies of the data, but generally speaking, we have this delay even when we have a synchronous call. Once you have received the data from the main service, it can be updated and you will have an outdated version of the data.<\/p>\n

2) When we need to produce some events\/messages from our service It’s recommended to use the\u00a0transactional outbox pattern<\/a>.<\/p>\n

We can use this pattern even if we need to do some synchronous calls. For example, we want to update some information in another service after some action.<\/p>\n

In this case, we can run this call\u00a0on hit<\/u>\u00a0and lost this call if the second service is in downtime (of course we can use retries but we don’t know when the target service will be ready) OR we can create a task in our database with the specific type and parameters and perform this task in the background. if we cannot perform the task we just return it to the queue.\u00a0The advantages of this pattern are in the use of transactions and therefore in maintaining consistency.<\/p>\n<\/div>\n<\/div>\n<\/div>\n

<\/div>\n


\u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 https:\/\/habr.com\/ru\/articles\/675002\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/div>\n
\n
\n
\n
\n
<\/figcaption><\/figure>\n

This is an article that describes my vision of building a system that actively uses Go as the main programming language and SOA\/microservices as a design paradigm.\u00a0<\/p>\n

Here I will try to cover 4 chapters that together allow us to build a solid and reliable system.<\/p>\n

1. System design<\/h4>\n

Let’s start from the global view of a system that can handle web requests. This schema shows which layers our system has.<\/p>\n

<\/figcaption><\/figure>\n

On the left side, we can see\u00a0Users<\/strong>. They can be real\u00a0persons<\/u>\u00a0or third-party\u00a0machines<\/u>\u00a0that use us as a provider of some service. They interact with\u00a0the Clients layer<\/strong>\u00a0and in some cases with\u00a0the API layer.\u00a0<\/strong><\/p>\n

The Clients layer\u00a0<\/strong>contains applications like FrontOffice, BackOffice, iosApp, etc. Those applications can be written in JS\/Kotlin\/Swift\/etc. Generally, they’re UI applications.\u00a0This is what is called FrontEnd. The purpose of this layer is to provide a good user experience.<\/p>\n

The API layer<\/strong>\u00a0receives requests from\u00a0the Clients layer\u00a0<\/strong>or directly from\u00a0the Users<\/strong>. This layer contains the API Gateway(GTW) and SingleSignOn(Auth) services. The API Gateway allows us to terminate SSL Requests, do caching, do authentication\/etc. The SSO(Auth) service encapsulates information about users, roles, and permissions granted to entities and operations. It can use RBAC\/ABAC approach. When we receive a request in GTW we retrieve information about the user, its roles, permissions from SSO and inject this information into the request and pass it to the next layer.<\/p>\n

The BFF(BackendForFrontend) layer\u00a0<\/strong>contains specific APIs that were built for specific clients cause sometimes we want to have different behavior in different clients. We can create a separate API for the ios application or for the integration with a big third-party client. If we don’t need to have some specific behavior for clients we can use Generic API for simplicity.\u00a0<\/p>\n

The Deep Backend layer.\u00a0<\/strong>This is the place where general magic happens. This block contains our services that handle business needs. They receive requests from\u00a0BFF<\/strong>\u00a0and go to\u00a0the External Service Gateways.\u00a0<\/strong>This layer cannot interact with the external world directly.\u00a0Deep Backend\u00a0<\/strong>contains some business services and some utility services.<\/p>\n

Lock Coordinator<\/u>\u00a0is a service using for distributed transactions(should be avoided as much as possible) and distributed locks(when you want to guarantee that some operation with the specific parameters will not be executed multiple at the same time).<\/p>\n

Feature Hub<\/u>\u00a0is a service using for enabling or disabling some functionality of our system. It’s a single point of this truth that can say is a specific feature is enabled or not. We can even build AB\/MV tests here. In some cases, this service can be moved to\u00a0the API layer.<\/strong><\/p>\n

Object Storage<\/u>\u00a0is a service using for file handling. This is the only place where our files are stored. It can be backed by S3\/EFS\/etc. If some service wants to store or retrieve a file it should interact with this service.<\/p>\n

Message Bus<\/u>\u00a0is a service(it can be just a Kafka cluster ) using for asynchronous communications. Every service can send events(something that happened) or just async request\/response to communicate with another service.<\/p>\n

The External Service Gateway<\/strong>\u00a0layer sends requests out of our system. When we have the integration with a third-party service we place a service here. For example, if you want to communicate with Google you place google-eg svc here. When your services want to communicate with Google they come to this service and it goes to the real world. This approach allows you to mock this part of request execution in some cases and switch to another provider in a short period of time if it’s needed.<\/p>\n

2. Service structure<\/h4>\n

Having looked at our system from a higher perspective, we can now dive into services and their structure.<\/p>\n

It is recommended to put the service source code into a separate directory and the API of this service into a directory with the\u00a0_sdk<\/strong>\u00a0suffix.<\/p>\n

<\/figcaption><\/figure>\n

The SDK repo contains information about how other services can communicate with ours.<\/p>\n

We put the interface of the service into the proto directory and generate clients for other languages by Makefile.<\/p>\n

The service is placed in another directory. This design allows clients to include only the part with the interface and not the whole service.<\/p>\n

3. Basis repo<\/h4>\n

Sometimes we have the functionality that can be shared between services. This is the basis repo. The basis is a set of utility libraries like a framework.<\/p>\n

<\/figcaption><\/figure>\n

It can contain common HTTP-middlewares, logger, RDBMS-connectors, etc.<\/p>\n

It can be mono-repo like\u00a0\/basis<\/strong>\u00a0or split into multiple repositories like\u00a0\/basis-core,<\/strong>\u00a0\/basis-network<\/strong>,\u00a0\/basis-storage, etc.<\/strong><\/p>\n

Creating this repository allows us to extract common modules and reuse them. At the same time, this block of code can be\u00a0transferred to the support of the infra team.<\/p>\n

4.\u00a0Useful patterns<\/h4>\n

1) When we have a system where every service goes to another service we have a distributed monolith. It means that our system is very unstable. When one service goes down it can affect multiple parts of the system.<\/p>\n

The better way is to use the\u00a0CQRS pattern<\/a>. When we update some data in some service that is the master of those data we produce an event. This event describes what was changed.<\/p>\n

Other services listen to this event and update their copy of the data. When they need to fetch some information they don’t go to the master-service they go to their own copy. It reduces network calls and improves the stability of the system.<\/p>\n

Obviously, we may have some delay between updating data in the main service and copies of the data, but generally speaking, we have this delay even when we have a synchronous call. Once you have received the data from the main service, it can be updated and you will have an outdated version of the data.<\/p>\n

2) When we need to produce some events\/messages from our service It’s recommended to use the\u00a0transactional outbox pattern<\/a>.<\/p>\n

We can use this pattern even if we need to do some synchronous calls. For example, we want to update some information in another service after some action.<\/p>\n

In this case, we can run this call\u00a0on hit<\/u>\u00a0and lost this call if the second service is in downtime (of course we can use retries but we don’t know when the target service will be ready) OR we can create a task in our database with the specific type and parameters and perform this task in the background. if we cannot perform the task we just return it to the queue.\u00a0The advantages of this pattern are in the use of transactions and therefore in maintaining consistency.<\/p>\n<\/div>\n<\/div>\n<\/div>\n

<\/div>\n


\u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 https:\/\/habr.com\/ru\/articles\/675002\/<\/a>
<\/br><\/br><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-398810","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/398810","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=398810"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/398810\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=398810"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=398810"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=398810"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}