A sample of a Microservices architecture for the domain of a simple Mobile Phone/Telecoms domain. To explore some of the different aspects of a microservices architecture.
(Warning: This code is not suitable for Production use)
You can try out the system by executing a Walkthrough of the Sceanrio - Order a Mobile, as follows:
Prerequissites: Install and launch Docker Install Visual Studio Code Install REST Client Visual Studio Code plugin
- Launch the system using docker:
$ docker-compose up - Wait a minute
- Verify the test system has successfully launched by executing the checks in the file
\docs\ManualTesting\CheckSystem.http - Connect to the SQL Server database using the Admin account:
localhost,5433, Login:SA, Password:Pass@word - Verify the database is running by running the SQL script
\docs\ManualTesting\CheckDatabase.sql - Open the file
docs\ManualTesting\ExecuteScenarios.http - Create a Customer - Click
Send Requestunder### Step 1 - Create a Customer - Observe the Customer was created with the SQL:
select * from Mobiles.Customers - Order a Mobile - Click
Send Requestunder### Step 2 - Order a Mobile - Observe the Mobile was created with the SQL:
select * from Mobiles.Mobiles - Observe the Mobile Order was created with the SQL:
select * from Mobiles.Orders - Simulate the External SIM Card System completing the Mobile Provision Order - Click
Send Requestunder### Step 3 - The External Service has completed the Mobile Provision Order - Observe that the Mobile is
WaitingForActivatewith the SQL:select * from Mobiles.Mobiles - When the External SIM Card System has processed the Order the Activation Code would then be mailed to the Customer to complete the Activaion
Microservices - also known as the microservice architecture - is an architectural style that structures an application as a collection of services that are
- Highly maintainable and testable
- Loosely coupled
- Independently deployable
- Organized around business capabilities
- Owned by a small team
The microservice architecture enables the rapid, frequent and reliable delivery of large, complex applications. It also enables an organization to evolve its technology stack.
From microservices.io
| Aspect | Implementation | |
|---|---|---|
| Data | SQL Server, Dapper, Entity Framework | |
| Sovereignty | Seperate database schemas and logins | |
| Communication | ||
| Event Bus | AWS SNQ, SQS. JustSaying. Minimal Event Bus. GoAws | |
| Outbox Pattern | In Mobiles.API, EventPublisherService and Checkers | |
| API Gateway | Not Implemented | |
| Resiliency | ||
| Retry | Polly/HttpClientFactory | |
| Health Monitoring | ASP.NET Core Health Checks Middleware | |
| Observability | Promethues, Grafana | |
| Structured Logging | Serilog, Seq | |
| Distributed Tracing | Not Implemented | |
| Security | Not Implemented | |
| Testing | ||
| Manual | Walkthroughs using the REST Client Visual Studio Code plugin | |
| Unit tests | xUnit, Moq, Fluent assertions | |
| End-to-end tests | Scenario Scripts in EndToEndApiLevelTests | |
| Load tests | k6 Load Testing tool | |
| Deployment | ||
| Containers | docker-compose | |
| Documentation | Markdown, Readme.md (this document) | |
| Domain | Sequence diagrams (sequencediagram.org), Statecharts (smcat) | |
| Architecture | 4+1 Model, C4 Model, C4 PlantUML Visual Studio Code plugin, LADRs (TODO) |
Each microservice should be independent from all the others, so it should own it's own data, this is known as Data Soveriengty. This allows the schema of the data to evolve without impacting any other services.
As the services should be loosley coupled, we avoid long chains of service calls which could result in temporal coupling. Instead services use the Publish/Subscribe pattern to communicate asynchronously using an Event Bus (also known as Message Bus).
If the Front end apps call the microservices directly, then they are coupled and changes to the back end will also require updates to the front end. An API Gateway (also known as a Back-end For Front-end (BFF)) can sit inbetween and act as a facade for the front end. It can perform re-routing of inbound requests and aggregate together the responses into a single response. The back end microservices can then evolve independently from the front end.
To keep the system resilient we should provide:
- Retry mechnism - retry failed requests, so that temporary or intermittent faults do not cause downtime
- Health checks - a way to check on the health of services
- Structured logging - log the details of what the service has done, in a format which ca be easily searched
- Metrics - to observe the functioning of the system as a whole
Microservices should perform the standard security practices of Authentication (verifying identify) and Authroization (verifying the principal has the correct claims). Performance shoud also be considered since multiple microservices may be involved in processing a request, each needing to do security checks. Also microservices should not be abe to access each others data.
Individual microservices can be tested with automated, fast running unit tests. These can keep the code from breaking and also help improve the design and keep components loosely coupled. Testing multiple services interacting with each other is more complicated. There are multiple ideas about how to do this, such as:
- End-to-end tests driven through a UI automation tool
- Testing services using faked dependencies such as the EF Core In-Memory database
- Running tests againat a simulated system running in Docker containers. For example using GoAws to simulate AWS SNS/SQS
Ultimately whatever approach is taken the goal is to have tests which can give confidence that the system as a whole works (and has not regressed) and are automated so they minimise the need for manual testing.
Microservices can be deployed in many ways. One way is by using containers, which allow each instance of a service to run in isolation from all others. The container can also be restarted if it crashes, by an Orchestrator. Another benefit of this approach is that test systems can be easily created and deployed using the same techniques.
Not directly related to microservices, but documentation is essential for future developers to quickly familiarise themselves with the system and for long term maintainability. We should document:
- The domain - record the business processes so future developers can see what problem the system is solving
- The architecture - record the high level structure of the system so future developers can see how the pieces fit together
The system is a Mobile Phone/Telecoms system which supports the ordering and cancelling of Mobile Phones. The Ordering process is described by the following diagram and User Stories:
User Story - Order a Mobile
As a SalesAgent I want to Order a Mobile
Given: The Order details are captured
When: The Order is placed
Then: A SimCard Order is sent to the External Sim Cards Provider
Then: The Mobile is saved in state WaitingForActivation
User Story - Activate a Mobile
As a Sales Agent I want to Activate a Mobile
Given: The Mobile is in state AwaitingActivation
When: The Activation Code is supplied
Then: The Activation Code is sent to the External Mobile Telecoms Network
Then: The Mobile is in state Live
During it's lifecycle a Mobile can transition between a number of states. These states such as New, ProcessingProvision, Live, etc. are shown in the Mobile States statechart.
In order to transition to a new state an Order must be sent to an External system. This In-flight order then progresses through it's own states of New, Processing, Sent and Completed. Adding these into the diagram results in the Mobile States with In-flight Order States statechart.
The greyed out states have not been implemented at present.
For information on statecharts, see Welcome to Statecharts and Statecharts.
- The User submits the MobileRequest in the Front-end
- The Front-end posts the MobileRequest to the BFF
- The BFF posts the MobileRequest to the Mobiles API
- The Mobiles API writes the MobileRequest to it's DB, Mobiles.Orders table
- The Mobiles API publishes the event to the MobileRequested topic
- The SimCards Microservice is subscribed to the MobileRequested topic so receives the event
- The SimCards Microservice writes the MobileRequest it's DB, SimCards.Orders table
- The SimCards Microservice posts a SmCardOrder to the External SimCards Provider API
The following diagrams map the architecture of the system. They following the 4+1 Model, the C4 Model and us the C4 PlantUML Visual Studio Code plugin.
Source: \docs\MobileC4Context.puml
Source: \docs\MobileC4Containers.puml
Source: \docs\MobileC4Components.puml
The goals of the tests are to ensure that the system is Reliable, Robust, Modifiable, Understandable and has acceptable Performance.
- Reliable - the users can use the system as intended and without encountering bugs.
- Robust - the system operates for an extended period of time without crashing.
- Modifiable - the system can be adapted to meet future requirements.
- Understandable - the system design is not more complicated than it needs to be.
- Performance - the system has acceptable performance.
To achieve these goals, my plan is to test the system from the bottom up with unit tests, and from the top down with manual and automated tests. These are illustrated on the standard testing pyramid:
The approximate positions of the different types of tests are shown with the numbers:
- Unit Tests
- API level Automated End-to-end Tests
- API level Manual Tests
- Load Tests
- Manual Tests
These are standard unit tests which test individual units in isolation and are fast running.
These support the testing goals as follows:
- Reliable - ensure each unit is reliable when run in isolation
- Modifiable - they are fast to run and so can be run after every code change to prevent regressions
- Understandable - the code can be refactored and simplified. These tests can then be re-run to check for and prevent regressions
These test each of the most important Scenarios which the system can execute.
These are not true "End-to-end" tests since they are at the API level rather than the UI. I chose to use the API level as a seam to test against, as this should provide confidence that the back end system has not regressed.
The tests are executed against a test system which is started using docker-compose. This launches the following:
- each service running in a seperate container
- the SQL Server database running in a container
- GoAws - a simulation of AWS SNS/SQS running in a container
The tests are then executed against this test system and verified by querying the database.
These support the testing goals as follows:
- Reliable - they match the primary Use Cases of the system, and so verify that the system works as intended
- Modifiable - after the system is modified, the tests can be re-run to check for and prevent regressions
- Understandable - the system can be refactored and simplified, then the tests can be re-run to check for and prevent regressions
These also test the most important Scenarios which the system can execute.
They are run against the same test system as above, launched through docker-compose. Once the system is running they are executed manually by using the REST Client Visual Studio Code plugin and executing the Scenarios in the file: \docs\ManualTesting\ExecuteScenarios.http
These support the testing goals as follows:
- Reliable - they can be run manaully to aid in troubleshooting and diagnosing
- Understandable - they allow each step of a Scenario to be run and checked
The Load Tests are performed using k6, which is a command line Load testing tool. The tests to be executed by k6 are defined in a JavaScript file (/docs/LoadTesting/LoadTest.js). This script details the actions to be performed (the Scenarios) and defines the number of Virtual Users and iterations. It is currently set to launch 5 simultaneous Virtual Users each performing 3 iterations of one of 5 Scenarios. So 5 * 5 * 3 = 75 tests in total.
During the test run, the Virtual User needs data to use for the current test iteration. This is pre-generated into a JSON file by the LoadTestingWebService. This also allows each Virtual User to request an identifier, which it can use to ensure that it gets it's own specific data for each test iteration it runs.
The Load Tests support the testing goals as follows:
- Reliable - the Load Test Scenarios match the primary use cases of the system, and so verify that the system works as intended
- Robust - they simulate a number of users simultaneously using the system and verify that it does not crash or have concurrency bugs
- Modifiable - after the system is modified, the tests can be re-run to check for and prevent regressions
- Understandable - the system can be refactored and simplified, then the tests can be re-run to check for and prevent regressions
- Performance - check the performance of the system (and benchmark it) with a number of simultaneous users
Manual Tests can be performed by through the Front End application. The main focus of these is to catch higher level, conceptual types of errors (have we built the right thing) and usability problems.
The Manual Tests support the testing goals as follows:
- Reliable - they verify that the system works as intended
Execute using the Visual Studio test runner.
- Start Docker Desktop
- Build the test system:
docker-compose -f docker-compose-test.yml -f docker-compose-test.override.yml build
- Launch the test system
docker-compose -f docker-compose-test.yml -f docker-compose-test.override.yml up
- Wait a minute
- Verify the test system has successfully launched by executing the checks in the file \docs\ManualTesting\CheckSystem.http
- Verify the database is running by running the SQL script \docs\ManualTesting\CheckDatabase.sql
Open the EndToEndTests.sln solution in Visual Studio. Execute the tests in the EndToEndApiLevelTests project using the Visual Studio test runner.
- Start Docker Desktop
- Build the test system:
docker-compose -f docker-compose-test.yml -f docker-compose-testoverride.yml build
- Launch the test system
docker-compose -f docker-compose-test.yml -f docker-compose-testoverride.yml up
-
Wait a minute
-
Verify the test system has successfully launched (see above)
-
Open \docs\ManualTesting\ExecuteScenarios.http in the REST Client Visual Studio Code plugin
-
Click 'Send Request' against the first Step of the first Scenario
-
Verify the changes in the database by running the SQL script: \docs\ManualTesting\CheckDatabase.sql
-
Repeat for the other Steps and Scenarios
- Install k6 and Cmder (optional)
- Start Docker Desktop
- Launch the test system
λ docker-compose -f docker-compose-test.yml -f docker-compose-test.override.yml up
- Launch the LoadTestingWebApp
λ cd docs\LoadTesting\LoadTestingWebService\LoadTestingWebService
λ dotnet run
- Run the k6 Load Tests (will take some time). Open a new Cmder tab:
λ cd docs\LoadTesting
λ k6 run LoadTest.js
- Create a Customer
- Order a Mobile
- Mobile Order Completed
- Activate a Mobile
- Activate Order Completed
Inputs:
- POST Customer to the Mobiles Web Service
Outputs:
- Updates Mobiles database
- Creates a Customer
Inputs:
- POST Order to the Mobiles Web Service
Outputs:
- Updates Mobiles database
- Mobile State to ProcessingProvisioning
- Order State to Sent
- Updates SIM Cards database
- Order State to Sent
- Calls External SIM Card system
Inputs:
- Complete the Mobile Order in the External SIM Card system
Outputs:
- Updates SIM Cards database
- Order State to Completed
- Updates Mobiles database
- Mobile State to WaitingForActivation
- Order State to Completed
Inputs:
- POST Activate Order to the Mobiles Web Service
Outputs:
- Updates Mobiles database
- Mobile State to ProcessingActivation
- Order State to Sent
- Updates Mobile Telecoms Network database
- Order State to Sent
- Calls External Mobile Telecoms Network system
Inputs:
- Complete the Activate Order in the External Mobile Telecoms Network system
Outputs:
- Updates Mobile Telecoms Network database
- Order State to Completed
- Updates Mobiles database
- Mobile State to Live
- Order State to Completed
I have implemented a basic Front End to aid in understanding how the system works. It consists of an Angular app which talks directly to the microservice APIs (rather than a Back End for Front End which has not been implemented at this time).
To launch the Front End:
- Launch the system
λ docker-compose -f docker-compose.yml -f docker-compose-override.yml up
- Wait a minute
- Launch the Front End
λ cd src/FrontEnd/FrontEnd.WebApp
λ dotnet run
- Open a web browser and go to http://localhost:5000
The scenarios described above can be performed as follows:
Observe that the new Mobile PhoneNumber has now been ordered with the state of 'New' and 'Order in progress'
Access the fake External SIM Card System by clicking the 'External SIM Card System' button at the top. Then complete the SIM Card order by clicking on 'COMPLETE'
Switch back to the Mobiles System by clicking the 'Mobiles System' button at the top. Observe that the Mobile state is now 'WaitingForActivate'
Access the fake External Mobile Telecoms System by clicking the 'External Mobile Telecoms System' button at the top. Then complete the Activate order by clicking on 'COMPLETE'
Switch back to the Mobiles System by clicking the 'Mobiles System' button at the top. Then observe that the Mobile state is now 'Live'
Logging in the system uses the Serilog library. This supports strutured logging in which log entries include data, rather than being just plain text. This allows the logs to be more easily searched, filtered and analysed to assist in diagnosing problems.
Theses logs are also pushed to Seq which produces a dashboard where they can be viewed and queried.
The Seq Dashboard can be viewed at the address:
http://localhost:5341
The system is monitored, so we can verify that it is functioning correctly and prevent problems before they escalate. System Metrics are gathered using Prometheus and displayed in Grafana Dashboards.
To test the system and generate metrics, run the End-to-end tests, then observe in the Prometheus Control Panel and Grafana Dashboard.
The Prometheus Control Panel can be viewed at:
http://localhost:9090
View the metrics using the following PromQL queries:
| PromQL | Description |
|---|---|
| mobile_provisions | Total number of Mobile Provisions requested |
| mobile_provisions_completed | Total number of Mobile Provisions completed |
| mobile_provisions_inprogress | Current number of Mobile Provisions in progress |
| mobile_activates | Total number of Mobile Activates requested |
| mobile_activates_completed | Total number of Mobile Activates requested |
| mobile_activates_inprogress | Current number of Mobile Activates in progress |
| mobile_ceases | Total number of Mobile Ceases requested |
| mobile_ceases_completed | Total number of Mobile Ceases requested |
| mobile_ceases_inprogress | Current number of Mobile Ceases in progress |
| PromQL | Description |
|---|---|
| simcard_orders_sent | Total number of SIM Card orders sent |
| simcard_orders_completed | Total number of SIM Card orders completed |
| simcard_orders_inprogress | Current number of SIM Card orders in progress |
| simcard_orders_failed | Total number of SIM Card orders which failed |
| PromQL | Description |
|---|---|
| mobiletelecomsnetwork_activate_orders_sent | Total number of Activate orders sent |
| mobiletelecomsnetwork_activate_orders_completed | Total number of Activate orders completed |
| mobiletelecomsnetwork_activate_orders_inprogress | Current number of Activate orders in progress |
| mobiletelecomsnetwork_activate_orders_failed | Total number of Activate orders which failed |
| mobiletelecomsnetwork_cease_orders_sent | Total number of Cease orders sent |
| mobiletelecomsnetwork_cease_orders_completed | Total number of Cease orders completed |
| mobiletelecomsnetwork_cease_orders_inprogress | Current number of Cease orders in progress |
| mobiletelecomsnetwork_cease_orders_failed | Total number of Cease orders which failed |
The Grafana Dashboard can be viewed at:
http://localhost:3000
[microservices.io]
Microservice Architecture by Chris Richardson
https://microservices.io
[4+1 Model]
The "4+1" View Model of Software Architecture
Architectural Blueprints - The "4+1" View Model of Software Architecture by Kruchten, Philippe (1995)
https://www.cs.ubc.ca/~gregor/teaching/papers/4+1view-architecture.pdf
https://en.wikipedia.org/wiki/4%2B1_architectural_view_model
[C4 Model]
The C4 model for visualising software architecture
https://c4model.com/
[C4 PlantUML]
The C4 model using PlantUML
C4-PlantUML
https://github.com/RicardoNiepel/C4-PlantUML
[REST Client]
REST Client Visual Studio plugin
by Huachao Mao
https://marketplace.visualstudio.com/items?itemName=humao.rest-client
[Welcome to Statecharts]
Welcome to the world of Statecharts
https://statecharts.github.io/
[Statecharts]
Statecharts: A visual formalism for complex systems by David Harel (1986)
http://www.inf.ed.ac.uk/teaching/courses/seoc/2005_2006/resources/statecharts.pdf
[k6]
k6 - Load Testing tool
https://k6.io