Handling reporting on Microservices Architecture.
Ask around and most developers will tell you how frightening Microservices are. It shouldn't be this way. Below we handle the reporting aspect of the Microservices. This is one of the most important pillars of Microservices architecture. Microservices communicate with each other. There needs to be a mechanism that allows them to understand the state and other essential information from the other Microservices.
The Microservices architecture style migrates the data to separate databases(schemas). While this works well for services, it isn't good for reporting. There are four main techniques for handling reporting in Microservices.
Database pull model.
HTTP pull model.
Batch pull model.
Event-based push model.
The first three techniques pull data from each service's databases. These are examples of anti-pattern names called "reach-in reporting". The problem with pulling data is that it creates dependencies between services. The Solution to this anti-pattern is to use an event-based push model. This model uses asynchronous events to send the information to the reporting database.
Instead of pulling data, each service sends its data updates as an event to a data-capture service. The data capture service updates the reporting database. The below image puts the model quite simply.
Arguably, the batch pull model could be acceptable if the data freshness is not an issue.
Other factors need to be considered:
Batch operations can be resource-intensive.
If one batch fails, it could potentially hold up or corrupt subsequent batches.
Depending on how much data you're pulling think about storage and database performance.
It is therefore, recommended that dedicated data pipeline should be used.
Thank you for your time.
Special shoutout to @RaulJuncoV on Twitter(x).