Service Integration & challenges
Services have been around for a while now and more recently they have taken the shape of Microservices and concepts like Domain-Driven Design, and others have been the guiding principles in their development. More often than not when an enterprise application is built along the lines of microservices architecture, the application ends up having multiple of them.
In this article, we will not delve into the intricacies of how to define the boundaries of a microservice and how many we should have, what granularity are we seeking, etc. Our focus in this article would be around the challenges or the pitfalls that one needs to be wary of while integrating with multiple services (within and outside of the enterprise application)
Let’s assume Microservice A is talking to another service B. The integration between these two can be either synchronous or asynchronous depending upon the nature of the use case that one is dealing with. But whatever it may be, there are chances that microservice B may get overwhelmed and starts deteriorating in performance by virtue of calls coming from A. What this will do is it will impact the performance of not just microservice A but the entire application in general. Do we have some remedy for this?
One thought that comes to the forefront is to upscale microservice B so that it can handle the load it is getting from service A. But if this load increases beyond what microservice B is upgraded to handle, we will end up circling back to deal with the same situation. What next?
Every microservice has limited resources (irrespective of how much we upscale the infrastructure), with this thought in the back of our minds, what naturally succeeds the previous thought is either microservice B starts rejecting calls beyond its capacity or somehow microservice A only send that many number of requests to B which it can tolerate and work with. In a distributed system, these concepts are implemented using Backpressure or by Throttling.
Throttling is an important concept where the microservice starts rejecting the calls based on how many it can safely handle without suffering the quality. There are various ways in which this can be implemented as an example. Throttling the Rate, Throttling the IP, Throttling the Scope, etc.
There’s another aspect that one needs to be aware of and that is Timeouts. When there are calls happening across microservices, especially when some of the calls are going beyond the logical and physical boundaries of the application, it is a defacto standard of defining the timeouts at various levels. Timeouts for the Read requests, Timeouts for Write requests, Timeouts for connection, etc.
Timeouts can occur for multiple reasons like the service is too busy doing something and hence any further requests landing on it get timed out, it could also occur due to service being not available at all and hence the caller is not receiving the expected response.
One more thing in line with this that one can think of is to use Circuit breakers. They can help in diverting the traffic from the service which is unavailable for any reason to some default implementation. What this will do is the application will not be unavailable completely and will continue to serve the requests which are possible with the default implementation.
Decoupling is another approach one can think of. Instead of processing the complete request in one go (there are obvious reasons if one does it), one can think of splitting the overall transactions into multiple parts and let some part of it be done asynchronously. It will give two benefits, the first being the response generation would be faster and secondly, the system will be relatively more available to process any new request.
Bulkhead is another approach where the service dedicates resources to different categories of request processing components. This way the problem in one category is isolated and doesn't impede the availability and performance of other components
As we can see there is so much that happens while integrations and equally important what we can do in order to keep our application resilient.
