Handling timeouts in a microservice architecture

[@ankitanautiyal1851]

Overview - Approach 1 : Ignore Approach 2 : configure and use defaults Approach 3 : Retry Request is non idempotent Request is expensive Request is over loaded Do like exponential backoff Approach 4 : retry only if needed Ask user it’s already done so we need again ? Approach 5 : rearchitect Remove synchronous communication whenever possible - go eventdriven architecture Retry - if too short - false +ve - if too long - perf bottleneck Make Retries safe using idempotency

[@ankk98]

# Summary 1. Keep operations idempotent 2. Retry only if needed and with exponential back offs 3. Remove synchronous dependencies, either by using queues or by removing the need for a request altogether 4. Always have timeouts when possible

[@shishirchaurasiya7374]

Amazing information Arpit sir Was using this from a long time back but after this video video got actualy realizations why they are used for and most importantly the pros and cons thanks a lot

[@shuvomondal3995]

This an example of read request, how can we handle create or update time out like you gave an example of transferring money from A to B how we handle that ?

[@vinaydixit81]

The way you described was awesome. But for Timeout you are describing that should fall under Resilience (Fault Tolerance) to make system reliable. Which will include--> timeout, Retry, Circuit breaker & Fallback. You can also add these things. But anyway knowledgeable session.

[@imhiteshgarg]

Hi Arpit, Nice video, I didn't understand your point about rearchitecting the solution in point 5. In earlier videos you mentioned that microservice A should not have access to data of B and it should always interact with B for its data but here you said, duplicating B's data in A to remove dependency of synchronous communication with B. Also, i get that event driven architecture helps us in removing synchronous communication but doesn't it incur latency issues because at the end of day, A is still waiting for the response!

[@LaxmikantPawar7]

Great Explaination with real life use cases.

[@MaruhanPark]

At 16:23, so what should we do when a request has to be non-idempotent? The money transfer thing will have to be non-idempotent no matter what

[@musababdullah9539]

Excellently explained

[@rahulbera454]

Amazing video arpit !!

[@kumarprateek1279]

Great Video. Saw one video by Hussein Nasser on the same topic.

[@RahulPal-iz4ev]

Really helpful and great explanation 🔥

[@kushalkamra3803]

Awesome! thank you

[@barebears289]

Microservices come with a cost, as everything does. In microservices architecture, there are more things to go wrong, as there are more network calls that need to be handled.

[@shubharthidey8248]

As usual, great video and awesome content. One request, can you please make a video on feature flags and what are the best scenarios to implement feature flags.

[@AsliEngineering]

Noted. Adding it to my pipeline.

[@LazyTechie-f2z]

Great Video..easy to understand. Keep going

[@logeshsuresh9759]

Hey, I did not understand approach 5 where you rearchitect 21:20 . What do you mean by "duplicate the data into your service"? If you duplicate it in your service and if the analytics service has massive load won't it still take time to send back the response to the search. Can you please explain this part alone ? Great video btw, learning a lot!

[@DHARSHANSASTRYBN]

How about using circuit breakers as approach 6 ?

[@yogeshedekar6078]

Arpit I feel addition of Circuit breakers would probbaly complete this discussion depending upon target audience. This content is cetainly great for some one new with microservices comcept but for a mid-seinor level engineer who has already seen your in deapth videos this seems to slightly (about 5% less) not meet the expectation in terms od detailing. If furhter explanation of circuit breakers could be added this would certainly make this video complete.

[@AsliEngineering]

Noted. Thanks for suggesting.

[@sreekarv6282]

Inbox outbox pattern can be used for approach 4

[@hiteshbitscs]

Good explanation. Does circuit breaker also can be used in case of time outs? So that client won't even call downstream if so many calls are timing it out leading to bad UX.

[@abhishekdutta6189]

Great in-depth video! Hope this playlist keeps on growing. Please do make videos on async architectures.

[@snigdhagupta4476]

Should the timeout depend on the latency of downstream services? That will create a dependency on them. So, what is a good way to choose the timeout value?

[@shauryaverma8780]

The timeout value for a service should depend on the expected latency of downstream services, as well as the expected response time of the service itself. This will help ensure that the service is able to handle any delays caused by the downstream services, without causing unnecessary delays for the end user. A good way to choose the timeout value is to first determine the expected response time of the service, taking into account any delays caused by downstream services, and then adding a buffer to account for any unexpected delays. Additionally, monitoring the actual performance of the service and its downstream dependencies can help identify any issues and inform future timeout value decisions.

[@karthikb4866]

Great informational video on Handling timeouts between microservices. Just curious to know the USBscriber you are using to visualize the notes

[@AsliEngineering]

ipad + GoodNotes.

[@SumanBishnoi-qu7ys]

Hi @Arpit, Thank you for the easy to understand explanation of Microservices Concept. Are your notes available anywhere to go through once after watching videos?

[@AsliEngineering]

Present on my website. ArpitBhayani.me

[@SumanBishnoi-qu7ys]

@@AsliEngineering Thank you! :)

[@MaruhanPark]

For 21:38, even if you decide to use an asynchronous architecture using Event driven architecture, wouldn't you usually still need to keep track of which event is a response to which? For example, you might want to publish an event with some sort of an id and a response event would also include that id. And only when you receive an event with the same id, would you continue proceeding, otherwise, you would retry. So that would have the same problems as synchronous architectures, no?

[@balasravandindukurthi4702]

Asynchronous architecture here doesn't mean the way one service fired an event and waiting on response for the event. It is synchronous again. Asynchronous here mean that one service has got all information it needed to serve a request asynchronously upfront. For use-case of search and analytics service shown in video - we can have asynchronous architecture where in elastic search db(used by search service) would already have required data from analytics service. In this case there is no need to call analytics service. Hope it helped clear your query

[@MaruhanPark]

@@balasravandindukurthi4702 Sure you wouldn't need to query the analytics service at the point of query, but that same information had to have arrived at ES before the query. So you would use an event driven architecture (such as Kafka) where upon there being an update to the DB, not only would you update the DB, but you would publish that event, so that search service would consume that event and update elasticsearch. (You would be duplicating information because you wouldn't really want to use ES as your main db as the purpose of ES isn't really to be used as a DB) However, the issue with failed communication can still occur to receive the events, so there would still be concerns about timeouts and retries. However, at least this is done before query, which is what he might have been alluding to as what the benefit is.

[@anshulgera221]

Great points. Although curious, what should be the max response time for an API after which we consider making the process as asynchronous, considering that API call is being made between 2 backend services.

[@AsliEngineering]

There is no textbook answer for this. You should make it async when 1. Response time from other service is high. Eg image processing. 2. When you have an engineering bandwidth to do it.

[@coderkashif]

Amazing..

[@MANISHRAUT]

Awesome content. It would be great if you could show the practical at the same time.

[@AsliEngineering]

Great idea. Added it to my pipeline. Thanks

[@adityab2805]

Great informative video

[@AsliEngineering]

Thank you

[@yadneshkhode3091]

awesome please make more videos

[@AsliEngineering]

Please keep watching them 😉

[@jivanmainali1742]

but how do we know other service is overloaded in case of retries? @15:25

[@AsliEngineering]

Monitoring the key vitals of the service. When any service is under load its CPU/Memory usage would have been shot up. You can monitor these vitals and then understand if it is under load or not.

[@rabindradocument8934]

Istio and envoy