How to build a robust Payments service?

System Design for SDE-2 and above: arpitbhayani.me/masterclass
System Design for Beginners: arpitbhayani.me/sys-design
Redis Internals: arpitbhayani.me/redis
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafters.io/join?via=...
In the video, I explained the concept of idempotence in APIs, emphasizing its importance in ensuring consistent outcomes even with multiple executions. I highlighted scenarios like banking transactions and social media interactions to illustrate the significance of idempotence. I discussed the need for unique identifiers to maintain transaction integrity and prevent duplicate processing. Implementing idempotence involves strategic design choices to avoid unnecessary retries and potential errors. By incorporating common identifiers across services, robust and reliable APIs can be developed.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
Bittorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.me/knowledge-base
Bookshelf: arpitbhayani.me/bookshelf
Papershelf: arpitbhayani.me/papershelf
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack.com
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

[@gomzysharma]

I work at American Express, and yeah we use exactly same thing at backEnd in various flows and microservices, we call it as batch schedular jobs and re-trigger flows for retry and check mechanism, It helps to re-trigger(with definite counts only) the NOT-PROCESSED or Failure can be due to not-so mandatory checks or may be due to multiple reasons or system downtime/failure.

[@ankitbinjola]

Bro which tech stack you are using in American Express

[@YashHalgaonkar]

I work in a team at Amazon that is responsible for paying employees and I can testify that this is exactly how we have designed our APIs.

[@tusharabbott9946]

loving the content. thank you.

[@harshal4877]

You're doing great ,excellent content

[@abhinavanand6089]

Great concept, explained very well.

[@utsavprabhakar5072]

Second approach is pretty awesome. Started following you recently, quality content !

[@AsliEngineering]

Thanks 🙌

[@utsavprabhakar5072]

Can you also do an LLD+ HLd for a payment service and related microservices, i am fascinated by payments architecture and how these companies handle the humongous amount of transactions and minimal API failures etc maintaining very low downtimes

[@msddhoni8356]

Great content. Expecting more on the PG domain...

[@dinakaranonline]

great video. can you probably expand more on how the unique payment id can be generated and shared between front end / consumer and back end services? also how to avoid / handle potential duplicate requests coming from front end / consumers ? there can be a scenario where front end / consumer is sending two different payment id for the same transaction - how can that be avoided ? or is the onus is on front end/ consumer to ensure correct unique payment id is shared all times ? If so, how can that be managed ?

[@romokdas8494]

What happens when a second request comes from the frontend for the same transaction even before the creation of the unique id in the payment gateway service?

[@bharathmshetty]

So if a single entity is supposed to be updated, you have to make sure there is a transactions/an external record that keeps the status check? For example : Say you want to a update balance of a particular user, to make it idempotent you first create a get transaction record, update balance, update transaction status.What would happen if ther transaction status update fails? Is there way you could do this without making a network call say for example a high frequency system?

[@NikunjGupta9]

Awesome content 😎, I just have one query, second approach will work if retries happen from backend side, what if client (user) retries payment from front end, how does payment service know which payment id to use ? Or payment service will pass this payment id to front end in first call. I wanted to understand this flow

[@AsliEngineering]

Client will send the payment ID along with the request. The ID is stitched across all the services and the clients. This is also mentioned in the video, should have stretched explaining it. Noted.

[@shayanhati2325]

I think the Approach 1 to not retry is not full proof - even if you don't retry at the client end there can still be network retries which might mean that the request was still send to the server. For e.g the use-case of instagram like, if some error is thrown due to network or timeout and you don't retry from client it might have still received in the backend, so you would need to make the API idempotent in any case. So the next time the user click, the backend has to implement some kind of idempotence based on the userId - postId

[@barebears289]

What language/stack would u recommend to someone new for backend development?

[@AsliEngineering]

Golang with any db of your choice

[@kumarsaurav2626]

Sql DB should be sufficient for managing the transaction Id for huge number of requests(like 1 M trans every minute) with Sharding?

[@AsliEngineering]

Yes. Never a concern. There are 100s of ways of scaling it

[@nihalkashyap647]

Can't the payment service itself generate the payment id instead of requesting the payment gateway to generate one? This would save one network call.

[@AsliEngineering]

This should be done at the sink - the one who does the actual operation because it is the only component who would know the status of it and can atomically update in its system. No one else could be trusted.

[@crimsoncad3230]

I have a doubt with Approach 1 "Do not retry" What if the payment is processed in the backend but we don't receive a response and it gets timed out. In that case, we will show and error on frontend and there is a possibility that the user will try again after seeing the error, this will cause Double Payment.

[@itssatvik]

Double Payment is a scenario one has to handle themselves if their PG gateway doesn't provide a solution themselves. PG like razorpay support the functionality that by using a parent called order_id against which payments are created, so if user retires just making sure that a new payment is created against the same order_id will, try to create a new payment for same order_id at PG end and if in the meantime PG received response from PSP and an existing payment gets updated to paid, PG will not let the client create a new payment and will raise and if in the case it did not get an update and a new payment if done, this double payment case is handled by PG and they refund one of the payment back to user. If your PG doesn't support such functionality one can use a similar approach in their service and call the refund API of your partner PG gateway. Hope this made sense and gave some insight... You can look it up, you'll find quite interesting blogs

[@amansaxena4446]

he loves hypothetical word

[@AsliEngineering]

Hehehhehe 🤣🤣

[@rahuljain3374]

How to handle concurrency?

[@AsliEngineering]

Database

[@rjarora8372]

You still didn't explain how to Implement Idempotence in your own service. You just explained how to use idempotence feature provided by payment gateways

[@AsliEngineering]

Take inspiration from how PGs do it. If you want to have idempotence string your services through a common ID like how PGs do.

[@khushaltrivedi9829]

Courses are bit expensive 😅