Stork Delivery
The integration has to survive a retry
A timeout does not tell you whether an operation failed. Design integrations so trying again does not create a second business event.
The problem: a timeout can turn one order into two
Your application sends an order to a supplier. The connection times out before a response arrives. Should it send the order again?
The supplier may have received nothing. It may also have accepted the order and lost the response on the way back. Treating both situations as a simple failure can create duplicate orders.
This is a business problem hidden inside a network problem.
A practical approach
Give the operation an identity
Use a stable identifier for the intended operation, and keep it when retrying that same operation. The receiving system needs an agreed way to recognize it and avoid repeating the side effect.
A new order needs a new identity. A retry of the same order does not. Keep that distinction separate from the individual HTTP request, because one business operation may involve several requests.
AWS describes this approach in its article on making retries safe with idempotent APIs. The important principle is that repeating a request should not accidentally repeat the intended business effect.
Record what you know
Keep a durable record of the operation, its identifier, the last attempt, and any confirmed downstream reference. Avoid reducing every outcome to “success” or “failed.” An uncertain result may need reconciliation before another action is safe.
Suppose an order exists at the supplier but your application has not recorded its reference. A status lookup or review queue may be the right next step. Blindly creating a replacement order is not.
Put limits around retries
Retry only errors that are plausibly temporary. Space attempts apart, cap the number of attempts, and decide what happens when the limit is reached. Repeated invalid requests will not become valid through persistence.
Be careful when several layers retry independently. A browser, application, queue worker, and downstream client can multiply attempts without anyone intending that behavior.
Test the missing response
A useful integration test accepts the operation downstream, then interrupts the response. Another delivers the same operation twice. A third restarts the worker between receiving confirmation and recording it locally.
Check the business outcome, not just the HTTP response. There should be one intended order, with enough information to explain how it got there.
If an external API does not offer safe retry semantics, design a reconciliation process around its actual capabilities. Reliability starts with understanding the uncertainty you have, not hiding it behind another retry button.
The result: one intended operation with a traceable outcome
For one supplier operation, document how to distinguish confirmed success, confirmed failure, and an unknown result. In a test environment, interrupt the response after acceptance and retry using the same operation identity. Confirm there is only one downstream order. Where that cannot be guaranteed by the API, demonstrate the lookup or review process that resolves uncertainty before another order is created.