When an AI API you depend on goes down: a checklist for developers

If your product calls an AI API, that API will have bad days. These steps come from what the providers themselves recommend.

1. Only retry what can succeed

Google’s troubleshooting guide separates errors worth retrying (429, 408 and the 5xx errors) from errors about the request (400, 402 and 403), which will fail again. Perplexity says the same: do not retry 4xx errors, and retry 5xx errors with backoff. Retrying a bad key or an empty balance only adds noise.

2. Back off, and add jitter

Wait longer after each failure, and add a random element so many clients do not retry in step. Google and Perplexity both recommend exponential backoff with jitter. Anthropic’s SDK retries transient failures twice by default, with exponential backoff, and respects a retry-after header when there is one. OpenAI tells you to follow its retry headers.

3. Know which 429 you have

A rate limit will clear if you slow down. A spend cap or used-up credit will not clear until you change your account. Anthropic notes that a spend-cap 429 has no retry delay and keeps failing until access resumes, so a retry loop on it is wasted effort. See the error-codes guide.

4. Handle long requests properly

Anthropic advises streaming, or its batch interface, for long requests, because some networks drop idle connections and a long wait without a response can fail without any answer coming back. It also suggests keeping a TCP keep-alive on direct integrations.

5. Log the request ID

Anthropic returns a request ID on every response and asks you to include it when contacting support. Keeping the IDs of failed calls saves time later. Other providers have equivalents, so check their documentation.

6. Keep a fallback

Perplexity’s documentation recommends graceful degradation with fallbacks. DeepSeek’s error page suggests, for the rate-limit error, temporarily using another provider’s API. At minimum, decide in advance what your product shows when the AI is unavailable, such as a clear message instead of a spinner that never ends.

7. Check before you debug

Before spending an hour on your own code, look at the provider’s status page and ours. If the provider reports a problem with the API, you can stop. Our JSON feed gives the current verdict for each service, free for reasonable use. Remember that our verdict follows the consumer products for some services, and the API can fail separately.

Sources

Read on 8 October 2026. Providers change their documentation, so check it for anything important.