Four 504s on one session: rotate, don't retry

Retrying a failed proxy session and rotating to a new one are different questions on a sticky residential gateway. The session token in the username decides which exit you get, so a retry sends the same token to the same dead exit. A rotation mints a new token and asks for a different exit. Mosaic used to retry. It now rotates, and only when the session was its own to replace.

The report: a red launch under a green provider

The bug came in as a screenshot. A launch was refused four times with "The proxy gate-eu… did not answer … Proxy returned HTTP 504". A few centimetres above it, the provider's own status dot was green. The person who sent it put it plainly: they knew the proxy worked, and the app seemed to know it too.

Both readings were true. The provider check and the launch were talking to the same gateway on two different sessions, so they reached two different exits. The check's session worked. The profile's session did not.

Retry or rotate a proxy session?

Mosaic checks a proxy before every launch, and that check has retried three times since a residential pool was first pointed at it, because one observation of somebody else's network is not evidence. The retry was sound for a gateway that hiccups. It was useless here, because all three attempts carried the same session token, and on a sticky endpoint a session token is an exit.

To see how often this happens, we asked the seller for forty fresh sessions in a row. The twenty-ninth was dead:

session 889053 → 504    the attempt the launch refused on
session 889053 → 504    retry 1, as it was written
session 889053 → 504    retry 2
session 889053 → 504    retry 3

session 758767 → 504    a fresh one, also bad
session 195609 → 200
session 131797 → 200

A 504 there belongs to the session, not to the gateway. Three more tries on 889053 could only ever say 504 again. One new token was all it took, and the second new one answered.

How it works: Provider minting

What a rotation is allowed to move

A failure now costs a different exit instead of a longer wait. But moving somebody's exit is a decision, so the rule is narrow. Mosaic rotates a session in two cases:

• The session was minted from a provider. Every profile made from a provider holds a token Mosaic minted for it. The replacement is minted from the same provider with the same pick, so a profile asked for Germany gets another German exit. The token is written in the seller's own grammar, read off the gateway line you pasted, because a token we invented would authenticate and then be ignored.
• The proxy is marked to rotate on every launch. There, the operator has already said the session is disposable.

A session somebody pinned by hand is never moved. Quietly relocating it would be the silent repair this codebase refuses everywhere else: the profile would come out somewhere it was not asked to be, and nothing would say so. In that case the launch stops and the message names the proxy and quotes the gateway.

When a rotation works on a provider-minted proxy, the new token is written back to the profile. Otherwise every later launch would start by failing on the dead one again. A rotate-on-launch token is never written back, because storing it is what would stop it rotating.

Proving it without waiting for a bad exit

A real pool will not fail on demand, so the end-to-end check uses a stub gateway that answers 504 for one named session and forwards every other. Both halves in one run:

A1. session the profile holds, killed at the gateway: 33396200
A2. start: running
A3. session it holds now: 90537583
    proxy gate: the first session did not answer, retried on a fresh one

B1. a session pinned by hand, killed the same way: 77777777
B2. start: refused, correctly

The rule is written once and walked by check:rotate at all three places that mint a session: the launch, the provider check, and creating profiles from a provider. The last one matters most. One dead sample there would have left a whole batch drawn for no exit at all, the same kind of mismatch we wrote up in a profile that claimed New York from Austria.

One small thing that made the refusal read like a bug

The refusal quotes the gateway's own reason, and gateways do not punctuate. Our sentence ran straight into theirs: "Proxy returned HTTP 504 Launching anyway would open a browser…". A message doing its job read like a broken one. The quoted reason now gets its full stop before our sentence begins.

If you run residential proxies anywhere

The lesson is not specific to Mosaic. When a residential proxy session fails, ask which part failed before retrying. A timeout or a refused connection can be the gateway; a 504 on one session while others answer is that session's exit. Retry the gateway; replace the session. Checking that a proxy answers and tunnels before you trust it is covered step by step in how to check a proxy before you use it.

In Mosaic, the provider side of this, one pasted gateway line that mints a separate session for every profile, is on proxy providers.