Anatomy of a Silent 502: How Four HAProxy Characters Exposed a K8s Upload Failure

A 502 Bad Gateway tells you that one proxy failed to get a valid response from another system. It does not tell you which layer failed, whether the application ever saw the request, or which timeout actually expired.

We learned that distinction during a production incident involving failed file uploads through a Kubernetes platform.

The request path looked like this:

Internet |Firewall |HAProxy |ingress-nginx |Apache APISIX |Customer pod

Customers were seeing failures while uploading files such as PDFs and JPEGs. HAProxy was returning 502 responses, but the downstream evidence was strangely clean.

There were no corresponding requests in the ingress-nginx access logs.

Nothing appeared in APISIX.

Nothing reached the application.

For a while, it looked as though the traffic had simply vanished between HAProxy and Nginx.

It had not. The most useful evidence was hiding in four characters:

SH--

The Evidence in the HAProxy Logs

Two representative failures looked like...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE