Here is the day as usual, you got the bug report from FE team or QA about request can not be processed from BE side. As a BE engineer, you requested a curl as proper way to reproduce the bug. You pasted that curl to postman and started requesting, but got this strange error:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>400 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Bad request.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: 6ErPcNvTBbZ2q174iyLonbqphQBNHsRCRiOupa3eVZw3y9ZRB6v0PA==
</PRE>
<ADDRESS>
</ADDRESS>
</BODY></HTML>
As a senior BE engineer, you started triaging this from the request ID and found out:
- Backend still operates as usual
- No backend got not log for that request ID
- Cloudfront got no log for that request ID
What happened?
- First of all, request ID is generated from edge location and might not yet come to WAF or cloudfront log, so the request ID is not trustable in this case
- Second, the request got blocked from WAF, as in the strange response said
- Last resource, taking a look to curl itself, there is a header that silently drop the request at edge location and no proper error return, there is 'content-length'
Content-Length
- Content length is the header that be calculated on the client side to hint the server about the size of request payload
- Content length can be inspected by WAF or Cloudfront to mitigate the HTTP Request Smuggling (malform HTTP request to make server resources like CPU/RAM overload)
- CloudFront, ALB, nginx, HAProxy, Envoy, etc. are very strict about malformed framing headers specifically because of this attack class.
So, next time, better clean the curl request first by dropping some false positive headers, in this case: Content-Length