Thursday, April 7, 2022

[SOLVED] Why does curl send a Proxy-Connection header, even though the RFC seems to discourage it?

Issue

RFC 7230, Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing states in the href="https://www.rfc-editor.org/rfc/rfc7230#appendix-A.1.2" rel="nofollow noreferrer">appendix:

As a result, clients are encouraged not to send the Proxy-Connection header field in any requests.

Why is curl sending this header then, when using a proxy?

$ http_proxy=0.0.0.0:8080 curl -v http://google.com
...
> Accept: */*
> Referer:
> Proxy-Connection: Keep-Alive
>
...

I am using curl 7.71.1 on x86_64-pc-linux-gnu.


Addendum: It's another protocol, but HTTP/2 explicitly disallows Connection and related fields, as per RFC 7540 Section 8.1.2.2


Solution

What an awesome question! Thanks for asking.

This is one example of the things on the web where the specs and what we should do doesn't really match up with reality and what happens if we try to follow those written guidelines!

Back in 2016 we (in the curl project) actually removed the Proxy-Connection: Keep-Alive header from curl requests done to proxies for exactly that reason: it isn't necessary since the protocol implies keep-alive and the spec says so!

Then (after that change) we immediately got a stream of bug reports from people whose proxy connections broke down completely and persistent connections simply failed to work... and once we reverted that change everything went back to working again.

So maybe around the year 2026 things have changed enough so we can run that experiment again. Until then we keep this header in proxy requests so that all those crappy legacy proxies in use out there don't go bananas!

The world wild web is a crazy place.



Answered By - Daniel Stenberg
Answer Checked By - Willingham (WPSolving Volunteer)