Recent searches
Search options
Administered by:
On Linux (and I can only imagine on other platforms) `curl -I https://example.com` will output the headers with the caret return (\r) character at the end.
So if you're piping it, say, into `sed` with a regex that matches the end of the line with `$`, you might want to do `\r$` instead.
You know, if you want it to work as expected.
That was a weird Bash script bug to figure out, that's for sure.
@rysiek Not a Linux thing. HTML specification says that you're supposed to use CR LF in the wire protocol. Curl just carries it over when the server does that, which most do, because there's benefits to being 8-bit clean.
@riley oh I am not saying it is a curl bug! Yup, it's because of the HTTP spec, that's right.
It's still unexpected. And difficult to debug: copy-pasting the header one is trying to target into an `echo '<header>' | sed` type thing to try and debug the regex will not help you – the regex will work, even though just a moment ago the exact same regex with exact same (it would seem) input data failed.
Took me a while to figure it out. Thought I'm going mad for a sec there.
@rysiek sorry, s/HTML/HTTP/. HTML doesn't care; HTTP specifies CR LF.
I avoid sed and regex because of the difficulty in debugging.
But this is not something weird about curl. It's something weird about HTTP headers: they are defined as using \r\n as the line ending. See link.
You could say it's something weird about sed. Especially since (based on the link below) sed has a different behavior on windows, and your regex won't work there.
https://stackoverflow.com/questions/5757290/http-header-line-break-style
https://askubuntu.com/questions/1171030/sed-not-acting-as-end-of-line-character
https://stackoverflow.com/questions/4652652/preserve-line-endings
Do you really call it a "caret return"?
