| Share |
Updated: September 8th, 2011
Today, I was looking for a quick way to see HTTP response codes of a bunch of urls. Naturally, I turned to the curl command, which I would usually use like this:
curl -IL "URL"
This command would send a HEAD request (-I), follow through all redirects (-L), and display some useful information in the end. Most of the time it's ideal:
curl -IL "http://www.google.com" HTTP/1.1 200 OK Date: Fri, 11 Jun 2010 03:58:55 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Server: gws X-XSS-Protection: 1; mode=block Transfer-Encoding: chunked
However, the server I was curling didn't support HEAD requests explicitly. Additionally, I was really only interested in HTTP status codes and not in the rest of the output. This means I would have to change my strategy and issue GET requests, ignoring HTML output completely.
Curl manual to the rescue. A few minutes later, I came up with the following, which served my needs perfectly:
curl -sL -w "%{http_code} %{url_effective}\\n" "URL" -o /dev/nullHere is a sample of what comes out:
curl -sL -w "%{http_code} %{url_effective}\\n" "http://www.amazon.com/Kindle-Wireless-Reading-Display-Generation/dp/B0015T963C?tag=androidpolice-20" -o /dev/null
200 http://www.amazon.com/Kindle-Wireless-Reading-Display-Generation/dp/B0015T963CHere, -s silences curl's progress output, -L follows all redirects as before, -w prints the report using a custom format, and -o redirects curl's HTML output to /dev/null.
Here are the other special variables available in case you want to customize the output some more:
- url_effective
- http_code
- http_connect
- time_total
- time_namelookup
- time_connect
- time_pretransfer
- time_redirect
- time_starttransfer
- size_download
- size_upload
- size_header
- size_request
- speed_download
- speed_upload
- content_type
- num_connects
- num_redirects
- ftp_entry_path
Is there a better way to do this with curl? Perhaps, but this way offers the most flexibility, as I am in control of all the formatting.
Artem Russakovskii is a San Francisco programmer, blogger, and future millionaire (that last part is in the works). Follow Artem on Twitter (@ArtemR) or subscribe to the RSS feed.
In the meantime, if you found this article useful, feel free to buy me a cup of coffee below.
beer planet is a blog about technology, programming, computers, and geek life. It is run by Artem Russakovskii - a local San Francisco geek who is currently pursuing his own projects and regularly enjoys hacking Android, PHP, CSS, Javascript, AJAX, Perl, and regular expressions, working on Wordpress plugins and tools, tweaking MySQL queries and server settings, administering Linux machines, blogging, learning new things, and other geeky stuff.
Artem loves commenting on other people's blogs and reading comments on his own, so go ahead and leave one.
Hey –
Thanks for this; I needed a minimum-effort way of checking some returned headers, and this really got me up and running in no time at all. Cheers!
JB
Perfect – exactly what I needed. Thanks for sharing!
Hi man, that worked fine, but i can't get the status code if the request is POST, is that normal?
Hi, This is a cool thing you've shown. One question though .. can the output of -w argument be redirected into a text file when using this curl command in a batch script ?
Thank you! Just what I needed!
How I can the last modified time?
Thanks!
Exactly what I was looking for
Thanks Artem — this is a really helpful tool!
just what I was looking for, thanks!
Thanks !
Exactly what I was looking for
Amazing, dude! Thank you!
thanks for very simple and informative tutorial! helped me a lot!
Thank you _so_ much for sharing this tip !
Saved me a lot of time.
Keep them coming :I
Thanks for the very helpful tutorial! Just wondering if there is a way do determine if 301 redirects are in place properly working without querying one and the same URL multiple times?
What tops your tree. As with most other out of doors Christmas decorations,.
5.
Also visit my web site; Jeannette
Not sure how to suppress the status codes for each response. Here's what I used to manually take them out,
`curl -is dict://dict.org/d:$1 | awk '{ split("250|221|220|100",a,"|"); for (i=0; i<length(a); i++) { b[a[i]]=1; } if (!($1 in b)) { if ($1=="151") { $1=""; } print; } }'`
# some dictionary lookup on $1
# split ignorable status codes into a/b
# replace good status code (151) with a blank
[...] How To Display Just The HTTP Response Code In Command Line Curl [...]