Automatically resume interrupted downloads in OSX with curl

I recently found myself at the wrong end of a crappy internet connection and needing to download a 97MB file. Safari and Chrome would both choke during the download and leave me with a unusable, truncated 4MB file, with no way to resume the download. The OSX command-line download utility, curl, can resume downloads, but I had to check manually each time the download was interrupted and re-run the command.

After typing “up, enter” more times than I care to admit, I decided to figure out the automatic way. Here’s my bash one-liner to get automatic download resume in curl:

export ec=18; while [ $ec -eq 18 ]; do /usr/bin/curl -O -C - "http://www.example.com/a-big-archive.zip"; export ec=$?; done

Explanation: the exit code curl chucks when a download is interrupted is 18, and $? gives you the exit code of the last command in bash. So, while the exit code is 18, keep trying to download the file, maintaining the filename (-O) and resuming where the previous download left off (-C).

Update: As Jan points out in the comments, depending what is going wrong with the download, your error code may be different. Just change “18” to whatever error you’re seeing to get it to work for you! (If you’re feeling adventurous, you could change the condition to while [ $ec -ne 0 ], but that feels like using a bare except in Python, which is bad. ;)

6 thoughts on “Automatically resume interrupted downloads in OSX with curl

  1. Thanks, this helped me squeezing a 2GB OSX update through a shaky Colombian wifi connection. I had some “(56) Recv failure: Operation timed out” error codes and had to press enter manually. Perhaps the script could be improved to include multiple error codes in the future.

  2. Hello I am not a computer expert but I have lots of huge files to download and I have been hitting resume for days. Please help! I opened terminal and copied the link (inserting the website link) and this is what appears. What am I doing wrong?

    Last login: Sat Jan 14 01:53:16 on ttys000
    MGBLON04D9004old:~ nayana$ export ec=18; while [ $ec -eq 18 ]; do /usr/bin/curl -O -C – https://insidetheedit.com/shop/; export ec=$?; done
    curl: Remote file name has no length!
    curl: try ‘curl –help’ or ‘curl –manual’ for more information
    MGBLON04D9004old:~ nayana$

    Thanks in advance!

    1. Hi Nayana,

      curl can’t download a whole folder of stuff, but rather individual files. So, for example:

      curl -O -C https://insidetheedit.com/shop/file.mp4

      should work, but

      curl -O -C https://insidetheedit.com/shop/

      won’t.

      Additionally, if you need to log in to download the files, I suspect you need to pass additional options to curl:

      curl -O -C -u myusername:mypassword https://insidetheedit.com/shop/file.mp4

      I hope this helps!

  3. Hi There! I definitely not a programmer but I’m trying to tweak some code someone gave me to bring down a file that keeps timing out…

    Any help would be amazing! Here is what I usually do which works unless I have to work from some horrible internet connection.

    curl -u ‘user:pass’ -o /Users/me/Desktop/test.apk -O https://repo.apk

    I tried to integrate your super duper piece to no avail. This is my best guess. Clearly I’m missing some key concept…

    export ec=18; while [ $ec -eq 18 ]; do curl -u ‘user:pass’ -o /Users/me/Desktop/test.apk -O https://repo.apk

    It just freezes. As you can see I usually pass in my credentials and rename the file…. Help oh wise one!

    1. I don’t think I deserve the “wise one” moniker but I do think I picked the error(s) in your code. =)

      • first, don’t forget the “; export ec=$?; done” at the end. This makes sure that any error code is recorded before retrying, and that there is a loop. (Otherwise it’ll just wait for you to finish the command.) The syntax here, if you’re not familiar with bash loops, is “while [condition]; do command1; command2; command3; done”. This will repeat the three commands in sequence while the condition is true. Make sense?

      • second, and most important, you appear to have forgotten the -C, which is the important bit! Read the blog post for an explanation of that option.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s