Forums

Discuss all things Remember The Milk.

Automatically download/archive all events ical with 'curl'

andrew.reinartz says:
I like to have all of my webapp data backed up locally somewhere. I treat it like backing up any other important piece of data, and don't trust that the provider will be backing up my data and/or making it available to me forever (access goes down, products die out, etc).

I don't want to have to sync my ical feed to a calendar program soley to backup my data, especially since I have a lot of undated tasks that many calendar programs don't do anything with. So instead I decided to use the utility 'curl' to download the raw 'All Events ICalendar' feed. I had seen uses curl for posting data to RTM (e.g. http://www.rememberthemilk.com/forums/tips/2484/ and http://lifehacker.com/software/hack-attack/take-launchy-beyond-application-launching-284127.php), but not for downloading. It's actually dirt simple, so I thought I'd share it quick.

DISCLAIMER: I'm doing this on a linux box - the specific steps should work on most *NIX derivatives. That being said, curl is available for windows (see the forum post above), and you could easily automate the task in windows. The only thing I'm not sure how to do in windows is automatically name the file with the 'date' command.

1) Install 'cURL' however you normally install programs in your distro (or google for curl.exe if you're on windows).

2) Find your ical feed link in RTM: Go to 'Settings' -> 'Info' and copy the link from 'iCalendar Service (All Lists)'. You'll need to change the prefix of "webcal://..." to "https://..." to download with curl.

3) Test downloading with curl:
* At the command-line, type in something like the following (-o sets the output file, and -u sets the username and password that will be sent to the server):
curl -o rtm_ical_`date +\%Y\%m\%d`.ics -u RTMUSER:RTMPASSWROD https://www.rememberthemilk.com/icalendar/RTMUSER/
* Curl should download your ical feed to a file named something like rtm_ical_20080809.ics
* Open the file, just to make sure it looks like it has all of the events you expect.
*WINDOWS: if you're on windows, change the bit after -o to not include the `date....` bit, something like "-o rtm_ical_backup.ics"

4) You can either just run this command periodically yourself, or schedule it to run daily/weekly/etc with cron on *nix, or something else on windows. My computer runs all night, so I set it download at 3am every day with the following crontab line:
"00 3 * * * curl -o /home/azwr/backups/rtm_ical_`date +\%Y\%m\%d`.ics -u ......(rest the same as above)"

Every so often I remove the old dated backups. There's probably a smart way to have only 3 rotating backups or the like, but I didn't feel like figuring it out at the moment :)

The instructions are pretty vague, if you need more details (e.g. on editing your crontab) google or ask here, I might be able to help.

Hope someone finds this helpful!

~Andrew
Posted at 4:01pm on August 9, 2008
daliinfo says:
gooooooooooooooooooooood. Thank's
Posted 15 years ago
Log in to post a reply.