Automatic backup (Windows script)
dhenage says:
I wrote a little backup batch program that I though might be useful to others. This backs up the iCal file with all your tasks as well as a printer friendly version of any lists you select. It also compresses the files to save space. You can set it to run hourly or whatever using the Windows task scheduler.
Here's how to use the script:
1. Copy the script text below and paste into a text file such as rtm_backup.bat.
2. Do a find and replace on my_username and my_password to put in your own info.
3. Add as many lists as you'd like for the printer friendly viewing by (a) changing the ####### to the number corresponding to the list (you can find that number on the "Print" link in RTM when you have a list selected), and (b) changing the list name for the saved file.
4. If you dont' already have them, download and install the open source Windows binaries for wget (http://gnuwin32.sourceforge.net/downlinks/wget.php) and for 7-zip (http://downloads.sourceforge.net/sevenzip/7z457.msi). Make sure they are in your PATH (easy way: copy wget.exe and 7z.exe to C:\Windows\System32). If you don't want to compress the backups then you don't need 7-zip and must delete the last two lines of the script.
5. Add rtm_backup.bat to the Windows "Scheduled Tasks." If you want it to run more than once a day go into advanced options. Mine is set to "Every 1 hour(s) from 8:00 AM for 15 hour(s) every day, starting 6/1/2008." Try hstart (http://www.ntwind.com/software/utilities/hstart.html) if you don't want the command prompt popping up each time it runs. I use hstart and changed the scheduled task to this: "hstart /NOCONSOLE /IDLE C:\backup\RTM\rtm_backup.bat"
REM ######################################
@echo off
REM -- Note 1: make sure you have the latest version of wget installed (as of this writing 1.10.1)
REM -- Note 2: wget and 7z must be in your path
REM -- create a new directory with a timestamp for all the files
FOR /F "tokens=1-8 delims=/:. " %%J IN ("%date% %time%") DO SET timestamp=%%M-%%L-%%K_at_%%N.%%O.%%P
MKDIR %timestamp%
CD %timestamp%
REM -- backup all tasks using iCal
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/icalendar/my_username/ -O iCal.ics
REM -- printer friendly backup of selected lists for easier viewing
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "-.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "Projects.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "Waiting For.html"
http://www.rememberthemilk.com/print/my_username/#######/ -O "@any.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "@errands.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "@home.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "@office.html"
CD ..
REM -- delete the two lines below if you don't want to compress with 7-zip (WARNING: if you don't have 7-zip in your PATH then the 2nd command below will delete the files you are trying to backed up!)
7z a -tzip %timestamp%.zip %timestamp% > NUL
rmdir /S /Q %timestamp%
REM #########################################
Here's how to use the script:
1. Copy the script text below and paste into a text file such as rtm_backup.bat.
2. Do a find and replace on my_username and my_password to put in your own info.
3. Add as many lists as you'd like for the printer friendly viewing by (a) changing the ####### to the number corresponding to the list (you can find that number on the "Print" link in RTM when you have a list selected), and (b) changing the list name for the saved file.
4. If you dont' already have them, download and install the open source Windows binaries for wget (http://gnuwin32.sourceforge.net/downlinks/wget.php) and for 7-zip (http://downloads.sourceforge.net/sevenzip/7z457.msi). Make sure they are in your PATH (easy way: copy wget.exe and 7z.exe to C:\Windows\System32). If you don't want to compress the backups then you don't need 7-zip and must delete the last two lines of the script.
5. Add rtm_backup.bat to the Windows "Scheduled Tasks." If you want it to run more than once a day go into advanced options. Mine is set to "Every 1 hour(s) from 8:00 AM for 15 hour(s) every day, starting 6/1/2008." Try hstart (http://www.ntwind.com/software/utilities/hstart.html) if you don't want the command prompt popping up each time it runs. I use hstart and changed the scheduled task to this: "hstart /NOCONSOLE /IDLE C:\backup\RTM\rtm_backup.bat"
REM ######################################
@echo off
REM -- Note 1: make sure you have the latest version of wget installed (as of this writing 1.10.1)
REM -- Note 2: wget and 7z must be in your path
REM -- create a new directory with a timestamp for all the files
FOR /F "tokens=1-8 delims=/:. " %%J IN ("%date% %time%") DO SET timestamp=%%M-%%L-%%K_at_%%N.%%O.%%P
MKDIR %timestamp%
CD %timestamp%
REM -- backup all tasks using iCal
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/icalendar/my_username/ -O iCal.ics
REM -- printer friendly backup of selected lists for easier viewing
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "-.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "Projects.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "Waiting For.html"
http://www.rememberthemilk.com/print/my_username/#######/ -O "@any.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "@errands.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "@home.html"
wget -q --http-user="my_username" --http-password="my_password" http://www.rememberthemilk.com/print/my_username/#######/ -O "@office.html"
CD ..
REM -- delete the two lines below if you don't want to compress with 7-zip (WARNING: if you don't have 7-zip in your PATH then the 2nd command below will delete the files you are trying to backed up!)
7z a -tzip %timestamp%.zip %timestamp% > NUL
rmdir /S /Q %timestamp%
REM #########################################
dwr816 says:
Thanks a lot for sharing this. I had something similar on my Windows machine. Have moved recently to Mac and want to get a backup like this going there.
(closed account) says:
Yep, thanks for sharing dhenage, it's very useful.
Or at least potentially so, since I can't get it to work by following the above instructions.
I can get the printer friendly list, and I also get an ical file, but this file is empty (0KB). Strangely, by going to the very same address as is listed in the script (http://www.rememberthemilk.com/icalendar/my_username/) I do manage to get the full ical file.
Any ideas as to where I'm going wrong would be gratefully appreciated
Or at least potentially so, since I can't get it to work by following the above instructions.
I can get the printer friendly list, and I also get an ical file, but this file is empty (0KB). Strangely, by going to the very same address as is listed in the script (http://www.rememberthemilk.com/icalendar/my_username/) I do manage to get the full ical file.
Any ideas as to where I'm going wrong would be gratefully appreciated
benster says:
johaclimacticus, turn up the log level in the WGet command(s) to find out what the issue is. Maybe something with the check of a certificate could be causing your issue - but whatever it is, verbose logging is your friend.
(closed account) says:
Thanks for your help benster. You're quite right, it was a certificate issue. Part of the log file said:
ERROR: cannot verify www.rememberthemilk.com's certificate, issued by `/C=US/O=Equifax/OU=Equifax Secure Certificate Authority':
Unable to locally verify the issuer's authority.
To connect to www.rememberthemilk.com insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.
So I appended the command with --no-check-certificate, and then received a slightly different error message:
WARNING: cannot verify www.rememberthemilk.com's certificate, issued by `/C=US/O=Equifax/OU=Equifax Secure Certificate Authority':
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.
OK, now I'm totally stumped. Anyone have any further ideas?
ERROR: cannot verify www.rememberthemilk.com's certificate, issued by `/C=US/O=Equifax/OU=Equifax Secure Certificate Authority':
Unable to locally verify the issuer's authority.
To connect to www.rememberthemilk.com insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.
So I appended the command with --no-check-certificate, and then received a slightly different error message:
WARNING: cannot verify www.rememberthemilk.com's certificate, issued by `/C=US/O=Equifax/OU=Equifax Secure Certificate Authority':
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.
OK, now I'm totally stumped. Anyone have any further ideas?
benster says:
That *looks* like the wrong credentials are being sent. I don't want to be the guy that says check your user/pass, but in this case I believe that's consistent with that error message. Other than that, I'm not sure...
tkiser says:
The correct coding for the (new?) https-setup is as follows. This solved all the problems described in this thread.
Thanks to dhenage who originally gave this very nice idea about backing up our lists!
Regards, T.
REM ######################################
@echo off
REM -- Note 1: make sure you have the latest version of wget installed (as of this writing 1.10.1)
REM -- Note 2: wget and 7z must be in your path
REM -- create a new directory with a timestamp for all the files
FOR /F "tokens=1-8 delims=/:. " %%J IN ("%date% %time%") DO SET timestamp=%%M-%%L-%%K_at_%%N.%%O.%%P
MKDIR %timestamp%
CD %timestamp%
REM -- backup all tasks using iCal
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/icalendar/my_username/ -O iCal.ics
REM -- printer friendly backup of selected lists for easier viewing
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/print/my_username/#######/ -O "#######.html"
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/print/my_username/#######/ -O "#######.html"
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/print/my_username/#######/ -O "#######.html"
CD ..
REM -- delete the two lines below if you don't want to compress with 7-zip (WARNING: if you don't have 7-zip in your PATH then the 2nd command below will delete the files you are trying to backed up!)
rem 7z a -tzip %timestamp%.zip %timestamp% > NUL
rem rmdir /S /Q %timestamp%
REM #########################################
Thanks to dhenage who originally gave this very nice idea about backing up our lists!
Regards, T.
REM ######################################
@echo off
REM -- Note 1: make sure you have the latest version of wget installed (as of this writing 1.10.1)
REM -- Note 2: wget and 7z must be in your path
REM -- create a new directory with a timestamp for all the files
FOR /F "tokens=1-8 delims=/:. " %%J IN ("%date% %time%") DO SET timestamp=%%M-%%L-%%K_at_%%N.%%O.%%P
MKDIR %timestamp%
CD %timestamp%
REM -- backup all tasks using iCal
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/icalendar/my_username/ -O iCal.ics
REM -- printer friendly backup of selected lists for easier viewing
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/print/my_username/#######/ -O "#######.html"
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/print/my_username/#######/ -O "#######.html"
wget -q --http-user="my_username" --http-password="my_password" --no-check-certificate https://www.rememberthemilk.com/print/my_username/#######/ -O "#######.html"
CD ..
REM -- delete the two lines below if you don't want to compress with 7-zip (WARNING: if you don't have 7-zip in your PATH then the 2nd command below will delete the files you are trying to backed up!)
rem 7z a -tzip %timestamp%.zip %timestamp% > NUL
rem rmdir /S /Q %timestamp%
REM #########################################
snbart1 says:
By using this 'backup' script the saved 'print-'webfiles only contain a link directing to the correspondig online site of RTM.
Does somebody know a way to save the real content of the sites?
Of course there is still the ics-file, but having the possibility to use the lists, could be usefull.
Just in case RTM is offline...
Does somebody know a way to save the real content of the sites?
Of course there is still the ics-file, but having the possibility to use the lists, could be usefull.
Just in case RTM is offline...
(closed account) says:
Wahey! That's done it.
Many thanks to the combined efforts of dhenage and tkiser - we now have a method to automatically back up RTM.
Invaluable if, like me, your life depends on that innocent looking cow ...
Many thanks to the combined efforts of dhenage and tkiser - we now have a method to automatically back up RTM.
Invaluable if, like me, your life depends on that innocent looking cow ...
dhenage says:
tkiser: Thanks for the corrections to get this https compatible! Guess they changed their cert since I put this up.
snbart1: Perhaps you could expound on what you mean by the files only containing a link. It sounds like something incorrect in your wget command. Make sure your wget lines look exactly like what tkiser posted, substituting only the ###.
Anyone know how to get a feed of comments on a post? I don't frequent the forums, so I only saw these new comments because I check back every couple months.
snbart1: Perhaps you could expound on what you mean by the files only containing a link. It sounds like something incorrect in your wget command. Make sure your wget lines look exactly like what tkiser posted, substituting only the ###.
Anyone know how to get a feed of comments on a post? I don't frequent the forums, so I only saw these new comments because I check back every couple months.
snbart1 says:
dhenage: The script works correctly.
You can see what I meant by creating a new list with some notes and making a backup. If you delete this list and try to open the html-backup file corresponding to this list, you will see an empty website.
This occurs because the html-sites created by the backup-script don't store the content of your lists, it's created dynamically while opening the pages.
You can verify this behaviour because you can't open the backup files without logging in to RTM or by having a closer look to the backup files source code.
You can see what I meant by creating a new list with some notes and making a backup. If you delete this list and try to open the html-backup file corresponding to this list, you will see an empty website.
This occurs because the html-sites created by the backup-script don't store the content of your lists, it's created dynamically while opening the pages.
You can verify this behaviour because you can't open the backup files without logging in to RTM or by having a closer look to the backup files source code.
dhenage says:
to all: First, just to clarify, the difference between the script I put above and the one by tkiser is that the former is regular HTTP and the latter is encrypted HTTPS. They should have the same functionality, but I recommend using the encrypted method if you want to maintain privacy.
snabrt1: If I am understanding you correctly, you are saying that the backup script is saving links to the pages for you, not the actual page content. Again, I'm guessing that is a wget issue.
It doesn't look like I'm having that problem. While disconnected from the internet I am able to open the backed up HTML files. When I look at the source of these HTML files in a text editor I see the actual lists.
Perhaps if you provide a sample wget line from your script and your wget version number and OS then we could track down the issue.
snabrt1: If I am understanding you correctly, you are saying that the backup script is saving links to the pages for you, not the actual page content. Again, I'm guessing that is a wget issue.
It doesn't look like I'm having that problem. While disconnected from the internet I am able to open the backed up HTML files. When I look at the source of these HTML files in a text editor I see the actual lists.
Perhaps if you provide a sample wget line from your script and your wget version number and OS then we could track down the issue.
ymel72 says:
Good idea but for the same issue I would suggest to use twitter to send RTM tasks (use D rtm xxxxxx). And you have a backup of all your tasks in twitter.
Sending mail was already suggested and it could be combined with twitter if needed.
Cheers
Sending mail was already suggested and it could be combined with twitter if needed.
Cheers
damen says:
Hi,
Ensure that if you use the second script the https:// is in the URL for RTM, when you do not have this it will only download a redirect file as described above.
When it is in there the downloaded file contains the real list, which is then used to backup.
Thanks for the script, this fits right in the backup mechanism :)
Martijn
Ensure that if you use the second script the https:// is in the URL for RTM, when you do not have this it will only download a redirect file as described above.
When it is in there the downloaded file contains the real list, which is then used to backup.
Thanks for the script, this fits right in the backup mechanism :)
Martijn
Awesome - works great.
If you are on locales that display time with decimals, I got an error. With my regional settings, I get >echo %date% %time%
14.01.2011 13:38:46,39
To get alpha-sortable names (yyyy-mm-dd__hh-mm), I changed to:
FOR /F "tokens=1-8 delims=/:. " %%J IN ("%date% %time%") DO SET timestamp=%%L-%%K-%%J__%%M-%%N
Cheers,
Henrik
If you are on locales that display time with decimals, I got an error. With my regional settings, I get >echo %date% %time%
14.01.2011 13:38:46,39
To get alpha-sortable names (yyyy-mm-dd__hh-mm), I changed to:
FOR /F "tokens=1-8 delims=/:. " %%J IN ("%date% %time%") DO SET timestamp=%%L-%%K-%%J__%%M-%%N
Cheers,
Henrik
nskoric says:
Here is bash oneliner I use:
wget -q --http-user="YOUR_USERNAME" --http-password="YOUR_PASS" https://www.rememberthemilk.com/icalendar/YOUR_USERNAME/ -O rememberthemilk.`date '+%Y-%m-%d'`.ics
wget -q --http-user="YOUR_USERNAME" --http-password="YOUR_PASS" https://www.rememberthemilk.com/icalendar/YOUR_USERNAME/ -O rememberthemilk.`date '+%Y-%m-%d'`.ics
Log in
to post a reply.