Take the 2-minute tour ×

I want to wget (or other download batch command) the latest file that is added to a large repository. The latest nightly build thru http. I could mirror all files, but the repository are huge so I want to be able to remove old files and only trigger when there is a new file.

share|improve this question
can u provide more specifics about your problem? like the repository from where you want to download the file? is it using anykind of versioning? is it a FTP site?? –  rao_555 Jan 4 '12 at 11:39
wget -A.deb -m nightly.openerp.com/6.1/deb –  Anders Wallenquist Jan 4 '12 at 12:58

2 Answers 2

A first simple approach would be to use the time stamping capacity of wget, with this, you could only download the new files from the repository. But it leads to make a mirror of the repository (but refreshed smartly with only the new files :) ).

share|improve this answer
Even though it mirrors the repository in local, but according to the wget documentation, But once you mirrow the repository in local after that it will only download the modified files. –  rao_555 Jan 4 '12 at 14:29
I still have to keep the whole repository at my local site, less then a TB but large enought. –  Anders Wallenquist Jan 8 '12 at 21:50

I would advise you use rsync for this task. You have several options on how to do this, there are scripts available here


And there is a full tutorial here:


And debmirror:


share|improve this answer
Sorry, I only have http-access to the site. –  Anders Wallenquist Jan 8 '12 at 21:48

Your Answer


