On hub, in /home/projects/pgsql/ftp/pub/dev I see

*.tar.gz.md5                            postgresql-opt-snapshot.tar.gz
doc                                     postgresql-opt-snapshot.tar.gz.md5
postgresql-base-snapshot.tar.gz         postgresql-snapshot.tar.gz
postgresql-base-snapshot.tar.gz.md5     postgresql-snapshot.tar.gz.md5
postgresql-docs-snapshot.tar.gz         postgresql-test-snapshot.tar.gz
postgresql-docs-snapshot.tar.gz.md5     postgresql-test-snapshot.tar.gz.md5

which agrees with the view at http://www.ca.postgresql.org/ftpsite/dev/.

However, it seems that the mirrors have a lot more stuff:
ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
of files back to 7.1beta6, and so do the other several I checked in
a random sample.  Is the update mechanism failing to cause old files
to be removed from the mirrors?

Also, some of the mirrors claimed to be up-to-date by 
http://www.postgresql.org/index.html aren't. Fr instance,
download.sourceforge.net doesn't have 7.1.1 nor 7.1.2.
I thought that the up-to-date check was automated?

			regards, tom lane

Mon, 04 Jun 2001 21:56:21 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Mirrors not tracking main ftp site?
On Mon, 4 Jun 2001, Tom Lane wrote:

> However, it seems that the mirrors have a lot more stuff:
> ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
> of files back to 7.1beta6, and so do the other several I checked in
> a random sample.  Is the update mechanism failing to cause old files
> to be removed from the mirrors?

I'm doing:

/usr/local/bin/rsync -avz --delete hub.org::postgresql-ftp
/mnt/ftpd/pub/postgresql

every 4 hours and still have these files too.  Not really sure where the
files are comming from,  but when I deleted a file from /dev it didn't
come back on a sync.

I will look into the rsync info a bit more,  but...

- Brandon

----------------------------------------------------------------------------
 b. palmer,  bpalmer@crimelabs.net            pgp:crimelabs.net/bpalmer.pgp5


Mon, 4 Jun 2001 22:16:34 -0400 (EDT)
bpalmer <bpalmer@crimelabs.net>
Re: Mirrors not tracking main ftp site?
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> However, it seems that the mirrors have a lot more stuff:
> ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
> of files back to 7.1beta6, and so do the other several I checked in
> a random sample.  Is the update mechanism failing to cause old files
> to be removed from the mirrors?

Found the problem.  Since rsync gets a perms denied from .hidden,  it
refuses to delete files.

root@seraph:/root# ./rsync-postgres-ftp
receiving file list ... opendir(.hidden): Permission denied
done
IO error encountered - skipping file deletion
wrote 105 bytes  read 20762 bytes  2782.27 bytes/sec
total size is 521221478  speedup is 24978.27

When I changed the script to:

#/usr/local/bin/rsync -avz --delete hub.org::postgresql-ftp
/mnt/ftpd/pub/postgresql
/usr/local/bin/rsync -avz --ignore-errors --delete
hub.org::postgresql-ftp /mnt/ftpd/pub/postgresql

It worked.  People need to either use the --ignore-errors or have the
.hidden folder on the server removed.

- - Brandon


- ----------------------------------------------------------------------------
 b. palmer,  bpalmer@crimelabs.net            pgp:crimelabs.net/bpalmer.pgp5

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
Charset: noconv

iQA/AwUBOxxDFPYgmKoG+YbuEQLeiACeIhRJQ0HTZQCJc+aqHzqSfTods7IAnjEO
m9vtW2WRh3PMPXdlWeEBzTzY
=u6ep
-----END PGP SIGNATURE-----



Mon, 4 Jun 2001 22:24:51 -0400 (EDT)
bpalmer <bpalmer@crimelabs.net>
Re: Mirrors not tracking main ftp site?
Tom Lane wrote:
> 
> which agrees with the view at http://www.ca.postgresql.org/ftpsite/dev/.

Thanks for the pointer to ftp-sites http url ;)

Now that I was able to verify (after not getting in to ftp:// for days)
that there are no 7.1.2 RPMS, I would like to inquire, if there are any 
plans to make RPMS's.

----------------
Hannu

Tue, 05 Jun 2001 08:27:57 +0500
Hannu Krosing <hannu@tm.ee>
Re: Mirrors not tracking main ftp site?
bpalmer <bpalmer@crimelabs.net> writes:
> Found the problem.  Since rsync gets a perms denied from .hidden,  it
> refuses to delete files.

Ah-hah.  And that directory seems to have appeared on 13 Apr, which is
right about the time that the oldest un-deleted files on the mirrors are
from:

> ls -ld ~pgsql/ftp/pub/.hidden
d--x--x--x  2 root  pgsql  512 Apr 13 14:58 /home/projects/pgsql/ftp/pub/.hidden

Marc, what is that thing?  Can we get rid of it?

> It worked.  People need to either use the --ignore-errors or have the
> .hidden folder on the server removed.

I don't much care for the first alternative ...

			regards, tom lane

Tue, 05 Jun 2001 00:29:20 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Mirrors not tracking main ftp site? 
On Mon, 4 Jun 2001, Tom Lane wrote:

> On hub, in /home/projects/pgsql/ftp/pub/dev I see
>
> *.tar.gz.md5                            postgresql-opt-snapshot.tar.gz
> doc                                     postgresql-opt-snapshot.tar.gz.md5
> postgresql-base-snapshot.tar.gz         postgresql-snapshot.tar.gz
> postgresql-base-snapshot.tar.gz.md5     postgresql-snapshot.tar.gz.md5
> postgresql-docs-snapshot.tar.gz         postgresql-test-snapshot.tar.gz
> postgresql-docs-snapshot.tar.gz.md5     postgresql-test-snapshot.tar.gz.md5
>
> which agrees with the view at http://www.ca.postgresql.org/ftpsite/dev/.
>
> However, it seems that the mirrors have a lot more stuff:
> ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
> of files back to 7.1beta6, and so do the other several I checked in
> a random sample.  Is the update mechanism failing to cause old files
> to be removed from the mirrors?

Here's the syntax we tell them to use:

   rsync -avz --delete hub.org::[remote]/ [destination directory]

If that's not what they're using I can't go into their cronjobs and
fix it.

>
> Also, some of the mirrors claimed to be up-to-date by
> http://www.postgresql.org/index.html aren't. Fr instance,
> download.sourceforge.net doesn't have 7.1.1 nor 7.1.2.
> I thought that the up-to-date check was automated?

It is and here's the directory from sourceforge:

227 Entering Passive Mode (64,28,67,101,18,128).
150 Opening ASCII mode data connection for file list
-rw-r--r--   1 root     root      8117016 May 24 16:37 postgresql-7.1.2.tar.gz
-rw-r--r--   1 root     root           65 May 24 16:38 postgresql-7.1.2.tar.gz.md5
-rw-r--r--   1 root     root      3240364 May 24 16:38 postgresql-base-7.1.2.tar.gz
-rw-r--r--   1 root     root           70 May 24 16:38 postgresql-base-7.1.2.tar.gz.md5
-rw-r--r--   1 root     root      2072096 May 24 16:38 postgresql-docs-7.1.2.tar.gz
-rw-r--r--   1 root     root           70 May 24 16:38 postgresql-docs-7.1.2.tar.gz.md5
-rw-r--r--   1 root     root      1803742 May 24 16:38 postgresql-opt-7.1.2.tar.gz
-rw-r--r--   1 root     root           69 May 24 16:38 postgresql-opt-7.1.2.tar.gz.md5
-rw-r--r--   1 root     root      1002166 May 24 16:38 postgresql-test-7.1.2.tar.gz
-rw-r--r--   1 root     root           70 May 24 16:38 postgresql-test-7.1.2.tar.gz.md5
226-Transfer complete.
226 Quotas off

What is it you find missing about 7.1.2?  What were you actually looking
at?

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net
         56K Nationwide Dialup from $16.00/mo at Pop4 Networking
        Online Campground Directory    http://www.camping-usa.com
       Online Giftshop Superstore    http://www.cloudninegifts.com
==========================================================================




Tue, 5 Jun 2001 06:16:35 -0400 (EDT)
Vince Vielhaber <vev@michvhf.com>
Re: Mirrors not tracking main ftp site?

okay, just removed the .hidden directory from the ftp server, which should
correct that ... I had setup that .hidden directory to be excluded though,
not sure why it was bothering things :(

On Mon, 4 Jun 2001, bpalmer wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> > However, it seems that the mirrors have a lot more stuff:
> > ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
> > of files back to 7.1beta6, and so do the other several I checked in
> > a random sample.  Is the update mechanism failing to cause old files
> > to be removed from the mirrors?
>
> Found the problem.  Since rsync gets a perms denied from .hidden,  it
> refuses to delete files.
>
> root@seraph:/root# ./rsync-postgres-ftp
> receiving file list ... opendir(.hidden): Permission denied
> done
> IO error encountered - skipping file deletion
> wrote 105 bytes  read 20762 bytes  2782.27 bytes/sec
> total size is 521221478  speedup is 24978.27
>
> When I changed the script to:
>
> #/usr/local/bin/rsync -avz --delete hub.org::postgresql-ftp
> /mnt/ftpd/pub/postgresql
> /usr/local/bin/rsync -avz --ignore-errors --delete
> hub.org::postgresql-ftp /mnt/ftpd/pub/postgresql
>
> It worked.  People need to either use the --ignore-errors or have the
> .hidden folder on the server removed.
>
> - - Brandon
>
>
> - ----------------------------------------------------------------------------
>  b. palmer,  bpalmer@crimelabs.net            pgp:crimelabs.net/bpalmer.pgp5
>
> -----BEGIN PGP SIGNATURE-----
> Version: PGPfreeware 5.0i for non-commercial use
> Charset: noconv
>
> iQA/AwUBOxxDFPYgmKoG+YbuEQLeiACeIhRJQ0HTZQCJc+aqHzqSfTods7IAnjEO
> m9vtW2WRh3PMPXdlWeEBzTzY
> =u6ep
> -----END PGP SIGNATURE-----
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org


Tue, 5 Jun 2001 08:23:43 -0300 (ADT)
The Hermit Hacker <scrappy@hub.org>
Re: [CORE] Re: Mirrors not tracking main ftp site?
Vince Vielhaber <vev@michvhf.com> writes:
> On Mon, 4 Jun 2001, Tom Lane wrote:
>> Also, some of the mirrors claimed to be up-to-date by
>> http://www.postgresql.org/index.html aren't. Fr instance,
>> download.sourceforge.net doesn't have 7.1.1 nor 7.1.2.

> What is it you find missing about 7.1.2?  What were you actually looking
> at?

I went to ftp://download.sourceforge.net/pub/mirrors/postgresql/
(the link given by our homepage) and didn't see the v7.1.2 symlink,
nor did the source subdirectory have a v7.1.2 subdirectory.

As of this morning, though, both are there.  I suppose they synced up
overnight.

			regards, tom lane

Tue, 05 Jun 2001 09:50:50 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Mirrors not tracking main ftp site? 
On Tue, 5 Jun 2001, Tom Lane wrote:

> Vince Vielhaber <vev@michvhf.com> writes:
> > On Mon, 4 Jun 2001, Tom Lane wrote:
> >> Also, some of the mirrors claimed to be up-to-date by
> >> http://www.postgresql.org/index.html aren't. Fr instance,
> >> download.sourceforge.net doesn't have 7.1.1 nor 7.1.2.
>
> > What is it you find missing about 7.1.2?  What were you actually looking
> > at?
>
> I went to ftp://download.sourceforge.net/pub/mirrors/postgresql/
> (the link given by our homepage) and didn't see the v7.1.2 symlink,
> nor did the source subdirectory have a v7.1.2 subdirectory.
>
> As of this morning, though, both are there.  I suppose they synced up
> overnight.

Don't know what could have happened to it, I'm fairly certain I downloaded
it from them less than a week ago when I did some upgrading here.

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net
         56K Nationwide Dialup from $16.00/mo at Pop4 Networking
        Online Campground Directory    http://www.camping-usa.com
       Online Giftshop Superstore    http://www.cloudninegifts.com
==========================================================================




Tue, 5 Jun 2001 09:55:49 -0400 (EDT)
Vince Vielhaber <vev@michvhf.com>
Re: Mirrors not tracking main ftp site? 