Hi all,

The following script is currently used to build our main website. Static
HTML versions of each of the pages are generated from php scripts on an
hourly basis by this script, but the other day for some reason it
generated an empty index page, but gave no errors that we have been able
to find.

We've more or less resigned ourselves to this being a glitch, but
figured it wouldn't hurt to get a few extra sets of eyes over the script
anyway. Any comments or suggestions for improvements would be
appreciated.

Regards, Dave.


#! /bin/sh

########################################################################
#######
# chksvr - Check the status of the specified server
#
# $1 = Server
# $2 = Port
# $3 = Database
# $4 = User Name
########################################################################
#######
chksvr()
{
  # Dummy query to check the server status
  status=`/usr/local/bin/psql -h $1 -p $2 -U $4 -d $3 -A -t -F '' -c
"SELECT 'SERVER_UP'"`

  if [ "$status" != "SERVER_UP" ]; then
    mail -s "WARNING: Server $1:$2 is down!!" webmaster@postgresql.org <
/tmp/svr-status
    rm /tmp/svr-status
    exit 1
  fi
}

########################################################################
#######
# buildpage - build an HTML page from a PHP page
#
# $1 = page name with path, but no extension
########################################################################
#######
buildpage()
{
  /usr/local/bin/lynx -source http://www.postgresql.com$1.php \
    | /usr/bin/perl -pe '$_ = " $_ "; tr/ \t/ /s; $_ = substr($_,1,-1)'
\
    > /usr/local/www/www.postgresql.org$1.html
}

########################################################################
#######
# Check the database servers
########################################################################
#######
echo Checking databases...

chksvr dbsvr1.postgresql.org 5432 186_portal portal > /tmp/svr-status
2>&1
chksvr dbsvr2.postgresql.org 5432 gborg www > /tmp/svr-status 2>&1
chksvr dbsvr3.postgresql.org 5432 mirrors vev > /tmp/svr-status 2>&1

########################################################################
#######
# Build the site
########################################################################
#######

echo Building site...

# The main portal
buildpage /index
buildpage /licence
buildpage /lists
buildpage /unavailable

# News/Events
buildpage /eventform
buildpage /newsform
buildpage /event-archive
buildpage /news-archive

# The Mirrors
buildpage /mirrors-ftp

# Docs
buildpage /docs/index
buildpage /docs/awbook
buildpage /docs/bookoneyearon
buildpage /docs/booktips

# User's Lounge
buildpage /users-lounge/index
buildpage /users-lounge/books
buildpage /users-lounge/features
buildpage /users-lounge/interfaces
buildpage /users-lounge/limitations
buildpage /users-lounge/related

# Cleanup
rm /tmp/svr-status
Tue, 11 Mar 2003 20:35:29 -0000
"Dave Page" <dpage@vale-housing.co.uk>
Website build script
Seems possible that Lynx timed out and generated and sh left only an
empty file.

Guard against this by writing the output of lynx to a temporary file and
either check it for stuff you expect to be there (<!-- comments -->?) or
for some reasonable size. There are better was to do this, of course...

Gavin


Wed, 12 Mar 2003 09:37:39 +1100 (EST)
Gavin Sherry <swm@linuxworld.com.au>
Re: Website build script

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



As far as the problem of generating an empty index file, you could have 
the buildpage() function redirect the lynx output to a temporary file 
($1.temp) and check that file for validity somehow (i.e. non-zero size 
or grep for a known string ("Postgres"?)) If it fails, send mail to 
webmaster and exit. If it succeeds, run mv $1.temp $1.html. Or use cp 
instead of mv and you have a copy of the previous page always preserved.


- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200303111731
-----BEGIN PGP SIGNATURE-----
Comment: http://www.turnstep.com/pgp.html

iD8DBQE+bmSnvJuQZxSWSsgRAv3RAJ9vJ1Jaupd1XivctHzDunVv5sbWgwCdHmCC
9ehCuTJI5QOn79aV6AlYAxo=
=2BZ3
-----END PGP SIGNATURE-----



Tue, 11 Mar 2003 22:40:08 -0000
greg@turnstep.com
Re: Website build script
On Tue, Mar 11, 2003 at 10:40:08PM -0000, greg@turnstep.com wrote:

> As far as the problem of generating an empty index file, you could have 
> the buildpage() function redirect the lynx output to a temporary file 
> ($1.temp) and check that file for validity somehow (i.e. non-zero size 
> or grep for a known string ("Postgres"?)) If it fails, send mail to 
> webmaster and exit. If it succeeds, run mv $1.temp $1.html. Or use cp 
> instead of mv and you have a copy of the previous page always preserved.

It seems much better to check $? on exit, and also to check stderr which
should be empty.

Same for the perl output.

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"El conflicto es el camino real hacia la union"

Tue, 11 Mar 2003 19:09:55 -0400
Alvaro Herrera <alvherre@dcc.uchile.cl>
Re: Website build script
I have a customer porting an application from informix to postgres. They
require 2 things:

1) Cursors outside of transactions.
2) For update cursors as well as where current of 

If anyone is interested in this work, please reply off list.

-- 
Dave Cramer <dave@fastcrypt.com>
Cramer Consulting


11 Mar 2003 18:20:09 -0500
Dave Cramer <dave@fastcrypt.com>
Request for quote
Thought I would change the title to see if anyone would respond?

It wasn't clear but the intent was to support the following in the
backend.


On Tue, 2003-03-11 at 18:20, Dave Cramer wrote:
> I have a customer porting an application from informix to postgres. They
> require 2 things:
> 
> 1) Cursors outside of transactions.
> 2) For update cursors as well as where current of 
> 
> If anyone is interested in this work, please reply off list.
-- 
Dave Cramer <dave@fastcrypt.com>
Cramer Consulting


13 Mar 2003 10:41:40 -0500
Dave Cramer <dave@fastcrypt.com>
Re: Request for quote looking to pay for work