
Does anyone have anything against me applying this to the current source
tree?  


Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 

---------- Forwarded message ----------
Date: Fri, 17 Dec 1999 13:51:50 -0800 (PST)
From: Alfred Perlstein <bright@wintelcom.net>
To: The Hermit Hacker <scrappy@hub.org>
Subject: Re: pctrackd updates and such

On Fri, 17 Dec 1999, The Hermit Hacker wrote:

> 
> Okay, first thing...can you redo these as context diffs?  We generally
> refuse *any* patches that aren't context...

sure.

> 
> Second, are these against a reasonably current snapshot of PostgreSQL
> (aka. the upcoming v7), or v6.5.3 release?  If v6.5.3, we're gonna need to
> get these v7.x ready before we can commit them...

they are against a checked out cvs copy as of a couple days ago,
and should apply cleanly to what's in the current repo.

> Once both of the above conditions are in place, and after I get back from
> BC, I'll work on  getting these into the v7.0 release...or, at least,
> talked/commented about if there are any objections...
> 
> I'm outta here for 10 days...Happy Holidays and talk with ya when I get
> back...

ok, cool see you soon. :)

-Alfred

don't forget the problem with sending queries that may occur:

i'm not sure if handlesendfailure() can cope with only sending
a 'Q' to the backend, we may have to work out reservations or
something for space, another idea would be to implement a 
pqWritev() of some sort that would take an array of pointers
and lengths to send to the backend and only allow any data to
go into the backend if the entire string can fit.

then again, handlesendfailure may work, but doing reservations
for the send buffer seems cleaner...

diff's contexted against pgsql-'current':


Index: fe-connect.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.108
diff -u -c -r1.108 fe-connect.c
cvs diff: conflicting specifications of output style
*** fe-connect.c	1999/12/02 00:26:15	1.108
--- fe-connect.c	1999/12/14 09:42:24
***************
*** 595,625 ****
  	return 0;
  }
  
- 
- /* ----------
-  * connectMakeNonblocking -
-  * Make a connection non-blocking.
-  * Returns 1 if successful, 0 if not.
-  * ----------
-  */
- static int
- connectMakeNonblocking(PGconn *conn)
- {
- #ifndef WIN32
- 	if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
- #else
- 	if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
- #endif
- 	{
- 		printfPQExpBuffer(&conn->errorMessage,
- 						  "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
- 						  errno, strerror(errno));
- 		return 0;
- 	}
- 
- 	return 1;
- }
- 
  /* ----------
   * connectNoDelay -
   * Sets the TCP_NODELAY socket option.
--- 595,600 ----
***************
*** 792,798 ****
  	 *   Ewan Mellor <eem21@cam.ac.uk>.
  	 * ---------- */
  #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
! 	if (!connectMakeNonblocking(conn))
  		goto connect_errReturn;
  #endif	
  
--- 767,773 ----
  	 *   Ewan Mellor <eem21@cam.ac.uk>.
  	 * ---------- */
  #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
! 	if (PQsetnonblocking(conn, TRUE) != 0)
  		goto connect_errReturn;
  #endif	
  
***************
*** 904,910 ****
  	/* This makes the connection non-blocking, for all those cases which forced us
  	   not to do it above. */
  #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! 	if (!connectMakeNonblocking(conn))
  		goto connect_errReturn;
  #endif	
  
--- 879,885 ----
  	/* This makes the connection non-blocking, for all those cases which forced us
  	   not to do it above. */
  #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! 	if (PQsetnonblocking(conn, TRUE) != 0)
  		goto connect_errReturn;
  #endif	
  
***************
*** 1702,1707 ****
--- 1677,1683 ----
  	conn->inBuffer = (char *) malloc(conn->inBufSize);
  	conn->outBufSize = 8 * 1024;
  	conn->outBuffer = (char *) malloc(conn->outBufSize);
+ 	conn->nonblocking = FALSE;
  	initPQExpBuffer(&conn->errorMessage);
  	initPQExpBuffer(&conn->workBuffer);
  	if (conn->inBuffer == NULL ||
***************
*** 1811,1816 ****
--- 1787,1793 ----
  	conn->lobjfuncs = NULL;
  	conn->inStart = conn->inCursor = conn->inEnd = 0;
  	conn->outCount = 0;
+ 	conn->nonblocking = FALSE;
  
  }
  
Index: fe-exec.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.86
diff -u -c -r1.86 fe-exec.c
cvs diff: conflicting specifications of output style
*** fe-exec.c	1999/11/11 00:10:14	1.86
--- fe-exec.c	1999/12/14 05:55:11
***************
*** 13,18 ****
--- 13,19 ----
   */
  #include <errno.h>
  #include <ctype.h>
+ #include <fcntl.h>
  
  #include "postgres.h"
  #include "libpq-fe.h"
***************
*** 24,30 ****
  #include <unistd.h>
  #endif
  
- 
  /* keep this in same order as ExecStatusType in libpq-fe.h */
  const char *const pgresStatus[] = {
  	"PGRES_EMPTY_QUERY",
--- 25,30 ----
***************
*** 574,580 ****
--- 574,588 ----
  	 * we will NOT block waiting for more input.
  	 */
  	if (pqReadData(conn) < 0)
+ 	{
+ 		/*
+ 		 * try to flush the send-queue otherwise we may never get a 
+ 		 * resonce for something that may not have already been sent
+ 		 * because it's in our write buffer!
+ 		 */
+ 		pqFlush(conn);
  		return 0;
+ 	}
  	/* Parsing of the data waits till later. */
  	return 1;
  }
***************
*** 1088,1095 ****
--- 1096,1112 ----
  {
  	PGresult   *result;
  	PGresult   *lastResult;
+ 	bool	savedblocking;
  
  	/*
+ 	 * we assume anyone calling PQexec wants blocking behaviour,
+ 	 * we force the blocking status of the connection to blocking
+ 	 * for the duration of this function and restore it on return
+ 	 */
+ 	savedblocking = PQisnonblocking(conn);
+ 	PQsetnonblocking(conn, FALSE);
+ 
+ 	/*
  	 * Silently discard any prior query result that application didn't
  	 * eat. This is probably poor design, but it's here for backward
  	 * compatibility.
***************
*** 1102,1115 ****
  			PQclear(result);
  			printfPQExpBuffer(&conn->errorMessage,
  				"PQexec: you gotta get out of a COPY state yourself.\n");
! 			return NULL;
  		}
  		PQclear(result);
  	}
  
  	/* OK to send the message */
  	if (!PQsendQuery(conn, query))
! 		return NULL;
  
  	/*
  	 * For backwards compatibility, return the last result if there are
--- 1119,1133 ----
  			PQclear(result);
  			printfPQExpBuffer(&conn->errorMessage,
  				"PQexec: you gotta get out of a COPY state yourself.\n");
! 			/* restore blocking status */
! 			goto errout;
  		}
  		PQclear(result);
  	}
  
  	/* OK to send the message */
  	if (!PQsendQuery(conn, query))
! 		goto errout;
  
  	/*
  	 * For backwards compatibility, return the last result if there are
***************
*** 1142,1148 ****
--- 1160,1172 ----
  			result->resultStatus == PGRES_COPY_OUT)
  			break;
  	}
+ 
+ 	PQsetnonblocking(conn, savedblocking);
  	return lastResult;
+ 
+ errout:
+ 	PQsetnonblocking(conn, savedblocking);
+ 	return NULL;
  }
  
  
***************
*** 1431,1438 ****
  			 "PQendcopy() -- I don't think there's a copy in progress.\n");
  		return 1;
  	}
  
! 	(void) pqFlush(conn);		/* make sure no data is waiting to be sent */
  
  	/* Return to active duty */
  	conn->asyncStatus = PGASYNC_BUSY;
--- 1455,1468 ----
  			 "PQendcopy() -- I don't think there's a copy in progress.\n");
  		return 1;
  	}
+ 
+ 	/* make sure no data is waiting to be sent */
+ 	if (pqFlush(conn))
+ 		return (1);
  
! 	/* non blocking connections may have to abort at this point. */
! 	if (PQisnonblocking(conn) && PQisBusy(conn))
! 		return (1);
  
  	/* Return to active duty */
  	conn->asyncStatus = PGASYNC_BUSY;
***************
*** 2025,2028 ****
--- 2055,2126 ----
  		return 1;
  	else
  		return 0;
+ }
+ 
+ /* PQsetnonblocking:
+ 	 sets the PGconn's database connection non-blocking if the arg is TRUE
+ 	 or makes it non-blocking if the arg is FALSE, this will not protect
+ 	 you from PQexec(), you'll only be safe when using the non-blocking
+ 	 API
+ 	 Needs to be called only on a connected database connection.
+ */
+ 
+ int
+ PQsetnonblocking(PGconn *conn, int arg)
+ {
+ 	int	fcntlarg;
+ 
+ 	arg = (arg == TRUE) ? 1 : 0;
+ 	if (arg == conn->nonblocking)
+ 		return (0);
+ 
+ #ifdef USE_SSL
+ 	if (conn->ssl)
+ 	{
+ 		printfPQExpBuffer(&conn->errorMessage,
+ 			"PQsetnonblocking() -- not supported when using SSL\n");
+ 		return (-1);
+ 	}
+ #endif /* USE_SSL */
+ 
+ #ifndef WIN32
+ 	fcntlarg = fcntl(conn->sock, F_GETFL, 0);
+ 	if (fcntlarg == -1)
+ 		return (-1);
+ 
+ 	if ((arg == TRUE && 
+ 		fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
+ 		(arg == FALSE &&
+ 		fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) 
+ #else
+ 	fcntlarg = arg;
+ 	if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
+ #endif
+ 	{
+ 		printfPQExpBuffer(&conn->errorMessage,
+ 			"PQsetblocking() -- unable to set nonblocking status to %s\n",
+ 			arg == TRUE ? "TRUE" : "FALSE");
+ 		return (-1);
+ 	}
+ 
+ 	conn->nonblocking = arg;
+ 	return (0);
+ }
+ 
+ /* return the blocking status of the database connection, TRUE == nonblocking,
+ 	 FALSE == blocking
+ */
+ int
+ PQisnonblocking(PGconn *conn)
+ {
+ 
+ 	return (conn->nonblocking);
+ }
+ 
+ /* try to force data out, really only useful for non-blocking users */
+ int
+ PQflush(PGconn *conn)
+ {
+ 
+ 	return (pqFlush(conn));
  }
Index: fe-misc.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
retrieving revision 1.33
diff -u -c -r1.33 fe-misc.c
cvs diff: conflicting specifications of output style
*** fe-misc.c	1999/11/30 03:08:19	1.33
--- fe-misc.c	1999/12/14 08:21:09
***************
*** 86,91 ****
--- 86,119 ----
  {
  	size_t avail = Max(conn->outBufSize - conn->outCount, 0);
  
+ 	/*
+ 	 * if we are non-blocking and the send queue is too full to buffer this
+ 	 * request then try to flush some and return an error 
+ 	 */
+ 	if (PQisnonblocking(conn) && nbytes > avail && pqFlush(conn))
+ 	{
+ 		/* 
+ 		 * even if the flush failed we may still have written some
+ 		 * data, recalculate the size of the send-queue relative
+ 		 * to the amount we have to send, we may be able to queue it
+ 		 * afterall even though it's not sent to the database it's
+ 		 * ok, any routines that check the data coming from the
+ 		 * database better call pqFlush() anyway.
+ 		 */
+ 		if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
+ 		{
+ 			printfPQExpBuffer(&conn->errorMessage,
+ 				"pqPutBytes --  pqFlush couldn't flush enough"
+ 				" data: space available: %d, space needed %d\n",
+ 				Max(conn->outBufSize - conn->outCount, 0), nbytes);
+ 			return EOF;
+ 		}
+ 	}
+ 
+ 	/* 
+ 	 * the non-blocking code above makes sure that this isn't true,
+ 	 * essentially this is no-op
+ 	 */
  	while (nbytes > avail)
  	{
  		memcpy(conn->outBuffer + conn->outCount, s, avail);
***************
*** 548,553 ****
--- 576,589 ----
  		return EOF;
  	}
  
+ 	/* 
+ 	 * don't try to send zero data, allows us to use this function
+ 	 * without too much worry about overhead
+ 	 */
+ 	if (len == 0)
+ 		return (0);
+ 
+ 	/* while there's still data to send */
  	while (len > 0)
  	{
  		/* Prevent being SIGPIPEd if backend has closed the connection. */
***************
*** 556,561 ****
--- 592,598 ----
  #endif
  
  		int sent;
+ 
  #ifdef USE_SSL
  		if (conn->ssl) 
  		  sent = SSL_write(conn->ssl, ptr, len);
***************
*** 585,590 ****
--- 622,629 ----
  				case EWOULDBLOCK:
  					break;
  #endif
+ 				case EINTR:
+ 					continue;
  
  				case EPIPE:
  #ifdef ECONNRESET
***************
*** 616,628 ****
  			ptr += sent;
  			len -= sent;
  		}
  		if (len > 0)
  		{
  			/* We didn't send it all, wait till we can send more */
  
- 			/* At first glance this looks as though it should block.  I think
- 			 * that it will be OK though, as long as the socket is
- 			 * non-blocking. */
  			if (pqWait(FALSE, TRUE, conn))
  				return EOF;
  		}
--- 655,685 ----
  			ptr += sent;
  			len -= sent;
  		}
+ 
  		if (len > 0)
  		{
  			/* We didn't send it all, wait till we can send more */
+ 
+ 			/* 
+ 			 * if the socket is in non-blocking mode we may need
+ 			 * to abort here 
+ 			 */
+ #ifdef USE_SSL
+ 			/* can't do anything for our SSL users yet */
+ 			if (conn->ssl == NULL)
+ 			{
+ #endif
+ 				if (PQisnonblocking(conn))
+ 				{
+ 					/* shift the contents of the buffer */
+ 					memmove(conn->outBuffer, ptr, len);
+ 					conn->outCount = len;
+ 					return EOF;
+ 				}
+ #ifdef USE_SSL
+ 			}
+ #endif
  
  			if (pqWait(FALSE, TRUE, conn))
  				return EOF;
  		}
Index: libpq-fe.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.53
diff -u -c -r1.53 libpq-fe.h
cvs diff: conflicting specifications of output style
*** libpq-fe.h	1999/11/30 03:08:19	1.53
--- libpq-fe.h	1999/12/14 01:30:01
***************
*** 269,274 ****
--- 269,281 ----
  	extern int	PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
  	extern int	PQendcopy(PGconn *conn);
  
+ 	/* Set blocking/nonblocking connection to the backend */
+ 	extern int	PQsetnonblocking(PGconn *conn, int arg);
+ 	extern int	PQisnonblocking(PGconn *conn);
+ 
+ 	/* Force the write buffer to be written (or at least try) */
+ 	extern int	PQflush(PGconn *conn);
+ 
  	/*
  	 * "Fast path" interface --- not really recommended for application
  	 * use
Index: libpq-int.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
retrieving revision 1.14
diff -u -c -r1.14 libpq-int.h
cvs diff: conflicting specifications of output style
*** libpq-int.h	1999/11/30 03:08:19	1.14
--- libpq-int.h	1999/12/14 01:30:01
***************
*** 215,220 ****
--- 215,223 ----
  	int			inEnd;			/* offset to first position after avail
  								 * data */
  
+ 	int			nonblocking;	/* whether this connection is using a blocking
+ 								 * socket to the backend or not */
+ 
  	/* Buffer for data not yet sent to backend */
  	char	   *outBuffer;		/* currently allocated buffer */
  	int			outBufSize;		/* allocated size of buffer */





Sat, 8 Jan 2000 17:15:57 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
LIBPQ patches ...
Looks fine.  I have talked to someone about doing no-blocking
connections in the past.  Maybe this the same person.

I will let someone else comment on whether the protocol changes are
correct.


> 
> Does anyone have anything against me applying this to the current source
> tree?  
> 
> 
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org 
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> 
> ---------- Forwarded message ----------
> Date: Fri, 17 Dec 1999 13:51:50 -0800 (PST)
> From: Alfred Perlstein <bright@wintelcom.net>
> To: The Hermit Hacker <scrappy@hub.org>
> Subject: Re: pctrackd updates and such
> 
> On Fri, 17 Dec 1999, The Hermit Hacker wrote:
> 
> > 
> > Okay, first thing...can you redo these as context diffs?  We generally
> > refuse *any* patches that aren't context...
> 
> sure.
> 
> > 
> > Second, are these against a reasonably current snapshot of PostgreSQL
> > (aka. the upcoming v7), or v6.5.3 release?  If v6.5.3, we're gonna need to
> > get these v7.x ready before we can commit them...
> 
> they are against a checked out cvs copy as of a couple days ago,
> and should apply cleanly to what's in the current repo.
> 
> > Once both of the above conditions are in place, and after I get back from
> > BC, I'll work on  getting these into the v7.0 release...or, at least,
> > talked/commented about if there are any objections...
> > 
> > I'm outta here for 10 days...Happy Holidays and talk with ya when I get
> > back...
> 
> ok, cool see you soon. :)
> 
> -Alfred
> 
> don't forget the problem with sending queries that may occur:
> 
> i'm not sure if handlesendfailure() can cope with only sending
> a 'Q' to the backend, we may have to work out reservations or
> something for space, another idea would be to implement a 
> pqWritev() of some sort that would take an array of pointers
> and lengths to send to the backend and only allow any data to
> go into the backend if the entire string can fit.
> 
> then again, handlesendfailure may work, but doing reservations
> for the send buffer seems cleaner...
> 
> diff's contexted against pgsql-'current':
> 
> 
> Index: fe-connect.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
> retrieving revision 1.108
> diff -u -c -r1.108 fe-connect.c
> cvs diff: conflicting specifications of output style
> *** fe-connect.c	1999/12/02 00:26:15	1.108
> --- fe-connect.c	1999/12/14 09:42:24
> ***************
> *** 595,625 ****
>   	return 0;
>   }
>   
> - 
> - /* ----------
> -  * connectMakeNonblocking -
> -  * Make a connection non-blocking.
> -  * Returns 1 if successful, 0 if not.
> -  * ----------
> -  */
> - static int
> - connectMakeNonblocking(PGconn *conn)
> - {
> - #ifndef WIN32
> - 	if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
> - #else
> - 	if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
> - #endif
> - 	{
> - 		printfPQExpBuffer(&conn->errorMessage,
> - 						  "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
> - 						  errno, strerror(errno));
> - 		return 0;
> - 	}
> - 
> - 	return 1;
> - }
> - 
>   /* ----------
>    * connectNoDelay -
>    * Sets the TCP_NODELAY socket option.
> --- 595,600 ----
> ***************
> *** 792,798 ****
>   	 *   Ewan Mellor <eem21@cam.ac.uk>.
>   	 * ---------- */
>   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! 	if (!connectMakeNonblocking(conn))
>   		goto connect_errReturn;
>   #endif	
>   
> --- 767,773 ----
>   	 *   Ewan Mellor <eem21@cam.ac.uk>.
>   	 * ---------- */
>   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! 	if (PQsetnonblocking(conn, TRUE) != 0)
>   		goto connect_errReturn;
>   #endif	
>   
> ***************
> *** 904,910 ****
>   	/* This makes the connection non-blocking, for all those cases which forced us
>   	   not to do it above. */
>   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! 	if (!connectMakeNonblocking(conn))
>   		goto connect_errReturn;
>   #endif	
>   
> --- 879,885 ----
>   	/* This makes the connection non-blocking, for all those cases which forced us
>   	   not to do it above. */
>   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! 	if (PQsetnonblocking(conn, TRUE) != 0)
>   		goto connect_errReturn;
>   #endif	
>   
> ***************
> *** 1702,1707 ****
> --- 1677,1683 ----
>   	conn->inBuffer = (char *) malloc(conn->inBufSize);
>   	conn->outBufSize = 8 * 1024;
>   	conn->outBuffer = (char *) malloc(conn->outBufSize);
> + 	conn->nonblocking = FALSE;
>   	initPQExpBuffer(&conn->errorMessage);
>   	initPQExpBuffer(&conn->workBuffer);
>   	if (conn->inBuffer == NULL ||
> ***************
> *** 1811,1816 ****
> --- 1787,1793 ----
>   	conn->lobjfuncs = NULL;
>   	conn->inStart = conn->inCursor = conn->inEnd = 0;
>   	conn->outCount = 0;
> + 	conn->nonblocking = FALSE;
>   
>   }
>   
> Index: fe-exec.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
> retrieving revision 1.86
> diff -u -c -r1.86 fe-exec.c
> cvs diff: conflicting specifications of output style
> *** fe-exec.c	1999/11/11 00:10:14	1.86
> --- fe-exec.c	1999/12/14 05:55:11
> ***************
> *** 13,18 ****
> --- 13,19 ----
>    */
>   #include <errno.h>
>   #include <ctype.h>
> + #include <fcntl.h>
>   
>   #include "postgres.h"
>   #include "libpq-fe.h"
> ***************
> *** 24,30 ****
>   #include <unistd.h>
>   #endif
>   
> - 
>   /* keep this in same order as ExecStatusType in libpq-fe.h */
>   const char *const pgresStatus[] = {
>   	"PGRES_EMPTY_QUERY",
> --- 25,30 ----
> ***************
> *** 574,580 ****
> --- 574,588 ----
>   	 * we will NOT block waiting for more input.
>   	 */
>   	if (pqReadData(conn) < 0)
> + 	{
> + 		/*
> + 		 * try to flush the send-queue otherwise we may never get a 
> + 		 * resonce for something that may not have already been sent
> + 		 * because it's in our write buffer!
> + 		 */
> + 		pqFlush(conn);
>   		return 0;
> + 	}
>   	/* Parsing of the data waits till later. */
>   	return 1;
>   }
> ***************
> *** 1088,1095 ****
> --- 1096,1112 ----
>   {
>   	PGresult   *result;
>   	PGresult   *lastResult;
> + 	bool	savedblocking;
>   
>   	/*
> + 	 * we assume anyone calling PQexec wants blocking behaviour,
> + 	 * we force the blocking status of the connection to blocking
> + 	 * for the duration of this function and restore it on return
> + 	 */
> + 	savedblocking = PQisnonblocking(conn);
> + 	PQsetnonblocking(conn, FALSE);
> + 
> + 	/*
>   	 * Silently discard any prior query result that application didn't
>   	 * eat. This is probably poor design, but it's here for backward
>   	 * compatibility.
> ***************
> *** 1102,1115 ****
>   			PQclear(result);
>   			printfPQExpBuffer(&conn->errorMessage,
>   				"PQexec: you gotta get out of a COPY state yourself.\n");
> ! 			return NULL;
>   		}
>   		PQclear(result);
>   	}
>   
>   	/* OK to send the message */
>   	if (!PQsendQuery(conn, query))
> ! 		return NULL;
>   
>   	/*
>   	 * For backwards compatibility, return the last result if there are
> --- 1119,1133 ----
>   			PQclear(result);
>   			printfPQExpBuffer(&conn->errorMessage,
>   				"PQexec: you gotta get out of a COPY state yourself.\n");
> ! 			/* restore blocking status */
> ! 			goto errout;
>   		}
>   		PQclear(result);
>   	}
>   
>   	/* OK to send the message */
>   	if (!PQsendQuery(conn, query))
> ! 		goto errout;
>   
>   	/*
>   	 * For backwards compatibility, return the last result if there are
> ***************
> *** 1142,1148 ****
> --- 1160,1172 ----
>   			result->resultStatus == PGRES_COPY_OUT)
>   			break;
>   	}
> + 
> + 	PQsetnonblocking(conn, savedblocking);
>   	return lastResult;
> + 
> + errout:
> + 	PQsetnonblocking(conn, savedblocking);
> + 	return NULL;
>   }
>   
>   
> ***************
> *** 1431,1438 ****
>   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
>   		return 1;
>   	}
>   
> ! 	(void) pqFlush(conn);		/* make sure no data is waiting to be sent */
>   
>   	/* Return to active duty */
>   	conn->asyncStatus = PGASYNC_BUSY;
> --- 1455,1468 ----
>   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
>   		return 1;
>   	}
> + 
> + 	/* make sure no data is waiting to be sent */
> + 	if (pqFlush(conn))
> + 		return (1);
>   
> ! 	/* non blocking connections may have to abort at this point. */
> ! 	if (PQisnonblocking(conn) && PQisBusy(conn))
> ! 		return (1);
>   
>   	/* Return to active duty */
>   	conn->asyncStatus = PGASYNC_BUSY;
> ***************
> *** 2025,2028 ****
> --- 2055,2126 ----
>   		return 1;
>   	else
>   		return 0;
> + }
> + 
> + /* PQsetnonblocking:
> + 	 sets the PGconn's database connection non-blocking if the arg is TRUE
> + 	 or makes it non-blocking if the arg is FALSE, this will not protect
> + 	 you from PQexec(), you'll only be safe when using the non-blocking
> + 	 API
> + 	 Needs to be called only on a connected database connection.
> + */
> + 
> + int
> + PQsetnonblocking(PGconn *conn, int arg)
> + {
> + 	int	fcntlarg;
> + 
> + 	arg = (arg == TRUE) ? 1 : 0;
> + 	if (arg == conn->nonblocking)
> + 		return (0);
> + 
> + #ifdef USE_SSL
> + 	if (conn->ssl)
> + 	{
> + 		printfPQExpBuffer(&conn->errorMessage,
> + 			"PQsetnonblocking() -- not supported when using SSL\n");
> + 		return (-1);
> + 	}
> + #endif /* USE_SSL */
> + 
> + #ifndef WIN32
> + 	fcntlarg = fcntl(conn->sock, F_GETFL, 0);
> + 	if (fcntlarg == -1)
> + 		return (-1);
> + 
> + 	if ((arg == TRUE && 
> + 		fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
> + 		(arg == FALSE &&
> + 		fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) 
> + #else
> + 	fcntlarg = arg;
> + 	if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
> + #endif
> + 	{
> + 		printfPQExpBuffer(&conn->errorMessage,
> + 			"PQsetblocking() -- unable to set nonblocking status to %s\n",
> + 			arg == TRUE ? "TRUE" : "FALSE");
> + 		return (-1);
> + 	}
> + 
> + 	conn->nonblocking = arg;
> + 	return (0);
> + }
> + 
> + /* return the blocking status of the database connection, TRUE == nonblocking,
> + 	 FALSE == blocking
> + */
> + int
> + PQisnonblocking(PGconn *conn)
> + {
> + 
> + 	return (conn->nonblocking);
> + }
> + 
> + /* try to force data out, really only useful for non-blocking users */
> + int
> + PQflush(PGconn *conn)
> + {
> + 
> + 	return (pqFlush(conn));
>   }
> Index: fe-misc.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
> retrieving revision 1.33
> diff -u -c -r1.33 fe-misc.c
> cvs diff: conflicting specifications of output style
> *** fe-misc.c	1999/11/30 03:08:19	1.33
> --- fe-misc.c	1999/12/14 08:21:09
> ***************
> *** 86,91 ****
> --- 86,119 ----
>   {
>   	size_t avail = Max(conn->outBufSize - conn->outCount, 0);
>   
> + 	/*
> + 	 * if we are non-blocking and the send queue is too full to buffer this
> + 	 * request then try to flush some and return an error 
> + 	 */
> + 	if (PQisnonblocking(conn) && nbytes > avail && pqFlush(conn))
> + 	{
> + 		/* 
> + 		 * even if the flush failed we may still have written some
> + 		 * data, recalculate the size of the send-queue relative
> + 		 * to the amount we have to send, we may be able to queue it
> + 		 * afterall even though it's not sent to the database it's
> + 		 * ok, any routines that check the data coming from the
> + 		 * database better call pqFlush() anyway.
> + 		 */
> + 		if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
> + 		{
> + 			printfPQExpBuffer(&conn->errorMessage,
> + 				"pqPutBytes --  pqFlush couldn't flush enough"
> + 				" data: space available: %d, space needed %d\n",
> + 				Max(conn->outBufSize - conn->outCount, 0), nbytes);
> + 			return EOF;
> + 		}
> + 	}
> + 
> + 	/* 
> + 	 * the non-blocking code above makes sure that this isn't true,
> + 	 * essentially this is no-op
> + 	 */
>   	while (nbytes > avail)
>   	{
>   		memcpy(conn->outBuffer + conn->outCount, s, avail);
> ***************
> *** 548,553 ****
> --- 576,589 ----
>   		return EOF;
>   	}
>   
> + 	/* 
> + 	 * don't try to send zero data, allows us to use this function
> + 	 * without too much worry about overhead
> + 	 */
> + 	if (len == 0)
> + 		return (0);
> + 
> + 	/* while there's still data to send */
>   	while (len > 0)
>   	{
>   		/* Prevent being SIGPIPEd if backend has closed the connection. */
> ***************
> *** 556,561 ****
> --- 592,598 ----
>   #endif
>   
>   		int sent;
> + 
>   #ifdef USE_SSL
>   		if (conn->ssl) 
>   		  sent = SSL_write(conn->ssl, ptr, len);
> ***************
> *** 585,590 ****
> --- 622,629 ----
>   				case EWOULDBLOCK:
>   					break;
>   #endif
> + 				case EINTR:
> + 					continue;
>   
>   				case EPIPE:
>   #ifdef ECONNRESET
> ***************
> *** 616,628 ****
>   			ptr += sent;
>   			len -= sent;
>   		}
>   		if (len > 0)
>   		{
>   			/* We didn't send it all, wait till we can send more */
>   
> - 			/* At first glance this looks as though it should block.  I think
> - 			 * that it will be OK though, as long as the socket is
> - 			 * non-blocking. */
>   			if (pqWait(FALSE, TRUE, conn))
>   				return EOF;
>   		}
> --- 655,685 ----
>   			ptr += sent;
>   			len -= sent;
>   		}
> + 
>   		if (len > 0)
>   		{
>   			/* We didn't send it all, wait till we can send more */
> + 
> + 			/* 
> + 			 * if the socket is in non-blocking mode we may need
> + 			 * to abort here 
> + 			 */
> + #ifdef USE_SSL
> + 			/* can't do anything for our SSL users yet */
> + 			if (conn->ssl == NULL)
> + 			{
> + #endif
> + 				if (PQisnonblocking(conn))
> + 				{
> + 					/* shift the contents of the buffer */
> + 					memmove(conn->outBuffer, ptr, len);
> + 					conn->outCount = len;
> + 					return EOF;
> + 				}
> + #ifdef USE_SSL
> + 			}
> + #endif
>   
>   			if (pqWait(FALSE, TRUE, conn))
>   				return EOF;
>   		}
> Index: libpq-fe.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
> retrieving revision 1.53
> diff -u -c -r1.53 libpq-fe.h
> cvs diff: conflicting specifications of output style
> *** libpq-fe.h	1999/11/30 03:08:19	1.53
> --- libpq-fe.h	1999/12/14 01:30:01
> ***************
> *** 269,274 ****
> --- 269,281 ----
>   	extern int	PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
>   	extern int	PQendcopy(PGconn *conn);
>   
> + 	/* Set blocking/nonblocking connection to the backend */
> + 	extern int	PQsetnonblocking(PGconn *conn, int arg);
> + 	extern int	PQisnonblocking(PGconn *conn);
> + 
> + 	/* Force the write buffer to be written (or at least try) */
> + 	extern int	PQflush(PGconn *conn);
> + 
>   	/*
>   	 * "Fast path" interface --- not really recommended for application
>   	 * use
> Index: libpq-int.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
> retrieving revision 1.14
> diff -u -c -r1.14 libpq-int.h
> cvs diff: conflicting specifications of output style
> *** libpq-int.h	1999/11/30 03:08:19	1.14
> --- libpq-int.h	1999/12/14 01:30:01
> ***************
> *** 215,220 ****
> --- 215,223 ----
>   	int			inEnd;			/* offset to first position after avail
>   								 * data */
>   
> + 	int			nonblocking;	/* whether this connection is using a blocking
> + 								 * socket to the backend or not */
> + 
>   	/* Buffer for data not yet sent to backend */
>   	char	   *outBuffer;		/* currently allocated buffer */
>   	int			outBufSize;		/* allocated size of buffer */
> 
> 
> 
> 
> 
> ************
> 


-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sat, 8 Jan 2000 16:27:49 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] LIBPQ patches ...
On Sat, 8 Jan 2000, Bruce Momjian wrote:

> Looks fine.  I have talked to someone about doing no-blocking
> connections in the past.  Maybe this the same person.
> 
> I will let someone else comment on whether the protocol changes are
> correct.

Okay, if I haven't heard anything major by Sunday, I'm going to include
these, whic still gives us a month before beta (well, not quite, but
close) and then the beta period in order to clean it up...

 > 
> 
> > 
> > Does anyone have anything against me applying this to the current source
> > tree?  
> > 
> > 
> > Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> > Systems Administrator @ hub.org 
> > primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
> > 
> > ---------- Forwarded message ----------
> > Date: Fri, 17 Dec 1999 13:51:50 -0800 (PST)
> > From: Alfred Perlstein <bright@wintelcom.net>
> > To: The Hermit Hacker <scrappy@hub.org>
> > Subject: Re: pctrackd updates and such
> > 
> > On Fri, 17 Dec 1999, The Hermit Hacker wrote:
> > 
> > > 
> > > Okay, first thing...can you redo these as context diffs?  We generally
> > > refuse *any* patches that aren't context...
> > 
> > sure.
> > 
> > > 
> > > Second, are these against a reasonably current snapshot of PostgreSQL
> > > (aka. the upcoming v7), or v6.5.3 release?  If v6.5.3, we're gonna need to
> > > get these v7.x ready before we can commit them...
> > 
> > they are against a checked out cvs copy as of a couple days ago,
> > and should apply cleanly to what's in the current repo.
> > 
> > > Once both of the above conditions are in place, and after I get back from
> > > BC, I'll work on  getting these into the v7.0 release...or, at least,
> > > talked/commented about if there are any objections...
> > > 
> > > I'm outta here for 10 days...Happy Holidays and talk with ya when I get
> > > back...
> > 
> > ok, cool see you soon. :)
> > 
> > -Alfred
> > 
> > don't forget the problem with sending queries that may occur:
> > 
> > i'm not sure if handlesendfailure() can cope with only sending
> > a 'Q' to the backend, we may have to work out reservations or
> > something for space, another idea would be to implement a 
> > pqWritev() of some sort that would take an array of pointers
> > and lengths to send to the backend and only allow any data to
> > go into the backend if the entire string can fit.
> > 
> > then again, handlesendfailure may work, but doing reservations
> > for the send buffer seems cleaner...
> > 
> > diff's contexted against pgsql-'current':
> > 
> > 
> > Index: fe-connect.c
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
> > retrieving revision 1.108
> > diff -u -c -r1.108 fe-connect.c
> > cvs diff: conflicting specifications of output style
> > *** fe-connect.c	1999/12/02 00:26:15	1.108
> > --- fe-connect.c	1999/12/14 09:42:24
> > ***************
> > *** 595,625 ****
> >   	return 0;
> >   }
> >   
> > - 
> > - /* ----------
> > -  * connectMakeNonblocking -
> > -  * Make a connection non-blocking.
> > -  * Returns 1 if successful, 0 if not.
> > -  * ----------
> > -  */
> > - static int
> > - connectMakeNonblocking(PGconn *conn)
> > - {
> > - #ifndef WIN32
> > - 	if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
> > - #else
> > - 	if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
> > - #endif
> > - 	{
> > - 		printfPQExpBuffer(&conn->errorMessage,
> > - 						  "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
> > - 						  errno, strerror(errno));
> > - 		return 0;
> > - 	}
> > - 
> > - 	return 1;
> > - }
> > - 
> >   /* ----------
> >    * connectNoDelay -
> >    * Sets the TCP_NODELAY socket option.
> > --- 595,600 ----
> > ***************
> > *** 792,798 ****
> >   	 *   Ewan Mellor <eem21@cam.ac.uk>.
> >   	 * ---------- */
> >   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> > ! 	if (!connectMakeNonblocking(conn))
> >   		goto connect_errReturn;
> >   #endif	
> >   
> > --- 767,773 ----
> >   	 *   Ewan Mellor <eem21@cam.ac.uk>.
> >   	 * ---------- */
> >   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> > ! 	if (PQsetnonblocking(conn, TRUE) != 0)
> >   		goto connect_errReturn;
> >   #endif	
> >   
> > ***************
> > *** 904,910 ****
> >   	/* This makes the connection non-blocking, for all those cases which forced us
> >   	   not to do it above. */
> >   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> > ! 	if (!connectMakeNonblocking(conn))
> >   		goto connect_errReturn;
> >   #endif	
> >   
> > --- 879,885 ----
> >   	/* This makes the connection non-blocking, for all those cases which forced us
> >   	   not to do it above. */
> >   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> > ! 	if (PQsetnonblocking(conn, TRUE) != 0)
> >   		goto connect_errReturn;
> >   #endif	
> >   
> > ***************
> > *** 1702,1707 ****
> > --- 1677,1683 ----
> >   	conn->inBuffer = (char *) malloc(conn->inBufSize);
> >   	conn->outBufSize = 8 * 1024;
> >   	conn->outBuffer = (char *) malloc(conn->outBufSize);
> > + 	conn->nonblocking = FALSE;
> >   	initPQExpBuffer(&conn->errorMessage);
> >   	initPQExpBuffer(&conn->workBuffer);
> >   	if (conn->inBuffer == NULL ||
> > ***************
> > *** 1811,1816 ****
> > --- 1787,1793 ----
> >   	conn->lobjfuncs = NULL;
> >   	conn->inStart = conn->inCursor = conn->inEnd = 0;
> >   	conn->outCount = 0;
> > + 	conn->nonblocking = FALSE;
> >   
> >   }
> >   
> > Index: fe-exec.c
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
> > retrieving revision 1.86
> > diff -u -c -r1.86 fe-exec.c
> > cvs diff: conflicting specifications of output style
> > *** fe-exec.c	1999/11/11 00:10:14	1.86
> > --- fe-exec.c	1999/12/14 05:55:11
> > ***************
> > *** 13,18 ****
> > --- 13,19 ----
> >    */
> >   #include <errno.h>
> >   #include <ctype.h>
> > + #include <fcntl.h>
> >   
> >   #include "postgres.h"
> >   #include "libpq-fe.h"
> > ***************
> > *** 24,30 ****
> >   #include <unistd.h>
> >   #endif
> >   
> > - 
> >   /* keep this in same order as ExecStatusType in libpq-fe.h */
> >   const char *const pgresStatus[] = {
> >   	"PGRES_EMPTY_QUERY",
> > --- 25,30 ----
> > ***************
> > *** 574,580 ****
> > --- 574,588 ----
> >   	 * we will NOT block waiting for more input.
> >   	 */
> >   	if (pqReadData(conn) < 0)
> > + 	{
> > + 		/*
> > + 		 * try to flush the send-queue otherwise we may never get a 
> > + 		 * resonce for something that may not have already been sent
> > + 		 * because it's in our write buffer!
> > + 		 */
> > + 		pqFlush(conn);
> >   		return 0;
> > + 	}
> >   	/* Parsing of the data waits till later. */
> >   	return 1;
> >   }
> > ***************
> > *** 1088,1095 ****
> > --- 1096,1112 ----
> >   {
> >   	PGresult   *result;
> >   	PGresult   *lastResult;
> > + 	bool	savedblocking;
> >   
> >   	/*
> > + 	 * we assume anyone calling PQexec wants blocking behaviour,
> > + 	 * we force the blocking status of the connection to blocking
> > + 	 * for the duration of this function and restore it on return
> > + 	 */
> > + 	savedblocking = PQisnonblocking(conn);
> > + 	PQsetnonblocking(conn, FALSE);
> > + 
> > + 	/*
> >   	 * Silently discard any prior query result that application didn't
> >   	 * eat. This is probably poor design, but it's here for backward
> >   	 * compatibility.
> > ***************
> > *** 1102,1115 ****
> >   			PQclear(result);
> >   			printfPQExpBuffer(&conn->errorMessage,
> >   				"PQexec: you gotta get out of a COPY state yourself.\n");
> > ! 			return NULL;
> >   		}
> >   		PQclear(result);
> >   	}
> >   
> >   	/* OK to send the message */
> >   	if (!PQsendQuery(conn, query))
> > ! 		return NULL;
> >   
> >   	/*
> >   	 * For backwards compatibility, return the last result if there are
> > --- 1119,1133 ----
> >   			PQclear(result);
> >   			printfPQExpBuffer(&conn->errorMessage,
> >   				"PQexec: you gotta get out of a COPY state yourself.\n");
> > ! 			/* restore blocking status */
> > ! 			goto errout;
> >   		}
> >   		PQclear(result);
> >   	}
> >   
> >   	/* OK to send the message */
> >   	if (!PQsendQuery(conn, query))
> > ! 		goto errout;
> >   
> >   	/*
> >   	 * For backwards compatibility, return the last result if there are
> > ***************
> > *** 1142,1148 ****
> > --- 1160,1172 ----
> >   			result->resultStatus == PGRES_COPY_OUT)
> >   			break;
> >   	}
> > + 
> > + 	PQsetnonblocking(conn, savedblocking);
> >   	return lastResult;
> > + 
> > + errout:
> > + 	PQsetnonblocking(conn, savedblocking);
> > + 	return NULL;
> >   }
> >   
> >   
> > ***************
> > *** 1431,1438 ****
> >   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
> >   		return 1;
> >   	}
> >   
> > ! 	(void) pqFlush(conn);		/* make sure no data is waiting to be sent */
> >   
> >   	/* Return to active duty */
> >   	conn->asyncStatus = PGASYNC_BUSY;
> > --- 1455,1468 ----
> >   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
> >   		return 1;
> >   	}
> > + 
> > + 	/* make sure no data is waiting to be sent */
> > + 	if (pqFlush(conn))
> > + 		return (1);
> >   
> > ! 	/* non blocking connections may have to abort at this point. */
> > ! 	if (PQisnonblocking(conn) && PQisBusy(conn))
> > ! 		return (1);
> >   
> >   	/* Return to active duty */
> >   	conn->asyncStatus = PGASYNC_BUSY;
> > ***************
> > *** 2025,2028 ****
> > --- 2055,2126 ----
> >   		return 1;
> >   	else
> >   		return 0;
> > + }
> > + 
> > + /* PQsetnonblocking:
> > + 	 sets the PGconn's database connection non-blocking if the arg is TRUE
> > + 	 or makes it non-blocking if the arg is FALSE, this will not protect
> > + 	 you from PQexec(), you'll only be safe when using the non-blocking
> > + 	 API
> > + 	 Needs to be called only on a connected database connection.
> > + */
> > + 
> > + int
> > + PQsetnonblocking(PGconn *conn, int arg)
> > + {
> > + 	int	fcntlarg;
> > + 
> > + 	arg = (arg == TRUE) ? 1 : 0;
> > + 	if (arg == conn->nonblocking)
> > + 		return (0);
> > + 
> > + #ifdef USE_SSL
> > + 	if (conn->ssl)
> > + 	{
> > + 		printfPQExpBuffer(&conn->errorMessage,
> > + 			"PQsetnonblocking() -- not supported when using SSL\n");
> > + 		return (-1);
> > + 	}
> > + #endif /* USE_SSL */
> > + 
> > + #ifndef WIN32
> > + 	fcntlarg = fcntl(conn->sock, F_GETFL, 0);
> > + 	if (fcntlarg == -1)
> > + 		return (-1);
> > + 
> > + 	if ((arg == TRUE && 
> > + 		fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
> > + 		(arg == FALSE &&
> > + 		fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) 
> > + #else
> > + 	fcntlarg = arg;
> > + 	if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
> > + #endif
> > + 	{
> > + 		printfPQExpBuffer(&conn->errorMessage,
> > + 			"PQsetblocking() -- unable to set nonblocking status to %s\n",
> > + 			arg == TRUE ? "TRUE" : "FALSE");
> > + 		return (-1);
> > + 	}
> > + 
> > + 	conn->nonblocking = arg;
> > + 	return (0);
> > + }
> > + 
> > + /* return the blocking status of the database connection, TRUE == nonblocking,
> > + 	 FALSE == blocking
> > + */
> > + int
> > + PQisnonblocking(PGconn *conn)
> > + {
> > + 
> > + 	return (conn->nonblocking);
> > + }
> > + 
> > + /* try to force data out, really only useful for non-blocking users */
> > + int
> > + PQflush(PGconn *conn)
> > + {
> > + 
> > + 	return (pqFlush(conn));
> >   }
> > Index: fe-misc.c
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
> > retrieving revision 1.33
> > diff -u -c -r1.33 fe-misc.c
> > cvs diff: conflicting specifications of output style
> > *** fe-misc.c	1999/11/30 03:08:19	1.33
> > --- fe-misc.c	1999/12/14 08:21:09
> > ***************
> > *** 86,91 ****
> > --- 86,119 ----
> >   {
> >   	size_t avail = Max(conn->outBufSize - conn->outCount, 0);
> >   
> > + 	/*
> > + 	 * if we are non-blocking and the send queue is too full to buffer this
> > + 	 * request then try to flush some and return an error 
> > + 	 */
> > + 	if (PQisnonblocking(conn) && nbytes > avail && pqFlush(conn))
> > + 	{
> > + 		/* 
> > + 		 * even if the flush failed we may still have written some
> > + 		 * data, recalculate the size of the send-queue relative
> > + 		 * to the amount we have to send, we may be able to queue it
> > + 		 * afterall even though it's not sent to the database it's
> > + 		 * ok, any routines that check the data coming from the
> > + 		 * database better call pqFlush() anyway.
> > + 		 */
> > + 		if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
> > + 		{
> > + 			printfPQExpBuffer(&conn->errorMessage,
> > + 				"pqPutBytes --  pqFlush couldn't flush enough"
> > + 				" data: space available: %d, space needed %d\n",
> > + 				Max(conn->outBufSize - conn->outCount, 0), nbytes);
> > + 			return EOF;
> > + 		}
> > + 	}
> > + 
> > + 	/* 
> > + 	 * the non-blocking code above makes sure that this isn't true,
> > + 	 * essentially this is no-op
> > + 	 */
> >   	while (nbytes > avail)
> >   	{
> >   		memcpy(conn->outBuffer + conn->outCount, s, avail);
> > ***************
> > *** 548,553 ****
> > --- 576,589 ----
> >   		return EOF;
> >   	}
> >   
> > + 	/* 
> > + 	 * don't try to send zero data, allows us to use this function
> > + 	 * without too much worry about overhead
> > + 	 */
> > + 	if (len == 0)
> > + 		return (0);
> > + 
> > + 	/* while there's still data to send */
> >   	while (len > 0)
> >   	{
> >   		/* Prevent being SIGPIPEd if backend has closed the connection. */
> > ***************
> > *** 556,561 ****
> > --- 592,598 ----
> >   #endif
> >   
> >   		int sent;
> > + 
> >   #ifdef USE_SSL
> >   		if (conn->ssl) 
> >   		  sent = SSL_write(conn->ssl, ptr, len);
> > ***************
> > *** 585,590 ****
> > --- 622,629 ----
> >   				case EWOULDBLOCK:
> >   					break;
> >   #endif
> > + 				case EINTR:
> > + 					continue;
> >   
> >   				case EPIPE:
> >   #ifdef ECONNRESET
> > ***************
> > *** 616,628 ****
> >   			ptr += sent;
> >   			len -= sent;
> >   		}
> >   		if (len > 0)
> >   		{
> >   			/* We didn't send it all, wait till we can send more */
> >   
> > - 			/* At first glance this looks as though it should block.  I think
> > - 			 * that it will be OK though, as long as the socket is
> > - 			 * non-blocking. */
> >   			if (pqWait(FALSE, TRUE, conn))
> >   				return EOF;
> >   		}
> > --- 655,685 ----
> >   			ptr += sent;
> >   			len -= sent;
> >   		}
> > + 
> >   		if (len > 0)
> >   		{
> >   			/* We didn't send it all, wait till we can send more */
> > + 
> > + 			/* 
> > + 			 * if the socket is in non-blocking mode we may need
> > + 			 * to abort here 
> > + 			 */
> > + #ifdef USE_SSL
> > + 			/* can't do anything for our SSL users yet */
> > + 			if (conn->ssl == NULL)
> > + 			{
> > + #endif
> > + 				if (PQisnonblocking(conn))
> > + 				{
> > + 					/* shift the contents of the buffer */
> > + 					memmove(conn->outBuffer, ptr, len);
> > + 					conn->outCount = len;
> > + 					return EOF;
> > + 				}
> > + #ifdef USE_SSL
> > + 			}
> > + #endif
> >   
> >   			if (pqWait(FALSE, TRUE, conn))
> >   				return EOF;
> >   		}
> > Index: libpq-fe.h
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
> > retrieving revision 1.53
> > diff -u -c -r1.53 libpq-fe.h
> > cvs diff: conflicting specifications of output style
> > *** libpq-fe.h	1999/11/30 03:08:19	1.53
> > --- libpq-fe.h	1999/12/14 01:30:01
> > ***************
> > *** 269,274 ****
> > --- 269,281 ----
> >   	extern int	PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
> >   	extern int	PQendcopy(PGconn *conn);
> >   
> > + 	/* Set blocking/nonblocking connection to the backend */
> > + 	extern int	PQsetnonblocking(PGconn *conn, int arg);
> > + 	extern int	PQisnonblocking(PGconn *conn);
> > + 
> > + 	/* Force the write buffer to be written (or at least try) */
> > + 	extern int	PQflush(PGconn *conn);
> > + 
> >   	/*
> >   	 * "Fast path" interface --- not really recommended for application
> >   	 * use
> > Index: libpq-int.h
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
> > retrieving revision 1.14
> > diff -u -c -r1.14 libpq-int.h
> > cvs diff: conflicting specifications of output style
> > *** libpq-int.h	1999/11/30 03:08:19	1.14
> > --- libpq-int.h	1999/12/14 01:30:01
> > ***************
> > *** 215,220 ****
> > --- 215,223 ----
> >   	int			inEnd;			/* offset to first position after avail
> >   								 * data */
> >   
> > + 	int			nonblocking;	/* whether this connection is using a blocking
> > + 								 * socket to the backend or not */
> > + 
> >   	/* Buffer for data not yet sent to backend */
> >   	char	   *outBuffer;		/* currently allocated buffer */
> >   	int			outBufSize;		/* allocated size of buffer */
> > 
> > 
> > 
> > 
> > 
> > ************
> > 
> 
> 
> -- 
>   Bruce Momjian                        |  http://www.op.net/~candle
>   maillist@candle.pha.pa.us            |  (610) 853-3000
>   +  If your life is a hard drive,     |  830 Blythe Avenue
>   +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
> 

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


Sat, 8 Jan 2000 17:53:39 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] LIBPQ patches ...
The Hermit Hacker <scrappy@hub.org> writes:
> Does anyone have anything against me applying this to the current source
> tree?  

I'm not particularly comfortable with it --- it looks like the semantics
need more careful thought, particularly concerning when the output buffer
gets flushed and what happens if we can't send data right away.  The
insertion of a pqFlush into PQconsumeInput, in particular, looks like
an ill-thought-out hack that could break some applications.

I also object strongly to the lack of documentation.  Patches that
change public APIs and come without doco updates should be rejected
out of hand, IMNSHO.  Keeping the documentation up to date should
not be considered optional --- especially not when you're talking
about something that makes subtle and pervasive changes to library
behavior.

			regards, tom lane

Sat, 08 Jan 2000 17:27:02 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] LIBPQ patches ... 
On Sat, 8 Jan 2000, Tom Lane wrote:

> The Hermit Hacker <scrappy@hub.org> writes:
> > Does anyone have anything against me applying this to the current source
> > tree?  
> 
> I'm not particularly comfortable with it --- it looks like the semantics
> need more careful thought, particularly concerning when the output buffer
> gets flushed and what happens if we can't send data right away.  The
> insertion of a pqFlush into PQconsumeInput, in particular, looks like
> an ill-thought-out hack that could break some applications.

Well, at least we have more discussion on this then the previous two posts
about it, so it should give something for Alfred to address :)  Is there
anyone workign with libpq that could comment on possible a better way of
it being implemented?  

> I also object strongly to the lack of documentation.  Patches that
> change public APIs and come without doco updates should be rejected
> out of hand, IMNSHO.  Keeping the documentation up to date should
> not be considered optional --- especially not when you're talking
> about something that makes subtle and pervasive changes to library
> behavior.

Agreed here...Alfred and I talked about that on the phone tonight...I
posted the patches tonight so that he could get some feedback on them...if
we could figure out what needs to be fixed/improved, and he has an
indication that he's working in the right direction, then documentation is
forthcoming...

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


Sat, 8 Jan 2000 18:38:16 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] LIBPQ patches ... 
* Tom Lane <tgl@sss.pgh.pa.us> [000108 14:56] wrote:
> The Hermit Hacker <scrappy@hub.org> writes:
> > Does anyone have anything against me applying this to the current source
> > tree?  
> 
> I'm not particularly comfortable with it --- it looks like the semantics
> need more careful thought, particularly concerning when the output buffer
> gets flushed and what happens if we can't send data right away.  

Could you be more specific?  My patches address the fact that
although there is work to make libpq non-blocking you can easily
block while sending large queries expecially because of the select()
that is done in pqFlush().

The problem is that libpq doesn't reserve space in the send buffer
and will just block if waiting for the socket to the backend to
drain.  This needs to be fixed if libpq is trully going to offer
non-blocking behavior.

Unless you reserve the space in the buffer you have to block
otherwise if you abort (so as not to block) then libpq may have
sent a partial query down the pipe or just buffered part of some
data you've sent to the backend.  At this point you will be out of sync
with the backend.

If you are in 'normal mode' (blocking) then the behavior shouldn't
be any different, if you are non-blocking then if you attempt to
send data and it's not possible you'll get an error without
potentially sending a partial line to the backend.

> The
> insertion of a pqFlush into PQconsumeInput, in particular, looks like
> an ill-thought-out hack that could break some applications.

I think I agree, the code I was using would attempt an PQconsumeInput()
before doing a PQendcopy(), there could be data in the send buffer
that would make PQconsumeInput() never succeed hence the need for a
flush.

I'm going to try it without the PQconsumeInput() before the PQendcopy()
my modifications for PQendcopy() should make it non-blocking safe.
but in the meanwhile here's my (probably wrong) reasoning behind this
'hack': 

	No, IMHO it's needed, the problem is that there may be data
	in the send buffer that hasn't been sent yet, it could be
	part of a request to the backend that you are explicitly
	waiting for a result from.

	This can happen when doing a COPY into the database.

	What happens is that you send data, then when you send the
	'end copy' it can get buffered, then you loop forever
	attempting to read a result for a query that was never
	sent.

	In regards to it breaking applications, the send buffer
	should be opaque to the libpq user, libpq never has offered
	a truly non-blocking api, and even when using non-blocking
	the flush will fail if it can't be done and PQconsumeInput()
	will error out accordingly.

	Old applications can be snagged by the Flush since in theory
	PQconsumeInput shouldn't block, however I'm not sure if
	there's a real workaround for this except

	1.. saving the blocking status of the connection, 
	2.. setting it non-blocking and attempting a flush and then
	3.. restoring the blocking status.

	It seems that old applications can break (looping on an
	unsent line) regardless because of the not-flushed-query
	problem.

If you can figure an occasion where this might actually happen
(with the exception of my accidentaly abuse of libpq) then it
may need to be revisited.

I'll get back to you guys on the PQendcopy before PQconsumeInput
tests.

> I also object strongly to the lack of documentation.  Patches that
> change public APIs and come without doco updates should be rejected
> out of hand, IMNSHO.  Keeping the documentation up to date should
> not be considered optional --- especially not when you're talking
> about something that makes subtle and pervasive changes to library
> behavior.

I agree with you about the documentation issues, I will try to add
some documentation to the patches.

I think I can also take out the visibility of the PQflush() function
as normal applications really shouldn't need it.

How do you feel about the explicit PQsetnonblocking and PQisnonblocking
functions that I added as well as the additional field 'nonblocking' 
added to PGconn?   IMO the user shouldn't set the socket non-blocking
without informing the library about it otherwise it gets really ugly
because we have to constantly poll the socket's flags to make sure we
DTRT.

I also apologize for my not indented patches, is there a way to indent
according to postgresql standards on a FreeBSD system?  The patches
for pgindent are a bit out of date and I get floating point exceptions
when attempting to pgindent.

Thanks for the feedback.
-Alfred

Sat, 8 Jan 2000 16:01:48 -0800
Alfred Perlstein <bright@wintelcom.net>
Re: [HACKERS] LIBPQ patches ...
* Alfred Perlstein <bright@wintelcom.net> [000108 16:08] wrote:
> * Tom Lane <tgl@sss.pgh.pa.us> [000108 14:56] wrote:
> 
> > The
> > insertion of a pqFlush into PQconsumeInput, in particular, looks like
> > an ill-thought-out hack that could break some applications.
> 
> I think I agree, the code I was using would attempt an PQconsumeInput()
> before doing a PQendcopy(), there could be data in the send buffer
> that would make PQconsumeInput() never succeed hence the need for a
> flush.
> 
> I'm going to try it without the PQconsumeInput() before the PQendcopy()
> my modifications for PQendcopy() should make it non-blocking safe.
> but in the meanwhile here's my (probably wrong) reasoning behind this
> 'hack': 
> 
> 	No, IMHO it's needed, the problem is that there may be data
> 	in the send buffer that hasn't been sent yet, it could be
> 	part of a request to the backend that you are explicitly
> 	waiting for a result from.
> 
> 	This can happen when doing a COPY into the database.
> 
> 	What happens is that you send data, then when you send the
> 	'end copy' it can get buffered, then you loop forever
> 	attempting to read a result for a query that was never
> 	sent.
> 
> 	In regards to it breaking applications, the send buffer
> 	should be opaque to the libpq user, libpq never has offered
> 	a truly non-blocking api, and even when using non-blocking
> 	the flush will fail if it can't be done and PQconsumeInput()
> 	will error out accordingly.
> 
> 	Old applications can be snagged by the Flush since in theory
> 	PQconsumeInput shouldn't block, however I'm not sure if
> 	there's a real workaround for this except
> 
> 	1.. saving the blocking status of the connection, 
> 	2.. setting it non-blocking and attempting a flush and then
> 	3.. restoring the blocking status.
> 
> 	It seems that old applications can break (looping on an
> 	unsent line) regardless because of the not-flushed-query
> 	problem.
> 
> If you can figure an occasion where this might actually happen
> (with the exception of my accidentaly abuse of libpq) then it
> may need to be revisited.
> 
> I'll get back to you guys on the PQendcopy before PQconsumeInput
> tests.

I just remebered where the problem is (sorry it's been about 2 weeks
since i've read through the code) it's a bit different
and messier than I thought:

I begin my COPY commands with a PQsendQuery() which has this block
of code in it:

	/* send the query to the backend; */
	/* the frontend-backend protocol uses 'Q' to designate queries */
	if (pqPutnchar("Q", 1, conn) ||
		pqPuts(query, conn) ||
		pqFlush(conn))
	{
		handleSendFailure(conn);
		return 0;
	}

It can get really hairy for non-blocking connections if any of the
functions in the 'if' conditional fail, any ideas on a workaround?

One that comes to mind is using the low/high watermarks in sockets,
if we do that then a write-ready true condition would garantee that
we have X number of bytes available in our send buffer and we can
safely queue the data.  This doesn't seem portable and would be 
pretty complex.

Another is to attempt a flush beforehand aborting early if it fails
however we still need to flush after we pqPutnchar and pqPuts,
if that fails we are back to needing to call pqFlush from PQconsumeInput
beccause the only failure is that the backend's pipe is potentially full
and we may have queued something.

This 'hack' would be to allow the last flush to fail and always call
pqFlush in PQconsumeInput when the connection is non-blocking because
with the new code it shouldn't block, PQconsumeInput will function to
drive data to the backend in that situation.  We'll only do the
flush in PQconsumeInput() for non-blocking connections because
the conditional in PQsendQuery shouldn't fail for blocking connections
unless something is seriously wrong.

I hate to say it, but I like the hack approach, it is somewhat wierd
but looks like it would work quite well as any error returned from
the backend would be delayed until PQconsumeInput and a broken connection
would still be returned immediatly from PQsendQuery.

Your opinion?

-- 
-Alfred Perlstein - [bright@rush.net|alfred@freebsd.org]
Wintelcom systems administrator and programmer
   - http://www.wintelcom.net/ [bright@wintelcom.net]

Sat, 8 Jan 2000 16:34:52 -0800
Alfred Perlstein <bright@wintelcom.net>
Re: [HACKERS] LIBPQ patches ...

Hello all.  I wanted to throw up an idea for a
XML interface to PostgreSQL. 

The underlying information model for SGML documents
is called the "groves" model.  The model's primary
prupose is to handle multiple content hierarchites
within a single network; as this is the primary 
problem with multimedia hypertext markup (HyTime).
Anyway, what is interesting is that the core of the
groves model seems to be the following seemingly
simple production:

  node :=  character | map( string, list(node) )

It is an alternation between a map (unordered 
set) and a list (ordered bag).  Anyway, the discovery
made me sit back in my chair and go hmmmmmm. As it 
seems to me; this model is the essence of a multi-level 
result set, and could point towards a rather natural
mapping of a multi-hierarchy result set onto a tag
based markup language like XML.

For starters, here is a row...

  row-node := map (string, list(characters))

For example, here is an order, and two order-line rows:

  order-row-node  :=  { id       = '345'
                      , buyer    = 'Clark Evans'
                      }

  order-line-row  :=  { order-id = '345'
                      , product  = 'Bag of rice'  
                      , quantity = '1'
                      }

  order-line-row  :=  { order-id = '345'
                      , product  = 'Tofo'
                      , quantity = '3'

And here, is a table:

  relation-node   := map('relation-name',list(row-node))

  line-relation   := { order-line = 
                       [
                         { order-id = '345'
                         , product  = 'Bag of rice'
                         , quantity = '1'
                         }
                       ,
                          ...
                       ,
                         { order-id = '345'
                         , product  = 'Tofo'
                         , quantity = '1'
                         }
                       ]  
                     }

Here is the origonal production again:

  node :=  character | map( string, list(node) )      

It could then be used to return a nested
result set like:

   SELECT * 
     FROM ORDER 
    WHERE ID = '345'
     
    with this sub-query nested...

    SELECT PRODUCT, QUANTITY 
      FROM ORDER_LINE 
     WHERE ORDER_ID = ID;

  my-order  =:  { id = '345'
                , buyer = 'Clark Evans'
                , lines =
                       [
                         { order-id = '345'
                         , product = 'Bag of rice' }
                         , quantity = '1'
                         }
                       ,
                         { order-id = '345'
                         , product = 'Tofo'
                         , quantity = '3'
                         }
                       ]    
                }

Here is a mapping to a simple markup language 
(http://www.egroups.com/list/sml-dev/info.html)

<order>
  <id>345</id>
  <buyer>Clark Evans</buyer>
  <order-line-list>
     <order-line>
       <product>Bag of rice</product>
       <quantity>1</quantity>
     </order-line>
     <order-line>
       <product>Tofo</product>
       <quantity>3</quantity>
     </order-line>
  </order-line-list>
</order>

So, if you notice the even levels are maps, 
and the odd levels are lists.  Kinda cool.

Of course, you can shorten it with attribute notation...

<order id="345" buyer="Clark Evans">
  <order-line product="Bag of rice" quantity="1" />
  <order-line product="Tofo"        quantity="3" />
</order>

  (The syntax is more brief, but it only 
   allows for one list per map)

Anyway, this could allow for some nice automated
conversions in between a database and SML/XML/SGML.

Indeed, I was thinking that a binary version of
of the former syntax would make for a great 
interface into/outof the database.  Add a simple
text/binary adapter and PostgreSQL would be
XML ready... without the need for any particular
mapping tool.

Best,

Clark















Sat, 8 Jan 2000 22:57:34 -0500 (EST)
"Clark C. Evans" <clark.evans@manhattanproject.com>
A Markup Database Interface; Borrowing from Groves
At 05:27 PM 1/8/00 -0500, Tom Lane wrote:

>I also object strongly to the lack of documentation.  Patches that
>change public APIs and come without doco updates should be rejected
>out of hand, IMNSHO.  Keeping the documentation up to date should
>not be considered optional --- especially not when you're talking
>about something that makes subtle and pervasive changes to library
>behavior.

Boy, Tom's really laid it out in excellent style.  If the author of
such changes doesn't document them, chances are that the documentation
won't get done.  That's very bad.  

The automatic rejection of undocumented patches that change the API
or other user-visible behavior shouldn't be controversial.  I know
there are some folks who aren't native-english speakers, so perhaps
you don't want to require that the implementor of such patches provide
the final documentation wording.  But the information should be there
and spelled out in a form that can be very easily moved to the docs.



- Don Baccus, Portland OR <dhogaza@pacifier.com>
  Nature photos, on-line guides, Pacific Northwest
  Rare Bird Alert Service and other goodies at
  http://donb.photo.net.

Sun, 09 Jan 2000 07:01:02 -0800
Don Baccus <dhogaza@pacifier.com>
Re: [HACKERS] LIBPQ patches ... 
Don Baccus <dhogaza@pacifier.com> writes:
> At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
>> I also object strongly to the lack of documentation.

> ... I know there are some folks who aren't native-english speakers, so
> perhaps you don't want to require that the implementor of such patches
> provide the final documentation wording.  But the information should
> be there and spelled out in a form that can be very easily moved to
> the docs.

Oh, absolutely.  Thomas, our master of the docs, has always had the
policy of "give me some words, I'll take care of formatting and
editing..."

I was probably too harsh on Alfred last night, since in fact his code
was fairly well commented, and some minimal doco could have been
extracted from the routine headers.  But on a change like this, I think
some paragraphs of coherent high-level explanation are needed: what it
does, when and why you'd use it.  I didn't see that anywhere...

			regards, tom lane

Sun, 09 Jan 2000 10:50:28 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] LIBPQ patches ... 
> At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> 
> >I also object strongly to the lack of documentation.  Patches that
> >change public APIs and come without doco updates should be rejected
> >out of hand, IMNSHO.  Keeping the documentation up to date should
> >not be considered optional --- especially not when you're talking
> >about something that makes subtle and pervasive changes to library
> >behavior.
> 
> Boy, Tom's really laid it out in excellent style.  If the author of
> such changes doesn't document them, chances are that the documentation
> won't get done.  That's very bad.  
> 
> The automatic rejection of undocumented patches that change the API
> or other user-visible behavior shouldn't be controversial.  I know
> there are some folks who aren't native-english speakers, so perhaps
> you don't want to require that the implementor of such patches provide
> the final documentation wording.  But the information should be there
> and spelled out in a form that can be very easily moved to the docs.

If it is missing, we get back to them before final release and ask for
doc patches.  They get in there one way or another.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 9 Jan 2000 13:03:36 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] LIBPQ patches ...
On Sun, 9 Jan 2000, Don Baccus wrote:

> At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> 
> >I also object strongly to the lack of documentation.  Patches that
> >change public APIs and come without doco updates should be rejected
> >out of hand, IMNSHO.  Keeping the documentation up to date should
> >not be considered optional --- especially not when you're talking
> >about something that makes subtle and pervasive changes to library
> >behavior.
> 
> Boy, Tom's really laid it out in excellent style.  If the author of
> such changes doesn't document them, chances are that the documentation
> won't get done.  That's very bad.  
> 
> The automatic rejection of undocumented patches that change the API
> or other user-visible behavior shouldn't be controversial.  I know
> there are some folks who aren't native-english speakers, so perhaps
> you don't want to require that the implementor of such patches provide
> the final documentation wording.  But the information should be there
> and spelled out in a form that can be very easily moved to the docs.

These patches were originally submited before Xmas by Alfred, asking for
feedback on them and possibly pointing out errors in implementation...he
wanted to get a feel whether or not it was *worth* him putting further
work into them.  They fell on silent ears.

Personally, I wouldn't waste my time on documenting something that, in the
end, I'd be the only one using...I'd get feedback on the usefulness first,
and then deal with building up the documentation after I've found out that
its worth it...

Alfred didn't ask "do I need to add documentation?" ... he knew that ...
he asked whether or not the implementation was appropriate, and was worth
his time to continue working ...


Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 



Sun, 9 Jan 2000 15:32:21 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] LIBPQ patches ... 
The FreeBSD Documentation Project (FDP) has excellent references to get a
general idea on building sgml docs. First, you can install the
textproc/docproj port or, if you're not running freebsd, refer to the
website to see which programs you need. Second, you can read the FDP
Primer which details how everything comes together:
http://www.freebsd.org/tutorials/docproj-primer/

Furthermore, again if you happen to be running FreeBSD, you can grab the
doc src using cvsup. The proper reference is also documented somewhere in
the Primer or in the Synchronisation chapter in the Handbook.

Keep at it, sgml and the docbook stylesheets are really worthwhile when
you start getting the hang of it.
Marc

> * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote: 
> > Don Baccus <dhogaza@pacifier.com> writes:
> > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> > >> I also object strongly to the lack of documentation.
> > 
> > > ... I know there are some folks who aren't native-english speakers, so
> > > perhaps you don't want to require that the implementor of such patches
> > > provide the final documentation wording.  But the information should
> > > be there and spelled out in a form that can be very easily moved to
> > > the docs.
> > 
> > Oh, absolutely.  Thomas, our master of the docs, has always had the
> > policy of "give me some words, I'll take care of formatting and
> > editing..."
> > 
> > I was probably too harsh on Alfred last night, since in fact his code
> > was fairly well commented, and some minimal doco could have been
> > extracted from the routine headers.  But on a change like this, I think
> > some paragraphs of coherent high-level explanation are needed: what it
> > does, when and why you'd use it.  I didn't see that anywhere...
> 
> I've actually been trying to work on the sgml and failing miserably,
> I have no clue how this stuff works (sgml compilation) are you asking
> for a couple of paragraphs that describe the proposed changes?
> 
> If so I hope this suffices, if not some help on building the sgml
> would be much appreciated:
> 
> --------
> 
> Summary:
> 
> First and foremost, great pains have been taken so that there 
> are _no_ compatibility issues.
> 
> If a 6.5.3 libpq program should not behave any differently
> with this patches in place, all they do is offer a step closer
> to a truly non-blocking connection to the backend and address
> some issues with non-blocking connections.
> 
> ----
> 
> Added functions:
> 
> int PQisnonblocking(static PGconn *conn);
> 
>   returns whether or not the socket is in blocking mode, however...
>   it doesn't actually check the socket flags, it relies on the user
>   to call 'PQsetnonblocking()' to keep the internal state of libpq
>   sane.  users should no longer use 'PQsocket()' to retrieve the
>   socket and 'manually' ioctl/fcntl it to non-blocking
> 
>   returns TRUE if the socket has been set to blocking more, FALSE
>   if the socket is blocking
> 
> int PQflush(PGconn *conn);
> 
>   flush the send-queue to the backend, just make this visible to the
>   user for convience, as the internal function works, 0 for success,
>   EOF for any failure.
> 
> int PQsetnonblocking(PGconn *conn, int arg);
> 
>   actually set the connection to the backend to blocking or
>   non-blocking arg should be set to TRUE to set the connection to
>   non-blocking or FALSE to set it blocking.
> 
>   there's an implied blocking flush of the send-queue which is
>   really ok as the user is either 'going into' or 'returning from'
>   a blocking state
> 
>   returns 0 for success, -1 for failure
> 
> ---
> 
> New functionality:
> 
>   PQsetblocking() allows libpq to know what behavior the user really
>   wants, the user will not block sending data to the backend,
>   potentially if i had a constant stream of data and was doing a
>   COPYIN it'd never finish because unless the backend lost the
>   connection I would block while sending until the backend can take
>   more data.
> 
> ---
> 
> Implementation changes:
> 
>   none should be visible to programs based on 6.5.3's libpq.
> 
>   programs based on later versions of libpq will notice that
>   the non-blocking connection functions will set the state of
>   the connection to non-blocking automatically.
> 
>   when the connection is set non-blocking pqFlush() will not block
>   if the sendqueue would be filled by new data inserted into the
>   the queue.
> 
>   functions that poll for data from the backend implicitly _try_
>   flush the send queue if set to non-blocking.  This allows the
>   polling to act as a context for pushing queued data to the backend.
> 
> ---
> 
> Problems:
> 
>   We need some sort of send-queue commit reservations so that
>   there's no chance of us sending a partial queury down the pipe
>   to the backend, right now this is hacked around by potentially
>   blocking in non-blocking mode if something 'goes terribly wrong'
>   I plan to fix this.
> 
> ---
> 
> Quirks:
> 
>   PQexec() assumes the caller wants blocking behavior and will set the
>   connection to blocking for the duration of the PQexec() call, it will
>   then restore it
> 
> ---
> 
> Internal changes:
> 
>   new field in PGconn 'int nonblocking' set to 1 if the connection is 
>   nonblocking, 0 if blocking (default)
> 
>   macro pqIsnonblocking(PGconn) to avoid a function call to check blocking
>   status (only visible if libpq-int.h is included)
> 
>   the internal function connectMakeNonblocking() has been replaced with
>   PQsetblocking()
> 
>   restart a send if EINTR is reported during a flush.
> 
> ---
> 
> Lastly:
> 
>   This is work in progress, I will be working towards making libpq
>   better at not blocking.
> 
> here are the diffs:
> 
> Index: fe-connect.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
> retrieving revision 1.109
> diff -u -c -IHeader -I$Id:  -r1.109 fe-connect.c
> cvs diff: conflicting specifications of output style
> *** fe-connect.c	2000/01/14 05:33:15	1.109
> --- fe-connect.c	2000/01/14 18:36:54
> ***************
> *** 594,624 ****
>   	return 0;
>   }
>   
> - 
> - /* ----------
> -  * connectMakeNonblocking -
> -  * Make a connection non-blocking.
> -  * Returns 1 if successful, 0 if not.
> -  * ----------
> -  */
> - static int
> - connectMakeNonblocking(PGconn *conn)
> - {
> - #ifndef WIN32
> - 	if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
> - #else
> - 	if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
> - #endif
> - 	{
> - 		printfPQExpBuffer(&conn->errorMessage,
> - 						  "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
> - 						  errno, strerror(errno));
> - 		return 0;
> - 	}
> - 
> - 	return 1;
> - }
> - 
>   /* ----------
>    * connectNoDelay -
>    * Sets the TCP_NODELAY socket option.
> --- 594,599 ----
> ***************
> *** 789,795 ****
>   	 *   Ewan Mellor <eem21@cam.ac.uk>.
>   	 * ---------- */
>   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! 	if (!connectMakeNonblocking(conn))
>   		goto connect_errReturn;
>   #endif	
>   
> --- 764,770 ----
>   	 *   Ewan Mellor <eem21@cam.ac.uk>.
>   	 * ---------- */
>   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! 	if (PQsetnonblocking(conn, TRUE) != 0)
>   		goto connect_errReturn;
>   #endif	
>   
> ***************
> *** 898,904 ****
>   	/* This makes the connection non-blocking, for all those cases which forced us
>   	   not to do it above. */
>   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! 	if (!connectMakeNonblocking(conn))
>   		goto connect_errReturn;
>   #endif	
>   
> --- 873,879 ----
>   	/* This makes the connection non-blocking, for all those cases which forced us
>   	   not to do it above. */
>   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! 	if (PQsetnonblocking(conn, TRUE) != 0)
>   		goto connect_errReturn;
>   #endif	
>   
> ***************
> *** 1702,1707 ****
> --- 1677,1683 ----
>   	conn->inBuffer = (char *) malloc(conn->inBufSize);
>   	conn->outBufSize = 8 * 1024;
>   	conn->outBuffer = (char *) malloc(conn->outBufSize);
> + 	conn->nonblocking = FALSE;
>   	initPQExpBuffer(&conn->errorMessage);
>   	initPQExpBuffer(&conn->workBuffer);
>   	if (conn->inBuffer == NULL ||
> ***************
> *** 1812,1817 ****
> --- 1788,1794 ----
>   	conn->lobjfuncs = NULL;
>   	conn->inStart = conn->inCursor = conn->inEnd = 0;
>   	conn->outCount = 0;
> + 	conn->nonblocking = FALSE;
>   
>   }
>   
> Index: fe-exec.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
> retrieving revision 1.86
> diff -u -c -IHeader -I$Id:  -r1.86 fe-exec.c
> cvs diff: conflicting specifications of output style
> *** fe-exec.c	1999/11/11 00:10:14	1.86
> --- fe-exec.c	2000/01/14 22:47:07
> ***************
> *** 13,18 ****
> --- 13,19 ----
>    */
>   #include <errno.h>
>   #include <ctype.h>
> + #include <fcntl.h>
>   
>   #include "postgres.h"
>   #include "libpq-fe.h"
> ***************
> *** 24,30 ****
>   #include <unistd.h>
>   #endif
>   
> - 
>   /* keep this in same order as ExecStatusType in libpq-fe.h */
>   const char *const pgresStatus[] = {
>   	"PGRES_EMPTY_QUERY",
> --- 25,30 ----
> ***************
> *** 514,526 ****
>   	conn->curTuple = NULL;
>   
>   	/* send the query to the backend; */
> ! 	/* the frontend-backend protocol uses 'Q' to designate queries */
> ! 	if (pqPutnchar("Q", 1, conn) ||
> ! 		pqPuts(query, conn) ||
> ! 		pqFlush(conn))
>   	{
> ! 		handleSendFailure(conn);
> ! 		return 0;
>   	}
>   
>   	/* OK, it's launched! */
> --- 514,566 ----
>   	conn->curTuple = NULL;
>   
>   	/* send the query to the backend; */
> ! 
> ! 	/*
> ! 	 * in order to guarantee that we don't send a partial query 
> ! 	 * where we would become out of sync with the backend and/or
> ! 	 * block during a non-blocking connection we must first flush
> ! 	 * the send buffer before sending more data
> ! 	 *
> ! 	 * an alternative is to implement 'queue reservations' where
> ! 	 * we are able to roll up a transaction 
> ! 	 * (the 'Q' along with our query) and make sure we have
> ! 	 * enough space for it all in the send buffer.
> ! 	 */
> ! 	if (pqIsnonblocking(conn))
>   	{
> ! 		/*
> ! 		 * the buffer must have emptied completely before we allow
> ! 		 * a new query to be buffered
> ! 		 */
> ! 		if (pqFlush(conn))
> ! 			return 0;
> ! 		/* 'Q' == queries */
> ! 		/* XXX: if we fail here we really ought to not block */
> ! 		if (pqPutnchar("Q", 1, conn) ||
> ! 			pqPuts(query, conn))
> ! 		{
> ! 			handleSendFailure(conn);	
> ! 			return 0;
> ! 		}
> ! 		/*
> ! 		 * give the data a push, ignore the return value as
> ! 		 * ConsumeInput() will do any aditional flushing if needed
> ! 		 */
> ! 		(void) pqFlush(conn);	
> ! 	}
> ! 	else
> ! 	{
> ! 		/* 
> ! 		 * the frontend-backend protocol uses 'Q' to 
> ! 		 * designate queries 
> ! 		 */
> ! 		if (pqPutnchar("Q", 1, conn) ||
> ! 			pqPuts(query, conn) ||
> ! 			pqFlush(conn))
> ! 		{
> ! 			handleSendFailure(conn);
> ! 			return 0;
> ! 		}
>   	}
>   
>   	/* OK, it's launched! */
> ***************
> *** 574,580 ****
> --- 614,630 ----
>   	 * we will NOT block waiting for more input.
>   	 */
>   	if (pqReadData(conn) < 0)
> + 	{
> + 		/*
> + 		 * for non-blocking connections
> + 		 * try to flush the send-queue otherwise we may never get a 
> + 		 * responce for something that may not have already been sent
> + 		 * because it's in our write buffer!
> + 		 */
> + 		if (pqIsnonblocking(conn))
> + 			(void) pqFlush(conn);
>   		return 0;
> + 	}
>   	/* Parsing of the data waits till later. */
>   	return 1;
>   }
> ***************
> *** 1088,1093 ****
> --- 1138,1153 ----
>   {
>   	PGresult   *result;
>   	PGresult   *lastResult;
> + 	bool	savedblocking;
> + 
> + 	/*
> + 	 * we assume anyone calling PQexec wants blocking behaviour,
> + 	 * we force the blocking status of the connection to blocking
> + 	 * for the duration of this function and restore it on return
> + 	 */
> + 	savedblocking = pqIsnonblocking(conn);
> + 	if (PQsetnonblocking(conn, FALSE) == -1)
> + 		return NULL;
>   
>   	/*
>   	 * Silently discard any prior query result that application didn't
> ***************
> *** 1102,1115 ****
>   			PQclear(result);
>   			printfPQExpBuffer(&conn->errorMessage,
>   				"PQexec: you gotta get out of a COPY state yourself.\n");
> ! 			return NULL;
>   		}
>   		PQclear(result);
>   	}
>   
>   	/* OK to send the message */
>   	if (!PQsendQuery(conn, query))
> ! 		return NULL;
>   
>   	/*
>   	 * For backwards compatibility, return the last result if there are
> --- 1162,1176 ----
>   			PQclear(result);
>   			printfPQExpBuffer(&conn->errorMessage,
>   				"PQexec: you gotta get out of a COPY state yourself.\n");
> ! 			/* restore blocking status */
> ! 			goto errout;
>   		}
>   		PQclear(result);
>   	}
>   
>   	/* OK to send the message */
>   	if (!PQsendQuery(conn, query))
> ! 		goto errout;	/* restore blocking status */
>   
>   	/*
>   	 * For backwards compatibility, return the last result if there are
> ***************
> *** 1142,1148 ****
> --- 1203,1217 ----
>   			result->resultStatus == PGRES_COPY_OUT)
>   			break;
>   	}
> + 
> + 	if (PQsetnonblocking(conn, savedblocking) == -1)
> + 		return NULL;
>   	return lastResult;
> + 
> + errout:
> + 	if (PQsetnonblocking(conn, savedblocking) == -1)
> + 		return NULL;
> + 	return NULL;
>   }
>   
>   
> ***************
> *** 1431,1438 ****
>   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
>   		return 1;
>   	}
>   
> ! 	(void) pqFlush(conn);		/* make sure no data is waiting to be sent */
>   
>   	/* Return to active duty */
>   	conn->asyncStatus = PGASYNC_BUSY;
> --- 1500,1516 ----
>   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
>   		return 1;
>   	}
> + 
> + 	/*
> + 	 * make sure no data is waiting to be sent, 
> + 	 * abort if we are non-blocking and the flush fails
> + 	 */
> + 	if (pqFlush(conn) && pqIsnonblocking(conn))
> + 		return (1);
>   
> ! 	/* non blocking connections may have to abort at this point. */
> ! 	if (pqIsnonblocking(conn) && PQisBusy(conn))
> ! 		return (1);
>   
>   	/* Return to active duty */
>   	conn->asyncStatus = PGASYNC_BUSY;
> ***************
> *** 2025,2028 ****
> --- 2103,2192 ----
>   		return 1;
>   	else
>   		return 0;
> + }
> + 
> + /* PQsetnonblocking:
> + 	 sets the PGconn's database connection non-blocking if the arg is TRUE
> + 	 or makes it non-blocking if the arg is FALSE, this will not protect
> + 	 you from PQexec(), you'll only be safe when using the non-blocking
> + 	 API
> + 	 Needs to be called only on a connected database connection.
> + */
> + 
> + int
> + PQsetnonblocking(PGconn *conn, int arg)
> + {
> + 	int	fcntlarg;
> + 
> + 	arg = (arg == TRUE) ? 1 : 0;
> + 	/* early out if the socket is already in the state requested */
> + 	if (arg == conn->nonblocking)
> + 		return (0);
> + 
> + 	/*
> + 	 * to guarantee constancy for flushing/query/result-polling behavior
> + 	 * we need to flush the send queue at this point in order to guarantee
> + 	 * proper behavior.
> + 	 * this is ok because either they are making a transition
> + 	 *  _from_ or _to_ blocking mode, either way we can block them.
> + 	 */
> + 	/* if we are going from blocking to non-blocking flush here */
> + 	if (!pqIsnonblocking(conn) && pqFlush(conn))
> + 		return (-1);
> + 
> + 
> + #ifdef USE_SSL
> + 	if (conn->ssl)
> + 	{
> + 		printfPQExpBuffer(&conn->errorMessage,
> + 			"PQsetnonblocking() -- not supported when using SSL\n");
> + 		return (-1);
> + 	}
> + #endif /* USE_SSL */
> + 
> + #ifndef WIN32
> + 	fcntlarg = fcntl(conn->sock, F_GETFL, 0);
> + 	if (fcntlarg == -1)
> + 		return (-1);
> + 
> + 	if ((arg == TRUE && 
> + 		fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
> + 		(arg == FALSE &&
> + 		fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) 
> + #else
> + 	fcntlarg = arg;
> + 	if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
> + #endif
> + 	{
> + 		printfPQExpBuffer(&conn->errorMessage,
> + 			"PQsetblocking() -- unable to set nonblocking status to %s\n",
> + 			arg == TRUE ? "TRUE" : "FALSE");
> + 		return (-1);
> + 	}
> + 
> + 	conn->nonblocking = arg;
> + 
> + 	/* if we are going from non-blocking to blocking flush here */
> + 	if (pqIsnonblocking(conn) && pqFlush(conn))
> + 		return (-1);
> + 
> + 	return (0);
> + }
> + 
> + /* return the blocking status of the database connection, TRUE == nonblocking,
> + 	 FALSE == blocking
> + */
> + int
> + PQisnonblocking(const PGconn *conn)
> + {
> + 
> + 	return (pqIsnonblocking(conn));
> + }
> + 
> + /* try to force data out, really only useful for non-blocking users */
> + int
> + PQflush(PGconn *conn)
> + {
> + 
> + 	return (pqFlush(conn));
>   }
> Index: fe-misc.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
> retrieving revision 1.33
> diff -u -c -IHeader -I$Id:  -r1.33 fe-misc.c
> cvs diff: conflicting specifications of output style
> *** fe-misc.c	1999/11/30 03:08:19	1.33
> --- fe-misc.c	2000/01/12 03:12:14
> ***************
> *** 86,91 ****
> --- 86,122 ----
>   {
>   	size_t avail = Max(conn->outBufSize - conn->outCount, 0);
>   
> + 	/*
> + 	 * if we are non-blocking and the send queue is too full to buffer this
> + 	 * request then try to flush some and return an error 
> + 	 */
> + 	if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn))
> + 	{
> + 		/* 
> + 		 * even if the flush failed we may still have written some
> + 		 * data, recalculate the size of the send-queue relative
> + 		 * to the amount we have to send, we may be able to queue it
> + 		 * afterall even though it's not sent to the database it's
> + 		 * ok, any routines that check the data coming from the
> + 		 * database better call pqFlush() anyway.
> + 		 */
> + 		if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
> + 		{
> + 			printfPQExpBuffer(&conn->errorMessage,
> + 				"pqPutBytes --  pqFlush couldn't flush enough"
> + 				" data: space available: %d, space needed %d\n",
> + 				Max(conn->outBufSize - conn->outCount, 0), nbytes);
> + 			return EOF;
> + 		}
> + 	}
> + 
> + 	/* 
> + 	 * is the amount of data to be sent is larger than the size of the
> + 	 * output buffer then we must flush it to make more room.
> + 	 *
> + 	 * the code above will make sure the loop conditional is never 
> + 	 * true for non-blocking connections
> + 	 */
>   	while (nbytes > avail)
>   	{
>   		memcpy(conn->outBuffer + conn->outCount, s, avail);
> ***************
> *** 548,553 ****
> --- 579,592 ----
>   		return EOF;
>   	}
>   
> + 	/* 
> + 	 * don't try to send zero data, allows us to use this function
> + 	 * without too much worry about overhead
> + 	 */
> + 	if (len == 0)
> + 		return (0);
> + 
> + 	/* while there's still data to send */
>   	while (len > 0)
>   	{
>   		/* Prevent being SIGPIPEd if backend has closed the connection. */
> ***************
> *** 556,561 ****
> --- 595,601 ----
>   #endif
>   
>   		int sent;
> + 
>   #ifdef USE_SSL
>   		if (conn->ssl) 
>   		  sent = SSL_write(conn->ssl, ptr, len);
> ***************
> *** 585,590 ****
> --- 625,632 ----
>   				case EWOULDBLOCK:
>   					break;
>   #endif
> + 				case EINTR:
> + 					continue;
>   
>   				case EPIPE:
>   #ifdef ECONNRESET
> ***************
> *** 616,628 ****
>   			ptr += sent;
>   			len -= sent;
>   		}
>   		if (len > 0)
>   		{
>   			/* We didn't send it all, wait till we can send more */
>   
> - 			/* At first glance this looks as though it should block.  I think
> - 			 * that it will be OK though, as long as the socket is
> - 			 * non-blocking. */
>   			if (pqWait(FALSE, TRUE, conn))
>   				return EOF;
>   		}
> --- 658,688 ----
>   			ptr += sent;
>   			len -= sent;
>   		}
> + 
>   		if (len > 0)
>   		{
>   			/* We didn't send it all, wait till we can send more */
> + 
> + 			/* 
> + 			 * if the socket is in non-blocking mode we may need
> + 			 * to abort here 
> + 			 */
> + #ifdef USE_SSL
> + 			/* can't do anything for our SSL users yet */
> + 			if (conn->ssl == NULL)
> + 			{
> + #endif
> + 				if (pqIsnonblocking(conn))
> + 				{
> + 					/* shift the contents of the buffer */
> + 					memmove(conn->outBuffer, ptr, len);
> + 					conn->outCount = len;
> + 					return EOF;
> + 				}
> + #ifdef USE_SSL
> + 			}
> + #endif
>   
>   			if (pqWait(FALSE, TRUE, conn))
>   				return EOF;
>   		}
> Index: libpq-fe.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
> retrieving revision 1.54
> diff -u -c -IHeader -I$Id:  -r1.54 libpq-fe.h
> cvs diff: conflicting specifications of output style
> *** libpq-fe.h	2000/01/14 05:33:15	1.54
> --- libpq-fe.h	2000/01/14 22:45:33
> ***************
> *** 261,266 ****
> --- 261,273 ----
>   	extern int	PQgetlineAsync(PGconn *conn, char *buffer, int bufsize);
>   	extern int	PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
>   	extern int	PQendcopy(PGconn *conn);
> + 
> + 	/* Set blocking/nonblocking connection to the backend */
> + 	extern int	PQsetnonblocking(PGconn *conn, int arg);
> + 	extern int	PQisnonblocking(const PGconn *conn);
> + 
> + 	/* Force the write buffer to be written (or at least try) */
> + 	extern int	PQflush(PGconn *conn);
>   
>   	/*
>   	 * "Fast path" interface --- not really recommended for application
> Index: libpq-int.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
> retrieving revision 1.15
> diff -u -c -IHeader -I$Id:  -r1.15 libpq-int.h
> cvs diff: conflicting specifications of output style
> *** libpq-int.h	2000/01/14 05:33:15	1.15
> --- libpq-int.h	2000/01/14 18:32:51
> ***************
> *** 214,219 ****
> --- 214,222 ----
>   	int			inEnd;			/* offset to first position after avail
>   								 * data */
>   
> + 	int			nonblocking;	/* whether this connection is using a blocking
> + 								 * socket to the backend or not */
> + 
>   	/* Buffer for data not yet sent to backend */
>   	char	   *outBuffer;		/* currently allocated buffer */
>   	int			outBufSize;		/* allocated size of buffer */
> ***************
> *** 297,301 ****
> --- 300,310 ----
>   #define strerror(A) (sys_errlist[(A)])
>   #endif	 /* sunos4 */
>   #endif	 /* !strerror */
> + 
> + /* 
> +  * this is so that we can check is a connection is non-blocking internally
> +  * without the overhead of a function call
> +  */
> + #define pqIsnonblocking(conn)	(conn->nonblocking)
>   
>   #endif	 /* LIBPQ_INT_H */
> 
> 
> on a side note miscadmin.h causes problems on FreeBSD because it uses
> pid_t without having included sys/types.h
> 
> thanks!
> 
> -- 
> -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
> 
> ************
> 


Fri, 14 Jan 2000 14:11:52 +0000 (GMT)
admin <admin@wtbwts.com>
Re: [HACKERS] Revised nonblocking patches + quasi docs
* Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote:
> Don Baccus <dhogaza@pacifier.com> writes:
> > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> >> I also object strongly to the lack of documentation.
> 
> > ... I know there are some folks who aren't native-english speakers, so
> > perhaps you don't want to require that the implementor of such patches
> > provide the final documentation wording.  But the information should
> > be there and spelled out in a form that can be very easily moved to
> > the docs.
> 
> Oh, absolutely.  Thomas, our master of the docs, has always had the
> policy of "give me some words, I'll take care of formatting and
> editing..."
> 
> I was probably too harsh on Alfred last night, since in fact his code
> was fairly well commented, and some minimal doco could have been
> extracted from the routine headers.  But on a change like this, I think
> some paragraphs of coherent high-level explanation are needed: what it
> does, when and why you'd use it.  I didn't see that anywhere...

I've actually been trying to work on the sgml and failing miserably,
I have no clue how this stuff works (sgml compilation) are you asking
for a couple of paragraphs that describe the proposed changes?

If so I hope this suffices, if not some help on building the sgml
would be much appreciated:

--------

Summary:

First and foremost, great pains have been taken so that there 
are _no_ compatibility issues.

If a 6.5.3 libpq program should not behave any differently
with this patches in place, all they do is offer a step closer
to a truly non-blocking connection to the backend and address
some issues with non-blocking connections.

----

Added functions:

int PQisnonblocking(static PGconn *conn);

  returns whether or not the socket is in blocking mode, however...
  it doesn't actually check the socket flags, it relies on the user
  to call 'PQsetnonblocking()' to keep the internal state of libpq
  sane.  users should no longer use 'PQsocket()' to retrieve the
  socket and 'manually' ioctl/fcntl it to non-blocking

  returns TRUE if the socket has been set to blocking more, FALSE
  if the socket is blocking

int PQflush(PGconn *conn);

  flush the send-queue to the backend, just make this visible to the
  user for convience, as the internal function works, 0 for success,
  EOF for any failure.

int PQsetnonblocking(PGconn *conn, int arg);

  actually set the connection to the backend to blocking or
  non-blocking arg should be set to TRUE to set the connection to
  non-blocking or FALSE to set it blocking.

  there's an implied blocking flush of the send-queue which is
  really ok as the user is either 'going into' or 'returning from'
  a blocking state

  returns 0 for success, -1 for failure

---

New functionality:

  PQsetblocking() allows libpq to know what behavior the user really
  wants, the user will not block sending data to the backend,
  potentially if i had a constant stream of data and was doing a
  COPYIN it'd never finish because unless the backend lost the
  connection I would block while sending until the backend can take
  more data.

---

Implementation changes:

  none should be visible to programs based on 6.5.3's libpq.

  programs based on later versions of libpq will notice that
  the non-blocking connection functions will set the state of
  the connection to non-blocking automatically.

  when the connection is set non-blocking pqFlush() will not block
  if the sendqueue would be filled by new data inserted into the
  the queue.

  functions that poll for data from the backend implicitly _try_
  flush the send queue if set to non-blocking.  This allows the
  polling to act as a context for pushing queued data to the backend.

---

Problems:

  We need some sort of send-queue commit reservations so that
  there's no chance of us sending a partial queury down the pipe
  to the backend, right now this is hacked around by potentially
  blocking in non-blocking mode if something 'goes terribly wrong'
  I plan to fix this.

---

Quirks:

  PQexec() assumes the caller wants blocking behavior and will set the
  connection to blocking for the duration of the PQexec() call, it will
  then restore it

---

Internal changes:

  new field in PGconn 'int nonblocking' set to 1 if the connection is 
  nonblocking, 0 if blocking (default)

  macro pqIsnonblocking(PGconn) to avoid a function call to check blocking
  status (only visible if libpq-int.h is included)

  the internal function connectMakeNonblocking() has been replaced with
  PQsetblocking()

  restart a send if EINTR is reported during a flush.

---

Lastly:

  This is work in progress, I will be working towards making libpq
  better at not blocking.

here are the diffs:

Index: fe-connect.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.109
diff -u -c -IHeader -I$Id:  -r1.109 fe-connect.c
cvs diff: conflicting specifications of output style
*** fe-connect.c	2000/01/14 05:33:15	1.109
--- fe-connect.c	2000/01/14 18:36:54
***************
*** 594,624 ****
  	return 0;
  }
  
- 
- /* ----------
-  * connectMakeNonblocking -
-  * Make a connection non-blocking.
-  * Returns 1 if successful, 0 if not.
-  * ----------
-  */
- static int
- connectMakeNonblocking(PGconn *conn)
- {
- #ifndef WIN32
- 	if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
- #else
- 	if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
- #endif
- 	{
- 		printfPQExpBuffer(&conn->errorMessage,
- 						  "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
- 						  errno, strerror(errno));
- 		return 0;
- 	}
- 
- 	return 1;
- }
- 
  /* ----------
   * connectNoDelay -
   * Sets the TCP_NODELAY socket option.
--- 594,599 ----
***************
*** 789,795 ****
  	 *   Ewan Mellor <eem21@cam.ac.uk>.
  	 * ---------- */
  #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
! 	if (!connectMakeNonblocking(conn))
  		goto connect_errReturn;
  #endif	
  
--- 764,770 ----
  	 *   Ewan Mellor <eem21@cam.ac.uk>.
  	 * ---------- */
  #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
! 	if (PQsetnonblocking(conn, TRUE) != 0)
  		goto connect_errReturn;
  #endif	
  
***************
*** 898,904 ****
  	/* This makes the connection non-blocking, for all those cases which forced us
  	   not to do it above. */
  #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! 	if (!connectMakeNonblocking(conn))
  		goto connect_errReturn;
  #endif	
  
--- 873,879 ----
  	/* This makes the connection non-blocking, for all those cases which forced us
  	   not to do it above. */
  #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! 	if (PQsetnonblocking(conn, TRUE) != 0)
  		goto connect_errReturn;
  #endif	
  
***************
*** 1702,1707 ****
--- 1677,1683 ----
  	conn->inBuffer = (char *) malloc(conn->inBufSize);
  	conn->outBufSize = 8 * 1024;
  	conn->outBuffer = (char *) malloc(conn->outBufSize);
+ 	conn->nonblocking = FALSE;
  	initPQExpBuffer(&conn->errorMessage);
  	initPQExpBuffer(&conn->workBuffer);
  	if (conn->inBuffer == NULL ||
***************
*** 1812,1817 ****
--- 1788,1794 ----
  	conn->lobjfuncs = NULL;
  	conn->inStart = conn->inCursor = conn->inEnd = 0;
  	conn->outCount = 0;
+ 	conn->nonblocking = FALSE;
  
  }
  
Index: fe-exec.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.86
diff -u -c -IHeader -I$Id:  -r1.86 fe-exec.c
cvs diff: conflicting specifications of output style
*** fe-exec.c	1999/11/11 00:10:14	1.86
--- fe-exec.c	2000/01/14 22:47:07
***************
*** 13,18 ****
--- 13,19 ----
   */
  #include <errno.h>
  #include <ctype.h>
+ #include <fcntl.h>
  
  #include "postgres.h"
  #include "libpq-fe.h"
***************
*** 24,30 ****
  #include <unistd.h>
  #endif
  
- 
  /* keep this in same order as ExecStatusType in libpq-fe.h */
  const char *const pgresStatus[] = {
  	"PGRES_EMPTY_QUERY",
--- 25,30 ----
***************
*** 514,526 ****
  	conn->curTuple = NULL;
  
  	/* send the query to the backend; */
! 	/* the frontend-backend protocol uses 'Q' to designate queries */
! 	if (pqPutnchar("Q", 1, conn) ||
! 		pqPuts(query, conn) ||
! 		pqFlush(conn))
  	{
! 		handleSendFailure(conn);
! 		return 0;
  	}
  
  	/* OK, it's launched! */
--- 514,566 ----
  	conn->curTuple = NULL;
  
  	/* send the query to the backend; */
! 
! 	/*
! 	 * in order to guarantee that we don't send a partial query 
! 	 * where we would become out of sync with the backend and/or
! 	 * block during a non-blocking connection we must first flush
! 	 * the send buffer before sending more data
! 	 *
! 	 * an alternative is to implement 'queue reservations' where
! 	 * we are able to roll up a transaction 
! 	 * (the 'Q' along with our query) and make sure we have
! 	 * enough space for it all in the send buffer.
! 	 */
! 	if (pqIsnonblocking(conn))
  	{
! 		/*
! 		 * the buffer must have emptied completely before we allow
! 		 * a new query to be buffered
! 		 */
! 		if (pqFlush(conn))
! 			return 0;
! 		/* 'Q' == queries */
! 		/* XXX: if we fail here we really ought to not block */
! 		if (pqPutnchar("Q", 1, conn) ||
! 			pqPuts(query, conn))
! 		{
! 			handleSendFailure(conn);	
! 			return 0;
! 		}
! 		/*
! 		 * give the data a push, ignore the return value as
! 		 * ConsumeInput() will do any aditional flushing if needed
! 		 */
! 		(void) pqFlush(conn);	
! 	}
! 	else
! 	{
! 		/* 
! 		 * the frontend-backend protocol uses 'Q' to 
! 		 * designate queries 
! 		 */
! 		if (pqPutnchar("Q", 1, conn) ||
! 			pqPuts(query, conn) ||
! 			pqFlush(conn))
! 		{
! 			handleSendFailure(conn);
! 			return 0;
! 		}
  	}
  
  	/* OK, it's launched! */
***************
*** 574,580 ****
--- 614,630 ----
  	 * we will NOT block waiting for more input.
  	 */
  	if (pqReadData(conn) < 0)
+ 	{
+ 		/*
+ 		 * for non-blocking connections
+ 		 * try to flush the send-queue otherwise we may never get a 
+ 		 * responce for something that may not have already been sent
+ 		 * because it's in our write buffer!
+ 		 */
+ 		if (pqIsnonblocking(conn))
+ 			(void) pqFlush(conn);
  		return 0;
+ 	}
  	/* Parsing of the data waits till later. */
  	return 1;
  }
***************
*** 1088,1093 ****
--- 1138,1153 ----
  {
  	PGresult   *result;
  	PGresult   *lastResult;
+ 	bool	savedblocking;
+ 
+ 	/*
+ 	 * we assume anyone calling PQexec wants blocking behaviour,
+ 	 * we force the blocking status of the connection to blocking
+ 	 * for the duration of this function and restore it on return
+ 	 */
+ 	savedblocking = pqIsnonblocking(conn);
+ 	if (PQsetnonblocking(conn, FALSE) == -1)
+ 		return NULL;
  
  	/*
  	 * Silently discard any prior query result that application didn't
***************
*** 1102,1115 ****
  			PQclear(result);
  			printfPQExpBuffer(&conn->errorMessage,
  				"PQexec: you gotta get out of a COPY state yourself.\n");
! 			return NULL;
  		}
  		PQclear(result);
  	}
  
  	/* OK to send the message */
  	if (!PQsendQuery(conn, query))
! 		return NULL;
  
  	/*
  	 * For backwards compatibility, return the last result if there are
--- 1162,1176 ----
  			PQclear(result);
  			printfPQExpBuffer(&conn->errorMessage,
  				"PQexec: you gotta get out of a COPY state yourself.\n");
! 			/* restore blocking status */
! 			goto errout;
  		}
  		PQclear(result);
  	}
  
  	/* OK to send the message */
  	if (!PQsendQuery(conn, query))
! 		goto errout;	/* restore blocking status */
  
  	/*
  	 * For backwards compatibility, return the last result if there are
***************
*** 1142,1148 ****
--- 1203,1217 ----
  			result->resultStatus == PGRES_COPY_OUT)
  			break;
  	}
+ 
+ 	if (PQsetnonblocking(conn, savedblocking) == -1)
+ 		return NULL;
  	return lastResult;
+ 
+ errout:
+ 	if (PQsetnonblocking(conn, savedblocking) == -1)
+ 		return NULL;
+ 	return NULL;
  }
  
  
***************
*** 1431,1438 ****
  			 "PQendcopy() -- I don't think there's a copy in progress.\n");
  		return 1;
  	}
  
! 	(void) pqFlush(conn);		/* make sure no data is waiting to be sent */
  
  	/* Return to active duty */
  	conn->asyncStatus = PGASYNC_BUSY;
--- 1500,1516 ----
  			 "PQendcopy() -- I don't think there's a copy in progress.\n");
  		return 1;
  	}
+ 
+ 	/*
+ 	 * make sure no data is waiting to be sent, 
+ 	 * abort if we are non-blocking and the flush fails
+ 	 */
+ 	if (pqFlush(conn) && pqIsnonblocking(conn))
+ 		return (1);
  
! 	/* non blocking connections may have to abort at this point. */
! 	if (pqIsnonblocking(conn) && PQisBusy(conn))
! 		return (1);
  
  	/* Return to active duty */
  	conn->asyncStatus = PGASYNC_BUSY;
***************
*** 2025,2028 ****
--- 2103,2192 ----
  		return 1;
  	else
  		return 0;
+ }
+ 
+ /* PQsetnonblocking:
+ 	 sets the PGconn's database connection non-blocking if the arg is TRUE
+ 	 or makes it non-blocking if the arg is FALSE, this will not protect
+ 	 you from PQexec(), you'll only be safe when using the non-blocking
+ 	 API
+ 	 Needs to be called only on a connected database connection.
+ */
+ 
+ int
+ PQsetnonblocking(PGconn *conn, int arg)
+ {
+ 	int	fcntlarg;
+ 
+ 	arg = (arg == TRUE) ? 1 : 0;
+ 	/* early out if the socket is already in the state requested */
+ 	if (arg == conn->nonblocking)
+ 		return (0);
+ 
+ 	/*
+ 	 * to guarantee constancy for flushing/query/result-polling behavior
+ 	 * we need to flush the send queue at this point in order to guarantee
+ 	 * proper behavior.
+ 	 * this is ok because either they are making a transition
+ 	 *  _from_ or _to_ blocking mode, either way we can block them.
+ 	 */
+ 	/* if we are going from blocking to non-blocking flush here */
+ 	if (!pqIsnonblocking(conn) && pqFlush(conn))
+ 		return (-1);
+ 
+ 
+ #ifdef USE_SSL
+ 	if (conn->ssl)
+ 	{
+ 		printfPQExpBuffer(&conn->errorMessage,
+ 			"PQsetnonblocking() -- not supported when using SSL\n");
+ 		return (-1);
+ 	}
+ #endif /* USE_SSL */
+ 
+ #ifndef WIN32
+ 	fcntlarg = fcntl(conn->sock, F_GETFL, 0);
+ 	if (fcntlarg == -1)
+ 		return (-1);
+ 
+ 	if ((arg == TRUE && 
+ 		fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
+ 		(arg == FALSE &&
+ 		fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) 
+ #else
+ 	fcntlarg = arg;
+ 	if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
+ #endif
+ 	{
+ 		printfPQExpBuffer(&conn->errorMessage,
+ 			"PQsetblocking() -- unable to set nonblocking status to %s\n",
+ 			arg == TRUE ? "TRUE" : "FALSE");
+ 		return (-1);
+ 	}
+ 
+ 	conn->nonblocking = arg;
+ 
+ 	/* if we are going from non-blocking to blocking flush here */
+ 	if (pqIsnonblocking(conn) && pqFlush(conn))
+ 		return (-1);
+ 
+ 	return (0);
+ }
+ 
+ /* return the blocking status of the database connection, TRUE == nonblocking,
+ 	 FALSE == blocking
+ */
+ int
+ PQisnonblocking(const PGconn *conn)
+ {
+ 
+ 	return (pqIsnonblocking(conn));
+ }
+ 
+ /* try to force data out, really only useful for non-blocking users */
+ int
+ PQflush(PGconn *conn)
+ {
+ 
+ 	return (pqFlush(conn));
  }
Index: fe-misc.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
retrieving revision 1.33
diff -u -c -IHeader -I$Id:  -r1.33 fe-misc.c
cvs diff: conflicting specifications of output style
*** fe-misc.c	1999/11/30 03:08:19	1.33
--- fe-misc.c	2000/01/12 03:12:14
***************
*** 86,91 ****
--- 86,122 ----
  {
  	size_t avail = Max(conn->outBufSize - conn->outCount, 0);
  
+ 	/*
+ 	 * if we are non-blocking and the send queue is too full to buffer this
+ 	 * request then try to flush some and return an error 
+ 	 */
+ 	if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn))
+ 	{
+ 		/* 
+ 		 * even if the flush failed we may still have written some
+ 		 * data, recalculate the size of the send-queue relative
+ 		 * to the amount we have to send, we may be able to queue it
+ 		 * afterall even though it's not sent to the database it's
+ 		 * ok, any routines that check the data coming from the
+ 		 * database better call pqFlush() anyway.
+ 		 */
+ 		if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
+ 		{
+ 			printfPQExpBuffer(&conn->errorMessage,
+ 				"pqPutBytes --  pqFlush couldn't flush enough"
+ 				" data: space available: %d, space needed %d\n",
+ 				Max(conn->outBufSize - conn->outCount, 0), nbytes);
+ 			return EOF;
+ 		}
+ 	}
+ 
+ 	/* 
+ 	 * is the amount of data to be sent is larger than the size of the
+ 	 * output buffer then we must flush it to make more room.
+ 	 *
+ 	 * the code above will make sure the loop conditional is never 
+ 	 * true for non-blocking connections
+ 	 */
  	while (nbytes > avail)
  	{
  		memcpy(conn->outBuffer + conn->outCount, s, avail);
***************
*** 548,553 ****
--- 579,592 ----
  		return EOF;
  	}
  
+ 	/* 
+ 	 * don't try to send zero data, allows us to use this function
+ 	 * without too much worry about overhead
+ 	 */
+ 	if (len == 0)
+ 		return (0);
+ 
+ 	/* while there's still data to send */
  	while (len > 0)
  	{
  		/* Prevent being SIGPIPEd if backend has closed the connection. */
***************
*** 556,561 ****
--- 595,601 ----
  #endif
  
  		int sent;
+ 
  #ifdef USE_SSL
  		if (conn->ssl) 
  		  sent = SSL_write(conn->ssl, ptr, len);
***************
*** 585,590 ****
--- 625,632 ----
  				case EWOULDBLOCK:
  					break;
  #endif
+ 				case EINTR:
+ 					continue;
  
  				case EPIPE:
  #ifdef ECONNRESET
***************
*** 616,628 ****
  			ptr += sent;
  			len -= sent;
  		}
  		if (len > 0)
  		{
  			/* We didn't send it all, wait till we can send more */
  
- 			/* At first glance this looks as though it should block.  I think
- 			 * that it will be OK though, as long as the socket is
- 			 * non-blocking. */
  			if (pqWait(FALSE, TRUE, conn))
  				return EOF;
  		}
--- 658,688 ----
  			ptr += sent;
  			len -= sent;
  		}
+ 
  		if (len > 0)
  		{
  			/* We didn't send it all, wait till we can send more */
+ 
+ 			/* 
+ 			 * if the socket is in non-blocking mode we may need
+ 			 * to abort here 
+ 			 */
+ #ifdef USE_SSL
+ 			/* can't do anything for our SSL users yet */
+ 			if (conn->ssl == NULL)
+ 			{
+ #endif
+ 				if (pqIsnonblocking(conn))
+ 				{
+ 					/* shift the contents of the buffer */
+ 					memmove(conn->outBuffer, ptr, len);
+ 					conn->outCount = len;
+ 					return EOF;
+ 				}
+ #ifdef USE_SSL
+ 			}
+ #endif
  
  			if (pqWait(FALSE, TRUE, conn))
  				return EOF;
  		}
Index: libpq-fe.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.54
diff -u -c -IHeader -I$Id:  -r1.54 libpq-fe.h
cvs diff: conflicting specifications of output style
*** libpq-fe.h	2000/01/14 05:33:15	1.54
--- libpq-fe.h	2000/01/14 22:45:33
***************
*** 261,266 ****
--- 261,273 ----
  	extern int	PQgetlineAsync(PGconn *conn, char *buffer, int bufsize);
  	extern int	PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
  	extern int	PQendcopy(PGconn *conn);
+ 
+ 	/* Set blocking/nonblocking connection to the backend */
+ 	extern int	PQsetnonblocking(PGconn *conn, int arg);
+ 	extern int	PQisnonblocking(const PGconn *conn);
+ 
+ 	/* Force the write buffer to be written (or at least try) */
+ 	extern int	PQflush(PGconn *conn);
  
  	/*
  	 * "Fast path" interface --- not really recommended for application
Index: libpq-int.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
retrieving revision 1.15
diff -u -c -IHeader -I$Id:  -r1.15 libpq-int.h
cvs diff: conflicting specifications of output style
*** libpq-int.h	2000/01/14 05:33:15	1.15
--- libpq-int.h	2000/01/14 18:32:51
***************
*** 214,219 ****
--- 214,222 ----
  	int			inEnd;			/* offset to first position after avail
  								 * data */
  
+ 	int			nonblocking;	/* whether this connection is using a blocking
+ 								 * socket to the backend or not */
+ 
  	/* Buffer for data not yet sent to backend */
  	char	   *outBuffer;		/* currently allocated buffer */
  	int			outBufSize;		/* allocated size of buffer */
***************
*** 297,301 ****
--- 300,310 ----
  #define strerror(A) (sys_errlist[(A)])
  #endif	 /* sunos4 */
  #endif	 /* !strerror */
+ 
+ /* 
+  * this is so that we can check is a connection is non-blocking internally
+  * without the overhead of a function call
+  */
+ #define pqIsnonblocking(conn)	(conn->nonblocking)
  
  #endif	 /* LIBPQ_INT_H */


on a side note miscadmin.h causes problems on FreeBSD because it uses
pid_t without having included sys/types.h

thanks!

-- 
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]

Fri, 14 Jan 2000 11:14:30 -0800
Alfred Perlstein <bright@wintelcom.net>
Revised nonblocking patches + quasi docs
On Fri, 14 Jan 2000, Alfred Perlstein wrote:

> 
> If so then I'll be glad to update the docs myself, otherwise I'd
> also be happy to provide coupious amounts of plaintext docs and
> comments in my code like I have been so far.

That's all you need to do...as long as we have documentation that can be
included, it will be included ... if in sgml, all the better, but
plaintext works also...



Fri, 14 Jan 2000 15:58:36 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] Revised nonblocking patches + quasi docs
* admin <admin@wtbwts.com> [000114 11:35] wrote:
> Alfred wrote:
> > * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote: 
> > > Don Baccus <dhogaza@pacifier.com> writes:
> > > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> > > >> I also object strongly to the lack of documentation.
> > > 
> > > > ... I know there are some folks who aren't native-english speakers, so
> > > > perhaps you don't want to require that the implementor of such patches
> > > > provide the final documentation wording.  But the information should
> > > > be there and spelled out in a form that can be very easily moved to
> > > > the docs.
> > > 
> > > Oh, absolutely.  Thomas, our master of the docs, has always had the
> > > policy of "give me some words, I'll take care of formatting and
> > > editing..."
> > > 
> > > I was probably too harsh on Alfred last night, since in fact his code
> > > was fairly well commented, and some minimal doco could have been
> > > extracted from the routine headers.  But on a change like this, I think
> > > some paragraphs of coherent high-level explanation are needed: what it
> > > does, when and why you'd use it.  I didn't see that anywhere...
> > 
> > I've actually been trying to work on the sgml and failing miserably,
> > I have no clue how this stuff works (sgml compilation) are you asking
> > for a couple of paragraphs that describe the proposed changes?
> > 
> > If so I hope this suffices, if not some help on building the sgml
> > would be much appreciated:
> > 
> > --------
> > 
> The FreeBSD Documentation Project (FDP) has excellent references to get a
> general idea on building sgml docs. First, you can install the
> textproc/docproj port or, if you're not running freebsd, refer to the
> website to see which programs you need. Second, you can read the FDP
> Primer which details how everything comes together:
> http://www.freebsd.org/tutorials/docproj-primer/
> 
> Furthermore, again if you happen to be running FreeBSD, you can grab the
> doc src using cvsup. The proper reference is also documented somewhere in
> the Primer or in the Synchronisation chapter in the Handbook.
> 
> Keep at it, sgml and the docbook stylesheets are really worthwhile when
> you start getting the hang of it.
> Marc

'course I run freebsd. :)  I even have the docproj port installed,
however it seems that there's some things missing here, (see the
end of this message).

I really have no problem with commenting my code nor do I have a
problem with producing documentation for these changes, however
I'm _extremely_ pressed for time with this project, haven't slept
in 2 days and I and don't have time to fight with building the sgml
files to check that my changes/additions are valid, I'd much rather
focus on working on the rest of libpq for blocking issues and getting
my app into test mode.

Perhaps someone can offer a step-by-step to building _postgresql's_
doc files, or maybe there's a machine out there where this will
build properly and someone can give me an account on it?

If so then I'll be glad to update the docs myself, otherwise I'd
also be happy to provide coupious amounts of plaintext docs and
comments in my code like I have been so far.

thanks,
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]

~/pgcvs/pgsql/doc/src % gmake       
gmake all
gmake[1]: Entering directory `/home/bright/pgcvs/pgsql/doc/src'
gmake -C sgml clean
gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
(rm -rf HTML.manifest *.html *.htm *.1 *.l man1 manl manpage*)
gmake[2]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
gmake -C sgml admin.html
gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
(rm -rf *.htm)
jade -D ref -D ../graphics -V %use-id-as-filename% -d /home/users/t/thomas/db118.d/docbook/html/docbook.dsl -t sgml admin.sgml

   ^^^^^^^^^^^^---- huh?
   ~/pgcvs/pgsql/doc % find . -name "*.dsl"
   ~/pgcvs/pgsql/doc % 

   continues...

jade:admin.sgml:26:59:W: cannot generate system identifier for public text "-//Davenport//DTD DocBook V3.0//EN"
jade:admin.sgml:51:0:E: reference to entity "BOOK" for which no system identifier could be generated
jade:admin.sgml:26:0: entity was defined here
jade:admin.sgml:51:0:E: DTD did not contain element declaration for document type name
jade:admin.sgml:53:9:E: there is no attribute "ID"
jade:admin.sgml:53:16:E: element "BOOK" undefined
jade:admin.sgml:57:7:E: element "TITLE" undefined
jade:admin.sgml:58:10:E: element "BOOKINFO" undefined
jade:admin.sgml:59:14:E: element "RELEASEINFO" undefined
jade:admin.sgml:60:13:E: element "BOOKBIBLIO" undefined
jade:admin.sgml:61:15:E: element "AUTHORGROUP" undefined
jade:admin.sgml:62:15:E: element "CORPAUTHOR" undefined
jade:admin.sgml:67:10:E: element "EDITOR" undefined
jade:admin.sgml:68:14:E: element "FIRSTNAME" undefined
jade:admin.sgml:69:12:E: element "SURNAME" undefined
jade:admin.sgml:70:16:E: element "AFFILIATION" undefined
jade:admin.sgml:71:13:E: element "ORGNAME" undefined
jade:admin.sgml:82:8:E: element "DATE" undefined
jade:admin.sgml:85:14:E: element "LEGALNOTICE" undefined
jade:admin.sgml:86:8:E: element "PARA" undefined
jade:admin.sgml:87:16:E: element "PRODUCTNAME" undefined
jade:admin.sgml:87:56:E: general entity "copy" not defined and no default entity
jade:admin.sgml:107:13:E: there is no attribute "ID"
jade:admin.sgml:107:22:E: element "PREFACE" undefined
jade:admin.sgml:108:8:E: element "TITLE" undefined
jade:admin.sgml:110:7:E: element "PARA" undefined
jade:admin.sgml:111:15:E: element "PRODUCTNAME" undefined
jade:admin.sgml:117:15:E: element "PRODUCTNAME" undefined
jade:intro-ag.sgml:1:13:E: there is no attribute "ID"
jade:intro-ag.sgml:1:23:E: element "CHAPTER" undefined
jade:intro-ag.sgml:2:8:E: element "TITLE" undefined
jade:intro-ag.sgml:4:7:E: element "PARA" undefined
jade:intro-ag.sgml:6:14:E: there is no attribute "URL"
jade:intro-ag.sgml:6:38:E: element "ULINK" undefined
jade:intro-ag.sgml:6:51:E: element "PRODUCTNAME" undefined
jade:intro-ag.sgml:10:15:E: element "PRODUCTNAME" undefined
jade:intro-ag.sgml:11:74:E: element "ULINK" undefined
jade:intro-ag.sgml:12:16:E: element "PRODUCTNAME" undefined
jade:intro-ag.sgml:13:19:E: element "PRODUCTNAME" undefined
jade:intro-ag.sgml:15:55:E: element "ACRONYM" undefined
jade:intro-ag.sgml:16:33:E: element "ACRONYM" undefined
jade:intro-ag.sgml:17:23:E: element "ACRONYM" undefined
jade:info.sgml:1:6:E: element "SECT1" undefined
jade:info.sgml:2:7:E: element "TITLE" undefined
jade:info.sgml:4:6:E: element "PARA" undefined
jade:info.sgml:8:14:E: element "VARIABLELIST" undefined
jade:info.sgml:9:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:10:8:E: element "TERM" undefined
jade:info.sgml:11:12:E: element "LISTITEM" undefined
jade:info.sgml:12:9:E: element "PARA" undefined
jade:info.sgml:18:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:19:8:E: element "TERM" undefined
jade:info.sgml:20:12:E: element "LISTITEM" undefined
jade:info.sgml:21:9:E: element "PARA" undefined
jade:info.sgml:27:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:28:8:E: element "TERM" undefined
jade:info.sgml:29:12:E: element "LISTITEM" undefined
jade:info.sgml:30:9:E: element "PARA" undefined
jade:info.sgml:38:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:39:8:E: element "TERM" undefined
jade:info.sgml:40:12:E: element "LISTITEM" undefined
jade:info.sgml:41:9:E: element "PARA" undefined
jade:info.sgml:47:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:48:8:E: element "TERM" undefined
jade:info.sgml:49:12:E: element "LISTITEM" undefined
jade:info.sgml:50:9:E: element "PARA" undefined
jade:info.sgml:51:33:E: element "PRODUCTNAME" undefined
jade:info.sgml:53:17:E: element "PRODUCTNAME" undefined
jade:info.sgml:55:15:E: element "CITETITLE" undefined
jade:info.sgml:56:41:E: element "CITETITLE" undefined
jade:info.sgml:61:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:62:8:E: element "TERM" undefined
jade:info.sgml:63:12:E: element "LISTITEM" undefined
jade:info.sgml:64:9:E: element "PARA" undefined
jade:info.sgml:66:41:E: element "CITETITLE" undefined
jade:info.sgml:72:6:E: element "PARA" undefined
jade:info.sgml:74:14:E: element "PRODUCTNAME" undefined
jade:info.sgml:77:14:E: element "VARIABLELIST" undefined
jade:info.sgml:78:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:79:8:E: element "TERM" undefined
jade:info.sgml:80:12:E: element "LISTITEM" undefined
jade:info.sgml:81:9:E: element "PARA" undefined
jade:info.sgml:87:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:88:8:E: element "TERM" undefined
jade:info.sgml:89:12:E: element "LISTITEM" undefined
jade:info.sgml:90:9:E: element "PARA" undefined
jade:info.sgml:97:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:98:8:E: element "TERM" undefined
jade:info.sgml:99:12:E: element "LISTITEM" undefined
jade:info.sgml:100:9:E: element "PARA" undefined
jade:info.sgml:106:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:107:8:E: element "TERM" undefined
jade:info.sgml:108:12:E: element "LISTITEM" undefined
jade:info.sgml:109:9:E: element "PARA" undefined
jade:info.sgml:111:32:E: element "ULINK" undefined
jade:info.sgml:111:45:E: element "PRODUCTNAME" undefined
jade:info.sgml:113:28:E: element "PRODUCTNAME" undefined
jade:info.sgml:119:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:120:8:E: element "TERM" undefined
jade:info.sgml:121:12:E: element "LISTITEM" undefined
jade:info.sgml:122:9:E: element "PARA" undefined
jade:info.sgml:124:53:E: element "ULINK" undefined
jade:info.sgml:125:67:E: element "ULINK" undefined
jade:info.sgml:133:15:E: element "VARLISTENTRY" undefined
jade:info.sgml:134:8:E: element "TERM" undefined
jade:info.sgml:135:12:E: element "LISTITEM" undefined
jade:info.sgml:136:9:E: element "PARA" undefined
jade:info.sgml:137:17:E: element "PRODUCTNAME" undefined
jade:info.sgml:139:37:E: element "PRODUCTNAME" undefined
jade:info.sgml:147:9:E: element "PARA" undefined
jade:info.sgml:151:50:E: element "ULINK" undefined
jade:info.sgml:152:64:E: element "ULINK" undefined
jade:notation.sgml:1:10:E: there is no attribute "ID"
jade:notation.sgml:1:23:E: element "SECT1" undefined
jade:notation.sgml:2:7:E: element "TITLE" undefined
jade:notation.sgml:4:6:E: element "PARA" undefined
jade:notation.sgml:6:12:E: element "FIRSTTERM" undefined
jade:notation.sgml:8:14:E: element "PRODUCTNAME" undefined
jade:notation.sgml:10:14:E: element "PRODUCTNAME" undefined
jade:notation.sgml:13:14:E: element "PRODUCTNAME" undefined
jade:notation.sgml:16:6:E: element "PARA" undefined
jade:notation.sgml:18:14:E: element "PRODUCTNAME" undefined
jade:notation.sgml:18:48:E: element "FIRSTTERM" undefined
jade:notation.sgml:19:32:E: element "REPLACEABLE" undefined
jade:notation.sgml:20:27:E: element "PRODUCTNAME" undefined
jade:notation.sgml:24:31:E: element "PRODUCTNAME" undefined
jade:notation.sgml:26:28:E: element "PRODUCTNAME" undefined
jade:notation.sgml:27:11:E: element "EMPHASIS" undefined
jade:notation.sgml:28:73:E: element "FIRSTTERM" undefined
jade:notation.sgml:29:66:E: element "FIRSTTERM" undefined
jade:notation.sgml:33:6:E: element "PARA" undefined
jade:notation.sgml:35:12:E: element "FIRSTTERM" undefined
jade:notation.sgml:36:13:E: element "ACRONYM" undefined
jade:notation.sgml:37:14:E: element "PRODUCTNAME" undefined
jade:notation.sgml:41:14:E: element "APPLICATION" undefined
jade:notation.sgml:44:6:E: element "PARA" undefined
jade:notation.sgml:45:18:E: element "APPLICATION" undefined
jade:notation.sgml:47:21:E: element "PRODUCTNAME" undefined
jade:notation.sgml:48:51:E: element "APPLICATION" undefined
jade:notation.sgml:50:38:E: element "APPLICATION" undefined
jade:notation.sgml:56:6:E: element "PARA" undefined
jade:notation.sgml:57:18:E: element "PRODUCTNAME" undefined
jade:notation.sgml:58:45:E: element "APPLICATION" undefined
jade:notation.sgml:60:14:E: element "PRODUCTNAME" undefined
jade:notation.sgml:68:20:E: element "SECT1" undefined
jade:notation.sgml:69:7:E: element "TITLE" undefined
jade:notation.sgml:71:6:E: element "PARA" undefined
jade:notation.sgml:72:8:E: element "QUOTE" undefined
jade:notation.sgml:72:33:E: element "FILENAME" undefined
jade:notation.sgml:74:26:E: element "PRODUCTNAME" undefined
jade:notation.sgml:77:6:E: element "PARA" undefined
jade:notation.sgml:79:9:E: element "QUOTE" undefined
jade:notation.sgml:79:30:E: element "QUOTE" undefined
jade:notation.sgml:81:9:E: element "QUOTE" undefined
jade:notation.sgml:81:30:E: element "QUOTE" undefined
jade:notation.sgml:81:78:E: element "QUOTE" undefined
jade:notation.sgml:85:6:E: element "PARA" undefined
jade:notation.sgml:86:34:E: element "QUOTE" undefined
jade:notation.sgml:86:55:E: element "QUOTE" undefined
jade:notation.sgml:87:22:E: element "QUOTE" undefined
jade:notation.sgml:90:6:E: element "PARA" undefined
jade:notation.sgml:92:71:E: element "QUOTE" undefined
jade:notation.sgml:92:73:E: general entity "gt" not defined and no default entity
jade:notation.sgml:93:41:E: element "PRODUCTNAME" undefined
jade:notation.sgml:94:49:E: element "QUOTE" undefined
jade:notation.sgml:96:8:E: element "QUOTE" undefined
jade:notation.sgml:97:10:E: element "ACRONYM" undefined
jade:notation.sgml:97:63:E: element "QUOTE" undefined
jade:notation.sgml:101:6:E: element "NOTE" undefined
jade:notation.sgml:102:7:E: element "PARA" undefined
jade:notation.sgml:103:39:E: element "PRODUCTNAME" undefined
jade:notation.sgml:106:42:E: element "ULINK" undefined
jade:y2k.sgml:1:15:E: element "SECT1" undefined
jade:y2k.sgml:2:7:E: element "TITLE" undefined
jade:y2k.sgml:4:6:E: element "NOTE" undefined
jade:y2k.sgml:5:8:E: element "TITLE" undefined
jade:y2k.sgml:7:7:E: element "PARA" undefined
jade:y2k.sgml:9:50:E: element "ULINK" undefined
jade:y2k.sgml:14:6:E: element "PARA" undefined
jade:y2k.sgml:15:18:E: element "PRODUCTNAME" undefined
jade:y2k.sgml:16:18:E: element "PRODUCTNAME" undefined
jade:y2k.sgml:21:14:E: element "ITEMIZEDLIST" undefined
jade:y2k.sgml:22:11:E: element "LISTITEM" undefined
jade:y2k.sgml:23:8:E: element "PARA" undefined
jade:y2k.sgml:24:65:E: element "PRODUCTNAME" undefined
jade:y2k.sgml:26:36:E: element "PRODUCTNAME" undefined
jade:y2k.sgml:31:11:E: element "LISTITEM" undefined
jade:y2k.sgml:32:8:E: element "PARA" undefined
jade:y2k.sgml:36:19:E: element "PRODUCTNAME" undefined
jade:y2k.sgml:42:11:E: element "LISTITEM" undefined
jade:y2k.sgml:43:8:E: element "PARA" undefined
jade:y2k.sgml:47:65:E: element "ULINK" undefined
jade:y2k.sgml:50:15:E: element "QUOTE" undefined
jade:y2k.sgml:50:57:E: element "QUOTE" undefined
jade:y2k.sgml:51:18:E: element "QUOTE" undefined
jade:y2k.sgml:51:60:E: element "QUOTE" undefined
jade:y2k.sgml:55:11:E: element "LISTITEM" undefined
jade:y2k.sgml:56:8:E: element "PARA" undefined
jade:y2k.sgml:59:16:E: element "PRODUCTNAME" undefined
jade:y2k.sgml:64:6:E: element "PARA" undefined
jade:y2k.sgml:66:56:E: element "ULINK" undefined
jade:y2k.sgml:68:53:E: element "ULINK" undefined
jade:I: maximum number of errors (200) reached; change with -E option
jade:E: cannot open "/home/users/t/thomas/db118.d/docbook/html/docbook.dsl" (No such file or directory)
jade:E: specification document does not have the DSSSL architecture as a base architecture


 PostgreSQL Administrator's Guide
   Covering v6.5 for general release
         The PostgreSQL Development Team
   
       Thomas
    Lockhart
         Caltech/JPL
    
   
 

   (last updated 1999-06-01)
  

         PostgreSQL is Copyright  1996-9
    by the Postgres Global Development Group.
   
  

 



   Summary

     Postgres, 
   developed originally in the UC Berkeley Computer Science Department,
   pioneered many of the object-relational concepts
   now becoming available in some commercial databases.
   It provides SQL92/SQL3 language support,
   transaction integrity, and type extensibility.
   PostgreSQL is an open-source descendant
   of this original Berkeley code.
  
 

     Introduction

     This document is the Administrator's  Manual  for  the  
   PostgreSQL
   database  management system, originally developed at the University
   of California at  Berkeley.   

   PostgreSQL  is  based  on
       Postgres release 4.2. 
   The Postgres project, 
   led by Professor Michael Stonebraker, was sponsored  by  the
   Defense  Advanced Research Projects Agency (DARPA), the
   Army Research Office (ARO), the National Science  
   Foundation (NSF), and ESL, Inc.
  

    Resources

   This manual set is organized into several parts:
 

      Tutorial
            An introduction for new users. Does not cover advanced features.
    
   
  

     User's Guide
            General information for users, including available commands and data types.
    
   
  

     Programmer's Guide
            Advanced information for application programmers. Topics include
     type and function extensibility, library interfaces,
     and application design issues.
    
   
  

     Administrator's Guide
            Installation and management information. List of supported machines.
    
   
  

     Developer's Guide
            Information for Postgres developers.
     This is intended for those who are contributing to the
     Postgres project;
     application development information should appear in the 
     Programmer's Guide.
     Currently included in the Programmer's Guide.
    
   
  

     Reference Manual
            Detailed reference information on command syntax.
     Currently included in the User's Guide.
    
   
  
 

   In addition to this manual set, there are other resources to help you with
  Postgres installation and use:
 

      man pages
            The man pages have general information on command syntax.
    
   
  

     FAQs
            The Frequently Asked Questions (FAQ) documents address both general issues
     and some platform-specific issues.
    
   
  

     READMEs
            README files are available for some contributed packages.
    
   
  

     Web Site
            The
     Postgres
     web site might have some information not appearing in the distribution.
     There is a mhonarc catalog of mailing list traffic
     which is a rich resource for many topics.
    
   
  

     Mailing Lists
            The
     pgsql-general
     (archive)
     mailing list is a good place to have user questions answered.
     Other mailing lists are available; consult the Info Central section of the
     PostgreSQL web site for details.
    
   
  

     Yourself!
            Postgres is an open source product. 
     As such, it depends on the user community for ongoing support.
     As you begin to use Postgres, 
     you will rely on others for help, either through the
     documentation or through the mailing lists. 
     Consider contributing your knowledge back. If you learn something
     which is not in the documentation, write it up and contribute it.
     If you add features to the code, contribute it.
    

         Even those without a lot of experience can provide corrections and
     minor changes in the documentation, and that is a good way to start.
     The 
     pgsql-docs
     (archive)
     mailing list is the place to get going.
    
   
  
 



    Terminology

   In the following documentation,
  site
  may be interpreted as the host machine on which 
  Postgres is installed.
  Since it is possible to install more than one set of 
  Postgres
  databases on a single host, this term more precisely denotes any
  particular set of installed 
  Postgres binaries and databases.
 

   The 
  Postgres superuser
  is the user named postgres
  who owns the Postgres
  binaries and database files.  As the database superuser, all
  protection mechanisms may be bypassed and any data accessed
  arbitrarily.  
  In addition, the Postgres superuser is allowed to execute
  some support programs which are generally not available to all users.
  Note that the Postgres superuser is
  not
  the same as the Unix superuser (which will be referred to as root).
  The superuser should have a non-zero user identifier (UID)
  for security reasons.
 

   The
  database administrator
  or DBA, is the person who is responsible for installing 
  Postgres with mechanisms to
  enforce a security policy for a site.  The DBA can add new users by
  the method described below 
  and maintain a set of template databases for use by
  createdb.
 

   The postmaster
  is the process that acts as a clearing-house for requests 
  to the Postgres system.
  Frontend applications connect to the postmaster,
  which keeps tracks of any system errors and communication between the
  backend processes.  The postmaster
  can take several command-line arguments to tune its behavior.
  However, supplying arguments is necessary only if you intend to run multiple
  sites or a non-default site.
 

   The Postgres backend
  (the actual executable program postgres) may be executed
  directly from the user shell by the 
  Postgres super-user 
  (with the database name as an argument).  However,
  doing this bypasses the shared buffer pool and lock table associated
  with a postmaster/site, therefore this is not recommended in a multiuser
  site.
 

 Notation

   ... or /usr/local/pgsql/ 
  at the front of a file name is used to represent the
  path to the Postgres superuser's home directory.
 

   In a command synopsis, brackets
  ([ and ]) indicate an optional phrase or keyword.
  Anything in braces
  ({ and }) and containing vertical bars (|)
  indicates that you must choose one.
 

   In examples, parentheses (( and )) are used to group boolean
  expressions.  | is the boolean operator OR.
 

   Examples will show commands executed from various accounts and programs.
  Commands executed from the root account will be preceeded with .
  Commands executed from the Postgres
  superuser account will be preceeded with %, while commands
  executed from an unprivileged user's account will be preceeded with
  $.
  SQL commands will be preceeded with =
  or will have no leading prompt, depending on the context.
 

      At the time of writing (Postgres v6.5) the notation for
   flagging commands is not universally consistant throughout the documentation set.
   Please report problems to
   the Documentation Mailing List.
  
 



    Y2K Statement

   Author

     Written by 
   Thomas Lockhart
   on 1998-10-22.
  
 

   The PostgreSQL Global Development Team provides
  the Postgres software code tree as a public service,
  without warranty and without liability for it's behavior or performance.
  However, at the time of writing:
 

          The author of this statement, a volunteer on the Postgres
    support team since November, 1996, is not aware of 
    any problems in the Postgres code base related
    to time transitions around Jan 1, 2000 (Y2K).
   
  

         The author of this statement is not aware of any reports of Y2K problems 
    uncovered in regression testing
    or in other field use of recent or current versions
    of Postgres. We might have expected
    to hear about problems if they existed, given the installed base and
    the active participation of users on the support mailing lists.
   
  

         To the best of the author's knowledge, the
    assumptions Postgres makes about dates specified with a two-digit year
    are documented in the current 
    User's Guide
    in the chapter on data types.
    For two-digit years, the significant transition year is 1970, not 2000;
    e.g. 70-01-01 is interpreted as 1970-01-01,
    whereas 69-01-01 is interpreted as 2069-01-01.
   
  

         Any Y2K problems in the underlying OS related to obtaining "the
    current time" may propagate into apparent Y2K problems in
    Postgres.
   
  
 

   Refer to 
  The Gnu Project
  and
  gmake[2]: *** [admin.html] Error 1
gmake[2]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
gmake[1]: *** [admin.tar] Error 2
gmake[1]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src'
gmake: *** [install] Error 2


oy!

Fri, 14 Jan 2000 12:05:17 -0800
Alfred Perlstein <bright@wintelcom.net>
Re: [HACKERS] Revised nonblocking patches + quasi docs
> > > I've actually been trying to work on the sgml and failing miserably,
> > > I have no clue how this stuff works (sgml compilation) are you asking
> > > for a couple of paragraphs that describe the proposed changes?
> > > If so I hope this suffices, if not some help on building the sgml
> > > would be much appreciated:
> 'course I run freebsd. :)  I even have the docproj port installed,
> however it seems that there's some things missing here, (see the
> end of this message).
> Perhaps someone can offer a step-by-step to building _postgresql's_
> doc files, or maybe there's a machine out there where this will
> build properly and someone can give me an account on it?

You are probably very near having something working. The Postgres docs
have an appendix on "Documentation", which contains some information
on getting jade built and running. The makefile in doc/src/sgml has
comments which indicate what probably needs to be changed, and how to
do it (the parameters are set so the stuff builds on postgresql.org,
but I have a couple of lines in my Makefile.custom to get things to
work at home).

In particular, the .dsl files are somewhere in your jade installation
tree (it is in /usr/lib/sgml/stylesheets/nwalsh-modular/{print,html}
on my Linux box).

Ask more specific questions and we'll help you through it, but only
after you get some sleep :)

I'm out of town through the weekend, but will be on-list Monday night
afaik.

                    - Thomas

> ~/pgcvs/pgsql/doc/src % gmake
> gmake all
> gmake[1]: Entering directory `/home/bright/pgcvs/pgsql/doc/src'
> gmake -C sgml clean
> gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
> (rm -rf HTML.manifest *.html *.htm *.1 *.l man1 manl manpage*)
> gmake[2]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
> gmake -C sgml admin.html
> gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
> (rm -rf *.htm)
> jade -D ref -D ../graphics -V %use-id-as-filename% -d /home/users/t/thomas/db118.d/docbook/html/docbook.dsl -t sgml admin.sgml
> 
>    ^^^^^^^^^^^^---- huh?
>    ~/pgcvs/pgsql/doc % find . -name "*.dsl"
>    ~/pgcvs/pgsql/doc %
> 
>    continues...


-- 
Thomas Lockhart				lockhart@alumni.caltech.edu
South Pasadena, California

Sat, 15 Jan 2000 03:19:15 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Re: [HACKERS] Revised nonblocking patches + quasi docs
* Thomas Lockhart <lockhart@alumni.caltech.edu> [000114 19:42] wrote:
> > > > I've actually been trying to work on the sgml and failing miserably,
> > > > I have no clue how this stuff works (sgml compilation) are you asking
> > > > for a couple of paragraphs that describe the proposed changes?
> > > > If so I hope this suffices, if not some help on building the sgml
> > > > would be much appreciated:
> > 'course I run freebsd. :)  I even have the docproj port installed,
> > however it seems that there's some things missing here, (see the
> > end of this message).
> > Perhaps someone can offer a step-by-step to building _postgresql's_
> > doc files, or maybe there's a machine out there where this will
> > build properly and someone can give me an account on it?
> 
> You are probably very near having something working. 

Yes, I can feel it, I just ran to the store to pick up the goat blood
and candles. :)

With the help of a friend Jeroen Ruigrok van der Werven, one of
the FreeBSD'doc folks I got it working here's what I needed to do:

install the textproc/docproj and textproc/docbook from /usr/ports,
maybe more packages are needed (dsssl-docbook-modular, dtd-catalogs),
i was installing everything hoping to get it to work...

setup my enviornment... (this ought to be mentioned in the docs)

export SMGL_ROOT=/usr/local/share/sgml
SGML_CATALOG_FILES=/usr/local/share/sgml/jade/catalog
SGML_CATALOG_FILES=/usr/local/share/sgml/html/catalog:$SGML_CATALOG_FILES
SGML_CATALOG_FILES=/usr/local/share/sgml/iso8879/catalog:$SGML_CATALOG_FILES
SGML_CATALOG_FILES=/usr/local/share/sgml/transpec/catalog:$SGML_CATALOG_FILES
SGML_CATALOG_FILES=/usr/local/share/sgml/docbook/catalog:$SGML_CATALOG_FILES
export SGML_CATALOG_FILES

then in the pgsql/doc/src dir:

gmake all \
	HSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/html/ \
	PSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/print/ \

wait a good long time...

viola.

Ok, I should have the docs for my code along with some help for 
hapless FreeBSD users trying to work on this stuff in a bit.

I guess this means that Marc can't weasel his way out of doing 
documentation anymore, or was that the point all along? :)

Btw, does anyone have some fixes so gvim doesn't barf doing syntax
highlighting on these sgml files?

-Alfred

Sat, 15 Jan 2000 06:06:08 -0800
Alfred Perlstein <bright@wintelcom.net>
FreeBSD postgresql doc-HOWTO was: Re: [HACKERS] Revised nonblocking
	patches + quasi docs
* Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote:
> Don Baccus <dhogaza@pacifier.com> writes:
> > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> >> I also object strongly to the lack of documentation.
> 
> > ... I know there are some folks who aren't native-english speakers, so
> > perhaps you don't want to require that the implementor of such patches
> > provide the final documentation wording.  But the information should
> > be there and spelled out in a form that can be very easily moved to
> > the docs.
> 
> Oh, absolutely.  Thomas, our master of the docs, has always had the
> policy of "give me some words, I'll take care of formatting and
> editing..."
> 
> I was probably too harsh on Alfred last night, since in fact his code
> was fairly well commented, and some minimal doco could have been
> extracted from the routine headers.  But on a change like this, I think
> some paragraphs of coherent high-level explanation are needed: what it
> does, when and why you'd use it.  I didn't see that anywhere...

Here's the revised patch, it includes sgml docs and changes to
ensure that old style connections behave the way they are expected
to:

Index: doc/src/sgml/libpq.sgml
===================================================================
RCS file: /home/pgcvs/pgsql/doc/src/sgml/libpq.sgml,v
retrieving revision 1.25
diff -u -c -r1.25 libpq.sgml
*** doc/src/sgml/libpq.sgml	2000/01/14 05:33:13	1.25
--- doc/src/sgml/libpq.sgml	2000/01/17 03:40:30
***************
*** 377,382 ****
--- 377,386 ----
     changed in the future.
    </para>
    <para>
+    These functions leave the socket in a non-blocking state as if 
+    <function>PQsetnonblocking</function> had been called.
+   </para>
+   <para>
     These functions are not thread-safe.
    </para>
   </listitem>
***************
*** 1168,1175 ****
--- 1172,1229 ----
  Applications that do not like these limitations can instead use the
  underlying functions that <function>PQexec</function> is built from:
  <function>PQsendQuery</function> and <function>PQgetResult</function>.
+ </para>
+ <para>
+ Older programs that used this functionality as well as 
+ <function>PQputline</function> and <function>PQputnbytes</function>
+ could block waiting to send data to the backend, to
+ address that issue, the function <function>PQsetnonblocking</function>
+ was added.
+ </para>
+ <para>
+ Old applications can neglect to use <function>PQsetnonblocking</function>
+ and get the older potentially blocking behavior.  Newer programs can use 
+ <function>PQsetnonblocking</function> to achieve a completely non-blocking
+ connection to the backend.
  
  <itemizedlist>
+  <listitem>
+    <para>
+     <function>PQsetnonblocking</function> Sets the state of the connection
+     to non-blocking.
+ <synopsis>
+ int PQsetnonblocking(PGconn *conn)
+ </synopsis>
+     this function will ensure that calls to 
+     <function>PQputline</function>, <function>PQputnbytes</function>,
+     <function>PQsendQuery</function> and <function>PQendcopy</function>
+     will not block but instead return an error if they need to be called
+     again.
+    </para>
+    <para>
+     When a database connection has been set to non-blocking mode and
+     <function>PQexec</function> is called, it will temporarily set the state
+     of the connection to blocking until the <function>PQexec</function> 
+     completes. 
+    </para>
+    <para>
+     More of libpq is expected to be made safe for 
+     <function>PQsetnonblocking</function> functionality in the near future.
+   </para>
+  </listitem>
+ 
+ <listitem>
+ <para>
+ <function>PQisnonblocking</function>
+        Returns the blocking status of the database connection.
+ <synopsis>
+ int PQisnonblocking(const PGconn *conn)
+ </synopsis>
+        Returns TRUE if the connection is set to non-blocking mode,
+        FALSE if blocking.
+ </para>
+ </listitem>
+ 
  <listitem>
  <para>
  <function>PQsendQuery</function>
***************
*** 1267,1286 ****
  
  <listitem>
  <para>
  <function>PQsocket</function>
  	  Obtain the file descriptor number for the backend connection socket.
! 	  A valid descriptor will be >= 0; a result of -1 indicates that
  	  no backend connection is currently open.
  <synopsis>
  int PQsocket(const PGconn *conn);
  </synopsis>
  <function>PQsocket</function> should be used to obtain the backend socket descriptor
  in preparation for executing <function>select</function>(2).  This allows an
! application to wait for either backend responses or other conditions.
  If the result of <function>select</function>(2) indicates that data can be read from
  the backend socket, then <function>PQconsumeInput</function> should be called to read the
  data; after which, <function>PQisBusy</function>, <function>PQgetResult</function>,
  and/or <function>PQnotifies</function> can be used to process the response.
  </para>
  </listitem>
  
--- 1321,1363 ----
  
  <listitem>
  <para>
+ <function>PQflush</function> Attempt to flush any data queued to the backend,
+ returns 0 if successful (or if the send queue is empty) or EOF if it failed for
+ some reason.
+ <synopsis>
+ int PQflush(PGconn *conn);
+ </synopsis>
+ <function>PQflush</function> needs to be called on a non-blocking connection 
+ before calling <function>select</function> to determine if a responce has
+ arrived.  If 0 is returned it ensures that there is no data queued to the 
+ backend that has not actually been sent.  Only applications that have used
+ <function>PQsetnonblocking</function> have a need for this.
+ </para>
+ </listitem>
+ 
+ <listitem>
+ <para>
  <function>PQsocket</function>
  	  Obtain the file descriptor number for the backend connection socket.
! 	  A valid descriptor will be &gt;= 0; a result of -1 indicates that
  	  no backend connection is currently open.
  <synopsis>
  int PQsocket(const PGconn *conn);
  </synopsis>
  <function>PQsocket</function> should be used to obtain the backend socket descriptor
  in preparation for executing <function>select</function>(2).  This allows an
! application using a blocking connection to wait for either backend responses or
! other conditions.
  If the result of <function>select</function>(2) indicates that data can be read from
  the backend socket, then <function>PQconsumeInput</function> should be called to read the
  data; after which, <function>PQisBusy</function>, <function>PQgetResult</function>,
  and/or <function>PQnotifies</function> can be used to process the response.
+ </para>
+ <para>
+ Non-blocking connections (that have used <function>PQsetnonblocking</function>)
+ should not use <function>select</function> until <function>PQflush</function>
+ has returned 0 indicating that there is no buffered data waiting to be sent
+ to the backend.
  </para>
  </listitem>
  
Index: src/interfaces/libpq/fe-connect.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.111
diff -u -c -r1.111 fe-connect.c
*** src/interfaces/libpq/fe-connect.c	2000/01/16 21:18:52	1.111
--- src/interfaces/libpq/fe-connect.c	2000/01/17 02:35:56
***************
*** 594,624 ****
  	return 0;
  }
  
- 
- /* ----------
-  * connectMakeNonblocking -
-  * Make a connection non-blocking.
-  * Returns 1 if successful, 0 if not.
-  * ----------
-  */
- static int
- connectMakeNonblocking(PGconn *conn)
- {
- #ifndef WIN32
- 	if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
- #else
- 	if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
- #endif
- 	{
- 		printfPQExpBuffer(&conn->errorMessage,
- 						  "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
- 						  errno, strerror(errno));
- 		return 0;
- 	}
- 
- 	return 1;
- }
- 
  /* ----------
   * connectNoDelay -
   * Sets the TCP_NODELAY socket option.
--- 594,599 ----
***************
*** 789,795 ****
  	 *   Ewan Mellor <eem21@cam.ac.uk>.
  	 * ---------- */
  #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
! 	if (!connectMakeNonblocking(conn))
  		goto connect_errReturn;
  #endif	
  
--- 764,770 ----
  	 *   Ewan Mellor <eem21@cam.ac.uk>.
  	 * ---------- */
  #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
! 	if (PQsetnonblocking(conn, TRUE) != 0)
  		goto connect_errReturn;
  #endif	
  
***************
*** 898,904 ****
  	/* This makes the connection non-blocking, for all those cases which forced us
  	   not to do it above. */
  #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! 	if (!connectMakeNonblocking(conn))
  		goto connect_errReturn;
  #endif	
  
--- 873,879 ----
  	/* This makes the connection non-blocking, for all those cases which forced us
  	   not to do it above. */
  #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! 	if (PQsetnonblocking(conn, TRUE) != 0)
  		goto connect_errReturn;
  #endif	
  
***************
*** 1720,1725 ****
--- 1695,1701 ----
  	conn->inBuffer = (char *) malloc(conn->inBufSize);
  	conn->outBufSize = 8 * 1024;
  	conn->outBuffer = (char *) malloc(conn->outBufSize);
+ 	conn->nonblocking = FALSE;
  	initPQExpBuffer(&conn->errorMessage);
  	initPQExpBuffer(&conn->workBuffer);
  	if (conn->inBuffer == NULL ||
***************
*** 1830,1835 ****
--- 1806,1812 ----
  	conn->lobjfuncs = NULL;
  	conn->inStart = conn->inCursor = conn->inEnd = 0;
  	conn->outCount = 0;
+ 	conn->nonblocking = FALSE;
  
  }
  
Index: src/interfaces/libpq/fe-exec.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.86
diff -u -c -r1.86 fe-exec.c
*** src/interfaces/libpq/fe-exec.c	1999/11/11 00:10:14	1.86
--- src/interfaces/libpq/fe-exec.c	2000/01/14 22:47:07
***************
*** 13,18 ****
--- 13,19 ----
   */
  #include <errno.h>
  #include <ctype.h>
+ #include <fcntl.h>
  
  #include "postgres.h"
  #include "libpq-fe.h"
***************
*** 24,30 ****
  #include <unistd.h>
  #endif
  
- 
  /* keep this in same order as ExecStatusType in libpq-fe.h */
  const char *const pgresStatus[] = {
  	"PGRES_EMPTY_QUERY",
--- 25,30 ----
***************
*** 514,526 ****
  	conn->curTuple = NULL;
  
  	/* send the query to the backend; */
! 	/* the frontend-backend protocol uses 'Q' to designate queries */
! 	if (pqPutnchar("Q", 1, conn) ||
! 		pqPuts(query, conn) ||
! 		pqFlush(conn))
  	{
! 		handleSendFailure(conn);
! 		return 0;
  	}
  
  	/* OK, it's launched! */
--- 514,566 ----
  	conn->curTuple = NULL;
  
  	/* send the query to the backend; */
! 
! 	/*
! 	 * in order to guarantee that we don't send a partial query 
! 	 * where we would become out of sync with the backend and/or
! 	 * block during a non-blocking connection we must first flush
! 	 * the send buffer before sending more data
! 	 *
! 	 * an alternative is to implement 'queue reservations' where
! 	 * we are able to roll up a transaction 
! 	 * (the 'Q' along with our query) and make sure we have
! 	 * enough space for it all in the send buffer.
! 	 */
! 	if (pqIsnonblocking(conn))
  	{
! 		/*
! 		 * the buffer must have emptied completely before we allow
! 		 * a new query to be buffered
! 		 */
! 		if (pqFlush(conn))
! 			return 0;
! 		/* 'Q' == queries */
! 		/* XXX: if we fail here we really ought to not block */
! 		if (pqPutnchar("Q", 1, conn) ||
! 			pqPuts(query, conn))
! 		{
! 			handleSendFailure(conn);	
! 			return 0;
! 		}
! 		/*
! 		 * give the data a push, ignore the return value as
! 		 * ConsumeInput() will do any aditional flushing if needed
! 		 */
! 		(void) pqFlush(conn);	
! 	}
! 	else
! 	{
! 		/* 
! 		 * the frontend-backend protocol uses 'Q' to 
! 		 * designate queries 
! 		 */
! 		if (pqPutnchar("Q", 1, conn) ||
! 			pqPuts(query, conn) ||
! 			pqFlush(conn))
! 		{
! 			handleSendFailure(conn);
! 			return 0;
! 		}
  	}
  
  	/* OK, it's launched! */
***************
*** 574,580 ****
--- 614,630 ----
  	 * we will NOT block waiting for more input.
  	 */
  	if (pqReadData(conn) < 0)
+ 	{
+ 		/*
+ 		 * for non-blocking connections
+ 		 * try to flush the send-queue otherwise we may never get a 
+ 		 * responce for something that may not have already been sent
+ 		 * because it's in our write buffer!
+ 		 */
+ 		if (pqIsnonblocking(conn))
+ 			(void) pqFlush(conn);
  		return 0;
+ 	}
  	/* Parsing of the data waits till later. */
  	return 1;
  }
***************
*** 1088,1093 ****
--- 1138,1153 ----
  {
  	PGresult   *result;
  	PGresult   *lastResult;
+ 	bool	savedblocking;
+ 
+ 	/*
+ 	 * we assume anyone calling PQexec wants blocking behaviour,
+ 	 * we force the blocking status of the connection to blocking
+ 	 * for the duration of this function and restore it on return
+ 	 */
+ 	savedblocking = pqIsnonblocking(conn);
+ 	if (PQsetnonblocking(conn, FALSE) == -1)
+ 		return NULL;
  
  	/*
  	 * Silently discard any prior query result that application didn't
***************
*** 1102,1115 ****
  			PQclear(result);
  			printfPQExpBuffer(&conn->errorMessage,
  				"PQexec: you gotta get out of a COPY state yourself.\n");
! 			return NULL;
  		}
  		PQclear(result);
  	}
  
  	/* OK to send the message */
  	if (!PQsendQuery(conn, query))
! 		return NULL;
  
  	/*
  	 * For backwards compatibility, return the last result if there are
--- 1162,1176 ----
  			PQclear(result);
  			printfPQExpBuffer(&conn->errorMessage,
  				"PQexec: you gotta get out of a COPY state yourself.\n");
! 			/* restore blocking status */
! 			goto errout;
  		}
  		PQclear(result);
  	}
  
  	/* OK to send the message */
  	if (!PQsendQuery(conn, query))
! 		goto errout;	/* restore blocking status */
  
  	/*
  	 * For backwards compatibility, return the last result if there are
***************
*** 1142,1148 ****
--- 1203,1217 ----
  			result->resultStatus == PGRES_COPY_OUT)
  			break;
  	}
+ 
+ 	if (PQsetnonblocking(conn, savedblocking) == -1)
+ 		return NULL;
  	return lastResult;
+ 
+ errout:
+ 	if (PQsetnonblocking(conn, savedblocking) == -1)
+ 		return NULL;
+ 	return NULL;
  }
  
  
***************
*** 1431,1438 ****
  			 "PQendcopy() -- I don't think there's a copy in progress.\n");
  		return 1;
  	}
  
! 	(void) pqFlush(conn);		/* make sure no data is waiting to be sent */
  
  	/* Return to active duty */
  	conn->asyncStatus = PGASYNC_BUSY;
--- 1500,1516 ----
  			 "PQendcopy() -- I don't think there's a copy in progress.\n");
  		return 1;
  	}
+ 
+ 	/*
+ 	 * make sure no data is waiting to be sent, 
+ 	 * abort if we are non-blocking and the flush fails
+ 	 */
+ 	if (pqFlush(conn) && pqIsnonblocking(conn))
+ 		return (1);
  
! 	/* non blocking connections may have to abort at this point. */
! 	if (pqIsnonblocking(conn) && PQisBusy(conn))
! 		return (1);
  
  	/* Return to active duty */
  	conn->asyncStatus = PGASYNC_BUSY;
***************
*** 2025,2028 ****
--- 2103,2192 ----
  		return 1;
  	else
  		return 0;
+ }
+ 
+ /* PQsetnonblocking:
+ 	 sets the PGconn's database connection non-blocking if the arg is TRUE
+ 	 or makes it non-blocking if the arg is FALSE, this will not protect
+ 	 you from PQexec(), you'll only be safe when using the non-blocking
+ 	 API
+ 	 Needs to be called only on a connected database connection.
+ */
+ 
+ int
+ PQsetnonblocking(PGconn *conn, int arg)
+ {
+ 	int	fcntlarg;
+ 
+ 	arg = (arg == TRUE) ? 1 : 0;
+ 	/* early out if the socket is already in the state requested */
+ 	if (arg == conn->nonblocking)
+ 		return (0);
+ 
+ 	/*
+ 	 * to guarantee constancy for flushing/query/result-polling behavior
+ 	 * we need to flush the send queue at this point in order to guarantee
+ 	 * proper behavior.
+ 	 * this is ok because either they are making a transition
+ 	 *  _from_ or _to_ blocking mode, either way we can block them.
+ 	 */
+ 	/* if we are going from blocking to non-blocking flush here */
+ 	if (!pqIsnonblocking(conn) && pqFlush(conn))
+ 		return (-1);
+ 
+ 
+ #ifdef USE_SSL
+ 	if (conn->ssl)
+ 	{
+ 		printfPQExpBuffer(&conn->errorMessage,
+ 			"PQsetnonblocking() -- not supported when using SSL\n");
+ 		return (-1);
+ 	}
+ #endif /* USE_SSL */
+ 
+ #ifndef WIN32
+ 	fcntlarg = fcntl(conn->sock, F_GETFL, 0);
+ 	if (fcntlarg == -1)
+ 		return (-1);
+ 
+ 	if ((arg == TRUE && 
+ 		fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
+ 		(arg == FALSE &&
+ 		fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) 
+ #else
+ 	fcntlarg = arg;
+ 	if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
+ #endif
+ 	{
+ 		printfPQExpBuffer(&conn->errorMessage,
+ 			"PQsetblocking() -- unable to set nonblocking status to %s\n",
+ 			arg == TRUE ? "TRUE" : "FALSE");
+ 		return (-1);
+ 	}
+ 
+ 	conn->nonblocking = arg;
+ 
+ 	/* if we are going from non-blocking to blocking flush here */
+ 	if (pqIsnonblocking(conn) && pqFlush(conn))
+ 		return (-1);
+ 
+ 	return (0);
+ }
+ 
+ /* return the blocking status of the database connection, TRUE == nonblocking,
+ 	 FALSE == blocking
+ */
+ int
+ PQisnonblocking(const PGconn *conn)
+ {
+ 
+ 	return (pqIsnonblocking(conn));
+ }
+ 
+ /* try to force data out, really only useful for non-blocking users */
+ int
+ PQflush(PGconn *conn)
+ {
+ 
+ 	return (pqFlush(conn));
  }
Index: src/interfaces/libpq/fe-misc.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
retrieving revision 1.33
diff -u -c -r1.33 fe-misc.c
*** src/interfaces/libpq/fe-misc.c	1999/11/30 03:08:19	1.33
--- src/interfaces/libpq/fe-misc.c	2000/01/12 03:12:14
***************
*** 86,91 ****
--- 86,122 ----
  {
  	size_t avail = Max(conn->outBufSize - conn->outCount, 0);
  
+ 	/*
+ 	 * if we are non-blocking and the send queue is too full to buffer this
+ 	 * request then try to flush some and return an error 
+ 	 */
+ 	if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn))
+ 	{
+ 		/* 
+ 		 * even if the flush failed we may still have written some
+ 		 * data, recalculate the size of the send-queue relative
+ 		 * to the amount we have to send, we may be able to queue it
+ 		 * afterall even though it's not sent to the database it's
+ 		 * ok, any routines that check the data coming from the
+ 		 * database better call pqFlush() anyway.
+ 		 */
+ 		if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
+ 		{
+ 			printfPQExpBuffer(&conn->errorMessage,
+ 				"pqPutBytes --  pqFlush couldn't flush enough"
+ 				" data: space available: %d, space needed %d\n",
+ 				Max(conn->outBufSize - conn->outCount, 0), nbytes);
+ 			return EOF;
+ 		}
+ 	}
+ 
+ 	/* 
+ 	 * is the amount of data to be sent is larger than the size of the
+ 	 * output buffer then we must flush it to make more room.
+ 	 *
+ 	 * the code above will make sure the loop conditional is never 
+ 	 * true for non-blocking connections
+ 	 */
  	while (nbytes > avail)
  	{
  		memcpy(conn->outBuffer + conn->outCount, s, avail);
***************
*** 548,553 ****
--- 579,592 ----
  		return EOF;
  	}
  
+ 	/* 
+ 	 * don't try to send zero data, allows us to use this function
+ 	 * without too much worry about overhead
+ 	 */
+ 	if (len == 0)
+ 		return (0);
+ 
+ 	/* while there's still data to send */
  	while (len > 0)
  	{
  		/* Prevent being SIGPIPEd if backend has closed the connection. */
***************
*** 556,561 ****
--- 595,601 ----
  #endif
  
  		int sent;
+ 
  #ifdef USE_SSL
  		if (conn->ssl) 
  		  sent = SSL_write(conn->ssl, ptr, len);
***************
*** 585,590 ****
--- 625,632 ----
  				case EWOULDBLOCK:
  					break;
  #endif
+ 				case EINTR:
+ 					continue;
  
  				case EPIPE:
  #ifdef ECONNRESET
***************
*** 616,628 ****
  			ptr += sent;
  			len -= sent;
  		}
  		if (len > 0)
  		{
  			/* We didn't send it all, wait till we can send more */
  
- 			/* At first glance this looks as though it should block.  I think
- 			 * that it will be OK though, as long as the socket is
- 			 * non-blocking. */
  			if (pqWait(FALSE, TRUE, conn))
  				return EOF;
  		}
--- 658,688 ----
  			ptr += sent;
  			len -= sent;
  		}
+ 
  		if (len > 0)
  		{
  			/* We didn't send it all, wait till we can send more */
+ 
+ 			/* 
+ 			 * if the socket is in non-blocking mode we may need
+ 			 * to abort here 
+ 			 */
+ #ifdef USE_SSL
+ 			/* can't do anything for our SSL users yet */
+ 			if (conn->ssl == NULL)
+ 			{
+ #endif
+ 				if (pqIsnonblocking(conn))
+ 				{
+ 					/* shift the contents of the buffer */
+ 					memmove(conn->outBuffer, ptr, len);
+ 					conn->outCount = len;
+ 					return EOF;
+ 				}
+ #ifdef USE_SSL
+ 			}
+ #endif
  
  			if (pqWait(FALSE, TRUE, conn))
  				return EOF;
  		}
Index: src/interfaces/libpq/libpq-fe.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.55
diff -u -c -r1.55 libpq-fe.h
*** src/interfaces/libpq/libpq-fe.h	2000/01/15 05:37:21	1.55
--- src/interfaces/libpq/libpq-fe.h	2000/01/17 02:35:56
***************
*** 263,268 ****
--- 263,275 ----
  	extern int	PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
  	extern int	PQendcopy(PGconn *conn);
  
+ 	/* Set blocking/nonblocking connection to the backend */
+ 	extern int	PQsetnonblocking(PGconn *conn, int arg);
+ 	extern int	PQisnonblocking(const PGconn *conn);
+ 
+ 	/* Force the write buffer to be written (or at least try) */
+ 	extern int	PQflush(PGconn *conn);
+ 
  	/*
  	 * "Fast path" interface --- not really recommended for application
  	 * use
Index: src/interfaces/libpq/libpq-int.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
retrieving revision 1.16
diff -u -c -r1.16 libpq-int.h
*** src/interfaces/libpq/libpq-int.h	2000/01/15 05:37:21	1.16
--- src/interfaces/libpq/libpq-int.h	2000/01/17 02:35:56
***************
*** 214,219 ****
--- 214,222 ----
  	int			inEnd;			/* offset to first position after avail
  								 * data */
  
+ 	int			nonblocking;	/* whether this connection is using a blocking
+ 								 * socket to the backend or not */
+ 
  	/* Buffer for data not yet sent to backend */
  	char	   *outBuffer;		/* currently allocated buffer */
  	int			outBufSize;		/* allocated size of buffer */
***************
*** 299,303 ****
--- 302,312 ----
  #define strerror(A) (sys_errlist[(A)])
  #endif	 /* sunos4 */
  #endif	 /* !strerror */
+ 
+ /* 
+  * this is so that we can check is a connection is non-blocking internally
+  * without the overhead of a function call
+  */
+ #define pqIsnonblocking(conn)	(conn->nonblocking)
  
  #endif	 /* LIBPQ_INT_H */


Sun, 16 Jan 2000 16:00:45 -0800
Alfred Perlstein <bright@wintelcom.net>
docs done Re: [HACKERS] LIBPQ patches ...
> setup my enviornment... (this ought to be mentioned in the docs)

But afaik this isn't required for me to run on postgresql.org, a
FreeBSD machine set up by Marc/scrappy.

> then in the pgsql/doc/src dir:
> gmake all \
>         HSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/html/ \
>         PSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/print/ \

That works too. I usually just set up a src/Makefile.custom with the
two lines defining HSTYLE and PSTYLE.

> Btw, does anyone have some fixes so gvim doesn't barf doing syntax
> highlighting on these sgml files?

Let us know when you find them; I can help with emacs...

                   - Thomas

-- 
Thomas Lockhart				lockhart@alumni.caltech.edu
South Pasadena, California

Tue, 18 Jan 2000 02:58:18 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Re: FreeBSD postgresql doc-HOWTO was: Re: [HACKERS] Revised
	nonblocking patches + quasi docs
Applied.

> * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote:
> > Don Baccus <dhogaza@pacifier.com> writes:
> > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> > >> I also object strongly to the lack of documentation.
> > 
> > > ... I know there are some folks who aren't native-english speakers, so
> > > perhaps you don't want to require that the implementor of such patches
> > > provide the final documentation wording.  But the information should
> > > be there and spelled out in a form that can be very easily moved to
> > > the docs.
> > 
> > Oh, absolutely.  Thomas, our master of the docs, has always had the
> > policy of "give me some words, I'll take care of formatting and
> > editing..."
> > 
> > I was probably too harsh on Alfred last night, since in fact his code
> > was fairly well commented, and some minimal doco could have been
> > extracted from the routine headers.  But on a change like this, I think
> > some paragraphs of coherent high-level explanation are needed: what it
> > does, when and why you'd use it.  I didn't see that anywhere...
> 
> Here's the revised patch, it includes sgml docs and changes to
> ensure that old style connections behave the way they are expected
> to:
> 
> Index: doc/src/sgml/libpq.sgml
> ===================================================================
> RCS file: /home/pgcvs/pgsql/doc/src/sgml/libpq.sgml,v
> retrieving revision 1.25
> diff -u -c -r1.25 libpq.sgml
> *** doc/src/sgml/libpq.sgml	2000/01/14 05:33:13	1.25
> --- doc/src/sgml/libpq.sgml	2000/01/17 03:40:30
> ***************
> *** 377,382 ****
> --- 377,386 ----
>      changed in the future.
>     </para>
>     <para>
> +    These functions leave the socket in a non-blocking state as if 
> +    <function>PQsetnonblocking</function> had been called.
> +   </para>
> +   <para>
>      These functions are not thread-safe.
>     </para>
>    </listitem>
> ***************
> *** 1168,1175 ****
> --- 1172,1229 ----
>   Applications that do not like these limitations can instead use the
>   underlying functions that <function>PQexec</function> is built from:
>   <function>PQsendQuery</function> and <function>PQgetResult</function>.
> + </para>
> + <para>
> + Older programs that used this functionality as well as 
> + <function>PQputline</function> and <function>PQputnbytes</function>
> + could block waiting to send data to the backend, to
> + address that issue, the function <function>PQsetnonblocking</function>
> + was added.
> + </para>
> + <para>
> + Old applications can neglect to use <function>PQsetnonblocking</function>
> + and get the older potentially blocking behavior.  Newer programs can use 
> + <function>PQsetnonblocking</function> to achieve a completely non-blocking
> + connection to the backend.
>   
>   <itemizedlist>
> +  <listitem>
> +    <para>
> +     <function>PQsetnonblocking</function> Sets the state of the connection
> +     to non-blocking.
> + <synopsis>
> + int PQsetnonblocking(PGconn *conn)
> + </synopsis>
> +     this function will ensure that calls to 
> +     <function>PQputline</function>, <function>PQputnbytes</function>,
> +     <function>PQsendQuery</function> and <function>PQendcopy</function>
> +     will not block but instead return an error if they need to be called
> +     again.
> +    </para>
> +    <para>
> +     When a database connection has been set to non-blocking mode and
> +     <function>PQexec</function> is called, it will temporarily set the state
> +     of the connection to blocking until the <function>PQexec</function> 
> +     completes. 
> +    </para>
> +    <para>
> +     More of libpq is expected to be made safe for 
> +     <function>PQsetnonblocking</function> functionality in the near future.
> +   </para>
> +  </listitem>
> + 
> + <listitem>
> + <para>
> + <function>PQisnonblocking</function>
> +        Returns the blocking status of the database connection.
> + <synopsis>
> + int PQisnonblocking(const PGconn *conn)
> + </synopsis>
> +        Returns TRUE if the connection is set to non-blocking mode,
> +        FALSE if blocking.
> + </para>
> + </listitem>
> + 
>   <listitem>
>   <para>
>   <function>PQsendQuery</function>
> ***************
> *** 1267,1286 ****
>   
>   <listitem>
>   <para>
>   <function>PQsocket</function>
>   	  Obtain the file descriptor number for the backend connection socket.
> ! 	  A valid descriptor will be >= 0; a result of -1 indicates that
>   	  no backend connection is currently open.
>   <synopsis>
>   int PQsocket(const PGconn *conn);
>   </synopsis>
>   <function>PQsocket</function> should be used to obtain the backend socket descriptor
>   in preparation for executing <function>select</function>(2).  This allows an
> ! application to wait for either backend responses or other conditions.
>   If the result of <function>select</function>(2) indicates that data can be read from
>   the backend socket, then <function>PQconsumeInput</function> should be called to read the
>   data; after which, <function>PQisBusy</function>, <function>PQgetResult</function>,
>   and/or <function>PQnotifies</function> can be used to process the response.
>   </para>
>   </listitem>
>   
> --- 1321,1363 ----
>   
>   <listitem>
>   <para>
> + <function>PQflush</function> Attempt to flush any data queued to the backend,
> + returns 0 if successful (or if the send queue is empty) or EOF if it failed for
> + some reason.
> + <synopsis>
> + int PQflush(PGconn *conn);
> + </synopsis>
> + <function>PQflush</function> needs to be called on a non-blocking connection 
> + before calling <function>select</function> to determine if a responce has
> + arrived.  If 0 is returned it ensures that there is no data queued to the 
> + backend that has not actually been sent.  Only applications that have used
> + <function>PQsetnonblocking</function> have a need for this.
> + </para>
> + </listitem>
> + 
> + <listitem>
> + <para>
>   <function>PQsocket</function>
>   	  Obtain the file descriptor number for the backend connection socket.
> ! 	  A valid descriptor will be &gt;= 0; a result of -1 indicates that
>   	  no backend connection is currently open.
>   <synopsis>
>   int PQsocket(const PGconn *conn);
>   </synopsis>
>   <function>PQsocket</function> should be used to obtain the backend socket descriptor
>   in preparation for executing <function>select</function>(2).  This allows an
> ! application using a blocking connection to wait for either backend responses or
> ! other conditions.
>   If the result of <function>select</function>(2) indicates that data can be read from
>   the backend socket, then <function>PQconsumeInput</function> should be called to read the
>   data; after which, <function>PQisBusy</function>, <function>PQgetResult</function>,
>   and/or <function>PQnotifies</function> can be used to process the response.
> + </para>
> + <para>
> + Non-blocking connections (that have used <function>PQsetnonblocking</function>)
> + should not use <function>select</function> until <function>PQflush</function>
> + has returned 0 indicating that there is no buffered data waiting to be sent
> + to the backend.
>   </para>
>   </listitem>
>   
> Index: src/interfaces/libpq/fe-connect.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
> retrieving revision 1.111
> diff -u -c -r1.111 fe-connect.c
> *** src/interfaces/libpq/fe-connect.c	2000/01/16 21:18:52	1.111
> --- src/interfaces/libpq/fe-connect.c	2000/01/17 02:35:56
> ***************
> *** 594,624 ****
>   	return 0;
>   }
>   
> - 
> - /* ----------
> -  * connectMakeNonblocking -
> -  * Make a connection non-blocking.
> -  * Returns 1 if successful, 0 if not.
> -  * ----------
> -  */
> - static int
> - connectMakeNonblocking(PGconn *conn)
> - {
> - #ifndef WIN32
> - 	if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
> - #else
> - 	if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
> - #endif
> - 	{
> - 		printfPQExpBuffer(&conn->errorMessage,
> - 						  "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
> - 						  errno, strerror(errno));
> - 		return 0;
> - 	}
> - 
> - 	return 1;
> - }
> - 
>   /* ----------
>    * connectNoDelay -
>    * Sets the TCP_NODELAY socket option.
> --- 594,599 ----
> ***************
> *** 789,795 ****
>   	 *   Ewan Mellor <eem21@cam.ac.uk>.
>   	 * ---------- */
>   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! 	if (!connectMakeNonblocking(conn))
>   		goto connect_errReturn;
>   #endif	
>   
> --- 764,770 ----
>   	 *   Ewan Mellor <eem21@cam.ac.uk>.
>   	 * ---------- */
>   #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! 	if (PQsetnonblocking(conn, TRUE) != 0)
>   		goto connect_errReturn;
>   #endif	
>   
> ***************
> *** 898,904 ****
>   	/* This makes the connection non-blocking, for all those cases which forced us
>   	   not to do it above. */
>   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! 	if (!connectMakeNonblocking(conn))
>   		goto connect_errReturn;
>   #endif	
>   
> --- 873,879 ----
>   	/* This makes the connection non-blocking, for all those cases which forced us
>   	   not to do it above. */
>   #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! 	if (PQsetnonblocking(conn, TRUE) != 0)
>   		goto connect_errReturn;
>   #endif	
>   
> ***************
> *** 1720,1725 ****
> --- 1695,1701 ----
>   	conn->inBuffer = (char *) malloc(conn->inBufSize);
>   	conn->outBufSize = 8 * 1024;
>   	conn->outBuffer = (char *) malloc(conn->outBufSize);
> + 	conn->nonblocking = FALSE;
>   	initPQExpBuffer(&conn->errorMessage);
>   	initPQExpBuffer(&conn->workBuffer);
>   	if (conn->inBuffer == NULL ||
> ***************
> *** 1830,1835 ****
> --- 1806,1812 ----
>   	conn->lobjfuncs = NULL;
>   	conn->inStart = conn->inCursor = conn->inEnd = 0;
>   	conn->outCount = 0;
> + 	conn->nonblocking = FALSE;
>   
>   }
>   
> Index: src/interfaces/libpq/fe-exec.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
> retrieving revision 1.86
> diff -u -c -r1.86 fe-exec.c
> *** src/interfaces/libpq/fe-exec.c	1999/11/11 00:10:14	1.86
> --- src/interfaces/libpq/fe-exec.c	2000/01/14 22:47:07
> ***************
> *** 13,18 ****
> --- 13,19 ----
>    */
>   #include <errno.h>
>   #include <ctype.h>
> + #include <fcntl.h>
>   
>   #include "postgres.h"
>   #include "libpq-fe.h"
> ***************
> *** 24,30 ****
>   #include <unistd.h>
>   #endif
>   
> - 
>   /* keep this in same order as ExecStatusType in libpq-fe.h */
>   const char *const pgresStatus[] = {
>   	"PGRES_EMPTY_QUERY",
> --- 25,30 ----
> ***************
> *** 514,526 ****
>   	conn->curTuple = NULL;
>   
>   	/* send the query to the backend; */
> ! 	/* the frontend-backend protocol uses 'Q' to designate queries */
> ! 	if (pqPutnchar("Q", 1, conn) ||
> ! 		pqPuts(query, conn) ||
> ! 		pqFlush(conn))
>   	{
> ! 		handleSendFailure(conn);
> ! 		return 0;
>   	}
>   
>   	/* OK, it's launched! */
> --- 514,566 ----
>   	conn->curTuple = NULL;
>   
>   	/* send the query to the backend; */
> ! 
> ! 	/*
> ! 	 * in order to guarantee that we don't send a partial query 
> ! 	 * where we would become out of sync with the backend and/or
> ! 	 * block during a non-blocking connection we must first flush
> ! 	 * the send buffer before sending more data
> ! 	 *
> ! 	 * an alternative is to implement 'queue reservations' where
> ! 	 * we are able to roll up a transaction 
> ! 	 * (the 'Q' along with our query) and make sure we have
> ! 	 * enough space for it all in the send buffer.
> ! 	 */
> ! 	if (pqIsnonblocking(conn))
>   	{
> ! 		/*
> ! 		 * the buffer must have emptied completely before we allow
> ! 		 * a new query to be buffered
> ! 		 */
> ! 		if (pqFlush(conn))
> ! 			return 0;
> ! 		/* 'Q' == queries */
> ! 		/* XXX: if we fail here we really ought to not block */
> ! 		if (pqPutnchar("Q", 1, conn) ||
> ! 			pqPuts(query, conn))
> ! 		{
> ! 			handleSendFailure(conn);	
> ! 			return 0;
> ! 		}
> ! 		/*
> ! 		 * give the data a push, ignore the return value as
> ! 		 * ConsumeInput() will do any aditional flushing if needed
> ! 		 */
> ! 		(void) pqFlush(conn);	
> ! 	}
> ! 	else
> ! 	{
> ! 		/* 
> ! 		 * the frontend-backend protocol uses 'Q' to 
> ! 		 * designate queries 
> ! 		 */
> ! 		if (pqPutnchar("Q", 1, conn) ||
> ! 			pqPuts(query, conn) ||
> ! 			pqFlush(conn))
> ! 		{
> ! 			handleSendFailure(conn);
> ! 			return 0;
> ! 		}
>   	}
>   
>   	/* OK, it's launched! */
> ***************
> *** 574,580 ****
> --- 614,630 ----
>   	 * we will NOT block waiting for more input.
>   	 */
>   	if (pqReadData(conn) < 0)
> + 	{
> + 		/*
> + 		 * for non-blocking connections
> + 		 * try to flush the send-queue otherwise we may never get a 
> + 		 * responce for something that may not have already been sent
> + 		 * because it's in our write buffer!
> + 		 */
> + 		if (pqIsnonblocking(conn))
> + 			(void) pqFlush(conn);
>   		return 0;
> + 	}
>   	/* Parsing of the data waits till later. */
>   	return 1;
>   }
> ***************
> *** 1088,1093 ****
> --- 1138,1153 ----
>   {
>   	PGresult   *result;
>   	PGresult   *lastResult;
> + 	bool	savedblocking;
> + 
> + 	/*
> + 	 * we assume anyone calling PQexec wants blocking behaviour,
> + 	 * we force the blocking status of the connection to blocking
> + 	 * for the duration of this function and restore it on return
> + 	 */
> + 	savedblocking = pqIsnonblocking(conn);
> + 	if (PQsetnonblocking(conn, FALSE) == -1)
> + 		return NULL;
>   
>   	/*
>   	 * Silently discard any prior query result that application didn't
> ***************
> *** 1102,1115 ****
>   			PQclear(result);
>   			printfPQExpBuffer(&conn->errorMessage,
>   				"PQexec: you gotta get out of a COPY state yourself.\n");
> ! 			return NULL;
>   		}
>   		PQclear(result);
>   	}
>   
>   	/* OK to send the message */
>   	if (!PQsendQuery(conn, query))
> ! 		return NULL;
>   
>   	/*
>   	 * For backwards compatibility, return the last result if there are
> --- 1162,1176 ----
>   			PQclear(result);
>   			printfPQExpBuffer(&conn->errorMessage,
>   				"PQexec: you gotta get out of a COPY state yourself.\n");
> ! 			/* restore blocking status */
> ! 			goto errout;
>   		}
>   		PQclear(result);
>   	}
>   
>   	/* OK to send the message */
>   	if (!PQsendQuery(conn, query))
> ! 		goto errout;	/* restore blocking status */
>   
>   	/*
>   	 * For backwards compatibility, return the last result if there are
> ***************
> *** 1142,1148 ****
> --- 1203,1217 ----
>   			result->resultStatus == PGRES_COPY_OUT)
>   			break;
>   	}
> + 
> + 	if (PQsetnonblocking(conn, savedblocking) == -1)
> + 		return NULL;
>   	return lastResult;
> + 
> + errout:
> + 	if (PQsetnonblocking(conn, savedblocking) == -1)
> + 		return NULL;
> + 	return NULL;
>   }
>   
>   
> ***************
> *** 1431,1438 ****
>   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
>   		return 1;
>   	}
>   
> ! 	(void) pqFlush(conn);		/* make sure no data is waiting to be sent */
>   
>   	/* Return to active duty */
>   	conn->asyncStatus = PGASYNC_BUSY;
> --- 1500,1516 ----
>   			 "PQendcopy() -- I don't think there's a copy in progress.\n");
>   		return 1;
>   	}
> + 
> + 	/*
> + 	 * make sure no data is waiting to be sent, 
> + 	 * abort if we are non-blocking and the flush fails
> + 	 */
> + 	if (pqFlush(conn) && pqIsnonblocking(conn))
> + 		return (1);
>   
> ! 	/* non blocking connections may have to abort at this point. */
> ! 	if (pqIsnonblocking(conn) && PQisBusy(conn))
> ! 		return (1);
>   
>   	/* Return to active duty */
>   	conn->asyncStatus = PGASYNC_BUSY;
> ***************
> *** 2025,2028 ****
> --- 2103,2192 ----
>   		return 1;
>   	else
>   		return 0;
> + }
> + 
> + /* PQsetnonblocking:
> + 	 sets the PGconn's database connection non-blocking if the arg is TRUE
> + 	 or makes it non-blocking if the arg is FALSE, this will not protect
> + 	 you from PQexec(), you'll only be safe when using the non-blocking
> + 	 API
> + 	 Needs to be called only on a connected database connection.
> + */
> + 
> + int
> + PQsetnonblocking(PGconn *conn, int arg)
> + {
> + 	int	fcntlarg;
> + 
> + 	arg = (arg == TRUE) ? 1 : 0;
> + 	/* early out if the socket is already in the state requested */
> + 	if (arg == conn->nonblocking)
> + 		return (0);
> + 
> + 	/*
> + 	 * to guarantee constancy for flushing/query/result-polling behavior
> + 	 * we need to flush the send queue at this point in order to guarantee
> + 	 * proper behavior.
> + 	 * this is ok because either they are making a transition
> + 	 *  _from_ or _to_ blocking mode, either way we can block them.
> + 	 */
> + 	/* if we are going from blocking to non-blocking flush here */
> + 	if (!pqIsnonblocking(conn) && pqFlush(conn))
> + 		return (-1);
> + 
> + 
> + #ifdef USE_SSL
> + 	if (conn->ssl)
> + 	{
> + 		printfPQExpBuffer(&conn->errorMessage,
> + 			"PQsetnonblocking() -- not supported when using SSL\n");
> + 		return (-1);
> + 	}
> + #endif /* USE_SSL */
> + 
> + #ifndef WIN32
> + 	fcntlarg = fcntl(conn->sock, F_GETFL, 0);
> + 	if (fcntlarg == -1)
> + 		return (-1);
> + 
> + 	if ((arg == TRUE && 
> + 		fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
> + 		(arg == FALSE &&
> + 		fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) 
> + #else
> + 	fcntlarg = arg;
> + 	if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
> + #endif
> + 	{
> + 		printfPQExpBuffer(&conn->errorMessage,
> + 			"PQsetblocking() -- unable to set nonblocking status to %s\n",
> + 			arg == TRUE ? "TRUE" : "FALSE");
> + 		return (-1);
> + 	}
> + 
> + 	conn->nonblocking = arg;
> + 
> + 	/* if we are going from non-blocking to blocking flush here */
> + 	if (pqIsnonblocking(conn) && pqFlush(conn))
> + 		return (-1);
> + 
> + 	return (0);
> + }
> + 
> + /* return the blocking status of the database connection, TRUE == nonblocking,
> + 	 FALSE == blocking
> + */
> + int
> + PQisnonblocking(const PGconn *conn)
> + {
> + 
> + 	return (pqIsnonblocking(conn));
> + }
> + 
> + /* try to force data out, really only useful for non-blocking users */
> + int
> + PQflush(PGconn *conn)
> + {
> + 
> + 	return (pqFlush(conn));
>   }
> Index: src/interfaces/libpq/fe-misc.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
> retrieving revision 1.33
> diff -u -c -r1.33 fe-misc.c
> *** src/interfaces/libpq/fe-misc.c	1999/11/30 03:08:19	1.33
> --- src/interfaces/libpq/fe-misc.c	2000/01/12 03:12:14
> ***************
> *** 86,91 ****
> --- 86,122 ----
>   {
>   	size_t avail = Max(conn->outBufSize - conn->outCount, 0);
>   
> + 	/*
> + 	 * if we are non-blocking and the send queue is too full to buffer this
> + 	 * request then try to flush some and return an error 
> + 	 */
> + 	if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn))
> + 	{
> + 		/* 
> + 		 * even if the flush failed we may still have written some
> + 		 * data, recalculate the size of the send-queue relative
> + 		 * to the amount we have to send, we may be able to queue it
> + 		 * afterall even though it's not sent to the database it's
> + 		 * ok, any routines that check the data coming from the
> + 		 * database better call pqFlush() anyway.
> + 		 */
> + 		if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
> + 		{
> + 			printfPQExpBuffer(&conn->errorMessage,
> + 				"pqPutBytes --  pqFlush couldn't flush enough"
> + 				" data: space available: %d, space needed %d\n",
> + 				Max(conn->outBufSize - conn->outCount, 0), nbytes);
> + 			return EOF;
> + 		}
> + 	}
> + 
> + 	/* 
> + 	 * is the amount of data to be sent is larger than the size of the
> + 	 * output buffer then we must flush it to make more room.
> + 	 *
> + 	 * the code above will make sure the loop conditional is never 
> + 	 * true for non-blocking connections
> + 	 */
>   	while (nbytes > avail)
>   	{
>   		memcpy(conn->outBuffer + conn->outCount, s, avail);
> ***************
> *** 548,553 ****
> --- 579,592 ----
>   		return EOF;
>   	}
>   
> + 	/* 
> + 	 * don't try to send zero data, allows us to use this function
> + 	 * without too much worry about overhead
> + 	 */
> + 	if (len == 0)
> + 		return (0);
> + 
> + 	/* while there's still data to send */
>   	while (len > 0)
>   	{
>   		/* Prevent being SIGPIPEd if backend has closed the connection. */
> ***************
> *** 556,561 ****
> --- 595,601 ----
>   #endif
>   
>   		int sent;
> + 
>   #ifdef USE_SSL
>   		if (conn->ssl) 
>   		  sent = SSL_write(conn->ssl, ptr, len);
> ***************
> *** 585,590 ****
> --- 625,632 ----
>   				case EWOULDBLOCK:
>   					break;
>   #endif
> + 				case EINTR:
> + 					continue;
>   
>   				case EPIPE:
>   #ifdef ECONNRESET
> ***************
> *** 616,628 ****
>   			ptr += sent;
>   			len -= sent;
>   		}
>   		if (len > 0)
>   		{
>   			/* We didn't send it all, wait till we can send more */
>   
> - 			/* At first glance this looks as though it should block.  I think
> - 			 * that it will be OK though, as long as the socket is
> - 			 * non-blocking. */
>   			if (pqWait(FALSE, TRUE, conn))
>   				return EOF;
>   		}
> --- 658,688 ----
>   			ptr += sent;
>   			len -= sent;
>   		}
> + 
>   		if (len > 0)
>   		{
>   			/* We didn't send it all, wait till we can send more */
> + 
> + 			/* 
> + 			 * if the socket is in non-blocking mode we may need
> + 			 * to abort here 
> + 			 */
> + #ifdef USE_SSL
> + 			/* can't do anything for our SSL users yet */
> + 			if (conn->ssl == NULL)
> + 			{
> + #endif
> + 				if (pqIsnonblocking(conn))
> + 				{
> + 					/* shift the contents of the buffer */
> + 					memmove(conn->outBuffer, ptr, len);
> + 					conn->outCount = len;
> + 					return EOF;
> + 				}
> + #ifdef USE_SSL
> + 			}
> + #endif
>   
>   			if (pqWait(FALSE, TRUE, conn))
>   				return EOF;
>   		}
> Index: src/interfaces/libpq/libpq-fe.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
> retrieving revision 1.55
> diff -u -c -r1.55 libpq-fe.h
> *** src/interfaces/libpq/libpq-fe.h	2000/01/15 05:37:21	1.55
> --- src/interfaces/libpq/libpq-fe.h	2000/01/17 02:35:56
> ***************
> *** 263,268 ****
> --- 263,275 ----
>   	extern int	PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
>   	extern int	PQendcopy(PGconn *conn);
>   
> + 	/* Set blocking/nonblocking connection to the backend */
> + 	extern int	PQsetnonblocking(PGconn *conn, int arg);
> + 	extern int	PQisnonblocking(const PGconn *conn);
> + 
> + 	/* Force the write buffer to be written (or at least try) */
> + 	extern int	PQflush(PGconn *conn);
> + 
>   	/*
>   	 * "Fast path" interface --- not really recommended for application
>   	 * use
> Index: src/interfaces/libpq/libpq-int.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
> retrieving revision 1.16
> diff -u -c -r1.16 libpq-int.h
> *** src/interfaces/libpq/libpq-int.h	2000/01/15 05:37:21	1.16
> --- src/interfaces/libpq/libpq-int.h	2000/01/17 02:35:56
> ***************
> *** 214,219 ****
> --- 214,222 ----
>   	int			inEnd;			/* offset to first position after avail
>   								 * data */
>   
> + 	int			nonblocking;	/* whether this connection is using a blocking
> + 								 * socket to the backend or not */
> + 
>   	/* Buffer for data not yet sent to backend */
>   	char	   *outBuffer;		/* currently allocated buffer */
>   	int			outBufSize;		/* allocated size of buffer */
> ***************
> *** 299,303 ****
> --- 302,312 ----
>   #define strerror(A) (sys_errlist[(A)])
>   #endif	 /* sunos4 */
>   #endif	 /* !strerror */
> + 
> + /* 
> +  * this is so that we can check is a connection is non-blocking internally
> +  * without the overhead of a function call
> +  */
> + #define pqIsnonblocking(conn)	(conn->nonblocking)
>   
>   #endif	 /* LIBPQ_INT_H */
> 
> 
> ************
> 


-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 18 Jan 2000 00:53:50 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [PATCHES] docs done Re: [HACKERS] LIBPQ patches ...
Applied, or did I already say that?


> * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote:
> > Don Baccus <dhogaza@pacifier.com> writes:
> > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> > >> I also object strongly to the lack of documentation.
> > 
> > > ... I know there are some folks who aren't native-english speakers, so
> > > perhaps you don't want to require that the implementor of such patches
> > > provide the final documentation wording.  But the information should
> > > be there and spelled out in a form that can be very easily moved to
> > > the docs.
> > 
> > Oh, absolutely.  Thomas, our master of the docs, has always had the
> > policy of "give me some words, I'll take care of formatting and
> > editing..."
> > 
> > I was probably too harsh on Alfred last night, since in fact his code
> > was fairly well commented, and some minimal doco could have been
> > extracted from the routine headers.  But on a change like this, I think
> > some paragraphs of coherent high-level explanation are needed: what it
> > does, when and why you'd use it.  I didn't see that anywhere...
> 
> Here's the revised patch, it includes sgml docs and changes to
> ensure that old style connections behave the way they are expected
> to:
> 

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 18 Jan 2000 14:09:46 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [PATCHES] docs done Re: [HACKERS] LIBPQ patches ...
> * Bruce Momjian <pgman@candle.pha.pa.us> [000118 11:49] wrote:
> > Applied, or did I already say that?
> 
> Just one mail was sent, but I cc'd patches and hackers as well as
> yourself on the message, sorry for duplicates, but since the mailing
> contained my revised patch I sent to -patches as well.
> 
> I'll be a bit less zealous in the future. :)

No, that is fine.  I usually catch that, but I was not sure in this
case.  Seems I only sent out one, and had not already sent it.  It is
good to hit multiple lists with something that has been discussed this
much.


-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 18 Jan 2000 14:37:31 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [PATCHES] docs done Re: [HACKERS] LIBPQ patches ...
* Bruce Momjian <pgman@candle.pha.pa.us> [000118 11:49] wrote:
> Applied, or did I already say that?

Just one mail was sent, but I cc'd patches and hackers as well as
yourself on the message, sorry for duplicates, but since the mailing
contained my revised patch I sent to -patches as well.

I'll be a bit less zealous in the future. :)

sorry,
-Alfred

Tue, 18 Jan 2000 11:53:04 -0800
Alfred Perlstein <bright@wintelcom.net>
Re: [PATCHES] docs done Re: [HACKERS] LIBPQ patches ...