[Forgive me if you got this already. I don't _think_ it got out last
time]..

Casting your minds back again to the discussion a few months ago. I was
talking about making changes to the fe/be protocol to accomodate the OO
extensions I was talking about. At the time I mentioned interest in
fixing some other things while I was there such as adding a streaming
interface, and perhaps fixing a few other things while I was at it.

Then someone said all the code was going to be discarded anyway and the
protocol moved to Corba. That threw a spanner in the works and I havn't
done anything since because I couldn't get any more details.
 
So here's the question again. Is Corba really a good thing for a
database, seeing as a db is concerned with transferring massive chunks
of
simply formatted data. I'm no Corba guru, but I would have thought (a)
Corba would be not very efficient at that sort of thing, probably adding
big overhead in bytes, and possibly a lot more protocol back and forth,
and (b) isn't the protocol simple enough anyway that Corba is overkill.
 
If you guys convince me that you really are going to move to Corba that
will influence how I approach this. I might even do work to implement
the Corba stuff.

Chris
Ok, I'll broach the subject again, and see where we get....

Casting your minds back again to the discussion a few months ago. I was
talking about making changes to the fe/be protocol to accomodate the OO
extensions I was talking about. At the time I mentioned interest in
fixing some other things while I was there such as adding a streaming
interface, and perhaps fixing a few other things while I was at it.

Then someone said all the code was going to be discarded anyway and the
protocol moved to Corba. That threw a spanner in the works and I havn't
done anything since because I couldn't get any more details.

So here's the question again. Is Corba really a good thing for a
database, seeing as it is concerned with transferring massive chunks of
basic formatted data. I'm no Corba guru, but I would have thought (a)
Corba would be not very efficient at that sort of thing, probably adding
big overhead in bytes, and possibly a lot more protocol back and forth,
and (b) isn't the protocol simple enough anyway that Corba is overkill.

If you guys convince me that you really are going to move to Corba that
will influence how I approach this. I might even do work to implement
the Corba stuff.

Chris.
Fri, 19 May 2000 15:01:59 +1000
Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>
OO / fe-be protocol
Chris Bitmead wrote:
> 
> [Forgive me if you got this already. I don't _think_ it got out last
> time]..
> 
> Casting your minds back again to the discussion a few months ago. I was
> talking about making changes to the fe/be protocol to accomodate the OO
> extensions I was talking about. At the time I mentioned interest in
> fixing some other things while I was there such as adding a streaming
> interface, and perhaps fixing a few other things while I was at it.

I did an clien in pure python for v6.2 an I sure found some things to fix ;)

I also have several other ideas for enchancing it.

So please contact me on this list when you start doing the actual work. 

> Then someone said all the code was going to be discarded anyway and the
> protocol moved to Corba.

Someone was contemplating (maybe even doing _some_ work on) Corba, sorry
but I don't remember who it was.

> So here's the question again. Is Corba really a good thing for a
> database, seeing as a db is concerned with transferring massive chunks
> of simply formatted data.

While it may be a good thing to have a Corba interface to PostgreSQL, 
I don't think it will ever be the main interface.

> I'm no Corba guru, but I would have thought (a)
> Corba would be not very efficient at that sort of thing, probably adding
> big overhead in bytes, and possibly a lot more protocol back and forth,
> and (b) isn't the protocol simple enough anyway that Corba is overkill.

Definitely.

----------------
Hannu

Fri, 19 May 2000 10:38:59 +0300
Hannu Krosing <hannu@tm.ee>
Re: OO / fe-be protocol
Chris Bitmead <chrisb@nimrod.itg.telstra.com.au> writes:
> Then someone said all the code was going to be discarded anyway and the
> protocol moved to Corba. That threw a spanner in the works and I havn't
> done anything since because I couldn't get any more details.

There's been some talk of using Corba, but it's certainly not a done
deal; in fact I don't think anyone's actively working on it right now.
 
> So here's the question again. Is Corba really a good thing for a
> database, seeing as a db is concerned with transferring massive chunks
> of simply formatted data. I'm no Corba guru, but I would have thought
> (a) Corba would be not very efficient at that sort of thing, probably
> adding big overhead in bytes, and possibly a lot more protocol back
> and forth, and (b) isn't the protocol simple enough anyway that Corba
> is overkill.

The attraction of Corba to my mind is that it might save us from the
convert-everything-to-text bottleneck of the current protocol (by
providing cross-platform byte order translation and so forth).  That
should give us a performance boost, hopefully more than enough to cancel
out any added overhead.  I won't be very excited about switching to
Corba if it turns out to be a performance dog compared to what we have.

I'm not a Corba guru (at the moment anyway...) so someone else might be
able to offer a more-informed opinion here.

The alternative is to stick with the present protocol and perhaps try
to sandpaper off some of its uglier corners.  It'd probably be worth
discussing what we might want in that direction, if only so we can get
a feel for how much work would be involved if we go that route rather
than the Corba route.

(Or we could do neither, instead inventing a brand-new protocol that's
still Postgres-only, but that seems like it has no particular
attraction... there's a lot of work invested in the current frontends
and if we're going to throw it away we probably ought to adopt a
standards-based protocol.  IMHO anyway.)

			regards, tom lane

Fri, 19 May 2000 16:50:31 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: OO / fe-be protocol 
> Ok, I'll go back to reading about Corba and see if I can figure out if
> it can do the job.

It can, and it is appropriate.

The devil is in the details, which include concerns on portability of
the ORB among our > 20 platforms, additional levels of complexity for
the minimum, small installation (Naming Service, etc etc), and general
unfamiliarity with CORBA. I'm sure there are other concerns too.

I've got some experience with C++ ORBs (TAO and Mico), but am not
familiar with the C mapping and how clean it may or may not be.

The "transform only if necessary" philosophy of CORBA (that is,
recipients are responsible for changing byte order if required, but do
not if not) should minimize overhead. And the support for dynamic data
definition and data handling should be a real winner, at least for
communications to outside the server. Inside the server it could help
us clean up our interfaces, and start thinking about distributing
portions onto multiple platforms. Should be fun :)

                      - Thomas

-- 
Thomas Lockhart				lockhart@alumni.caltech.edu
South Pasadena, California

Sat, 20 May 2000 04:08:28 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Re: OO / fe-be protocol
Tom Lane wrote:

> I'm not a Corba guru (at the moment anyway...) so someone else might be
> able to offer a more-informed opinion here.

Ok, I'll go back to reading about Corba and see if I can figure out if
it can do the job.

> (Or we could do neither, instead inventing a brand-new protocol that's
> still Postgres-only, but that seems like it has no particular
> attraction... there's a lot of work invested in the current frontends
> and if we're going to throw it away we probably ought to adopt a
> standards-based protocol.  IMHO anyway.)

But if Corba is not appropriate, what else is there?

Sat, 20 May 2000 18:48:47 +1000
Chris <chris@bitmead.com>
Re: OO / fe-be protocol
As an alternative you might consider XDR (RFC 1832, RFC 1014).

http://landfield.com/rfcs/rfc1832.html
http://landfield.com/rfcs/rfc1014.html

We are using this since many years. It is quite effective. XDR is available
on a lot of platforms (on QNX too).

In short:
The data structures/unions to be transfered between different platforms must
be defined in a normal C header. This header has to be proceeded by the tool
rpcgen. This rpcgen tool generates encoding/decoding functions for these
structures. These functions can than be called in the code before send and
after receive to encode/decode message buffers.

No things like ORBs etc. are required.

Regards,
Andreas Kardos

-----Ursprï¿½ngliche Nachricht-----
Von: Tom Lane <tgl@sss.pgh.pa.us>
An: Chris Bitmead <chrisb@nimrod.itg.telstra.com.au>
Cc: <pgsql-hackers@postgreSQL.org>
Gesendet: Freitag, 19. Mai 2000 22:50
Betreff: Re: [HACKERS] OO / fe-be protocol


> (Or we could do neither, instead inventing a brand-new protocol that's
> still Postgres-only, but that seems like it has no particular
> attraction... there's a lot of work invested in the current frontends
> and if we're going to throw it away we probably ought to adopt a
> standards-based protocol.  IMHO anyway.)
>
> regards, tom lane
>


Mon, 22 May 2000 11:47:07 +0200
"Kardos, Dr. Andreas" <kardos@repas-aeg.de>
Re: OO / fe-be protocol 
> I've been doing a little reading on Corba. It seems like a tuple would
> have to be returned as something like a sequence of any. But I believe
> that any might not be as efficient as one would like. Have you had any
> thoughts about what the interface should look like?

Someone put some CORBA stuff into src/corba/, including two files from
OMG which define a query interface afaict. At one level we would
certainly want to implement that, and then perhaps also implement a
lower-level interface which is more specific to Postgres.

One problem/feature with CORBA is that long responses are usually
handled with an interator interface, which requires more handshaking
than our current "streaming" interface. otoh, it is more robust, in the
sense that clients and servers have some control over their local
resources such as memory.

                    - Thomas

Wed, 31 May 2000 15:41:34 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Re: OO / fe-be protocol