Hi All,

I'm developing (currently in pre-alfa stage) a Acucobol interface for the 
Postgresql.
The Acucobol runtime have a generic FS API interface that handle the work 
with the
record oriented files, defining the open, close, read, write and so on low 
level function I can
extend the runtime to talk with any file and database.

My current work translate each Acucobol FS command in a relative Postgresql 
query and
the returned tuple will be translated in a record oriented view.
After some performance tests I've notice that this path have much overhead 
and because
this I was thinking to redesign the interface.

My first think was to bypass the SQL translation and use the Postgresql low 
level routines.
I need to see the tables as record oriented archive, so I can scan 
sequentially (forward and
backward) each record, lock/unlock it, insert and delete it and start to 
read the records with
a match of a specific key.

Does anyone know where can I start to search/read/learn/study some 
document/code of the
Postgresql low level routines ?

If need some detail, please ask ;-)!

Thanks in advance.


Roberto Fichera.


Thu, 31 May 2001 18:26:17 +0200
Roberto Fichera <robyf@tekno-soft.it>
Acucobol interface
Roberto Fichera <robyf@tekno-soft.it> writes:
> My first think was to bypass the SQL translation and use the Postgresql low 
> level routines.
> I need to see the tables as record oriented archive, so I can scan 
> sequentially (forward and
> backward) each record, lock/unlock it, insert and delete it and start to 
> read the records with
> a match of a specific key.

I don't think you want an SQL database at all.  Possibly something like
Sleepycat's Berkeley DB package is closer to what you are looking for...

			regards, tom lane

Tue, 05 Jun 2001 18:13:41 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Acucobol interface 
At 18.13 05/06/01 -0400, Tom Lane wrote:

>Roberto Fichera <robyf@tekno-soft.it> writes:
> > My first think was to bypass the SQL translation and use the Postgresql 
> low
> > level routines.
> > I need to see the tables as record oriented archive, so I can scan
> > sequentially (forward and
> > backward) each record, lock/unlock it, insert and delete it and start to
> > read the records with
> > a match of a specific key.
>
>I don't think you want an SQL database at all.  Possibly something like
>Sleepycat's Berkeley DB package is closer to what you are looking for...
>
>                         regards, tom lane

I know the Sleepycat's Berkelay DB packages but isn't what I need.
I need a relational database that can be used outside our Acucobol program,
like Excel, Access, Apache and in general a SQL view of our data for external
analysis and presentation. This is why I'm thinking to use SQL and in 
particular
the PostgreSQL. Currently there is only one direct interface from Acucobol and
a SQL server and was developed by DBMaker for their SQL server, but have
some limitation that I want bypass.

regards,


Roberto Fichera.


Wed, 06 Jun 2001 09:46:31 +0200
Roberto Fichera <kernel@tekno-soft.it>
Re: Acucobol interface 
Roberto Fichera wrote:
> 
> Hi All,
> 
> I'm developing (currently in pre-alfa stage) a Acucobol interface for the
> Postgresql.
> The Acucobol runtime have a generic FS API interface that handle the work
> with the
> record oriented files, defining the open, close, read, write and so on low
> level function I can
> extend the runtime to talk with any file and database.
> 
> My current work translate each Acucobol FS command in a relative Postgresql
> query and
> the returned tuple will be translated in a record oriented view.
> After some performance tests I've notice that this path have much overhead
> and because
> this I was thinking to redesign the interface.
> 
> My first think was to bypass the SQL translation and use the Postgresql low
> level routines.
> I need to see the tables as record oriented archive, so I can scan
> sequentially (forward and
> backward) each record, lock/unlock it, insert and delete it and start to
> read the records with
> a match of a specific key.
> 
> Does anyone know where can I start to search/read/learn/study some
> document/code of the
> Postgresql low level routines ?
> 
> If need some detail, please ask ;-)!
> 
> Thanks in advance.
> 
> Roberto Fichera.

What you are looking for is a very powerful database back-end, as Tom Lane
suggests, something like Berkeley DB might do, but then what you want is a SQL
interface over that.

I am reticent to admit that I have done a little COBOL and the interface for
data paradigms is very good for a dBase like package. If you can live without a
client/server interface ala Postgres, and can live with a file based access
methodology, then what you want is doable.

I'm not aware what platform you wish to run your program, I am assuming
Windows. The old dBase format is currently being used under the name "xbase."
There are many libraries that conform to this file format and offer the type of
access which you wish to have. On top of that, there are ODBC drivers (in UNIX
and Windows, btw) for these xBase files.

You write acucobol extensions using some generic xbase access layer, and use
the ODBC xbase driver for applications like Access and Excel.

You'll have to sort out all the issues like concurrent access, and stuff like
that, but it should come pretty close to what you want to do.

Thu, 07 Jun 2001 08:03:55 -0400
mlw <markw@mohawksoft.com>
Re: Acucobol interface
At 08.03 07/06/01 -0400, mlw wrote:
>Roberto Fichera wrote:
> >
> > Hi All,
> >
> > I'm developing (currently in pre-alfa stage) a Acucobol interface for the
> > Postgresql.
> > The Acucobol runtime have a generic FS API interface that handle the work
> > with the
> > record oriented files, defining the open, close, read, write and so on low
> > level function I can
> > extend the runtime to talk with any file and database.
> >
> > My current work translate each Acucobol FS command in a relative Postgresql
> > query and
> > the returned tuple will be translated in a record oriented view.
> > After some performance tests I've notice that this path have much overhead
> > and because
> > this I was thinking to redesign the interface.
> >
> > My first think was to bypass the SQL translation and use the Postgresql low
> > level routines.
> > I need to see the tables as record oriented archive, so I can scan
> > sequentially (forward and
> > backward) each record, lock/unlock it, insert and delete it and start to
> > read the records with
> > a match of a specific key.
> >
> > Does anyone know where can I start to search/read/learn/study some
> > document/code of the
> > Postgresql low level routines ?
> >
> > If need some detail, please ask ;-)!
> >
> > Thanks in advance.
> >
> > Roberto Fichera.
>
>What you are looking for is a very powerful database back-end, as Tom Lane
>suggests, something like Berkeley DB might do, but then what you want is a SQL
>interface over that.

I've already evaluated the Berkeley DB interface, and surely it's a good
interface, it maybe superior of the current Acucobol proprietary format
when the archive is very large (several Gb).

>I am reticent to admit that I have done a little COBOL and the interface for
>data paradigms is very good for a dBase like package. If you can live 
>without a
>client/server interface ala Postgres, and can live with a file based access
>methodology, then what you want is doable.

The main problem is that we want see our data as relational database
and we want continue to use the current programs. Currently we have
some customer that have their company archive large around 50Gb.

>I'm not aware what platform you wish to run your program, I am assuming
>Windows. The old dBase format is currently being used under the name "xbase."
>There are many libraries that conform to this file format and offer the 
>type of
>access which you wish to have. On top of that, there are ODBC drivers (in UNIX
>and Windows, btw) for these xBase files.

Acucobol runtime, currently is present in around 650 different platform 
(HW/SW)
so we can run the same programs in different environment. We use Linux
and WNT/W2K as server and W9x/WME as client. Use the xBase format
isn't a good choice when we have a several Gb of data, this is why I'm 
thinking
to the PostgreSQL. The current Acucobol's "flat file" isn't adequate to manage
such large files, we need a way to see that files as relational DB.

>You write acucobol extensions using some generic xbase access layer, and use
>the ODBC xbase driver for applications like Access and Excel.
>
>You'll have to sort out all the issues like concurrent access, and stuff like
>that, but it should come pretty close to what you want to do.

I have already done some work. I've implemented an extension of the generic
Acucobol FS layer that talk with a PostgreSQL. This lowlevel layer 
translate each
FS primitive in a query. The acucobol's record is translated in attribute 
(and vice versa)
using a XFD file (eXtended Fields Description) which describe each record's 
field and
that is cached in memory. This file is generated by the acucobol compiler 
for each used file.
With this informations I'm able to perform a complete query to the 
PostgreSQL, the
returned tuples will be translated in the expected "flat record" and 
finally returned to the
runtime for its work.

I know, the cobol use a different philosophy than a relational DB but this 
my work
show that this two different world could talk. Also, I know the difficulty 
of the major lowlevel
integration. I need some doc/indication/files-to-read of PostgreSQL 
lowlevel routines
to see if this different world could have a major integration bypassing the 
"overhead" of
the query accessing directly to the DB.


Roberto Fichera.


Thu, 07 Jun 2001 20:21:36 +0200
Roberto Fichera <kernel@tekno-soft.it>
Re: Re: Acucobol interface
Roberto Fichera wrote:

>
> >I am reticent to admit that I have done a little COBOL and the interface for
> >data paradigms is very good for a dBase like package. If you can live
> >without a
> >client/server interface ala Postgres, and can live with a file based access
> >methodology, then what you want is doable.
>
> The main problem is that we want see our data as relational database
> and we want continue to use the current programs. Currently we have
> some customer that have their company archive large around 50Gb.

In what format is this data?

> >I'm not aware what platform you wish to run your program, I am assuming
> >Windows. The old dBase format is currently being used under the name "xbase."
> >There are many libraries that conform to this file format and offer the
> >type of
> >access which you wish to have. On top of that, there are ODBC drivers (in UNIX
> >and Windows, btw) for these xBase files.
>
> Acucobol runtime, currently is present in around 650 different platform
> (HW/SW)
> so we can run the same programs in different environment. We use Linux
> and WNT/W2K as server and W9x/WME as client. Use the xBase format
> isn't a good choice when we have a several Gb of data, this is why I'm
> thinking
> to the PostgreSQL. The current Acucobol's "flat file" isn't adequate to manage
> such large files, we need a way to see that files as relational DB.

Just out of curiosity, why is the xbase format not a good choice?

> >You write acucobol extensions using some generic xbase access layer, and use
> >the ODBC xbase driver for applications like Access and Excel.
> >
> >You'll have to sort out all the issues like concurrent access, and stuff like
> >that, but it should come pretty close to what you want to do.

I think you are missing the point. A good xbase library will allow you to perform
"joins" on data across tables. It doesn't have a SQL syntax, but that does not
mean you can't code that way.

Also, ODBC drivers for xbase use SQL format queries.

> I have already done some work. I've implemented an extension of the generic
> Acucobol FS layer that talk with a PostgreSQL. This lowlevel layer
> translate each
> FS primitive in a query. The acucobol's record is translated in attribute
> (and vice versa)
> using a XFD file (eXtended Fields Description) which describe each record's
> field and
> that is cached in memory. This file is generated by the acucobol compiler
> for each used file.
> With this informations I'm able to perform a complete query to the
> PostgreSQL, the
> returned tuples will be translated in the expected "flat record" and
> finally returned to the
> runtime for its work.
>
> I know, the cobol use a different philosophy than a relational DB but this
> my work
> show that this two different world could talk. Also, I know the difficulty
> of the major lowlevel
> integration. I need some doc/indication/files-to-read of PostgreSQL
> lowlevel routines
> to see if this different world could have a major integration bypassing the
> "overhead" of
> the query accessing directly to the DB.

I think, strongly, you are going down the wrong track.  Take a look at:

http://sourceforge.net/projects/xdb/
http://www.unixodbc.org/



>
>
> Roberto Fichera.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/users-lounge/docs/faq.html


Thu, 07 Jun 2001 15:38:17 -0400
mlw <markw@mohawksoft.com>
Re: Acucobol interface
At 15.38 07/06/01 -0400, mlw wrote:

>Roberto Fichera wrote:
>
> >
> > >I am reticent to admit that I have done a little COBOL and the 
> interface for
> > >data paradigms is very good for a dBase like package. If you can live
> > >without a
> > >client/server interface ala Postgres, and can live with a file based 
> access
> > >methodology, then what you want is doable.
> >
> > The main problem is that we want see our data as relational database
> > and we want continue to use the current programs. Currently we have
> > some customer that have their company archive large around 50Gb.
>
>In what format is this data?

It's a VISION format, a proprietary variant of a B*Tree format.

> > >I'm not aware what platform you wish to run your program, I am assuming
> > >Windows. The old dBase format is currently being used under the name 
> "xbase."
> > >There are many libraries that conform to this file format and offer the
> > >type of
> > >access which you wish to have. On top of that, there are ODBC drivers 
> (in UNIX
> > >and Windows, btw) for these xBase files.
> >
> > Acucobol runtime, currently is present in around 650 different platform
> > (HW/SW)
> > so we can run the same programs in different environment. We use Linux
> > and WNT/W2K as server and W9x/WME as client. Use the xBase format
> > isn't a good choice when we have a several Gb of data, this is why I'm
> > thinking
> > to the PostgreSQL. The current Acucobol's "flat file" isn't adequate to 
> manage
> > such large files, we need a way to see that files as relational DB.
>
>Just out of curiosity, why is the xbase format not a good choice?

Because the DBF format didn't perform well on a large archive of serveral 
million
records. It's don't reuse the deleted records, have a limitation of 255 
fields and
the char() field can be max 255 char in length and the max record size is 4k.

> > >You write acucobol extensions using some generic xbase access layer, 
> and use
> > >the ODBC xbase driver for applications like Access and Excel.
> > >
> > >You'll have to sort out all the issues like concurrent access, and 
> stuff like
> > >that, but it should come pretty close to what you want to do.
>
>I think you are missing the point. A good xbase library will allow you to 
>perform
>"joins" on data across tables. It doesn't have a SQL syntax, but that does not
>mean you can't code that way.
>
>Also, ODBC drivers for xbase use SQL format queries.

I know.

> > I have already done some work. I've implemented an extension of the generic
> > Acucobol FS layer that talk with a PostgreSQL. This lowlevel layer
> > translate each
> > FS primitive in a query. The acucobol's record is translated in attribute
> > (and vice versa)
> > using a XFD file (eXtended Fields Description) which describe each record's
> > field and
> > that is cached in memory. This file is generated by the acucobol compiler
> > for each used file.
> > With this informations I'm able to perform a complete query to the
> > PostgreSQL, the
> > returned tuples will be translated in the expected "flat record" and
> > finally returned to the
> > runtime for its work.
> >
> > I know, the cobol use a different philosophy than a relational DB but this
> > my work
> > show that this two different world could talk. Also, I know the difficulty
> > of the major lowlevel
> > integration. I need some doc/indication/files-to-read of PostgreSQL
> > lowlevel routines
> > to see if this different world could have a major integration bypassing the
> > "overhead" of
> > the query accessing directly to the DB.
>
>I think, strongly, you are going down the wrong track.

Could be ;-)! But I want try to do some work before abandon this
solution.

>Take a look at:
>
>http://sourceforge.net/projects/xdb/

I've take a look on it, and I'd the confirm of the aspected limitations. 
Another
problem of this library is that we can't perform concurrent locks with other
applications different than xdb library. So it's unusable for me.

>http://www.unixodbc.org/

I know it, it's good link!


Roberto Fichera.


Fri, 08 Jun 2001 10:44:48 +0200
Roberto Fichera <kernel@tekno-soft.it>
Re: Re: Acucobol interface