

On Linux now exist project for raw I/O device 
(http://oss.sgi.com/projects/rawio/). Exist any plan (for far future)
with raw device for PgSQL? (TODO be quiet for this.)

						Karel

----------------------------------------------------------------------
Karel Zak <zakkr@zf.jcu.cz>              http://home.zf.jcu.cz/~zakkr/

Docs:        http://docs.linux.cz                    (big docs archive)	
Kim Project: http://home.zf.jcu.cz/~zakkr/kim/        (process manager)
FTP:         ftp://ftp2.zf.jcu.cz/users/zakkr/        (C/ncurses/PgSQL)
-----------------------------------------------------------------------


Mon, 6 Dec 1999 10:59:23 +0100 (CET)
Karel Zak - Zakkr <zakkr@zf.jcu.cz>
RAW I/O device
> On Linux now exist project for raw I/O device
> (http://oss.sgi.com/projects/rawio/). Exist any plan (for far future)
> with raw device for PgSQL? (TODO be quiet for this.)

    Up to now we kept the storage manager overhead in the system.
    Actually there is no way to tell which storage manager to use
    for a particular table/index, so anything goes to the default
    which is the magnetic disk one that  uses  single  files  for
    each relation.

    There   was  a  discussion  about  simplifying  it,  but  the
    consensus was to let it as is because it is the  base  for  a
    tablespace and/or raw device manager.

    AFAIK,  noone  is  working  on  it,  so it must be really FAR
    future. But the plan is still alive.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#========================================= wieck@debis.com (Jan Wieck) #



Mon, 6 Dec 1999 12:40:12 +0100 (MET)
wieck@debis.com (Jan Wieck)
Re: [HACKERS] RAW I/O device



On Mon, 6 Dec 1999, Jan Wieck wrote:

> > On Linux now exist project for raw I/O device
> > (http://oss.sgi.com/projects/rawio/). Exist any plan (for far future)
> > with raw device for PgSQL? (TODO be quiet for this.)
> 
>     Up to now we kept the storage manager overhead in the system.
>     Actually there is no way to tell which storage manager to use
>     for a particular table/index, so anything goes to the default
>     which is the magnetic disk one that  uses  single  files  for
>     each relation.
> 
>     There   was  a  discussion  about  simplifying  it,  but  the
>     consensus was to let it as is because it is the  base  for  a
>     tablespace and/or raw device manager.
> 
>     AFAIK,  noone  is  working  on  it,  so it must be really FAR
>     future. But the plan is still alive.

 I raise the question, because the linux kernel opening with raw-device  
new way for a faster and better database engine. I know (and agree) 
that it not is priority for next year(s?). But it is interesting, and
is prabably good remember it during development, and not write (in future)
features which close this good way. 

						Karel
 



Mon, 6 Dec 1999 13:47:18 +0100 (CET)
Karel Zak - Zakkr <zakkr@zf.jcu.cz>
Re: [HACKERS] RAW I/O device
I may be out of place on this one,  but remember that postgres runs on
other systems besides linux.  Making the DB work w/ one FS (and write the
storage code for it) seems pointless if we are still stuck on normal FSs
on other machines.  

- merlin

> > On Linux now exist project for raw I/O device
> > (http://oss.sgi.com/projects/rawio/). Exist any plan (for far future)
> > with raw device for PgSQL? (TODO be quiet for this.)
> 
>     Up to now we kept the storage manager overhead in the system.
>     Actually there is no way to tell which storage manager to use
>     for a particular table/index, so anything goes to the default
>     which is the magnetic disk one that  uses  single  files  for
>     each relation.
> 
>     There   was  a  discussion  about  simplifying  it,  but  the
>     consensus was to let it as is because it is the  base  for  a
>     tablespace and/or raw device manager.
> 
>     AFAIK,  noone  is  working  on  it,  so it must be really FAR
>     future. But the plan is still alive.




                    ------++++======++++------
 Smith Computer Lab Administrator,  Case Western Reserve University
     bap@scl.cwru.edu | 216.368.5066 | http://home.cwru.edu/~bap
                    ------++++======++++------



Mon, 6 Dec 1999 08:03:25 -0500 (EST)
merlin@scl.cwru.edu
Re: [HACKERS] RAW I/O device
> other systems besides linux.  Making the DB work w/ one FS (and write the
> storage code for it) seems pointless if we are still stuck on normal FSs
> on other machines.

	Yes, of course, but storing databases directly to RAW device - not
through the filesystem - is one feature of modern DB engines...

--------------------------------------------------------------------------
Ing. Pavel Janousek (PaJaSoft)                   FoNet, spol. s r. o.
Vyvoj software, sprava siti, Unix, Web, Y2K    Anenska 11, 602 00  Brno
E-mail: mailto:Janousek@FoNet.Cz               Tel.: +420  5  4324 4749
SMS: mailto:P.Janousek@SMS.Paegas.Cz           Fax.: +420  5  4324 4751
WWW: http://WWW.FoNet.Cz/                    E-mail:
mailto:Info@FoNet.Cz
--------------------------------------------------------------------------

Mon, 06 Dec 1999 14:26:07 +0100
"Ing. Pavel PaJaSoft Janousek" <pajasoft@fonet.cz>
Re: [HACKERS] RAW I/O device
On Mon, 6 Dec 1999, Ing. Pavel PaJaSoft Janousek wrote:

> > other systems besides linux.  Making the DB work w/ one FS (and write the
> > storage code for it) seems pointless if we are still stuck on normal FSs
> > on other machines.
> 
> 	Yes, of course, but storing databases directly to RAW device - not
> through the filesystem - is one feature of modern DB engines...

Actually, Oracle has been moving *away* from this...more recent versions
of Oracle recommend using the Operating System file systems, since, in
most cases, the Operating System does a better job, and its too difficult
to have Oracle itself optimize internal for all the different variants
that it supports....

At work, we use Oracle extensively, and I sat down one day last year with
our Oracle DBA to discuss exactly this...and, if I recall correctly, it
was prompted by a similar thread here...

If Linux is providing an Interface into the RAW file system, then this may
change things for Oracle, since it wouldn't have to "learn" all the
different OSs, as long as the API is the same across them all...and, my
experience with Linux is that the API for Linux will most likely be
different then everyone else *roll eyes*

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


Mon, 6 Dec 1999 10:44:55 -0400 (AST)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] RAW I/O device
The Hermit Hacker writes:
> If Linux is providing an Interface into the RAW file system, then this may
> change things for Oracle, since it wouldn't have to "learn" all the
> different OSs, as long as the API is the same across them all...and, my
> experience with Linux is that the API for Linux will most likely be
> different then everyone else *roll eyes*

The system admin side is slightly different from others: raw devices
have their own major number and devices, /dev/rawN and you "bind" a
raw device to any existing block device /dev/blockdev by doing
    # raw /dev/rawN /dev/blockdev

However, the DBA side (or software) side isn't any different: provided
you access /dev/rawN *only* in sector chunks (i.e. multiples of 512
bytes that are 512-byte aligned) then the software doesn't care
whether it's /dev/rawN or an ordinary block device. If PostgreSQL can
guarantee (or be tweaked/enhanced to guarantee) that it only ever
reads/writes in multiples of 512 byte chunks and never does anything
"weird" (truncates, file-specific, ioctls, mmap, needing O_CREAT to
start with etc.) then it should be perfectly happy when presented with
a /dev/rawN instead of an ordinary file.

--Malcolm

-- 
Malcolm Beattie <mbeattie@sable.ox.ac.uk>
Unix Systems Programmer
Oxford University Computing Services

Mon, 6 Dec 1999 17:25:29 +0000 (GMT)
Malcolm Beattie <mbeattie@sable.ox.ac.uk>
Re: [HACKERS] RAW I/O device
Sorry for the previous note...I'd tried stopping and restarting
postmaster and that didn't help, so I posted - this web site
gets low but steady use and was suddenly acting weird on me.

Anyway, a quick look at the source made it clear that Postgres
wasn't at fault, as the node's a T_Query and apply_RIR_view
clearly handles it.

So, I rebooted linux and it works fine now.  

Sorry to bother folks...



- Don Baccus, Portland OR <dhogaza@pacifier.com>
  Nature photos, on-line guides, Pacific Northwest
  Rare Bird Alert Service and other goodies at
  http://donb.photo.net.

Mon, 06 Dec 1999 11:52:10 -0800
Don Baccus <dhogaza@pacifier.com>
Re: view dying out of the blue...
>  I raise the question, because the linux kernel opening with raw-device  
> new way for a faster and better database engine. I know (and agree) 
> that it not is priority for next year(s?). But it is interesting, and
> is prabably good remember it during development, and not write (in future)
> features which close this good way. 

I would be very surprised to see any significant change in raw vs.
filesystem i/o on modern file systems, and I am sorry, but Linux ext2
does not count as modern.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 6 Dec 1999 22:24:24 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] RAW I/O device
> On Mon, 6 Dec 1999, Ing. Pavel PaJaSoft Janousek wrote:
> 
> > > other systems besides linux.  Making the DB work w/ one FS (and write the
> > > storage code for it) seems pointless if we are still stuck on normal FSs
> > > on other machines.
> > 
> > 	Yes, of course, but storing databases directly to RAW device - not
> > through the filesystem - is one feature of modern DB engines...
> 
> Actually, Oracle has been moving *away* from this...more recent versions
> of Oracle recommend using the Operating System file systems, since, in
> most cases, the Operating System does a better job, and its too difficult
> to have Oracle itself optimize internal for all the different variants
> that it supports....


Ding, ding, ding.  Give that man a cigar.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 6 Dec 1999 22:28:49 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] RAW I/O device

On Mon, 6 Dec 1999, Bruce Momjian wrote:

> >  I raise the question, because the linux kernel opening with raw-device  
> > new way for a faster and better database engine. I know (and agree) 
> > that it not is priority for next year(s?). But it is interesting, and
> > is prabably good remember it during development, and not write (in future)
> > features which close this good way. 
> 
> I would be very surprised to see any significant change in raw vs.
> filesystem i/o on modern file systems, and I am sorry, but Linux ext2
> does not count as modern.

Yes. The ext2's limitation and unavailable is public secret and use it for
raw is crazy idea. On a raw device can be implement specific data organization
(specific for DB demand). Raw's advantage is non-universal organization.

A raw is not only about filesystem, this feature remove full control from 
OS kernel to DB (example data caching - kernel not has information 
how/why/what remove to cache but DB has this information... etc).

						Karel     




   


Tue, 7 Dec 1999 12:55:03 +0100 (CET)
Karel Zak - Zakkr <zakkr@zf.jcu.cz>
Re: [HACKERS] RAW I/O device