FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: Pickett, David
> Sent: Thursday, December 08, 2005 9:40 AM
> 
> If it is not a flat file, seeks and mmap are both out, so you are
> going to need space somewhere in a file system: swap, /tmp, /var/tmp
> or the like.  It seems silly to have to check available space in
> each arena to pick one and have code for all three or more.  You
> could specify workspace by an argument or environmental variable,
> like sort.  However, virtual memory is rarely scarce any more, so
> if not a flat file, realloc buffer seems like a good choice that
> will only get better.
> 
> You only get messed up if they have both a pipe and huge fields,
> and then there is nowhere to hide on the client side!

Hi David, 

'Tis true.  I thought tmpfile(3) was controlled by TMPDIR.  It is in
NetBSD, but
http://www.opengroup.org/onlinepubs/009695399/xrat/xbd_chap10.html says
only "Implementations are encouraged to provide suitable directory names
in the environment variable TMPDIR".  (BTW, the standard sort(1) doesn't
provide for user-determined temp space, either.  I remember trying a
variety of sort utilities on a 16-million line file.  The ones that
didn't segfault took forever.  One might have thought that's a solved
problem in this day and age.) 

That said, I think it's a lot to ask of a virtual memory system to
reliably handle gigabytes of space for buffering.  A temporary file is
much more straightforward, and, while it may tax the client's disk
somewhat, will still be oodles faster than the server.  How many oodles
will of course depend on the client.  

IMO it's reasonable to require someone bcping huge PDFs into a database
to use a client with sufficient space in /tmp or wherever such that
tmpfile(3) won't fail for any single field.  You could say, and I might
agree, that if input isn't a pipe, there's no need for a temporary file.
My inclination is to make one anyway, because the code will be simpler.
Simpler code is more robust, and robustness trumps parsimony every time.


On the other hand, that's just a noodle, and working code trumps
noodles.  Patches welcome. :-)

Regards, 

--jkl



-----------------------------------------
The information contained in this transmission may be privileged and
confidential and is intended only for the use of the person(s) named
above. If you are not the intended recipient, or an employee or agent 
responsible
for delivering this message to the intended recipient, any review, 
dissemination,
distribution or duplication of this communication is strictly prohibited. If 
you are
not the intended recipient, please contact the sender immediately by reply 
e-mail
and destroy all copies of the original message. Please note that we do not 
accept
account orders and/or instructions by e-mail, and therefore will not be 
responsible
for carrying out such orders and/or instructions.  If you, as the intended 
recipient
of this message, the purpose of which is to inform and update our clients, 
prospects
and consultants of developments relating to our services and products, would 
not
like to receive further e-mail correspondence from the sender, please "reply" 
to the
sender indicating your wishes.  In the U.S.: 1345 Avenue of the Americas, New 
York,
NY 10105.