FreeTDS Development Group

["James K. Lowden" <jklowden AT freetds.org>]

Frediano Ziglio wrote:
> > I need to understand this very clearly to be useful to you.  I've
> > never seen the data stream.  What does it look like?  How is it
> > chunked?  
> 
> Well... assume you are returning a string 'hello!'
> with varchar(100)
>  Data info
>   type XSYBVARCHAR
>   len (16bit) 100
>  Data
>   len (16bit) 6
>   string 'hello!'
> with varchar(max)
>  Data info
>   type XSYBVARCHAR
>   len (16bit) -1
>  Data
>   total len (64bit) 6
>   chunk len (32bit) 6
>   string 'hello!'
>   chunk len (32bit) 0
> 
> note that type has same constant. 

Thanks, that's clear.  Is it the same for bcp in?  

Suggestion
----------

1.  As soon as XSYBVARCHAR arrives from the server, check the length.  If
-1, set coltype to XSYBMAXCHAR.  That tells the rest of the system the
column is really varchar(max) and lets is_blob_type work (if that's what
we want).  

2.  Like BLOB, allocate space according to total len.  Initialize textptr
to 0xbadfaced.  

3.  Read in all chunks, continue normally.  

> > My idea is to synthesize a textptr in libtds to make varchar(max) look
> > like TEXT to the client libraries.  I'm not worried about wasting a
> > few bytes, especially because -- if we're lucky -- the client
> > libraries won't have to change.  
> 
> Yes, TDSBLOB reuse is feasible, I don't think ODBC will have problems,
> problems raise in dblib where textptr is important to use in
> READTEXT/WRITETEXT. 

Right you are.  We can't synthesize textptr because it belongs to the
server.  :-(  

It's OK for now if db-lib has no way to update this kind of column, as
long as it can fetch it.  The user application always construct SQL to
update the column if need be.  

I don't think there's a great demand for extending db-lib.  But there's no
technical reason we couldn't.  We have dbdata(); we could add dbputdata()
along the lines of SQLPutData.  We'd have to add parameterized queries
first.... 

> I think is also the right time to add support for variant! This strange
> type is a sort of "any" type that is a single column can contain
> floats/integers/strings and so on. In its data representation it
> contains the real type used. This will require a new structure like
> TDSBLOB to handle these additional informations (it can't contain LOBs
> but allow varchar(8000) to be contained).  

Do you really think anyone cares about variants?  About the only place
variants are useful after you fetch them is in Visual Basic.  

I think they're really *dumb* from a relational calculus point of view. 
The whole point of predicate logic is to say something verifiable about
the key.  What kind of attribute is sometimes a number, sometimes a
string, and sometimes a date?  

If you want one man's opinion, I think a much better use of your
exceptional talents would be to add bcp to ODBC.  You'd surely make more
people happier that way.  (Including me!)  

> Another problem is the space wasted if table contains a lot of
> varchar(N) where N is quite big. 

I just don't think that is a big problem.  It's very rare that a row has
so many giant varchar columns.  You'd need 10 tables to get 80,000 bytes,
because each table's row can be only 8,060 bytes.  

A much bigger problem is that we allocate memory for BLOBs when fetching,
and really big blobs exhaust virtual memory pretty easily. I think we'd be
smarter to cache the row in a temporary file and seek on that if the data
are too big, say, over 1 MB.  

Regards, 

--jkl