FreeTDS Development Group

[Frediano Ziglio <freddy77 AT gmail.com>]

Il giorno mer, 19/03/2008 alle 18.03 -0400, James K. Lowden ha scritto:
> ZIGLIO, Frediano, VF-IT wrote:
> > > Of course we'll need to add XSYBVARCHARMAX or something like that.  
> > > 
> > 
> > This is the problem, there is no XSYBVARCHARMAX but XSYBVARCHAR is 2
> > byte long for varchar(N) where N <= 8000 while is "chunked" when
> > column_size is -1...
> 
> I need to understand this very clearly to be useful to you.  I've never
> seen the data stream.  What does it look like?  How is it chunked?  
> 

Well... assume you are returning a string 'hello!'
with varchar(100)
 Data info
  type XSYBVARCHAR
  len (16bit) 100
 Data
  len (16bit) 6
  string 'hello!'
with varchar(max)
 Data info
  type XSYBVARCHAR
  len (16bit) -1
 Data
  total len (64bit) 6
  chunk len (32bit) 6
  string 'hello!'
  chunk len (32bit) 0

note that type has same constant. The chunked part can be different like

 Data
  total len (64bit) -2 (unknown len)
  chunk len (32bit) 6
  string 'hello!'
  chunk len (32bit) 0

or

 Data
  total len (64bit) 6
  chunk len (32bit) 3
  string 'hel'
  chunk len (32bit) 3
  string 'lo!'
  chunk len (32bit) 0

> > So is_blob_type does not work. I agree that varchar(max) is a LOB but
> > has no textptr/timestamp so use TDSBLOB structure waste some space and
> > confuse programs.
> 
> My idea is to synthesize a textptr in libtds to make varchar(max) look
> like TEXT to the client libraries.  I'm not worried about wasting a few
> bytes, especially because -- if we're lucky -- the client libraries won't
> have to change.  
> 
> Maybe I should work from the outside in.  How does ODBC (or native client)
> handle varchar(max)?  Are you concerned that by converting varchar(max) to
> TEXT, client programs will not get the datatype they expect?  
> 

Yes, TDSBLOB reuse is feasible, I don't think ODBC will have problems,
problems raise in dblib where textptr is important to use in
READTEXT/WRITETEXT. However is_blob_type(col->column_type) is not
sufficient for XSYBVARCHAR. 
I think is also the right time to add support for variant! This strange
type is a sort of "any" type that is a single column can contain
floats/integers/strings and so on. In its data representation it
contains the real type used. This will require a new structure like
TDSBLOB to handle these additional informations (it can't contain LOBs
but allow varchar(8000) to be contained).
Another problem is the space wasted if table contains a lot of
varchar(N) where N is quite big. Considering a table with 10
varchar(8000) fields row use 80000 bytes but usually strings are quite
small. One solution is to remove the concept of fixed offset in the row
and allocate space in the row as required (like db does).

freddy77