Skip to Content.
Sympa Menu

freetds - Re: [freetds] Row problem

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "James K. Lowden" <jklowden AT freetds.org>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] Row problem
  • Date: Thu, 20 Mar 2008 17:40:55 -0400

Frediano Ziglio wrote:
> > I need to understand this very clearly to be useful to you. I've
> > never seen the data stream. What does it look like? How is it
> > chunked?
>
> Well... assume you are returning a string 'hello!'
> with varchar(100)
> Data info
> type XSYBVARCHAR
> len (16bit) 100
> Data
> len (16bit) 6
> string 'hello!'
> with varchar(max)
> Data info
> type XSYBVARCHAR
> len (16bit) -1
> Data
> total len (64bit) 6
> chunk len (32bit) 6
> string 'hello!'
> chunk len (32bit) 0
>
> note that type has same constant.

Thanks, that's clear. Is it the same for bcp in?

Suggestion
----------

1. As soon as XSYBVARCHAR arrives from the server, check the length. If
-1, set coltype to XSYBMAXCHAR. That tells the rest of the system the
column is really varchar(max) and lets is_blob_type work (if that's what
we want).

2. Like BLOB, allocate space according to total len. Initialize textptr
to 0xbadfaced.

3. Read in all chunks, continue normally.

> > My idea is to synthesize a textptr in libtds to make varchar(max) look
> > like TEXT to the client libraries. I'm not worried about wasting a
> > few bytes, especially because -- if we're lucky -- the client
> > libraries won't have to change.
>
> Yes, TDSBLOB reuse is feasible, I don't think ODBC will have problems,
> problems raise in dblib where textptr is important to use in
> READTEXT/WRITETEXT.

Right you are. We can't synthesize textptr because it belongs to the
server. :-(

It's OK for now if db-lib has no way to update this kind of column, as
long as it can fetch it. The user application always construct SQL to
update the column if need be.

I don't think there's a great demand for extending db-lib. But there's no
technical reason we couldn't. We have dbdata(); we could add dbputdata()
along the lines of SQLPutData. We'd have to add parameterized queries
first....

> I think is also the right time to add support for variant! This strange
> type is a sort of "any" type that is a single column can contain
> floats/integers/strings and so on. In its data representation it
> contains the real type used. This will require a new structure like
> TDSBLOB to handle these additional informations (it can't contain LOBs
> but allow varchar(8000) to be contained).

Do you really think anyone cares about variants? About the only place
variants are useful after you fetch them is in Visual Basic.

I think they're really *dumb* from a relational calculus point of view.
The whole point of predicate logic is to say something verifiable about
the key. What kind of attribute is sometimes a number, sometimes a
string, and sometimes a date?

If you want one man's opinion, I think a much better use of your
exceptional talents would be to add bcp to ODBC. You'd surely make more
people happier that way. (Including me!)

> Another problem is the space wasted if table contains a lot of
> varchar(N) where N is quite big.

I just don't think that is a big problem. It's very rare that a row has
so many giant varchar columns. You'd need 10 tables to get 80,000 bytes,
because each table's row can be only 8,060 bytes.

A much bigger problem is that we allocate memory for BLOBs when fetching,
and really big blobs exhaust virtual memory pretty easily. I think we'd be
smarter to cache the row in a temporary file and seek on that if the data
are too big, say, over 1 MB.

Regards,

--jkl




Archive powered by MHonArc 2.6.24.

Top of Page