Skip to Content.
Sympa Menu

freetds - Re: NTEXT support (again)

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: <camber AT ais.org>
  • To: TDS Development Group <freetds AT franklin.oit.unc.edu>
  • Subject: Re: NTEXT support (again)
  • Date: Mon, 5 Jun 2000 08:15:54 -0400 (EDT)



Hi Bob,

Thanks for the patches, I'll try to get them merged in tonight.

On Sun, 4 Jun 2000, Bob Kline wrote:

> On Sun, 4 Jun 2000, Bob Kline wrote:
>
> > On Sat, 3 Jun 2000, Bob Kline wrote:
> >
> > So I made a set of diffs against the 0.50 sources. [snip ...]
>
> I'm having a public conversation with myself. :->}
>
> The patches I sent in earlier this morning were flawed. Here are the
> flaws:
>
> 1. I was mistaken in my observation that the gcc wcslen() is calculating
> lengths correctly for UTF-16 strings. It doesn't, because in the
> gcc world, wide characters are four bytes wide, not two.
> 2. I had hoped that I would be able to avoid touching anything but the
> tds layer, but Microsoft isn't null-terminating the NTEXT data that
> it sends. The tds layer is obtaining the correct length from the tds
> stream, but the dblib layer was discarding this information. So the
> tds conversion code can't calculate the length and it can't get the
> length without some modifications to dblib.c.
dblib layer should really care about the length. The application is
responsible for ensure 1) there is a max length (in bytes) 2) there is
sufficient space for the returning text. MS must introduce a new binding
type to handle unicode right? (Something like NSTRINGBIND instead of
STRINGBIND???) Does anyone know what this might be? I believe ctlib
support multichar codes.

> 3. I believe I guessed incorrectly about the return value from
> tds_convert(), thinking that for string values it should represent
> the number of characters. It seems more likely that it is supposed
> to represent the number of bytes converted for all types. Haven't
> been able to find the comments documenting what the real answer is.
The return from tds_convert() is the size of converted data...ie it
includes the null.

> 4. The upper layers don't provide any way to bind to a wide-character
> string variable, so the destination type comes through as -1 when
> I pass through the data type byte for NTEXT (99). The real fix
> for this, of course, is to extend all the upper layers to fill
> this gap, but I'm reluctant to disturb any layers above tds more
> than I have to, so as a stopgap solution, I've added -1 to the
> switch statement in the new tds_convert_ntext() function, preserving
> the 16-bit characters from the original transmission.
again, a new bind type is needed. We need to know if MS has put one it
their dblib, otherwise I nominate NSTRINGBIND and NTBNSTRINGBIND.

>
> The replacement patches below fix all of these flaws. Please use them
> instead of those sent in earlier today. Sorry for the premature
> submission.
>
>
> *** dblib.c.orig Tue Nov 30 22:20:34 1999
> --- dblib.c Sun Jun 4 21:05:45 2000
> ***************
> *** 325,332 ****
> --- 325,334 ----
> }
> else
> {
> + DBINT srclen = -1;
>
> if (is_blob_type(curcol->column_type)) {
> + srclen = curcol->column_textsize;
> src = curcol->column_textvalue;
> } else {
> src = ((char*)buffer_row_address(buf, index))
> ***************
> *** 339,345 ****
> dbconvert(dbproc,
> srctype, /* srctype */
> src, /* src */
> ! -1, /* srclen */
> desttype, /* desttype */
> curcol->varaddr, /* dest */
> curcol->column_bindlen); /* destlen */
> --- 341,347 ----
> dbconvert(dbproc,
> srctype, /* srctype */
> src, /* src */
> ! srclen, /* srclen */
> desttype, /* desttype */
> curcol->varaddr, /* dest */
> curcol->column_bindlen); /* destlen */
>
>
> *** tds.h.orig Sun Jun 4 08:28:18 2000
> --- tds.h Sun Jun 4 08:24:39 2000
> ***************
> *** 155,160 ****
> --- 155,161 ----
> #define SYBDATETIME 61 /* 0x3D */
> #define SYBBIT 50 /* 0x32 */
> #define SYBTEXT 35 /* 0x23 */
> + #define SYBNTEXT 99 /* 0x63 */
> #define SYBIMAGE 34 /* 0x22 */
> #define SYBMONEY4 122 /* 0x7A */
> #define SYBMONEY 60 /* 0x3C */
> ***************
> *** 196,203 ****
> x==SYBVARBINARY || \
> x==SYBMONEYN || \
> x==SYBTEXT || \
> ! x==SYBIMAGE)
> ! #define is_blob_type(x) (x==SYBTEXT || x==SYBIMAGE)
> /* large type means it has a two byte size field */
> #define is_large_type(x) (x>128)
> #define is_numeric_type(x) (x==SYBNUMERIC || x==SYBDECIMAL)
> --- 197,205 ----
> x==SYBVARBINARY || \
> x==SYBMONEYN || \
> x==SYBTEXT || \
> ! x==SYBIMAGE || \
> ! x==SYBNTEXT)
> ! #define is_blob_type(x) (x==SYBTEXT || x==SYBIMAGE || x==SYBNTEXT)
> /* large type means it has a two byte size field */
> #define is_large_type(x) (x>128)
> #define is_numeric_type(x) (x==SYBNUMERIC || x==SYBDECIMAL)
>
> *** convert.c.orig Fri Dec 3 20:06:07 1999
> --- convert.c Sun Jun 4 20:42:59 2000
> ***************
> *** 23,28 ****
> --- 23,29 ----
> #include <stdlib.h>
> #include <unistd.h>
> #include <time.h>
> + #include <assert.h>
>
>
> static char software_version[] = "$Id: convert.c,v 1.35 1999/12/04
> 01:06:07 camber Exp $";
> ***************
> *** 103,108 ****
> --- 104,167 ----
> return cplen;
> }
> }
> +
> + typedef unsigned short utf16_t;
> +
> + static TDS_UINT utf16len(const utf16_t* s)
> + {
> + const utf16_t* p = s;
> + while (*p++)
> + ;
> + return p - s;
> + }
> +
> + TDS_INT tds_convert_ntext(int srctype,unsigned char *src,TDS_UINT srclen,
> + int desttype,unsigned char *dest,TDS_UINT destlen)
> + {
> + /*
> + * XXX Limit of 255 + 1 needs to be fixed by determining what the
> + * real limit of [N][VAR]CHAR columns is.
> + * XXX What about NCHAR? Don't see a constant for it in tds.h.
> + * XXX Case for -1 in switch statement because upper levels don't
> + * have a way to bind to wide-character types.
> + */
> + TDS_UINT i, cplen, char_limit = 255;
> + utf16_t* wsrc = (utf16_t*)src;
> + utf16_t* wdest = (utf16_t*)dest;
> + assert(sizeof(utf16_t) == 2);
> + switch (desttype) {
> + case SYBNVARCHAR:
> + if (destlen > char_limit * sizeof(utf16_t))
> + destlen = char_limit * sizeof(utf16_t);
> + /* Fall through ... */
> + case SYBNTEXT:
> + case -1:
> + cplen = srclen > destlen ? destlen : srclen;
> + memcpy(dest, src, cplen);
> + if (destlen < srclen + sizeof(utf16_t)) {
> + size_t term_pos = destlen - sizeof(utf16_t);
> + size_t odd_bytes = term_pos % sizeof(utf16_t);
> + term_pos -= odd_bytes;
> + wdest[term_pos / sizeof(utf16_t)] = 0;
> + }
> + else
> + wdest[cplen / sizeof(utf16_t)] = 0;
> + return utf16len(wdest) * 2;
> + default:
> + /* Assume caller wants conversion to narrow string. */
> + if (destlen > char_limit && desttype != SYBTEXT)
> + destlen = char_limit;
> + cplen = srclen > destlen ? destlen : srclen;
> + for (i = 0; i < cplen; ++i)
> + dest[i] = (unsigned char)wsrc[i];
> + if (destlen < srclen)
> + dest[destlen - 1] = 0;
> + else
> + dest[cplen] = 0;
> + return strlen(dest);
> + }
> + }
> +
> TDS_INT tds_convert_binary(int srctype,unsigned char *src,TDS_INT srclen,
> int desttype,unsigned char *dest,TDS_INT destlen)
> {
> ***************
> *** 678,683 ****
> --- 737,746 ----
> break;
> case SYBTEXT:
> return tds_convert_text(srctype,src,srclen,
> + desttype,dest,destlen);
> + break;
> + case SYBNTEXT:
> + return tds_convert_ntext(srctype,src,srclen,
> desttype,dest,destlen);
> break;
> default:
>
>
> --
> Bob Kline
> mailto:bkline AT rksystems.com
> http://www.rksystems.com
>
>





Archive powered by MHonArc 2.6.24.

Top of Page