FreeTDS Development Group

[Frediano Ziglio <freddy77 AT gmail.com>]

I saw patch about CTLib and nvarchar. I forgot to comment and review
this patch but looking at unitests results it seems that this patch
fix nothing and break too much. The patch should try to fix ct_param
with wide charset source but I think is not that correct. libTDS does
not handle wide charset that much. tds_convert doesn't like wide
charset. libTDS just convert wide charset to multi-byte/single-byte
and back. There are some optimization (mostly on odbc layer) that
allow wide charset to be stored in libTDS recordset but only after
many checks. A more deeper review of the patch follow

From: jklowden <jklowden>
Date: Sat, 6 Jun 2009 01:42:28 +0000 (+0000)
Subject: Applied patch from Andrew Victor ML 24 November 2008 "CTLib
implementation handles ...

Applied patch from Andrew Victor ML 24 November 2008 "CTLib
implementation handles NVARCHAR strings as VARCHAR"
---

diff --git a/src/ctlib/cs.c b/src/ctlib/cs.c
index 3646016..ecbb332 100644
--- a/src/ctlib/cs.c
+++ b/src/ctlib/cs.c
@@ -601,6 +601,7 @@ cs_convert(CS_CONTEXT * ctx, CS_DATAFMT * srcfmt,
CS_VOID * srcdata, CS_DATAFMT

                case SYBCHAR:
                case SYBVARCHAR:
+               case SYBNVARCHAR:
                case SYBTEXT:
                        tdsdump_log(TDS_DBG_FUNC, "cs_convert() desttype = 
character\n");

@@ -812,6 +813,7 @@ cs_convert(CS_CONTEXT * ctx, CS_DATAFMT * srcfmt,
CS_VOID * srcdata, CS_DATAFMT
                break;
        case SYBCHAR:
        case SYBVARCHAR:
+       case SYBNVARCHAR:
        case SYBTEXT:
                if (len > destlen) {
                        fprintf(stderr, "error_handler: Data-conversion 
resulted in overflow.\n");


useless, type came from _ct_get_server_type which never returns SYBNVARCHAR.

diff --git a/src/ctlib/ct.c b/src/ctlib/ct.c
index def81f3..f4e8b6b 100644
--- a/src/ctlib/ct.c
+++ b/src/ctlib/ct.c
@@ -2016,6 +2016,9 @@ _ct_get_client_type(int datatype, int usertype, int 
size)
                        return CS_UNICHAR_TYPE;
                return CS_CHAR_TYPE;
                break;
+       case SYBNVARCHAR:
+               return CS_UNICHAR_TYPE;
+               break;
        }

        return CS_FAIL;
@@ -4034,7 +4037,10 @@ paraminfoalloc(TDSSOCKET * tds, CS_PARAM * first_param)
                                return NULL;
                        pcol->on_server.column_size = pcol->column_size = 
p->maxlen;
                        pcol->column_cur_size = temp_datalen;
-                       if (temp_datalen > 0 && temp_datalen > p->maxlen)
+                       if (p->maxlen == 0 && pcol->column_type == 
SYBNVARCHAR) {
+                               pcol->on_server.column_size = 
pcol->column_size = temp_datalen*2;
+                       }
+                       else if (temp_datalen > 0 && temp_datalen > p->maxlen)
                                pcol->on_server.column_size = 
pcol->column_size = temp_datalen;
                } else {
                        pcol->column_cur_size = pcol->column_size;

diff --git a/src/tds/convert.c b/src/tds/convert.c
index 2c0dad4..0cdb5fa 100644
--- a/src/tds/convert.c
+++ b/src/tds/convert.c
@@ -340,6 +340,7 @@ tds_convert_char(int srctype, const TDS_CHAR *
src, TDS_UINT srclen, int desttyp
                return srclen;

        case CASE_ALL_CHAR:
+       case SYBNVARCHAR:               /* CHAR return as Unicode */
                cr->c = (TDS_CHAR *) malloc(srclen + 1);
                test_alloc(cr->c);
                memcpy(cr->c, src, srclen);


tds_convert doesn't handle wide charset so this copy is not that fine

diff --git a/src/tds/data.c b/src/tds/data.c
index 3c81ff5..042337b 100644
--- a/src/tds/data.c
+++ b/src/tds/data.c
@@ -85,6 +85,9 @@ tds_set_param_type(TDSSOCKET * tds, TDSCOLUMN *
curcol, TDS_SERVER_TYPE type)
                case SYBBINARY:
                        type = XSYBBINARY;
                        break;
+               case SYBNVARCHAR:
+                       type = XSYBNVARCHAR;
+                       break;
                        /* avoid warning on other types */
                default:
                        break;


Personally I don't know where SYBNVARCHAR constant came. I tried all
protocols but MS never return SYBNVARCHAR, only SYBVARCHAR (protocol <
7.0) or XSYBNVARCHAR (protocol >= 7.0). However it should be ok.

@@ -152,6 +155,7 @@ tds_get_cardinal_type(int datatype)
        case SYBNTEXT:
                return SYBTEXT;
        case XSYBNVARCHAR:
+               return SYBNVARCHAR;
        case XSYBVARCHAR:
                return SYBVARCHAR;
        case XSYBNCHAR:


here the purpose of tds_get_cardinal_type is to convert wide character
to multi/single-byte encoding. ODBC use another way where libTDS
doesn't apply any conversion and just store wide characters as readed
from server. ODBC has been deeply tested for this but CTLib require
extensive testing.

Well... the question is now. What this patch is expected to fix and
how to fix the patch ?? Perhaps it would be better to convert wide
character in CTLib. IMO SYBNVARCHAR should not be used by tds_convert
nor in TDSCOLUMN->column_type.

freddy77