Skip to Content.
Sympa Menu

freetds - [freetds] Bulk loading not recognizing nvarchar columns perfectly

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Will McQueen <wmcqueen AT athenahealth.com>
  • To: "freetds AT lists.ibiblio.org" <freetds AT lists.ibiblio.org>
  • Subject: [freetds] Bulk loading not recognizing nvarchar columns perfectly
  • Date: Fri, 8 Jul 2011 20:32:44 -0400

Freddy,

I think I have another bug fix for bcp.c, so let me know what you think. The
patch I recommend is:

--- freetds-0.92.dev.20110707/src/dblib/bcp.c 2011-06-18 13:52:24.000000000
-0400
+++ freetds-0.92.dev.20110619-mod/src/dblib/bcp.c 2011-07-08
20:00:50.352462000 -0400
@@ -1375,7 +1380,7 @@
bcpcol->bcp_column_data->datalen = 0;
} else {
bcpcol->bcp_column_data->is_null = 0;
- desttype = tds_get_conversion_type(bcpcol->column_type,
bcpcol->column_size);
+ desttype = tds_get_conversion_type(bcpcol->on_server.column_type,
bcpcol->on_server.column_size);

/* special hack for text columns */

This doesn’t seem to change much on calls to dbconvert later on in bcp.c, but
the big thing it changes is it skips the following block of code for NVARCHAR
columns:

/* trim trailing blanks from character data */
if (desttype == SYBCHAR || desttype == SYBVARCHAR) {
bcpcol->bcp_column_data->datalen = rtrim((char *)
bcpcol->bcp_column_data->data,

bcpcol->bcp_column_data->datalen);
}

The rtrim already tries to skip the rtrim if the desttype is XSYBNVARCHAR,
but if the client-side column type is SYBVARCHAR it doesn’t skip this block
the way it should.

When I run my import with the above change, I’m able to load certain
characters that I couldn’t previously (i.e., ‰) and the following snippets
change in my tdsdump:


-dblib.c:2204:dbconvert(0x19eff490, SYBCHAR, 0x19f36470, 3, SYBINT4,
0x19f023b0, 4)
+dblib.c:2204:dbconvert(0x10493490, SYBCHAR, 0x104ca470, 3, SYBINT4,
0x104963b0, 4)

-dblib.c:2204:dbconvert(0x19eff490, SYBCHAR, 0x19eff360, 84, SYBVARCHAR,
0x19f00110, 320)
+dblib.c:2204:dbconvert(0x10493490, SYBCHAR, 0x10493360, 84, XSYBNVARCHAR,
0x10494110, 320)

-dblib.c:2204:dbconvert(0x19eff490, SYBCHAR, 0x19f36470, 6, SYBINT8,
0x19f00280, 8)
+dblib.c:2204:dbconvert(0x10493490, SYBCHAR, 0x104ca470, 6, SYBINT8,
0x10494280, 8)

-dblib.c:2204:dbconvert(0x19eff490, SYBCHAR, 0x19f36470, 10, SYBVARCHAR,
0x19f48770, 160)
+dblib.c:2204:dbconvert(0x10493490, SYBCHAR, 0x104ca470, 10, XSYBNVARCHAR,
0x104dc770, 160)

-dblib.c:2204:dbconvert(0x19eff490, SYBCHAR, 0x19f36470, 10, SYBDATETIME,
0x19f48a50, 8)
+dblib.c:2204:dbconvert(0x10493490, SYBCHAR, 0x104ca470, 10, SYBDATETIME,
0x104dca50, 8)


Thanks,
Will



  • [freetds] Bulk loading not recognizing nvarchar columns perfectly, Will McQueen, 07/08/2011

Archive powered by MHonArc 2.6.24.

Top of Page