FreeTDS Development Group

[Burkhard AT habmalnefrage.de]

Hi James,

Thanks for your response. The log file indicates that the server reports the 
column type as UCS-2:


net.c:671:Received packet
0000 81 01 00 00 00 09 00 63-00 00 20 00 09 04 d0 00 |.......c .. .....|
0010 34 0d 00 62 00 75 00 72-00 6b 00 68 00 61 00 72 |4..b.u.r .k.h.a.r|
0020 00 64 00 5f 00 74 00 65-00 73 00 74 00 04 6e 00 |.d._.t.e .s.t..n.|
0030 61 00 6d 00 65 00 d1 10-64 75 6d 6d 79 20 74 65 |a.m.e... dummy te|
0040 78 74 70 74 72 00 00 00-64 75 6d 6d 79 54 53 00 |xtptr... dummyTS.|
0050 08 00 00 00 41 00 00 d8-d0 dd 42 00 ff 11 00 c1 |....A... ..B.....|
0060 00 01 00 00 00 79 00 00-00 00 fe 00 00 e0 00 00 |.....y.. ........|
0070 00 00 00               -                        |...|

token.c:510:processing result tokens.  marker is  81(TDS7_RESULT)
token.c:1520:processing TDS7 result metadata.
mem.c:563:tds_free_all_results()
token.c:1544:set current_results (1 column) to tds->res_info
token.c:1555:setting up column 0
token.c:3294:adjust_character_column_size:
        Server charset: UCS-2LE
        Server column_size: 2097152
        Client charset: UTF-8
        Client column_size: 4194304
token.c:1490:tds7_get_data_info:
        colname = name (4 bytes)
        type = 35 (text)
        server's type = 99 (UCS-2 text)
        column_varint_size = 4
        column_size = 4194304 (2097152 on server)


 (and in the TDS documentation I don't see other ways of sending unicode 
characters than UCS-2). On the SQL Server type, there is no difference in the 
column type, whether a string contains "surrogate characters" (and needs to 
be interpreted as UTF-16) or not. This seems OK to me as UCS-2 is a subset of 
UTF-16. 

Nevertheless, when I make a query in SQL Server Management Studio, it 
correctly converts the characters from UTF-16 to UTF-8 when writing the 
results to a report file.

In the example above, the value retrieved from the database is a 
three-UTF-16-character string: character "A", the character U+101D0 ("    
PHAISTOS DISC SIGN PEDESTRIAN" :-), and "B".

I have also tried using the XML data type instead of NVARCHAR, the FreeTDS 
log file looks similar.

>From all that I learned about this, it seems it would be safe intepreting 
>the UCS-2 characters as UTF-16, for the conversion to UTF-8 - since UTF-16 
>is backwards compatible to UCS-2. (see 
>http://en.wikipedia.org/wiki/UTF-16/UCS-2 ). Maybe for now as an option to 
>the configure script? 

I would appreciate your help with such a change... for sure I can help with 
testing it.

Best Regards,
  Burkhard
-- 
Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief!  
Jetzt De-Mail-Adresse reservieren: http://portal.gmx.net/de/go/demail