FreeTDS Development Group

[Marc Abramowitz <msabramo AT gmail.com>]

Interesting. John, I wonder if you can write a little C program that 
illustrates the bug? I think that the FreeTDS authors are most comfortable in 
C, so this will probably illustrate the problem best for them and show beyond 
a shadow of a doubt that there's nothing happening in pymssql. 

-Marc
http://marc-abramowitz.com
Sent from my iPhone 4S


> On Aug 29, 2014, at 7:07 AM, John Anderson <sontek AT gmail.com> wrote:
> 
>> On Fri, Aug 29, 2014 at 9:56 AM, Frediano Ziglio <freddy77 AT gmail.com> 
>> wrote:
>> 
>> 2014-08-29 17:50 GMT+01:00 John Anderson <sontek AT gmail.com>:
>> 
>>> Hello!  I'm wondering if you had any tips on how to work with characters
>>> larger than 0xFFFF?  We currently have the problem where we try to
>>> insert/update a row in the database and it gets truncated at the
>> character
>>> and creates an invalid syntax:
>>> 
>>> For example this character:   \U0001f44d
>>> 
>>> We've tracked this down to the fact that FreeTDS is trying to convert
>> this
>>> to UCS-2 rather than UTF16-LE, which is what SQL Server 2012 uses.
>>> 
>>> For example if we say:
>>> 
>>> INSERT INTO table(name) VALUES(N' Hello \U0001f44d')
>> How are you using this string?
> Sorry, this is the python representation of the string which is a "Thumbs
> Up", It's UTF-8 hex representation is 0xf09f918d and what needs to be
> stored in the database is UTF-16LE hex 0x3dd84ddc.
> 
> This is the character:
> 
> http://www.fileformat.info/info/unicode/char/1F44D/index.htm
> 
> But it is any character above 0xFFFF will cause the issue.
> 
> So the actually query ends up looking like N'Hello <display of a thumbs up>'
> 
> 
> 
>> 
>>> We get the error:
>>> 
>>> Unclosed quotation mark after the character string 'Hello ' because it
>>> stops processing the rest of the query.
>>> 
>>> Is there a freetds.conf or environment setting we can use to tell it to
>> use
>>> UTF-16LE instead of UCS2 so that we can accept these larger character
>> sets?
>>> 
>>> As a temporary solution we've started sending the raw UTF-16LE bytes to
>>> FreeTDS but we prefer not having to do this.
>>> 
>>> Thanks,
>>> John
>> 
>> Frediano
>> _______________________________________________
>> FreeTDS mailing list
>> FreeTDS AT lists.ibiblio.org
>> http://lists.ibiblio.org/mailman/listinfo/freetds
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds