Unicode, UTF-8, and Greek

Curt Hagenlocher curt at hagenlocher.org
Thu Oct 11 09:51:09 EDT 2001


> Try this in isql on NT:
> 	select substring( N'world', 1, 3 )
> You will see (from memory; I'm at home):
> 
> ------
> wor

Yep.  You don't even have to go to the trouble of a substring --
select N'foo' has exactly the same effect.  Interestingly, isql/w
doesn't have this problem, though it will show four dashes for both
select 'foo' and select N'foo'.

If the FreeTDS code uses UTF-8, and I know that the column is of
type nvarchar(x), how big a buffer do I have to allocate to
guarantee that I can load the entire field?  3x + 1 (assuming null
termination), because I know that the maximum length of a UTF-8
representation of a string that can be stored as UCS-2 is 3 bytes.

What happens if I pass a smaller buffer?

> SQL Server 7 returned a column six bytes wide, 2 bytes/character. 
> I don't know what FreeTDS does with that today, but I'd guess it would
> indicate a 3-byte column.  

The Java code handles this correctly. :)
> 
> (It's a little maddening, if you spend much time in Query Analyzer
> looking at system tables, whose "name" columns are now nvarchar(255),
> meaning they take up 512 positions on the screen.  That's a lot of
> whitespace, even on a 1600x1200 screen.

... or having to scroll to column 344 for the cmd column of sp_who.

> if the client binds varchar to the column, return ascii.  If he
> binds nvarchar to it, return UTF-8?

That sounds right to me.

> As you've pointed out before, the principle of least surprise should
> apply, and we don't know what perl/python/php expect/want/do with
> UTF-8.

Python seems to use UTF-16, and there are definitely built-in
functions to convert to/from UTF-8.

> I don't think we've seen one question ever on this list about Unicode
> data on a Sybase system, which I take as zero interest. 

But then, there aren't that many questions specific to Sybase in
general.

--
Curt Hagenlocher
curt at hagenlocher.org



More information about the FreeTDS mailing list