Skip to Content.
Sympa Menu

freetds - Re: [freetds] unicode

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "James K. Lowden" <jklowden AT freetds.org>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] unicode
  • Date: Tue, 22 Jul 2008 19:08:15 -0400

Jason Young (Morgon) wrote:
>
> In my short experience (and through confirmation with SQL Server MVPs at
>
> Microsoft), inserting UTF8 strings into nvarchar fields require N''
> prefixed strings.
>
> I know that FreeTDS doesn't care about the N prefix, but when doing my
> tests, I was only able to fully extract inserted 'UTF8' data when
> inserting using the N prefix into an nvarchar field.

Well, there are a lot of moving parts.

I have no experience with ideographic scripts. Using Microsoft's Query
Analyzer and sqsh, this works just fine with a varchar column:

insert u values ('Å')

If that doesn't work for you, I'd be interested in seeing a TDSDUMP log to
make sure it's not a client-side configuration issue.

The form N'utf8text' is *invalid* as far as the server is concerned. The
server has no way of knowing the string is utf-8 encoded. It *assumes* --
it has to assume something -- it's UCS-2 encoded.

The server cannot store UTF-8 encoded text per se. It can store it, just
as it stores a PDF file: as a blob. But it can't index it or sort it or
compare it.

When you do:

mssql_query("INSERT INTO utf8test (textfield) VALUES ('<UTF8 Text>')");

FreeTDS converts the SQL text from your encoding to UCS-2. The server
never sees the UTF-8 data. It should see UCS-2 data *if* your client
encoding is set to UTF-8.

You can store random byte-pairs verbatim in an nvarchar column. That's
what the server promises to do: not to infer an encoding And they'll
sort and compare, just not meaningfully.

--jkl





Archive powered by MHonArc 2.6.24.

Top of Page