Skip to Content.
Sympa Menu

freetds - Re: [freetds] SQLGetData() returns len 4095 for empty TEXT on AIX 6.1 without ClientCharset defined

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Re: [freetds] SQLGetData() returns len 4095 for empty TEXT on AIX 6.1 without ClientCharset defined

Chronological Thread  
    < Chronological >       < Thread >
  • From: Sebastien FLAESCH <sf AT 4js.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] SQLGetData() returns len 4095 for empty TEXT on AIX 6.1 without ClientCharset defined
  • Date: Thu, 14 Mar 2013 14:54:19 +0100
On 03/14/2013 10:37 AM, Sebastien FLAESCH wrote:
Hi,

I have already reported an issue regarding SQLGetData()/iconv, see mails with
title:

"SQLGetData() returns TEXT length minus one on Solaris 64b with SQL Server
2005"

But now we have a similar problem on AIX 6.1, still with version 0.92.377.

After inserting an empty TEXT, we fetch it back (see mail history for
details),
and SQLGetData() returns a len of 4095 bytes, when zero is expected.

This occurs if no ClientCharset is defined in the ODBC data source...

When setting:

ClientCharset = en_US.8859-15

or:

ClientCharset = en_US.UTF-8

Sorry that was stupid, I mixed locale -l names and iconv -l names ...

In fact SQLGetData() returns a len of zero when

ClientCharset = <invalid_iconv_locale>

In such case the FreeTDS driver should stop with a fatal error.

This is subject of a lot of mistakes and then good luck once you have crap in
your db ;-)


The len returned by SQLGetData() is zero, as expected.

Some news about this?

As I suggested, if there is some iconv conversion issues, FreeTDS should stop
with a fatal error, to avoid any data loss / mis-conversion. It should not
silently send or return invalid character data.

Seb




On 02/06/2013 11:21 AM, Sebastien FLAESCH wrote:
FYI:

We have tested with this configure option:

--disable-libiconv

and now it works...

Whatever iconv issue it is, I believe that FreeTDS should check iconv
status
and stop with a fatal error if iconv is not able to handle charset
conversions.

Character set conversion is critical, when subject of data loss or
corruption
with no special warning for the user.

Seb

On 02/05/2013 03:22 PM, Sebastien FLAESCH wrote:
It appears that compiling with libiconv does not help.
Seb


On 02/05/2013 12:46 PM, Sebastien FLAESCH wrote:
On 02/04/2013 06:57 PM, Frediano Ziglio wrote:
2013/2/4 Sebastien FLAESCH<sf AT 4js.com>:
Frediano,

I believe it's related to the client charset conversions...

I did following simple test:

My database collation is CP1252, so to avoid iconv usage, in the ODBC
data source definition, I have now:

ClientCharset = CP1252

The program uses only pure ASCII chars, so this is ok.

Now there is no more iconv error in the log, and the first call to
SQLGetData() returns 10000 bytes as expected...

If fact for character data, ODBC returns the char pieces with an
additional \0 terminator.

So, with a buffer of 4096 bytes:

Call #1 to SQLGetData() returns len=10000 bytes (ok)
Call #2 to SQLGetData() returns len=5905 bytes
there was 5905 bytes left to read, because only 4095 bytes could
be written to the buffer, because of additional \0 terminator.
10000 - 4095 = 5905 ...
Call #3 to SQLGetData() returns len=1810 bytes
5905 - 4095 = 1810 ...
Last call: 1810 bytes fit in the 4096 buffer.


Well, this is expected. But I don't understand what's going wrong
with Solaris.
Could be possible that Solaris do not convert some characters and so
it convert a byte characters to 0 bytes ??

Possible...

Are you sure that there is only ASCII characters (0-127) ??

Pure ASCII.

Why should Solaris report some warning converting ASCII ??

My C locale is set to POSIX/C, I don't know why it ends as ISO-8859-1
for the iconv conversions in FreeTDS. Maybe it's the default.

Is there some mapping between the C locale (LANG/LC_ALL) and the iconv
locale name (I know these can be different names, how is this mapped?)

A dump would be helpful. Send privately to me if it contains sensitive
information.

I send you a dump.

You could try using libiconv.

We'll try this too.

I hope this helps...

Seb


Frediano



On 02/03/2013 12:12 PM, Frediano Ziglio wrote:
Hi,
where is SQL_DATA_AT_EXEC in your code?

Which DM are you using? Is possible that for some reason DM call some
additional function to do some checks. This could discovered enabling
FreeTDS dump.

Quire strange it works correctly on Linux but not on Solaris.
Does data contains no-ASCII characters?
How long are usually these TEXT fields?
Which TDS protocol version are you using?
There is a specific getdata test for this stuff and should work.

Regards,
Frediano

2013/2/2 Sebastien FLAESCH<sf AT 4js.com>:
Hi Frediano,

FYI, we use a single-byte encoding (LANG=POSIX) ...

Here is the code we use, nothing particular, this is the same
code we
use for different ODBC drivers...

#define BLOB_BLOCK_SIZE 4096
....
static SQLRETURN getBlobPieces(SqlStatement * st, SQLUSMALLINT fpos,
SQLSMALLINT ctype, MyLocator * loc,
FILE * out, SQLLEN * plen)
{
SQLRETURN r = 0;
char buf[BLOB_BLOCK_SIZE];
SQLLEN len;
int i = 0;
while (1) {
r = SQLGetData(st->stmtHandle, fpos, ctype,
(SQLPOINTER) buf, sizeof(buf),&len);
if (!SQL_SUCCEEDED(r) || len == 0 || len == SQL_NULL_DATA)
break;
if (len> (SQLLEN) sizeof(buf) || len == SQL_NO_TOTAL)
len = sizeof(buf);
/* Char data piece get \0, these must be removed! */
if (ctype == SQL_C_CHAR) {
if (buf[len - 1] == '\0') {
len--;
}
}
fwrite(buf, 1, len, out);
}
*plen = len;
return r;
}

Am I missusing SQLGetData()? As you can see, I modify the len after
Does this affect the next call to SQLGetData()?

Anyway, the INITIAL len returned by SQLGetData() is for sure:

(total-bytes - 1)

When using UTF-8, I get SQL_NO_TOTAL in the first call, but in
ASCII,
I get the (total size - 1)...

Seb

On 02/02/2013 08:52 AM, Frediano Ziglio wrote:
Il giorno 31/gen/2013, alle ore 14:28, Sebastien
FLAESCH<sf AT 4js.com> ha scritto:

Hi all,

Context:

$ uname -a
SunOS s64x100 5.10 Generic_120012-14 i86pc i386 i86pc

FreeTDS versions tested:
- freetds-0.92.dev.20120312
- freetds-dev.0.92.377

When fetching a TEXT from the database (SQL Server 2005), using
SQL_DATA_AT_EXEC,
it appears that the SQLGetData() function returns the size of
the text data,
minus one...

I have tested the same FreeTDS version on a Linux 32b, and
SQLGetData() returns
the correct length as expected...


Strange! How do you read the length?

I will continue to investigate, but I wanted to ask if this is
a know issue.

When connecting to SQL Server 2008 R2, this does not occur,
because SQLGetData()
returns SQL_NO_TOTAL and we fetch text pieces in a different
way...


This is done to deal with different encodings.

Thanks
Seb

Frediano

_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds



_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds




Archive powered by MHonArc 2.6.24.

Top of Page