Skip to Content.
Sympa Menu

freetds - Re: [freetds] SQLGetData() returns TEXT length minus one on Solaris 64b with SQL Server 2005

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Sebastien FLAESCH <sf AT 4js.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] SQLGetData() returns TEXT length minus one on Solaris 64b with SQL Server 2005
  • Date: Wed, 06 Feb 2013 11:21:35 +0100

FYI:

We have tested with this configure option:

--disable-libiconv

and now it works...

Whatever iconv issue it is, I believe that FreeTDS should check iconv status
and stop with a fatal error if iconv is not able to handle charset
conversions.

Character set conversion is critical, when subject of data loss or corruption
with no special warning for the user.

Seb

On 02/05/2013 03:22 PM, Sebastien FLAESCH wrote:
It appears that compiling with libiconv does not help.
Seb


On 02/05/2013 12:46 PM, Sebastien FLAESCH wrote:
On 02/04/2013 06:57 PM, Frediano Ziglio wrote:
2013/2/4 Sebastien FLAESCH<sf AT 4js.com>:
Frediano,

I believe it's related to the client charset conversions...

I did following simple test:

My database collation is CP1252, so to avoid iconv usage, in the ODBC
data source definition, I have now:

ClientCharset = CP1252

The program uses only pure ASCII chars, so this is ok.

Now there is no more iconv error in the log, and the first call to
SQLGetData() returns 10000 bytes as expected...

If fact for character data, ODBC returns the char pieces with an
additional \0 terminator.

So, with a buffer of 4096 bytes:

Call #1 to SQLGetData() returns len=10000 bytes (ok)
Call #2 to SQLGetData() returns len=5905 bytes
there was 5905 bytes left to read, because only 4095 bytes could
be written to the buffer, because of additional \0 terminator.
10000 - 4095 = 5905 ...
Call #3 to SQLGetData() returns len=1810 bytes
5905 - 4095 = 1810 ...
Last call: 1810 bytes fit in the 4096 buffer.


Well, this is expected. But I don't understand what's going wrong with
Solaris.
Could be possible that Solaris do not convert some characters and so
it convert a byte characters to 0 bytes ??

Possible...

Are you sure that there is only ASCII characters (0-127) ??

Pure ASCII.

Why should Solaris report some warning converting ASCII ??

My C locale is set to POSIX/C, I don't know why it ends as ISO-8859-1
for the iconv conversions in FreeTDS. Maybe it's the default.

Is there some mapping between the C locale (LANG/LC_ALL) and the iconv
locale name (I know these can be different names, how is this mapped?)

A dump would be helpful. Send privately to me if it contains sensitive
information.

I send you a dump.

You could try using libiconv.

We'll try this too.

I hope this helps...

Seb


Frediano



On 02/03/2013 12:12 PM, Frediano Ziglio wrote:
Hi,
where is SQL_DATA_AT_EXEC in your code?

Which DM are you using? Is possible that for some reason DM call some
additional function to do some checks. This could discovered enabling
FreeTDS dump.

Quire strange it works correctly on Linux but not on Solaris.
Does data contains no-ASCII characters?
How long are usually these TEXT fields?
Which TDS protocol version are you using?
There is a specific getdata test for this stuff and should work.

Regards,
Frediano

2013/2/2 Sebastien FLAESCH<sf AT 4js.com>:
Hi Frediano,

FYI, we use a single-byte encoding (LANG=POSIX) ...

Here is the code we use, nothing particular, this is the same code we
use for different ODBC drivers...

#define BLOB_BLOCK_SIZE 4096
....
static SQLRETURN getBlobPieces(SqlStatement * st, SQLUSMALLINT fpos,
SQLSMALLINT ctype, MyLocator * loc,
FILE * out, SQLLEN * plen)
{
SQLRETURN r = 0;
char buf[BLOB_BLOCK_SIZE];
SQLLEN len;
int i = 0;
while (1) {
r = SQLGetData(st->stmtHandle, fpos, ctype,
(SQLPOINTER) buf, sizeof(buf),&len);
if (!SQL_SUCCEEDED(r) || len == 0 || len == SQL_NULL_DATA)
break;
if (len> (SQLLEN) sizeof(buf) || len == SQL_NO_TOTAL)
len = sizeof(buf);
/* Char data piece get \0, these must be removed! */
if (ctype == SQL_C_CHAR) {
if (buf[len - 1] == '\0') {
len--;
}
}
fwrite(buf, 1, len, out);
}
*plen = len;
return r;
}

Am I missusing SQLGetData()? As you can see, I modify the len after
Does this affect the next call to SQLGetData()?

Anyway, the INITIAL len returned by SQLGetData() is for sure:

(total-bytes - 1)

When using UTF-8, I get SQL_NO_TOTAL in the first call, but in ASCII,
I get the (total size - 1)...

Seb

On 02/02/2013 08:52 AM, Frediano Ziglio wrote:
Il giorno 31/gen/2013, alle ore 14:28, Sebastien FLAESCH<sf AT 4js.com> ha
scritto:

Hi all,

Context:

$ uname -a
SunOS s64x100 5.10 Generic_120012-14 i86pc i386 i86pc

FreeTDS versions tested:
- freetds-0.92.dev.20120312
- freetds-dev.0.92.377

When fetching a TEXT from the database (SQL Server 2005), using
SQL_DATA_AT_EXEC,
it appears that the SQLGetData() function returns the size of the text data,
minus one...

I have tested the same FreeTDS version on a Linux 32b, and SQLGetData()
returns
the correct length as expected...


Strange! How do you read the length?

I will continue to investigate, but I wanted to ask if this is a know issue.

When connecting to SQL Server 2008 R2, this does not occur, because
SQLGetData()
returns SQL_NO_TOTAL and we fetch text pieces in a different way...


This is done to deal with different encodings.

Thanks
Seb

Frediano

_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds






Archive powered by MHonArc 2.6.24.

Top of Page