Skip to Content.
Sympa Menu

freetds - Re: [freetds] SQLGetData() returns TEXT length minus one on Solaris 64b with SQL Server 2005

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Frediano Ziglio <freddy77 AT gmail.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] SQLGetData() returns TEXT length minus one on Solaris 64b with SQL Server 2005
  • Date: Wed, 6 Feb 2013 16:18:45 +0000

2013/2/6 Sebastien FLAESCH <sf AT 4js.com>:
> FYI:
>
> We have tested with this configure option:
>
> --disable-libiconv
>
> and now it works...
>
> Whatever iconv issue it is, I believe that FreeTDS should check iconv status
> and stop with a fatal error if iconv is not able to handle charset
> conversions.
>
> Character set conversion is critical, when subject of data loss or
> corruption
> with no special warning for the user.
>
> Seb
>

Mmm... not that good. It means that Solaris iconv does something that
is not expected by FreeTDS. Probably FreeTDS expect a 1 to 1
conversion while lib iconv provided by Solaris does not respect this.
I should write a test if libiconv linked in is FreeTDS-aware. Probably
using GNU libiconv would fix your issues. However is hard to reproduce
using a libiconv that works as expected.

Frediano

> On 02/05/2013 03:22 PM, Sebastien FLAESCH wrote:
>> It appears that compiling with libiconv does not help.
>> Seb
>>
>>
>> On 02/05/2013 12:46 PM, Sebastien FLAESCH wrote:
>>> On 02/04/2013 06:57 PM, Frediano Ziglio wrote:
>>>> 2013/2/4 Sebastien FLAESCH<sf AT 4js.com>:
>>>>> Frediano,
>>>>>
>>>>> I believe it's related to the client charset conversions...
>>>>>
>>>>> I did following simple test:
>>>>>
>>>>> My database collation is CP1252, so to avoid iconv usage, in the ODBC
>>>>> data source definition, I have now:
>>>>>
>>>>> ClientCharset = CP1252
>>>>>
>>>>> The program uses only pure ASCII chars, so this is ok.
>>>>>
>>>>> Now there is no more iconv error in the log, and the first call to
>>>>> SQLGetData() returns 10000 bytes as expected...
>>>>>
>>>>> If fact for character data, ODBC returns the char pieces with an
>>>>> additional \0 terminator.
>>>>>
>>>>> So, with a buffer of 4096 bytes:
>>>>>
>>>>> Call #1 to SQLGetData() returns len=10000 bytes (ok)
>>>>> Call #2 to SQLGetData() returns len=5905 bytes
>>>>> there was 5905 bytes left to read, because only 4095 bytes could
>>>>> be written to the buffer, because of additional \0 terminator.
>>>>> 10000 - 4095 = 5905 ...
>>>>> Call #3 to SQLGetData() returns len=1810 bytes
>>>>> 5905 - 4095 = 1810 ...
>>>>> Last call: 1810 bytes fit in the 4096 buffer.
>>>>>
>>>>
>>>> Well, this is expected. But I don't understand what's going wrong with
>>>> Solaris.
>>>> Could be possible that Solaris do not convert some characters and so
>>>> it convert a byte characters to 0 bytes ??
>>>
>>> Possible...
>>>
>>>> Are you sure that there is only ASCII characters (0-127) ??
>>>
>>> Pure ASCII.
>>>
>>>> Why should Solaris report some warning converting ASCII ??
>>>
>>> My C locale is set to POSIX/C, I don't know why it ends as ISO-8859-1
>>> for the iconv conversions in FreeTDS. Maybe it's the default.
>>>
>>> Is there some mapping between the C locale (LANG/LC_ALL) and the iconv
>>> locale name (I know these can be different names, how is this mapped?)
>>>
>>>> A dump would be helpful. Send privately to me if it contains sensitive
>>>> information.
>>>
>>> I send you a dump.
>>>
>>>> You could try using libiconv.
>>>
>>> We'll try this too.
>>>
>>>>> I hope this helps...
>>>>>
>>>>> Seb
>>>>>
>>>>
>>>> Frediano
>>>>
>>>>
>>>>>
>>>>> On 02/03/2013 12:12 PM, Frediano Ziglio wrote:
>>>>>> Hi,
>>>>>> where is SQL_DATA_AT_EXEC in your code?
>>>>>>
>>>>>> Which DM are you using? Is possible that for some reason DM call some
>>>>>> additional function to do some checks. This could discovered enabling
>>>>>> FreeTDS dump.
>>>>>>
>>>>>> Quire strange it works correctly on Linux but not on Solaris.
>>>>>> Does data contains no-ASCII characters?
>>>>>> How long are usually these TEXT fields?
>>>>>> Which TDS protocol version are you using?
>>>>>> There is a specific getdata test for this stuff and should work.
>>>>>>
>>>>>> Regards,
>>>>>> Frediano
>>>>>>
>>>>>> 2013/2/2 Sebastien FLAESCH<sf AT 4js.com>:
>>>>>>> Hi Frediano,
>>>>>>>
>>>>>>> FYI, we use a single-byte encoding (LANG=POSIX) ...
>>>>>>>
>>>>>>> Here is the code we use, nothing particular, this is the same code we
>>>>>>> use for different ODBC drivers...
>>>>>>>
>>>>>>> #define BLOB_BLOCK_SIZE 4096
>>>>>>> ....
>>>>>>> static SQLRETURN getBlobPieces(SqlStatement * st, SQLUSMALLINT fpos,
>>>>>>> SQLSMALLINT ctype, MyLocator *
>>>>>>> loc,
>>>>>>> FILE * out, SQLLEN * plen)
>>>>>>> {
>>>>>>> SQLRETURN r = 0;
>>>>>>> char buf[BLOB_BLOCK_SIZE];
>>>>>>> SQLLEN len;
>>>>>>> int i = 0;
>>>>>>> while (1) {
>>>>>>> r = SQLGetData(st->stmtHandle, fpos, ctype,
>>>>>>> (SQLPOINTER) buf, sizeof(buf),&len);
>>>>>>> if (!SQL_SUCCEEDED(r) || len == 0 || len ==
>>>>>>> SQL_NULL_DATA)
>>>>>>> break;
>>>>>>> if (len> (SQLLEN) sizeof(buf) || len == SQL_NO_TOTAL)
>>>>>>> len = sizeof(buf);
>>>>>>> /* Char data piece get \0, these must be removed! */
>>>>>>> if (ctype == SQL_C_CHAR) {
>>>>>>> if (buf[len - 1] == '\0') {
>>>>>>> len--;
>>>>>>> }
>>>>>>> }
>>>>>>> fwrite(buf, 1, len, out);
>>>>>>> }
>>>>>>> *plen = len;
>>>>>>> return r;
>>>>>>> }
>>>>>>>
>>>>>>> Am I missusing SQLGetData()? As you can see, I modify the len after
>>>>>>> Does this affect the next call to SQLGetData()?
>>>>>>>
>>>>>>> Anyway, the INITIAL len returned by SQLGetData() is for sure:
>>>>>>>
>>>>>>> (total-bytes - 1)
>>>>>>>
>>>>>>> When using UTF-8, I get SQL_NO_TOTAL in the first call, but in ASCII,
>>>>>>> I get the (total size - 1)...
>>>>>>>
>>>>>>> Seb
>>>>>>>
>>>>>>> On 02/02/2013 08:52 AM, Frediano Ziglio wrote:
>>>>>>>> Il giorno 31/gen/2013, alle ore 14:28, Sebastien FLAESCH<sf AT 4js.com>
>>>>>>>> ha scritto:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> Context:
>>>>>>>>>
>>>>>>>>> $ uname -a
>>>>>>>>> SunOS s64x100 5.10 Generic_120012-14 i86pc i386 i86pc
>>>>>>>>>
>>>>>>>>> FreeTDS versions tested:
>>>>>>>>> - freetds-0.92.dev.20120312
>>>>>>>>> - freetds-dev.0.92.377
>>>>>>>>>
>>>>>>>>> When fetching a TEXT from the database (SQL Server 2005), using
>>>>>>>>> SQL_DATA_AT_EXEC,
>>>>>>>>> it appears that the SQLGetData() function returns the size of the
>>>>>>>>> text data,
>>>>>>>>> minus one...
>>>>>>>>>
>>>>>>>>> I have tested the same FreeTDS version on a Linux 32b, and
>>>>>>>>> SQLGetData() returns
>>>>>>>>> the correct length as expected...
>>>>>>>>>
>>>>>>>>
>>>>>>>> Strange! How do you read the length?
>>>>>>>>
>>>>>>>>> I will continue to investigate, but I wanted to ask if this is a
>>>>>>>>> know issue.
>>>>>>>>>
>>>>>>>>> When connecting to SQL Server 2008 R2, this does not occur, because
>>>>>>>>> SQLGetData()
>>>>>>>>> returns SQL_NO_TOTAL and we fetch text pieces in a different way...
>>>>>>>>>
>>>>>>>>
>>>>>>>> This is done to deal with different encodings.
>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Seb
>>>>>>>>
>>>>>>>> Frediano
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> FreeTDS mailing list
>>>>>>>> FreeTDS AT lists.ibiblio.org
>>>>>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> FreeTDS mailing list
>>>>>>> FreeTDS AT lists.ibiblio.org
>>>>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>>>> _______________________________________________
>>>>>> FreeTDS mailing list
>>>>>> FreeTDS AT lists.ibiblio.org
>>>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> FreeTDS mailing list
>>>>> FreeTDS AT lists.ibiblio.org
>>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>> _______________________________________________
>>>> FreeTDS mailing list
>>>> FreeTDS AT lists.ibiblio.org
>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>>
>>>
>>> _______________________________________________
>>> FreeTDS mailing list
>>> FreeTDS AT lists.ibiblio.org
>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>
>>
>> _______________________________________________
>> FreeTDS mailing list
>> FreeTDS AT lists.ibiblio.org
>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>
>
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds




Archive powered by MHonArc 2.6.24.

Top of Page