FreeTDS Development Group

[Sebastien FLAESCH <sf AT 4js.com>]

On 02/04/2013 06:57 PM, Frediano Ziglio wrote:
> 2013/2/4 Sebastien FLAESCH<sf AT 4js.com>:
> > Frediano,
> >
> > I believe it's related to the client charset conversions...
> >
> > I did following simple test:
> >
> > My database collation is CP1252, so to avoid iconv usage, in the ODBC
> > data source definition, I have now:
> >
> > ClientCharset = CP1252
> >
> > The program uses only pure ASCII chars, so this is ok.
> >
> > Now there is no more iconv error in the log, and the first call to
> > SQLGetData() returns 10000 bytes as expected...
> >
> > If fact for character data, ODBC returns the char pieces with an
> > additional \0 terminator.
> >
> > So, with a buffer of 4096 bytes:
> >
> > Call #1 to SQLGetData() returns len=10000 bytes (ok)
> > Call #2 to SQLGetData() returns len=5905 bytes
> >     there was 5905 bytes left to read, because only 4095 bytes could
> >     be written to the buffer, because of additional \0 terminator.
> >     10000 - 4095 = 5905 ...
> > Call #3 to SQLGetData() returns len=1810 bytes
> >      5905 - 4095 = 1810 ...
> > Last call: 1810 bytes fit in the 4096 buffer.
>
> Well, this is expected. But I don't understand what's going wrong with 
> Solaris.
> Could be possible that Solaris do not convert some characters and so
> it convert a byte characters to 0 bytes ??

Possible...

> Are you sure that there is only ASCII characters (0-127) ??

Pure ASCII.

> Why should Solaris report some warning converting ASCII ??

My C locale is set to POSIX/C, I don't know why it ends as ISO-8859-1
for the iconv conversions in FreeTDS. Maybe it's the default.

Is there some mapping between the C locale (LANG/LC_ALL) and the iconv
locale name (I know these can be different names, how is this mapped?)

> A dump would be helpful. Send privately to me if it contains sensitive
> information.

I send you a dump.

> You could try using libiconv.

We'll try this too.

> > I hope this helps...
> >
> > Seb
>
> Frediano
>
>
> > On 02/03/2013 12:12 PM, Frediano Ziglio wrote:
> > > Hi,
> > >     where is SQL_DATA_AT_EXEC in your code?
> > >
> > > Which DM are you using? Is possible that for some reason DM call some
> > > additional function to do some checks. This could discovered enabling
> > > FreeTDS dump.
> > >
> > > Quire strange it works correctly on Linux but not on Solaris.
> > > Does data contains no-ASCII characters?
> > > How long are usually these TEXT fields?
> > > Which TDS protocol version are you using?
> > > There is a specific getdata test for this stuff and should work.
> > >
> > > Regards,
> > >     Frediano
> > >
> > > 2013/2/2 Sebastien FLAESCH<sf AT 4js.com>:
> > > > Hi Frediano,
> > > >
> > > > FYI, we use a single-byte encoding (LANG=POSIX) ...
> > > >
> > > > Here is the code we use, nothing particular, this is the same code we
> > > > use for different ODBC drivers...
> > > >
> > > > #define BLOB_BLOCK_SIZE 4096
> > > > ....
> > > > static SQLRETURN getBlobPieces(SqlStatement * st, SQLUSMALLINT fpos,
> > > >                                   SQLSMALLINT ctype, MyLocator * loc,
> > > >                                   FILE * out, SQLLEN * plen)
> > > > {
> > > >        SQLRETURN r = 0;
> > > >        char buf[BLOB_BLOCK_SIZE];
> > > >        SQLLEN len;
> > > >        int i = 0;
> > > >        while (1) {
> > > >            r = SQLGetData(st->stmtHandle, fpos, ctype,
> > > >                           (SQLPOINTER) buf, sizeof(buf),&len);
> > > >            if (!SQL_SUCCEEDED(r) || len == 0 || len == SQL_NULL_DATA)
> > > >                break;
> > > >            if (len>   (SQLLEN) sizeof(buf) || len == SQL_NO_TOTAL)
> > > >                len = sizeof(buf);
> > > >            /* Char data piece get \0, these must be removed! */
> > > >            if (ctype == SQL_C_CHAR) {
> > > >                if (buf[len - 1] == '\0') {
> > > >                    len--;
> > > >                }
> > > >            }
> > > >            fwrite(buf, 1, len, out);
> > > >        }
> > > >        *plen = len;
> > > >        return r;
> > > > }
> > > >
> > > > Am I missusing SQLGetData()? As you can see, I modify the len after
> > > > Does this affect the next call to SQLGetData()?
> > > >
> > > > Anyway, the INITIAL len returned by SQLGetData() is for sure:
> > > >
> > > >       (total-bytes - 1)
> > > >
> > > > When using UTF-8, I get SQL_NO_TOTAL in the first call, but in ASCII,
> > > > I get the (total size - 1)...
> > > >
> > > > Seb
> > > >
> > > > On 02/02/2013 08:52 AM, Frediano Ziglio wrote:
> > > > > Il giorno 31/gen/2013, alle ore 14:28, Sebastien FLAESCH<sf AT 4js.com>    ha 
> > > > > scritto:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Context:
> > > > > >
> > > > > > $ uname -a
> > > > > > SunOS s64x100 5.10 Generic_120012-14 i86pc i386 i86pc
> > > > > >
> > > > > > FreeTDS versions tested:
> > > > > > - freetds-0.92.dev.20120312
> > > > > > - freetds-dev.0.92.377
> > > > > >
> > > > > > When fetching a TEXT from the database (SQL Server 2005), using 
> > > > > > SQL_DATA_AT_EXEC,
> > > > > > it appears that the SQLGetData() function returns the size of the text data,
> > > > > > minus one...
> > > > > >
> > > > > > I have tested the same FreeTDS version on a Linux 32b, and SQLGetData() 
> > > > > > returns
> > > > > > the correct length as expected...
> > > > >
> > > > > Strange! How do you read the length?
> > > > >
> > > > > > I will continue to investigate, but I wanted to ask if this is a know issue.
> > > > > >
> > > > > > When connecting to SQL Server 2008 R2, this does not occur, because 
> > > > > > SQLGetData()
> > > > > > returns SQL_NO_TOTAL and we fetch text pieces in a different way...
> > > > >
> > > > > This is done to deal with different encodings.
> > > > >
> > > > > > Thanks
> > > > > > Seb
> > > > >
> > > > > Frediano
> > > > >
> > > > > _______________________________________________
> > > > > FreeTDS mailing list
> > > > > FreeTDS AT lists.ibiblio.org
> > > > > http://lists.ibiblio.org/mailman/listinfo/freetds
> > > >
> > > > _______________________________________________
> > > > FreeTDS mailing list
> > > > FreeTDS AT lists.ibiblio.org
> > > > http://lists.ibiblio.org/mailman/listinfo/freetds
> > > _______________________________________________
> > > FreeTDS mailing list
> > > FreeTDS AT lists.ibiblio.org
> > > http://lists.ibiblio.org/mailman/listinfo/freetds
> >
> > _______________________________________________
> > FreeTDS mailing list
> > FreeTDS AT lists.ibiblio.org
> > http://lists.ibiblio.org/mailman/listinfo/freetds
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds