FreeTDS Development Group

["Dann Corbit" <DCorbit AT connx.com>]

First, a word of explanation:

We are using the FreeTDS driver in a strange way.  We need lots of
functionality not present in the FreeTDS driver, and so we are using it
as a sort of low level file source.  We need (for instance) updateable
cursors and many other things that are not present (we have already
implemented these things but we do not put them into the FreeTDS driver
because we are not currently familiar enough with the code base to do it
effectively and on time.)

We have another driver that sits over top of the FreeTDS driver and
gives it a very full capability as not only an ODBC driver but also
OLEDB, .NET, JDBC and with access to hundreds of functions like standard
deviation, etc. that do not exist in the original database.  Further, we
have to be able to transparently join Sybase and SQL*Server using the
driver with any other database like Adabas or IMS or Rdb, etc.
transparently and without programming.

You can see how our products work by looking at what you may find here:
http://www.connx.com/

And so our test cases will be impossible or at least very difficult for
you to reproduce because we have a very large layer that sits over top
of the actual FreeTDS driver.

> -----Original Message-----
> From: freetds-bounces AT lists.ibiblio.org [mailto:freetds-
> bounces AT lists.ibiblio.org] On Behalf Of ZIGLIO, Frediano, VF-IT
> Sent: Wednesday, January 16, 2008 1:20 AM
> To: FreeTDS Development Group
> Subject: Re: [freetds] Ongoing work for released version of FreeTDS
> version0, 64, 0, 6182
> 
> >
> > We are making corrections to FreeTDS driver version 64.
> >
> >
> >
> > We have found some important bugs and fixed them.
> >
> >
> >
> > Of course, we will contribute back the source so that anyone who
wants
> > to can back-patch the current development version.
> >
> >
> >
> > It is also possible that some of our changes cannot be used by the
> > standard version of FreeTDS, so any merge should be examined
closely.
> >
> >
> >
> > We are also implementing extensive testing over a large number of
> > platforms (there will be hundreds of thousands of tests performed
> > against the source).
> >
> >
> 
> Fine.
> 
> >
> > I guess what I am saying is that the FreeTDS group is
> > essentially going
> > to get free commercial support in a sense (we already have
> > and are going
> > to continue to expend resources to improve the driver).
> >
> >
> >
> > Of course we expect also to benefit from continued growth of
> > the FreeTDS
> > project.
> >
> 
> We are releasing version 0.82 and I noted that you are based on 0.64
> without post patch. I think it would be better to update. I manage to
> get a unified diff against 0.64. Included with notes

Your work was amazingly quick.  I am going to like working with this
organization.
 
> diff -ub ./convert_tds2sql.c new/convert_tds2sql.c
> --- ./convert_tds2sql.c       2006-02-24 22:14:24.000000000 +0100
> +++ new/convert_tds2sql.c     2008-01-16 09:40:37.938249787 +0100
> @@ -54,6 +54,9 @@
>       DATE_STRUCT *dsp;
>       TIME_STRUCT *tsp;
>       TIMESTAMP_STRUCT *tssp;
> +
> +     TIMESTAMP_STRUCT ts;
> +
>       SQL_NUMERIC_STRUCT *num;
> 
>       int ret = TDS_FAIL;
> @@ -140,11 +143,11 @@
> 
>               tds_datecrack(SYBDATETIME, &(ores.dt), &dr);
> 
> -             dsp = (DATE_STRUCT *) dest;
> +             ts.year = dr.year;
> +             ts.month = dr.month + 1;
> +             ts.day = dr.day;
> 
> -             dsp->year = dr.year;
> -             dsp->month = dr.month + 1;
> -             dsp->day = dr.day;
> +             memcpy(dest, &ts, sizeof(DATE_STRUCT));
> 
>               ret = sizeof(DATE_STRUCT);
>               break;
> @@ -157,11 +160,11 @@
> 
>               tds_datecrack(SYBDATETIME, &(ores.dt), &dr);
> 
> -             tsp = (TIME_STRUCT *) dest;
> +             ts.hour = dr.hour;
> +             ts.minute = dr.minute;
> +             ts.second = dr.second;
> 
> -             tsp->hour = dr.hour;
> -             tsp->minute = dr.minute;
> -             tsp->second = dr.second;
> +             memcpy(dest, &ts, sizeof(TIME_STRUCT));
> 
>               ret = sizeof(TIME_STRUCT);
>               break;
> @@ -174,15 +177,15 @@
> 
>               tds_datecrack(SYBDATETIME, &(ores.dt), &dr);
> 
> -             tssp = (TIMESTAMP_STRUCT *) dest;
> +             ts.year = dr.year;
> +             ts.month = dr.month + 1;
> +             ts.day = dr.day;
> +             ts.hour = dr.hour;
> +             ts.minute = dr.minute;
> +             ts.second = dr.second;
> +             ts.fraction = dr.millisecond * 1000000u;
> 
> -             tssp->year = dr.year;
> -             tssp->month = dr.month + 1;
> -             tssp->day = dr.day;
> -             tssp->hour = dr.hour;
> -             tssp->minute = dr.minute;
> -             tssp->second = dr.second;
> -             tssp->fraction = dr.millisecond * 1000000u;
> +             memcpy(dest, &ts, sizeof(TIMESTAMP_STRUCT));
> 
>               ret = sizeof(TIMESTAMP_STRUCT);
>               break;
> @@ -190,7 +193,7 @@
>  #ifdef SQL_C_SBIGINT
>       case SQL_C_SBIGINT:
>       case SQL_C_UBIGINT:
> -             *((TDS_INT8 *) dest) = ores.bi;
> +             memcpy(dest, &ores, sizeof(TDS_INT8));
>               ret = sizeof(TDS_INT8);
>               break;
>  #endif
> @@ -198,14 +201,14 @@
>       case SQL_C_LONG:
>       case SQL_C_SLONG:
>       case SQL_C_ULONG:
> -             *((TDS_INT *) dest) = ores.i;
> +             memcpy(dest, &ores, sizeof(TDS_INT));
>               ret = sizeof(TDS_INT);
>               break;
> 
>       case SQL_C_SHORT:
>       case SQL_C_SSHORT:
>       case SQL_C_USHORT:
> -             *((TDS_SMALLINT *) dest) = ores.si;
> +             memcpy(dest, &ores, sizeof(TDS_SMALLINT));
>               ret = sizeof(TDS_SMALLINT);
>               break;
> 
> @@ -218,12 +221,12 @@
>               break;
> 
>       case SQL_C_DOUBLE:
> -             *((TDS_FLOAT *) dest) = ores.f;
> +             memcpy(dest, &ores, sizeof(TDS_FLOAT));
>               ret = sizeof(TDS_FLOAT);
>               break;
> 
>       case SQL_C_FLOAT:
> -             *((TDS_REAL *) dest) = ores.r;
> +             memcpy(dest, &ores, sizeof(TDS_REAL));
>               ret = sizeof(TDS_REAL);
>               break;
> 
> 
> 
> This code only fixes unaligned binded data. Binded data should be
> aligned so I don't think should be merged. Also code for TIME_STRUCT
> introduce a read of uninitialized data, I don't think is what you
> wanted.

I will look into that, thanks.

> 
> diff -ub ./odbc.c new/odbc.c
> --- ./odbc.c  2006-06-29 23:38:13.000000000 +0200
> +++ new/odbc.c        2008-01-16 09:40:47.924301437 +0100
> @@ -448,7 +448,6 @@
>       tmp_size = stmt->ard->header.sql_desc_array_size;
>       stmt->ard->header.sql_desc_array_size = stmt->sql_rowset_size;
>       tmp_offset = stmt->ard->header.sql_desc_bind_offset_ptr;
> -     stmt->ard->header.sql_desc_bind_offset_ptr = NULL;
> 
>       /* TODO errors are sligthly different ... perhaps it's better to
> leave DM do this job ?? */
>       ret = _SQLFetch(stmt);
> 
> 
> 
> >From odbc documentation of SQLExetendedFetch:
> 
> SQLExtendedFetch does not support binding offsets (the
> SQL_ATTR_ROW_BIND_OFFSET_PTR statement attribute).
> 
> Did you have a test that state the opposite??

Until we made this correction, most test sets crashed when they got to
the point of using the bound offsets.  In fact, under Windows the
Microsoft metadata inquiries crashed (before we even got to our actual
queries).  I believe that those problems showed up initially when we
were using the OLEDB interface.  I am not sure how I can make the tests
meaningful, but I can undo the change and demonstrate the crash very
easily (we had literally hundreds of crashes caused by setting the
offset to NULL).

Because we use your driver as something underneath, it may be difficult
to easily produce a scenario that is easily reproducible for you but I
will try to do it when I get some time.  Perhaps if I turn on ODBC
tracing I can produce an exact sequence.

> @@ -2356,7 +2355,8 @@
>                       return SQL_ERROR;
> 
>               /* TODO other types for date ?? SQL_TYPE_DATE,
> SQL_TYPE_TIME */
> -             if (drec->sql_desc_concise_type == SQL_TIMESTAMP)
> +             if ((drec->sql_desc_concise_type == SQL_TIMESTAMP) ||
> +                 (drec->sql_desc_concise_type == SQL_TYPE_TIMESTAMP)
> )
>                       drec->sql_desc_length = sizeof("2000-01-01
> 12:00:00.0000")-1;
>               else
>                       drec->sql_desc_length = col->column_size;
> 
> 
> Fine, I think I can commit this without problems.
> 
> 
> @@ -3536,7 +3536,7 @@
>               }
> 
>               tdsdump_log(TDS_DBG_INFO1, "Creating prepared
> statement\n");
> -             if (tds_submit_prepare(tds, stmt->prepared_query, NULL,
> &stmt->dyn, params) == TDS_FAIL) {
> +             if (tds_submit_prepare(tds, stmt->prepared_query, NULL,
> &stmt->dyn, params) == TDS_FAIL && stmt->need_reprepare != 1) {
>                       tds_free_param_results(params);
>                       ODBC_RETURN(stmt, SQL_ERROR);
>               }
> 
> Mmm... this means deferring prepare on submit error... yes this can
> help!
> Do you have any test for this?

I can tell you how our finding came about.  We are creating updateable
cursors and we bind the columns after the prepare.  Without the change,
it always returns an error during the prepare phase.

> diff -ub ./odbc_util.c new/odbc_util.c
> --- ./odbc_util.c     2006-02-24 22:14:25.000000000 +0100
> +++ new/odbc_util.c   2008-01-16 09:39:26.306000956 +0100
> @@ -505,10 +505,12 @@
>               size = column_prec + 2;
>               break;
>       case SQL_DATE:
> +     case SQL_TYPE_DATE:
>               /* FIXME check always yyyy-mm-dd ?? */
>               size = 19;
>               break;
>       case SQL_TIME:
> +     case SQL_TYPE_TIME:
>               /* FIXME check always hh:mm:ss[.fff] */
>               size = 19;
>               break;
> 
> Fine, I think I can commit this without problems.
> 
> >
> >
> > At any rate, attached is a current (development) zip of the
> > odbc source.
> > I have not carefully examined if we made any changes to other
> > sub-projects or even what has been changed in this zip, but you can
> > easily determine it yourself by using diff.  We were not able
> > to use the
> > code without these changes.
> >
> >
> >
> > We are interested in the current work involving UNICODE.
> >
> >
> >
> > Is it possible to get a summary about the current state of that
work?
> >
> 
> 
> Depending on what you means by Unicode reply can change from
implemented
> to unimplemented.

We are sure that we can make it work with or without help (we have seen
the Unicode properly represented in the tabular data stream itself
before it was downgraded to char).  We need either UTF8 or 16 bit
Unicode from a Unicode database table to be delivered to the driver and
passed up the chain (we have data handlers for both).
 
> freddy77
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds