FreeTDS Development Group

[Frediano Ziglio <freddy77 AT gmail.com>]

2011/8/19  <jklowden AT schemamania.org>:
> I would like to re-engineer libtds from the ground up as a finite
> state machine.
>
> Fundamentally, libtds is a TDS parser.  Packets arrive in
> pre-determined sequences.  Packets, even variable-length fields
> within packets, are well defined.  I don't know if TDS is a *regular*
> language, but I suspect so.  It's definitely a parseable one.
>
> Why do this?  libtds is quite ad hoc; it reflects its origins as a
> discovery platform for understanding the TDS protocol.  Consequently,
> it's hard to understand and hasn't attracted a new developer in five
> years.  Clear principles and architecture would benefit the library
> and the project.
>
> Technically, I would put forward these goals:
>
> 0.  Reduce TDS description to tabular form, to be used as input the
> parser-generator.  Can also be used to generate structs for each
> packet type (see #5).  Can also be used to improve TDS documentation.
>

Good. This would reduce token.c code and query.c too.

> 1.  More predictable and rational flow of control, essential for
> thread safety.
>
> 2.  Clear factoring of protocol versions.
>

I don't understand these points that much, could you detail a bit?

> 3.  Better RAII, clearer memory ownership.
>
> 4.  Removal of iconv from TDS layer.  libtds should be transparent.
> Charset issues are binding issues; client libraries need to support
> access to the raw data.  Cf. dbdata() and nvarchar columns.
>

Yes, although clients does not expect column names encoded with wide
character sets. And also dblib does not support wide characters.

> 5.  Packet-based access to the network.  Data exchange between the
> client libraries and libtds should be by packet.
>
>    I do not mean that each client library should chase every
> packet 1:1 on the wire.  In particular, libtds has to scoop up the
> DONE_IN_PROC and compute packets after e.g. the final dbnextrow().
>

By packet do you mean stream here, am I right ?

> 6.  The same state machine can inform a server implementation.
>

???

> There are non-technical goals, too:
>
> 7.  Attract more developers by adopting a classic, computer-science
> approach.  I venture to say most programmers never get to work on a
> finite-state machine.
>
> 8.  Make FreeTDS portable to other RDBMSs.  Cf.
> http://freedb.schemamania.org.
>
> 9.  Make client library maintenance easier.  Consider this from
> dbnextrow():
>
> const int mask = 
> TDS_STOPAT_ROWFMT|TDS_RETURN_DONE|TDS_RETURN_ROW|TDS_RETURN_COMPUTE;
> ...
> switch (tds_process_tokens(tds, &res_type, NULL, mask)) {
>
> What would your mother say if she saw you writing C like that?
>
> Asynchronous calls comport with a by-packet interface design.  To
> implement dbpoll()/dbdataready(), db-lib would call something like:
>
>        int tds_next_packet_type(TDS*);
>
> which would return the TDS marker if a packet is available, or -1 if
> not.  The blocking calls would be:
>
>        int tds_get_XXX(TDS*, struct XXX *);
>

Mmm... you have to look at the story of libTDS code at least to not
commit same mistakes. Once you remove code that package and decode
streams/special packets what's left into libTDS? Although this
solutions seems linear it reminds me some old code. libTDS try to
unify some streams like rowfmt/rowfmt2/colinfo/colname. Old client
libraries contains monstrous code with large loops and switch to
couple with dozen of streams. After a first time of unification many
smaller loop was present and some of them was similar but some want to
stop before some stream types, other handle different streams so I
added a loop which take flags to decide which kind of stream to
handle.
The question is: how to handle tds_next_packet_type/tds_get_XXX in
order to avoid monstrous loops and switch?
I agree filling the same structure for every stream lead to the giant
TDSSOCKET structure and would be better if every generated stream
handle could return data readed from wire.

> I once thought C++ was the answer to libtds's problems, and I could
> still be convinced to use C++.  But I think the real answer is a
> formal approach.
>

I must admit C++ have very interesting feature missing in C.

> I've been reluctant to undertake all this myself, though.  I see no
> point in writing a new libtds unless the client libraries are updated
> to use it, and I don't want to take on more than db-lib.  If I *did*
> write libtds2 *and* change db-lib to use it, and that's all that
> happened, I'd have split the project, probably making things worse,
> not better.  For the sake of the project, I really need agreement
> before I can start.
>

Agree. Code grown and grown and some parts become ugly.

Frediano