FreeTDS Development Group

[jklowden AT schemamania.org]

I would like to re-engineer libtds from the ground up as a finite
state machine.  

Fundamentally, libtds is a TDS parser.  Packets arrive in 
pre-determined sequences.  Packets, even variable-length fields
within packets, are well defined.  I don't know if TDS is a *regular*
language, but I suspect so.  It's definitely a parseable one.  

Why do this?  libtds is quite ad hoc; it reflects its origins as a
discovery platform for understanding the TDS protocol.  Consequently,
it's hard to understand and hasn't attracted a new developer in five
years.  Clear principles and architecture would benefit the library
and the project.  

Technically, I would put forward these goals:  

0.  Reduce TDS description to tabular form, to be used as input the
parser-generator.  Can also be used to generate structs for each
packet type (see #5).  Can also be used to improve TDS documentation.  

1.  More predictable and rational flow of control, essential for
thread safety.  

2.  Clear factoring of protocol versions. 

3.  Better RAII, clearer memory ownership.  

4.  Removal of iconv from TDS layer.  libtds should be transparent. 
Charset issues are binding issues; client libraries need to support
access to the raw data.  Cf. dbdata() and nvarchar columns.  

5.  Packet-based access to the network.  Data exchange between the
client libraries and libtds should be by packet.  

    I do not mean that each client library should chase every
packet 1:1 on the wire.  In particular, libtds has to scoop up the
DONE_IN_PROC and compute packets after e.g. the final dbnextrow().  

6.  The same state machine can inform a server implementation.  

There are non-technical goals, too:

7.  Attract more developers by adopting a classic, computer-science
approach.  I venture to say most programmers never get to work on a
finite-state machine.  

8.  Make FreeTDS portable to other RDBMSs.  Cf.
http://freedb.schemamania.org.  

9.  Make client library maintenance easier.  Consider this from
dbnextrow():

const int mask = 
TDS_STOPAT_ROWFMT|TDS_RETURN_DONE|TDS_RETURN_ROW|TDS_RETURN_COMPUTE;
...
switch (tds_process_tokens(tds, &res_type, NULL, mask)) {

What would your mother say if she saw you writing C like that?  

Asynchronous calls comport with a by-packet interface design.  To
implement dbpoll()/dbdataready(), db-lib would call something like:

        int tds_next_packet_type(TDS*);
        
which would return the TDS marker if a packet is available, or -1 if
not.  The blocking calls would be:

        int tds_get_XXX(TDS*, struct XXX *); 

I once thought C++ was the answer to libtds's problems, and I could
still be convinced to use C++.  But I think the real answer is a
formal approach.  

I've been reluctant to undertake all this myself, though.  I see no
point in writing a new libtds unless the client libraries are updated
to use it, and I don't want to take on more than db-lib.  If I *did*
write libtds2 *and* change db-lib to use it, and that's all that
happened, I'd have split the project, probably making things worse,
not better.  For the sake of the project, I really need agreement 
before I can start.  

--jkl