FreeTDS Development Group

[Dann Corbit <DCorbit AT connx.com>]

This is Microsoft's TDS grammar:
http://msdn.microsoft.com/en-us/library/dd304582%28v=PROT.13%29.aspx

A grammar can be turned into a parser and tree walker with a tool like the 
Gold Parser:
http://www.devincook.com/goldparser/

-----Original Message-----
From: freetds-bounces AT lists.ibiblio.org 
[mailto:freetds-bounces AT lists.ibiblio.org] On Behalf Of 
jklowden AT schemamania.org
Sent: Thursday, August 18, 2011 3:02 PM
To: FreeTDS Development Group
Subject: [freetds] a better libtds

I would like to re-engineer libtds from the ground up as a finite state 
machine.  

Fundamentally, libtds is a TDS parser.  Packets arrive in pre-determined 
sequences.  Packets, even variable-length fields within packets, are well 
defined.  I don't know if TDS is a *regular* language, but I suspect so.  
It's definitely a parseable one.  


Why do this?  libtds is quite ad hoc; it reflects its origins as a discovery 
platform for understanding the TDS protocol.  Consequently, it's hard to 
understand and hasn't attracted a new developer in five years.  Clear 
principles and architecture would benefit the library and the project.  

Technically, I would put forward these goals:  

0.  Reduce TDS description to tabular form, to be used as input the 
parser-generator.  Can also be used to generate structs for each packet type 
(see #5).  Can also be used to improve TDS documentation.  

1.  More predictable and rational flow of control, essential for thread 
safety.  

2.  Clear factoring of protocol versions. 

3.  Better RAII, clearer memory ownership.  

4.  Removal of iconv from TDS layer.  libtds should be transparent. 
Charset issues are binding issues; client libraries need to support access to 
the raw data.  Cf. dbdata() and nvarchar columns.  

5.  Packet-based access to the network.  Data exchange between the client 
libraries and libtds should be by packet.  

    I do not mean that each client library should chase every packet 1:1 on 
the wire.  In particular, libtds has to scoop up the DONE_IN_PROC and compute 
packets after e.g. the final dbnextrow().  

6.  The same state machine can inform a server implementation.  

There are non-technical goals, too:

7.  Attract more developers by adopting a classic, computer-science approach. 
 I venture to say most programmers never get to work on a finite-state 
machine.  

8.  Make FreeTDS portable to other RDBMSs.  Cf.
http://freedb.schemamania.org.  

9.  Make client library maintenance easier.  Consider this from
dbnextrow():

const int mask = 
TDS_STOPAT_ROWFMT|TDS_RETURN_DONE|TDS_RETURN_ROW|TDS_RETURN_COMPUTE;
...
switch (tds_process_tokens(tds, &res_type, NULL, mask)) {

What would your mother say if she saw you writing C like that?  

Asynchronous calls comport with a by-packet interface design.  To implement 
dbpoll()/dbdataready(), db-lib would call something like:

        int tds_next_packet_type(TDS*);
        
which would return the TDS marker if a packet is available, or -1 if not.  
The blocking calls would be:

        int tds_get_XXX(TDS*, struct XXX *); 

I once thought C++ was the answer to libtds's problems, and I could still be 
convinced to use C++.  But I think the real answer is a formal approach.  

I've been reluctant to undertake all this myself, though.  I see no point in 
writing a new libtds unless the client libraries are updated to use it, and I 
don't want to take on more than db-lib.  If I *did* write libtds2 *and* 
change db-lib to use it, and that's all that happened, I'd have split the 
project, probably making things worse, not better.  For the sake of the 
project, I really need agreement before I can start.  

--jkl
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds