FreeTDS Development Group

["James K. Lowden" <james.k.lowden AT icloud.com>]

On Thu, 20 Oct 2016 12:40:49 -0400
Benjamin Moody <benjamin.moody AT gmail.com> wrote:

> Let's say that I have a black-box application which produces a
> massive, continuous stream of data and dumps it into an MS SQL Server
> database.  I want to pull the data *out* of this database as quickly
> as it is inserted, and pass it on for further processing in
> near-real-time.

By "black-box", I take it you mean the application cannot be changed,
and its current implementation writes to MS SQL Server.  

> Is it feasible to emulate the behavior of MS SQL Server

Probably not.  

> * Does FreeTDS include low-level protocol code that would be useful in
>   implementing such a stub server, or would it be simpler to start
>   from scratch?

Yes, and yes.  

The FreeTDS server code holds the rudiments of a beginning.  As far as
I know, it has never been used for anything serious.  It doesn't
implement any of the ACID features, nor does it recognize SQL.  

I have long thought that TDS is a very respectable protocol, and with
the right API (and implementation) would be convenient and efficient
peer-to-peer data exchange.  BCP is particularly lean.  It benefits from
its relational origins: everything is a table, and all datatypes are
strictly defined (with a blob escape valve).  Sybase thought so, too;
compared with their db-lib first draft, ct-lib is much more
peer-oriented.  

While libtds could be repurposed to support a server implemementation,
that was never its intended use.  It would be better to begin with a
state machine, and static definitions of all the varieties of packets
supported by different flavors of the protocol.  Client and server
become co-processes exchanging data as TDS pages.  

A generic table-exchange service would be a boon to the industry.
All popular data exchange protocols -- ftp, http, also protocol buffers
& similar -- are based on files or bytestreams.  None are based on the
TDS model: metadata once, rows follow, all bindings at runtime.  
What a shame the problem isn't more broadly understood, and addressed.  

--jkl