Skip to Content.
Sympa Menu

freetds - Re: [freetds] freetds async usage

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Frediano Ziglio <freddy77 AT gmail.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] freetds async usage
  • Date: Tue, 11 Nov 2014 10:06:38 +0000

Now is much cleaner. libTDS and mainly FreeTDS have not the
possibility of "returning instead of waiting". However it uses no
blocking sockets and poll (select for compatibility only today). To
implement green thread I would try to use coroutines. That is if we
detect that there is not data available at socket level queue the
socket for read/write select and switch the context to another thread
(yes, look like what kernels do!).

Actual code use two sockets, one is connected to mssql and another to
signal cancellation.

Frediano

2014-11-11 1:28 GMT+00:00 Tolga Ceylan <tolga.ceylan AT gmail.com>:
>
> That's nice. Just looked at pymssql code. I'm using ct_lib interface,
> so I didn't notice dbiordesc() before.
>
> But this is still a bit tricky, TCP send can also block. There's also
> risk of blocking during partial reads. These are implemented as
> while/for loops in freetds until desired buffer
> size is filled in.
>
> To get around this, we'd have to guarantee that there's always enough
> data to read/write. In case of recv, we can use MSG_PEEK to determine
> the size, but with the extra cost of additional system call. In case
> of send, I'm not sure there's any way...
>
> And then there's SSL and DNS resolver if applicable (luckily not in my
> case.)
>
> I also saw this:
>
> https://lists.ibiblio.org/sympa/arc/freetds/2013q4/028606.html
>
> Conceptually, for me the ideal way would be to perform all network I/O
> somewhere else (different thread, different library, etc.) and just
> feed this stream of data in/out to freetds. Sounds like significant
> amount of work. :-(
>
>
>
> On Mon, Nov 10, 2014 at 2:46 PM, Marc Abramowitz <msabramo AT gmail.com> wrote:
> > In the specific case of Python, I happen to know that pymssql supports
> > concurrency through gevent, because I added it. That was trivial to add,
> > because FreeTDS gives the file descriptor it's waiting on and gevent can
> > be told to run other greenlets until that file descriptor has data ready
> > to read.
> >
> > That's Python but if you're using some other platform (which?), you might
> > get some ideas by cribbing from pymssql. And perhaps I might get some
> > ideas by looking at what you do - maybe you'll improve upon it - so let
> > me know if you go that route.
> >
> > If you want more details, let me know. There was also a thread around
> > that time about async where I inquired similarly to you and James K.
> > Lowden responded with some thoughts. If you can't find it, let me know
> > and I'll see if I have it in my notes.
> >
> > -Marc
> > http://marc-abramowitz.com
> > Sent from my iPhone 4S
> >
> >
> >> On Nov 10, 2014, at 11:26 AM, Tolga Ceylan <tolga.ceylan AT gmail.com>
> >> wrote:
> >>
> >> It's more of an open question, any approach/implementation is OK.
> >>
> >> This could be via ct_poll() sort of approach or freetds could expose
> >> sockets
> >> for the application to monitor via epoll/poll, etc.
> >>
> >> I've got no strict requirements or any dependencies to any specific
> >> library.
> >>
> >> In other words, I want to manage multiple connections in a single thread
> >> where freetds API should not block.
> >>
> >>
> >>
> >>> On Mon, Nov 10, 2014 at 2:01 AM, Frediano Ziglio <freddy77 AT gmail.com>
> >>> wrote:
> >>> 2014-11-07 21:50 GMT+00:00 Tolga Ceylan <tolga.ceylan AT gmail.com>:
> >>>
> >>>> I know this is not possible with the current code, but I was wondering
> >>>> if anybody worked on this in the past.
> >>>>
> >>>> How hard would it be to add async support to freetds?
> >>>>
> >>>> For example, linux/epoll support?
> >>>>
> >>>> Cheers,
> >>>> Tolga Ceylan
> >>>
> >>> Hi,
> >>> different libraries have different ideas for async implementations.
> >>> What
> >>> do you mean by async with linux/epoll? Which library are you using?
> >>>
> >>> Regards,
> >>> Frediano




Archive powered by MHonArc 2.6.24.

Top of Page