Skip to Content.
Sympa Menu

freetds - Re: [freetds] freetds async usage

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Tolga Ceylan <tolga.ceylan AT gmail.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] freetds async usage
  • Date: Mon, 10 Nov 2014 17:28:32 -0800

That's nice. Just looked at pymssql code. I'm using ct_lib interface,
so I didn't notice dbiordesc() before.

But this is still a bit tricky, TCP send can also block. There's also
risk of blocking during partial reads. These are implemented as
while/for loops in freetds until desired buffer
size is filled in.

To get around this, we'd have to guarantee that there's always enough
data to read/write. In case of recv, we can use MSG_PEEK to determine
the size, but with the extra cost of additional system call. In case
of send, I'm not sure there's any way...

And then there's SSL and DNS resolver if applicable (luckily not in my case.)

I also saw this:

https://lists.ibiblio.org/sympa/arc/freetds/2013q4/028606.html

Conceptually, for me the ideal way would be to perform all network I/O
somewhere else (different thread, different library, etc.) and just
feed this stream of data in/out to freetds. Sounds like significant
amount of work. :-(



On Mon, Nov 10, 2014 at 2:46 PM, Marc Abramowitz <msabramo AT gmail.com> wrote:
> In the specific case of Python, I happen to know that pymssql supports
> concurrency through gevent, because I added it. That was trivial to add,
> because FreeTDS gives the file descriptor it's waiting on and gevent can be
> told to run other greenlets until that file descriptor has data ready to
> read.
>
> That's Python but if you're using some other platform (which?), you might
> get some ideas by cribbing from pymssql. And perhaps I might get some ideas
> by looking at what you do - maybe you'll improve upon it - so let me know
> if you go that route.
>
> If you want more details, let me know. There was also a thread around that
> time about async where I inquired similarly to you and James K. Lowden
> responded with some thoughts. If you can't find it, let me know and I'll
> see if I have it in my notes.
>
> -Marc
> http://marc-abramowitz.com
> Sent from my iPhone 4S
>
>
>> On Nov 10, 2014, at 11:26 AM, Tolga Ceylan <tolga.ceylan AT gmail.com> wrote:
>>
>> It's more of an open question, any approach/implementation is OK.
>>
>> This could be via ct_poll() sort of approach or freetds could expose
>> sockets
>> for the application to monitor via epoll/poll, etc.
>>
>> I've got no strict requirements or any dependencies to any specific
>> library.
>>
>> In other words, I want to manage multiple connections in a single thread
>> where freetds API should not block.
>>
>>
>>
>>> On Mon, Nov 10, 2014 at 2:01 AM, Frediano Ziglio <freddy77 AT gmail.com>
>>> wrote:
>>> 2014-11-07 21:50 GMT+00:00 Tolga Ceylan <tolga.ceylan AT gmail.com>:
>>>
>>>> I know this is not possible with the current code, but I was wondering
>>>> if anybody worked on this in the past.
>>>>
>>>> How hard would it be to add async support to freetds?
>>>>
>>>> For example, linux/epoll support?
>>>>
>>>> Cheers,
>>>> Tolga Ceylan
>>>
>>> Hi,
>>> different libraries have different ideas for async implementations. What
>>> do you mean by async with linux/epoll? Which library are you using?
>>>
>>> Regards,
>>> Frediano
>>> _______________________________________________
>>> FreeTDS mailing list
>>> FreeTDS AT lists.ibiblio.org
>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>> _______________________________________________
>> FreeTDS mailing list
>> FreeTDS AT lists.ibiblio.org
>> http://lists.ibiblio.org/mailman/listinfo/freetds
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds




Archive powered by MHonArc 2.6.24.

Top of Page