FreeTDS Development Group

[Brian Bruns <brian.bruns AT gmail.com>]

Checked in some more changes.  When a client exits in the middle of a
query it can leave the member in a confused state, so now we just
close it to be safe.

So please try the latest from CVS and let me know.

Brian

On Mon, 13 Dec 2004 15:02:00 +0000, craigs <s.craig AT andronics.com> wrote:
> Brian,
> 
> the only error message we get from our clients is that we cannot connect
> to database server, this happens in dbopen() which makes sense seen as
> the pool is down, your tests sound encouraging so maybe if i get a
> chance later today i will write a test program in a safe environment to
> hammer the pool, will run in gdb with tdsdump on and see how we go.
> 
> Shaun.
> 
> 
> 
> Brian Bruns wrote:
> 
> >Ok, I'm running now.  This is what I did to test:
> >
> >1. Built a query file with 'select * from <table>' for every table in
> >the database (about a 500 meg database) Good mix of small and large
> >tables.
> >2. Set up the pool for 2 connections
> >3. Spun off tsql to run the queries in parallel, averaging about 150
> >client connections to the pool at any one time.
> >4. Ran for half an hour so far and holding.
> >5. Ran clients up to 1000 and now letting it run for a bit.
> >
> >One thing I did notice is the need for a better scheduling algorithm
> >under heavy load to prevent starvation.
> >
> >Other than that, it seems we are having an issue with some sort of
> >abnormal case.  I did notice the pool go down when a "cancel" was
> >issued by the client.  I'm looking into that, but does that ring any
> >bells?
> >
> >Brian
> >
> >On Mon, 13 Dec 2004 13:28:22 +0000, craigs <s.craig AT andronics.com> wrote:
> >
> >
> >>Brian,
> >>
> >>we only had the new pool running for a short time and had to revert to
> >>an older version pretty quick when we seen there was problems so we
> >>never got a chance to get a stack trace, its very dangerous for us to
> >>put it back on just to get the trace as we have alot of input to our
> >>system. Let me know how the tests go using SQL Server, thanks for your 
> >>time.
> >>
> >>Shaun.
> >>
> >>
> >>
> >>Brian Bruns wrote:
> >>
> >>
> >>
> >>>Shaun,
> >>>
> >>>I ran it continuously for about 3 hours using a simulated workload.
> >>>This was against a Sybase 12.5.2 database, so that may be the issue.
> >>>I can try it from here against SQL Server, but didn't have that access
> >>>over the weekend.  I'll let you know if anything turns up, but so far
> >>>I'm not seeing it.
> >>>
> >>>BTW, I made two distinct fixes, one of which I was almost certain was
> >>>what you were seeing based on your description...oh well, we'll keep
> >>>plugging away.  Was the stack trace the same on the new code?
> >>>
> >>>Brian
> >>>
> >>>
> >>>On Mon, 13 Dec 2004 12:23:01 +0000, craigs <s.craig AT andronics.com> wrote:
> >>>
> >>>
> >>>
> >>>
> >>>>hi Brian,
> >>>>
> >>>>got the latest version off CVS, put it on our server and things didnt go
> >>>>to well! will test it more ourselfs but its really hard for us to
> >>>>duplicate live data but will try, did everything go ok when you tested 
> >>>>it?
> >>>>
> >>>>Shaun.
> >>>>
> >>>>
> >>>>
> >>>>Brian Bruns wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>I just applied a one liner (and your patch as well) to CVS.  Please
> >>>>>give it a try and let me know if it works.  As a bonus, this should
> >>>>>make it much easier to add TDS7/8 support to the pool.
> >>>>>
> >>>>>Cheers,
> >>>>>
> >>>>>Brian
> >>>>>
> >>>>>On Fri, 10 Dec 2004 11:24:59 +0000, craigs <s.craig AT andronics.com> 
> >>>>>wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>thats one hell of a film! thanks for looking into it, next year we 
> >>>>>>have
> >>>>>>predicted that we will average over 3000 clients hitting the pool 
> >>>>>>every
> >>>>>>hour so it would be good if we could track it down sooner rather than 
> >>>>>>later.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>Brian Bruns wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>The good: I was able to get it to crash on the same line.
> >>>>>>>The bad: Looks like a buffer overflow
> >>>>>>>The ugly: This means it originated elsewhere in the code and is
> >>>>>>>therefore hard to track down.
> >>>>>>>
> >>>>>>>Still plugging away at it, i'll let you know what i find.
> >>>>>>>
> >>>>>>>On Fri, 10 Dec 2004 10:30:24 +0000, craigs <s.craig AT andronics.com> 
> >>>>>>>wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>here is a detailed list of what we are running! we have a suse 9.2 
> >>>>>>>>linux
> >>>>>>>>server, using tds version 4.2 for connection pooling, snapshot
> >>>>>>>>freetds-0.64.dev.20041207, client programs written in C using 
> >>>>>>>>db-lib.  the
> >>>>>>>>pool goes down at least a few times every hour, we are using 3 max
> >>>>>>>>connections and receiving approx 1200 clients an hour and each 
> >>>>>>>>client does
> >>>>>>>>at least 2 querys and an insert with few doing 1 or 2 more.
> >>>>>>>>
> >>>>>>>>we have written a process watcher that watchs the connection pool 
> >>>>>>>>and
> >>>>>>>>restarts it if it crashes, before we added a patch it would take the
> >>>>>>>>connection pool nearly a minute to come back up, it wouldnt release 
> >>>>>>>>the
> >>>>>>>>socket, or the socket had to wait to time out, now the pool comes 
> >>>>>>>>back up
> >>>>>>>>again within a few seconds.
> >>>>>>>>
> >>>>>>>>Brian Bruns wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>Craig, Wanna send along the patch for that? I'll merge it. I've been
> >>>>>>>>planning on spending some time on this, and may get around to it 
> >>>>>>>>this
> >>>>>>>>weekend. What is your server/version? I'll try to do some load 
> >>>>>>>>testing and
> >>>>>>>>see if I can reproduce your error locally, and failing that scan 
> >>>>>>>>the code
> >>>>>>>>for something. Brian On Thu, 09 Dec 2004 09:17:32 +0000, craigs
> >>>>>>>><s.craig AT andronics.com> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>Frediano Ziglio wrote:
> >>>>>>>>Il mer, 2004-12-08 alle 13:52, craigs ha scritto:
> >>>>>>>>Brian Bruns wrote: Program received signal SIGSEGV, Segmentation 
> >>>>>>>>fault.
> >>>>>>>>0x0804b5b0 in pool_process_users (pool=0x804d008, fds=0xbfffdbd0) at
> >>>>>>>>user.c:131 131 if (FD_ISSET(puser->tds->s, fds)) { is this any 
> >>>>>>>>help? Shaun.
> >>>>>>>>Yes, probably puser->tds == NULL or puser->tds->s < 0. In other 
> >>>>>>>>words
> >>>>>>>>connection closed freddy77 Any ideas why the connection is closing, 
> >>>>>>>>would it
> >>>>>>>>be on my C clients side or maybe something to do with ms sql. when 
> >>>>>>>>it does
> >>>>>>>>crash it takes nearly a minute to come up again which can cause 
> >>>>>>>>problems for
> >>>>>>>>ourselfs, we have added a line in pool_main_loop which releases the 
> >>>>>>>>socket
> >>>>>>>>as soon it crashes, our up time again has been greatly reduced 
> >>>>>>>>because of
> >>>>>>>>this line, would this be of any use to anyone?
> >>>>>>>>_______________________________________________ FreeTDS mailing list
> >>>>>>>>FreeTDS AT lists.ibiblio.org 
> >>>>>>>>http://lists.ibiblio.org/mailman/listinfo/freetds
> >>>>>>>>_______________________________________________ FreeTDS mailing list
> >>>>>>>>FreeTDS AT lists.ibiblio.org 
> >>>>>>>>http://lists.ibiblio.org/mailman/listinfo/freetds
> >>>>>>>>_______________________________________________ FreeTDS mailing list
> >>>>>>>>FreeTDS AT lists.ibiblio.org 
> >>>>>>>>http://lists.ibiblio.org/mailman/listinfo/freetds
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
> 
>