FreeTDS Development Group

[Brian Bruns <brian.bruns AT gmail.com>]

Tests went fine.  I'm convinced we have no problems with normal
queries.  There were some issues with client disconnects and such. 
The other thing to look for would be RPCs/placeholder stuff and more
esoteric features like that.   What did you say your frontend was?

Also if we could look at the last few queries from the log before a
crash that might be helpful.  The new version removed some of the
excessive messages to stderr, so the log should be more intelligible.

Brian


On Tue, 14 Dec 2004 13:20:42 +0000, craigs <s.craig AT andronics.com> wrote:
> Brian,
> 
> Im flat out at the minute doing system admin so cant get a chance to
> test the new stuff, how did your tests go with ms sql
> 
> Shaun.
> 
> Brian Bruns wrote:
> 
> >Checked in some more changes.  When a client exits in the middle of a
> >query it can leave the member in a confused state, so now we just
> >close it to be safe.
> >
> >So please try the latest from CVS and let me know.
> >
> >Brian
> >
> >On Mon, 13 Dec 2004 15:02:00 +0000, craigs <s.craig AT andronics.com> wrote:
> >
> >
> >>Brian,
> >>
> >>the only error message we get from our clients is that we cannot connect
> >>to database server, this happens in dbopen() which makes sense seen as
> >>the pool is down, your tests sound encouraging so maybe if i get a
> >>chance later today i will write a test program in a safe environment to
> >>hammer the pool, will run in gdb with tdsdump on and see how we go.
> >>
> >>Shaun.
> >>
> >>
> >>
> >>Brian Bruns wrote:
> >>
> >>
> >>
> >>>Ok, I'm running now.  This is what I did to test:
> >>>
> >>>1. Built a query file with 'select * from <table>' for every table in
> >>>the database (about a 500 meg database) Good mix of small and large
> >>>tables.
> >>>2. Set up the pool for 2 connections
> >>>3. Spun off tsql to run the queries in parallel, averaging about 150
> >>>client connections to the pool at any one time.
> >>>4. Ran for half an hour so far and holding.
> >>>5. Ran clients up to 1000 and now letting it run for a bit.
> >>>
> >>>One thing I did notice is the need for a better scheduling algorithm
> >>>under heavy load to prevent starvation.
> >>>
> >>>Other than that, it seems we are having an issue with some sort of
> >>>abnormal case.  I did notice the pool go down when a "cancel" was
> >>>issued by the client.  I'm looking into that, but does that ring any
> >>>bells?
> >>>
> >>>Brian
> >>>
> >>>On Mon, 13 Dec 2004 13:28:22 +0000, craigs <s.craig AT andronics.com> wrote:
> >>>
> >>>
> >>>
> >>>
> >>>>Brian,
> >>>>
> >>>>we only had the new pool running for a short time and had to revert to
> >>>>an older version pretty quick when we seen there was problems so we
> >>>>never got a chance to get a stack trace, its very dangerous for us to
> >>>>put it back on just to get the trace as we have alot of input to our
> >>>>system. Let me know how the tests go using SQL Server, thanks for your 
> >>>>time.
> >>>>
> >>>>Shaun.
> >>>>
> >>>>
> >>>>
> >>>>Brian Bruns wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Shaun,
> >>>>>
> >>>>>I ran it continuously for about 3 hours using a simulated workload.
> >>>>>This was against a Sybase 12.5.2 database, so that may be the issue.
> >>>>>I can try it from here against SQL Server, but didn't have that access
> >>>>>over the weekend.  I'll let you know if anything turns up, but so far
> >>>>>I'm not seeing it.
> >>>>>
> >>>>>BTW, I made two distinct fixes, one of which I was almost certain was
> >>>>>what you were seeing based on your description...oh well, we'll keep
> >>>>>plugging away.  Was the stack trace the same on the new code?
> >>>>>
> >>>>>Brian
> >>>>>
> >>>>>
> >>>>>On Mon, 13 Dec 2004 12:23:01 +0000, craigs <s.craig AT andronics.com> 
> >>>>>wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>hi Brian,
> >>>>>>
> >>>>>>got the latest version off CVS, put it on our server and things didnt 
> >>>>>>go
> >>>>>>to well! will test it more ourselfs but its really hard for us to
> >>>>>>duplicate live data but will try, did everything go ok when you 
> >>>>>>tested it?
> >>>>>>
> >>>>>>Shaun.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>Brian Bruns wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>I just applied a one liner (and your patch as well) to CVS.  Please
> >>>>>>>give it a try and let me know if it works.  As a bonus, this should
> >>>>>>>make it much easier to add TDS7/8 support to the pool.
> >>>>>>>
> >>>>>>>Cheers,
> >>>>>>>
> >>>>>>>Brian
> >>>>>>>
> >>>>>>>On Fri, 10 Dec 2004 11:24:59 +0000, craigs <s.craig AT andronics.com> 
> >>>>>>>wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>thats one hell of a film! thanks for looking into it, next year we 
> >>>>>>>>have
> >>>>>>>>predicted that we will average over 3000 clients hitting the pool 
> >>>>>>>>every
> >>>>>>>>hour so it would be good if we could track it down sooner rather 
> >>>>>>>>than later.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>Brian Bruns wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>The good: I was able to get it to crash on the same line.
> >>>>>>>>>The bad: Looks like a buffer overflow
> >>>>>>>>>The ugly: This means it originated elsewhere in the code and is
> >>>>>>>>>therefore hard to track down.
> >>>>>>>>>
> >>>>>>>>>Still plugging away at it, i'll let you know what i find.
> >>>>>>>>>
> >>>>>>>>>On Fri, 10 Dec 2004 10:30:24 +0000, craigs <s.craig AT andronics.com> 
> >>>>>>>>>wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>here is a detailed list of what we are running! we have a suse 
> >>>>>>>>>>9.2 linux
> >>>>>>>>>>server, using tds version 4.2 for connection pooling, snapshot
> >>>>>>>>>>freetds-0.64.dev.20041207, client programs written in C using 
> >>>>>>>>>>db-lib.  the
> >>>>>>>>>>pool goes down at least a few times every hour, we are using 3 max
> >>>>>>>>>>connections and receiving approx 1200 clients an hour and each 
> >>>>>>>>>>client does
> >>>>>>>>>>at least 2 querys and an insert with few doing 1 or 2 more.
> >>>>>>>>>>
> >>>>>>>>>>we have written a process watcher that watchs the connection pool 
> >>>>>>>>>>and
> >>>>>>>>>>restarts it if it crashes, before we added a patch it would take 
> >>>>>>>>>>the
> >>>>>>>>>>connection pool nearly a minute to come back up, it wouldnt 
> >>>>>>>>>>release the
> >>>>>>>>>>socket, or the socket had to wait to time out, now the pool comes 
> >>>>>>>>>>back up
> >>>>>>>>>>again within a few seconds.
> >>>>>>>>>>
> >>>>>>>>>>Brian Bruns wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>Craig, Wanna send along the patch for that? I'll merge it. I've 
> >>>>>>>>>>been
> >>>>>>>>>>planning on spending some time on this, and may get around to it 
> >>>>>>>>>>this
> >>>>>>>>>>weekend. What is your server/version? I'll try to do some load 
> >>>>>>>>>>testing and
> >>>>>>>>>>see if I can reproduce your error locally, and failing that scan 
> >>>>>>>>>>the code
> >>>>>>>>>>for something. Brian On Thu, 09 Dec 2004 09:17:32 +0000, craigs
> >>>>>>>>>><s.craig AT andronics.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>Frediano Ziglio wrote:
> >>>>>>>>>>Il mer, 2004-12-08 alle 13:52, craigs ha scritto:
> >>>>>>>>>>Brian Bruns wrote: Program received signal SIGSEGV, Segmentation 
> >>>>>>>>>>fault.
> >>>>>>>>>>0x0804b5b0 in pool_process_users (pool=0x804d008, fds=0xbfffdbd0) 
> >>>>>>>>>>at
> >>>>>>>>>>user.c:131 131 if (FD_ISSET(puser->tds->s, fds)) { is this any 
> >>>>>>>>>>help? Shaun.
> >>>>>>>>>>Yes, probably puser->tds == NULL or puser->tds->s < 0. In other 
> >>>>>>>>>>words
> >>>>>>>>>>connection closed freddy77 Any ideas why the connection is 
> >>>>>>>>>>closing, would it
> >>>>>>>>>>be on my C clients side or maybe something to do with ms sql. 
> >>>>>>>>>>when it does
> >>>>>>>>>>crash it takes nearly a minute to come up again which can cause 
> >>>>>>>>>>problems for
> >>>>>>>>>>ourselfs, we have added a line in pool_main_loop which releases 
> >>>>>>>>>>the socket
> >>>>>>>>>>as soon it crashes, our up time again has been greatly reduced 
> >>>>>>>>>>because of
> >>>>>>>>>>this line, would this be of any use to anyone?
> >>>>>>>>>>_______________________________________________ FreeTDS mailing 
> >>>>>>>>>>list
> >>>>>>>>>>FreeTDS AT lists.ibiblio.org 
> >>>>>>>>>>http://lists.ibiblio.org/mailman/listinfo/freetds
> >>>>>>>>>>_______________________________________________ FreeTDS mailing 
> >>>>>>>>>>list
> >>>>>>>>>>FreeTDS AT lists.ibiblio.org 
> >>>>>>>>>>http://lists.ibiblio.org/mailman/listinfo/freetds
> >>>>>>>>>>_______________________________________________ FreeTDS mailing 
> >>>>>>>>>>list
> >>>>>>>>>>FreeTDS AT lists.ibiblio.org 
> >>>>>>>>>>http://lists.ibiblio.org/mailman/listinfo/freetds
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
> 
>