Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] Serializer performance patches

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Elliotte Harold <elharo AT metalab.unc.edu>
  • To: Wolfgang Hoschek <whoschek AT lbl.gov>
  • Cc: xom-interest AT lists.ibiblio.org
  • Subject: Re: [XOM-interest] Serializer performance patches
  • Date: Thu, 30 Jun 2005 08:46:44 -0400

Wolfgang Hoschek wrote:

Some of us learned programming on historic batch mainfraimes where a submitted hello world job would take one hour to complete, only to spit out a laconic "syntax error" on some printer in the computer center on the other side of campus. For most folks, this learning experience led to thinking hard about a program before doing anything, a style of programming that produces programs that are mostly bug free to begin with. Adding tests as a particular need arises seems sensible, thinking before testing is better, and refraining from mostly clueless arguments is best. Ah well, enjoy...

Just a thought, but maybe the software development practices that worked best in an era of 1 MHz batch processing and punch cards aren't necessarily the same as the ones that work well for 2 GHz computers with GUIs and acres of RAM? Or maybe those 1970s era techniques never worked so well in the first place. It was just the best that could be done with the available tools? I believe such programs are mostly bug free to begin with about as much as I believe it really was the tooth fairy leaving all those quarters under my pillow.

Your software development style seems to be very theoretical. That probably made sense in 1975, but in 2005 it's a lot quicker and easier to try three different approaches and measure which one works better than it is to carefully consider the details of the algorithm. Sadly computer science departments are mostly still teaching the theoretical method. Grad students arriving in my advanced class often know everything there is to know about carefully calculating big-O numbers for different algorithms but few if any of them have the foggiest clue what a profiler is. A couple know how to measure the wallclock time to benchmark something. I have to pick their jaws up off the floor after I demonstrate that arrays beat the pants off linked lists for performance since they've all proven the exact opposite in their data structures class.

Thinking before coding makes sense for design and architecture, but at the level of implementation detail, I no longer believe in it. Test driven development is just too damned effective. Maybe you're brighter than I am, but I find that only about 50% of the time are my thoughts about code correct. I can write a test case to prove myself right or wrong a lot faster than I can contrive some mathematical proof of program correctness that's probably flawed to begin with. Performance issues are no different. It's far more reliable to measure what actually happens than to reason about what you think will happen.

You've complained about some of the changes I've made in 1.1 that are hurting performance. It's not a coincidence that performance is the one area that's really not tested at all by XOM's unit test suite. Since performance tests still have have to be run manually, they tend not to get run. Consequently things tend to break. If performance testing could be integrated directly into XOM's automated test suite, then I could be a lot more confident about applying patches designed to fix and improve performance.

Perhaps in small 1970s Fortran programs that rarely exceeded 10,000 lines, it was possible to be reasonably confident in the code purely from an analysis of the algorithms. But in today's much larger object-oriented programs? I don't believe it's possible. Even with object boundaries, data encapsulation, careful attention to minimizing coupling, and more, there's still just way too much interconnectedness in even good code for a human to be able to carefully follow all possible paths through the code. The advantage of having a test suite that exercises most of the code base on many different paths is that you can be reasonably confident whether or not a change to fix one issue is likely to break anything else. Without a solid test suite, I'd be scared to touch anything in XOM for fear of what might break. Your proposed performance patch is a good example of that. The test suite made it immediately obvious that there were problems and it couldn't be accepted as is. The test suite also showed us exactly what broke and how so that you could fix the patch. Neither you nor I spotted the problems with failing to escape markup from a simple analysis of the code, but the test suite found it immediately. The larger and more complex a system is the more important it is that it be comprehensively tested. Anything less makes it impossible to safely change.

--
Elliotte Rusty Harold elharo AT metalab.unc.edu
XML in a Nutshell 3rd Edition Just Published!
http://www.cafeconleche.org/books/xian3/
http://www.amazon.com/exec/obidos/ISBN=0596007647/cafeaulaitA/ref=nosim




Archive powered by MHonArc 2.6.24.

Top of Page