Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] indexOf O(1) patch?

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Nils_Kilden-Pedersen AT Countrywide.Com
  • To: XOM-interest AT lists.ibiblio.org
  • Cc:
  • Subject: Re: [XOM-interest] indexOf O(1) patch?
  • Date: Wed, 2 Feb 2005 08:38:32 -0800

xom-interest-bounces AT lists.ibiblio.org wrote on 02/02/2005 04:29:47 AM:

> The question is perhaps what counts as a large document. I'd say it's a
> document that can't fit in available memory. The less memory XOM uses,
> the fewer documents count as large. They're some interesting ideas here,

> and I think they're worth exploring.

But how will you know in advance if the document will fit in memory? I
think Wolfgang's got a good point here. You don't want to find out by an
OutOfMemoryException, so if you're going to process "large" documents, you
need a different strategy anyway.
The scenarios I've seen are either static "small" documents, or dynamic
(growing) "large" documents.

Nils
>From elharo AT metalab.unc.edu Wed Feb 2 12:25:33 2005
Return-Path: <elharo AT metalab.unc.edu>
X-Original-To: XOM-interest AT lists.ibiblio.org
Delivered-To: XOM-interest AT lists.ibiblio.org
Received: from mail22.sea5.speakeasy.net (mail22.sea5.speakeasy.net
[69.17.117.24])
by lists.ibiblio.org (Postfix) with ESMTP id C95E24C008
for <XOM-interest AT lists.ibiblio.org>;
Wed, 2 Feb 2005 12:25:32 -0500 (EST)
Received: (qmail 5390 invoked from network); 2 Feb 2005 17:25:32 -0000
Received: from dsl254-067-087.nyc1.dsl.speakeasy.net (HELO [192.168.254.100])
(elharo@[216.254.67.87]) (envelope-sender <elharo AT metalab.unc.edu>)
by mail22.sea5.speakeasy.net (qmail-ldap-1.03) with AES256-SHA
encrypted SMTP
for <Nils_Kilden-Pedersen AT Countrywide.Com>; 2 Feb 2005 17:25:31 -0000
Message-ID: <42010D0A.9040008 AT metalab.unc.edu>
Date: Wed, 02 Feb 2005 12:25:30 -0500
From: Elliotte Harold <elharo AT metalab.unc.edu>
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US;
rv:1.7.5) Gecko/20041217
X-Accept-Language: en-us, en, fr
MIME-Version: 1.0
To: Nils_Kilden-Pedersen AT Countrywide.Com
Subject: Re: [XOM-interest] indexOf O(1) patch?
References:
<OF675D11FF.BD5D0330-ON88256F9C.005B1328-88256F9C.005B6B22 AT countrywide.com>
In-Reply-To:
<OF675D11FF.BD5D0330-ON88256F9C.005B1328-88256F9C.005B6B22 AT countrywide.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Cc: XOM-interest AT lists.ibiblio.org
X-BeenThere: xom-interest AT lists.ibiblio.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: xom-interest.lists.ibiblio.org
List-Unsubscribe: <http://lists.ibiblio.org/mailman/listinfo/xom-interest>,
<mailto:xom-interest-request AT lists.ibiblio.org?subject=unsubscribe>
List-Archive: <https://lists.ibiblio.org/sympa/arc/xom-interest>
List-Post: <mailto:xom-interest AT lists.ibiblio.org>
List-Help: <mailto:sympa AT lists.ibiblio.org?subject=HELP>
List-Subscribe: <http://lists.ibiblio.org/mailman/listinfo/xom-interest>,
<mailto:xom-interest-request AT lists.ibiblio.org?subject=subscribe>
X-List-Received-Date: Wed, 02 Feb 2005 17:25:33 -0000

Nils_Kilden-Pedersen AT Countrywide.Com wrote:


> But how will you know in advance if the document will fit in memory?

Experiment. The fact is I constantly see people having trouble
processing XML due to the size of their documents. I see this far more
often than I see people having problems with speed.

Less than two minutes ago I was reading a thread about this on the saxon
mailing list, where someone needed to process an 80MB document on a
512MB machine. What's big is all relative to the installed memory and
similar considerations. However, I think it's a big win for XOM to be
able to process documents as large as it possibly can. The fact that
there will always be larger documents than can comfortably be handled in
memory does not convince me that XOM shouldn't do the best job it can.

This is yet another example of the common binary fallacy. Here the
argument takes the form that XOM can't be perfect so it shouldn't try to
be better. The fact is, XOM can be better; and it's worth making it
better. Perhaps we'll eventually reach a point where the cost of making
XOM documents smaller outweighs the gains in so doing, but I don't think
XOM has reached that point yet. That Wolfgang can use an alternative
builder and come up with documents that are a third smaller than what
the standard builder produces tells me there's still a lot of room for
improvement.

--
Elliotte Rusty Harold elharo AT metalab.unc.edu
XML in a Nutshell 3rd Edition Just Published!
http://www.cafeconleche.org/books/xian3/
http://www.amazon.com/exec/obidos/ISBN=0596007647/cafeaulaitA/ref=nosim




Archive powered by MHonArc 2.6.24.

Top of Page