XOM API for Processing XML with Java

[Dmitry Katsubo <dma_k AT mail.ru>]

Dear Elliotte,

Thank you for your comments.

Elliotte Rusty Harold wrote on 17.09.2010 14:20:
> On Fri, Sep 17, 2010 at 7:11 AM, Dmitry Katsubo <dma_k AT mail.ru> wrote:
> 
>> I fully agree on the statement that parsing the XSLT into XOM tree is
>> fast and should not be considered as memory or time loss at all. However
>> we can imagine, that I cannot receive the stylesheet neither as
>> InputStream, File or String. It is passed to me from some other 3rd
>> party library X as javax.xml.stream.XMLEventReader or as
>> org.xmlpull.v1.XmlPullParser.
> 
> Supporting hypothetical use cases lead to bloated APIs. If there's a
> complelling need for XMLEventReader or XmlPullParser I'll look into
> it. So far I've never seen such a thing. Even you are not really
> saying you need it, just that you think it should be there.

I suggest you waiting for few more requests coming from several users,
and then make an appropriate extension to API. No pressure from my side.
Well, the drawback of extending the API *on request* is that some
developers will not come to this maillist to discuss the probable
improvements, because they are too lazy. Finally they will bloat the
application code, that is using XOM, with workarounds. So if you look at
final application as a whole, you will notice a bright and shining XOM
library in the center and back patches here and there when gluing with
it. So the overall picture is gray. And knowing that the majority
programmers will implement something "just to make it working" and never
come back for improvements, you will never know about the traps in using
XOM. It is just my feeling :)

>> Also as we have touched the question of serialization, why
>> nu.xom.Serializer does not have a constructor with Writer? Internally it
>> uses a writer. The only added value I see in nu.xom.Serializer is to
>> protect the user from using broken EBCDIC-family output streams. If I
>> use broken JDK OutputStream implementation, I won't blame XOM, really.
>> And if I have only a Writer, I need to think how to convert it to
>> OutputStream for XOM, who will convert it to Writer :)
> 
> This is painful but ties directly to another design flaw in Java, not
> XOM. Writers do not allow the client to determine the underlying
> encoding of the text (UTF-8, ISO-8859-1, etc.). Therefore with only a
> Writer I can't guarantee well-formed output. To answer your question
> in another thread this is one (of several) ways you can get malformed
> output from JDOM.

OK, I have looked the source code again. I see that the added value are
also the text escapers, which define the rules per charset.

Sorry, I don't understand the problem ground here. If I pass Writer to
XOM, it is true that you cannot determine the output encoding - it needs
to be supplied with 2nd argument the same way as for OutpuStream. But
XOM needs to feed the Writer with UTF strings (allowed by given
encoding) and it is up to Writer (or String) to convert the characters
to output encoding later on. There is kind of danger here, but I don't
think it is a stopper.

I attach a test case, that serializes two XML models via XOM and DOMv3
with same result. Please, point me exactly the line I am wrong or which
has a pitfall that most of programmers will hit.

Thank you!

-- 
With best regards,
Dmitry
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.StringWriter;
import java.io.UnsupportedEncodingException;

import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.FactoryConfigurationError;
import javax.xml.parsers.ParserConfigurationException;

import junit.framework.Assert;

import org.apache.xml.utils.DOMBuilder;
import org.junit.Test;
import org.w3c.dom.bootstrap.DOMImplementationRegistry;
import org.w3c.dom.ls.DOMImplementationLS;
import org.w3c.dom.ls.LSOutput;
import org.xml.sax.SAXException;

import nu.xom.Document;
import nu.xom.Element;
import nu.xom.Serializer;
import nu.xom.converters.SAXConverter;

public class SerializationTest {

        /*
         * This is a self-test for {@link 
#compareByteArraysIgnoringNewlinesAndCapitalization(byte[], byte[])}.
         */
        @Test
        public void selfTest() {
                byte[][][] testsForEquality = new byte[][][]{{new byte[]{1}, 
new byte[]{1}}, //
                                {new byte[]{1, 2}, new byte[]{1, 2}}, //
                                {new byte[]{1, 2}, new byte[]{1, 2}}, //
                                {new byte[]{1, 10, 2}, new byte[]{1, 2}}, //
                                {new byte[]{1, 10, 2, 13}, new byte[]{1, 2}}, 
//
                                {new byte[]{1, 10, 13, 2}, new byte[]{1, 2}}, 
//
                                {new byte[]{1, 2, 10, 13}, new byte[]{1, 2}}, 
//
                                {new byte[]{1, 10}, new byte[]{1}}, //
                                {new byte[]{13, 1}, new byte[]{1}}, //
                                {"abc".getBytes(), "abc\n".getBytes()}, //
                                {"Ab".getBytes(), "a\r\nB".getBytes()}, //
                };

                for (int i = 0; i < testsForEquality.length; i++) {
                        Assert.assertTrue(
                                                "Failed test#" + i,
                                                
compareByteArraysIgnoringNewlinesAndCapitalization(testsForEquality[i][0],
                                                                        
testsForEquality[i][1]));
                        Assert.assertTrue(
                                                "Failed test#" + i,
                                                
compareByteArraysIgnoringNewlinesAndCapitalization(testsForEquality[i][1],
                                                                        
testsForEquality[i][0]));
                }

                byte[][][] testsForNoneEquality = new byte[][][]{{new 
byte[]{1}, new byte[]{2}}, //
                                {new byte[]{1, 2}, new byte[]{1}}, //
                                {new byte[]{1, 2}, new byte[]{2, 1}}, //
                                {new byte[]{1, 10, 2}, new byte[]{13, 1, 10, 
3}}, //
                                {new byte[]{1, 10, 13, 2}, new byte[]{1, 2, 
10, 1}}, //
                                {new byte[]{1, 2, 13}, new byte[]{13, 1, 2, 
10, 1}}, //
                                {new byte[]{1, 10, 1}, new byte[]{1}}, //
                                {new byte[]{13, 1, 10, 1}, new byte[]{1, 
10}}, //
                                {"A".getBytes(), "B".getBytes()}, //
                                {"1".getBytes(), "0".getBytes()}, //
                };

                for (int i = 0; i < testsForNoneEquality.length; i++) {
                        Assert.assertFalse(
                                                "Failed test#" + i,
                                                
compareByteArraysIgnoringNewlinesAndCapitalization(testsForNoneEquality[i][0],
                                                                        
testsForNoneEquality[i][1]));
                        Assert.assertFalse(
                                                "Failed test#" + i,
                                                
compareByteArraysIgnoringNewlinesAndCapitalization(testsForNoneEquality[i][1],
                                                                        
testsForNoneEquality[i][0]));
                }
        }

        /*
         * XOM & DOM serialization test.
         */
        @Test
        public void testSerialization() throws UnsupportedEncodingException, 
IOException, ParserConfigurationException,
                                FactoryConfigurationError, SAXException, 
ClassNotFoundException, InstantiationException,
                                IllegalAccessException {
                // XOM Tree:
                Element root = new Element("test");
                // English translation: "Munchen - beautiful city"
                root.appendChild("München - красивый город"); 
// Mixture of ISO-8859-1 and ISO-8859-5

                Document xomDocument = new Document(root);

                org.w3c.dom.Document w3cDocument = 
DocumentBuilderFactory.newInstance().newDocumentBuilder().newDocument();

                SAXConverter saxConverter = new SAXConverter(new 
DOMBuilder(w3cDocument)); // DOMBuilder from Apache Xalan

                saxConverter.convert(xomDocument);

                // Using DOM level 3 serialization API for DOM W3C model:
                DOMImplementationLS lsImpl = (DOMImplementationLS) 
DOMImplementationRegistry.newInstance()
                                        .getDOMImplementation("LS");

                final String[] encodings = new String[]{"ISO-8859-1", 
"ISO-8859-5"};

                // Test for byte serialization:
                for (int i = 0; i < encodings.length; i++) {
                        // XOM:
                        ByteArrayOutputStream xomOutputStream = new 
ByteArrayOutputStream();

                        (new Serializer(xomOutputStream, 
encodings[i])).write(xomDocument);

                        // DOM:
                        ByteArrayOutputStream domOutputStream = new 
ByteArrayOutputStream();

                        LSOutput lsOutput = lsImpl.createLSOutput();
                        lsOutput.setByteStream(domOutputStream);
                        lsOutput.setEncoding(encodings[i]);

                        lsImpl.createLSSerializer().write(w3cDocument, 
lsOutput);

                        // byte-by-byte comparison:
                        Assert.assertTrue(
                                                "Failed byte test for 
encoding " + encodings[i],
                                                
compareByteArraysIgnoringNewlinesAndCapitalization(xomOutputStream.toByteArray(),
                                                                        
domOutputStream.toByteArray()));
                }

                // Test for character serialization:
                for (int i = 0; i < encodings.length; i++) {
                        // XOM:
                        ByteArrayOutputStream xomOutputStream = new 
ByteArrayOutputStream();

                        (new Serializer(xomOutputStream, 
encodings[i])).write(xomDocument);

                        // DOM:
                        StringWriter domWriter = new StringWriter();

                        LSOutput lsOutput = lsImpl.createLSOutput();
                        lsOutput.setCharacterStream(domWriter);
                        lsOutput.setEncoding(encodings[i]);

                        lsImpl.createLSSerializer().write(w3cDocument, 
lsOutput);

                        // byte-by-byte comparison:
                        Assert.assertTrue(
                                                "Failed char test for 
encoding " + encodings[i],
                                                
compareByteArraysIgnoringNewlinesAndCapitalization(xomOutputStream.toByteArray(),
 domWriter
                                                                        
.toString().getBytes(encodings[i])));
                }
        }

        private static boolean 
compareByteArraysIgnoringNewlinesAndCapitalization(byte[] a, byte[] b) {
                int i = 0;
                int j = 0;

                while (true) {
                        if (i < a.length && (a[i] == 10 || a[i] == 13)) {
                                i++;
                                continue;
                        }

                        if (j < b.length && (b[j] == 10 || b[j] == 13)) {
                                j++;
                                continue;
                        }

                        // Both reached the end successfully:
                        if (i == a.length && j == b.length) {
                                return true;
                        }

                        // One of two reached the end:
                        if (i == a.length ^ j == b.length) {
                                return false;
                        }

                        // Otherwise values should be equal, except the case 
for latin letters:
                        if (a[i] != b[j]) {
                                if ((a[i] < 'A' || a[i] > 'Z' || a[i] + 32 != 
b[j]) && (b[j] < 'A' || b[j] > 'Z' || b[j] + 32 != a[i])) {
                                        return false;
                                }
                        }

                        i++;
                        j++;
                }
        }
}