Biblical Hebrew Forum

["Karl Randolph" <kwrandolph AT email.com>]

Peter:

Content is exactly what is being lost when character entities are changed to 
html text strings.

That non-ASCII characters are stored in the archives as character entities 
means that they can not be quoted without losing content.

The encoding is defined by <META http-equiv="Content-Type" 
content="text/html; charset=us-ascii"> means that the only way to show 
non-ASCII characters is to have them in ASCII coding, such as character 
entities. There is no Windows-1255 or UTF-8 encoding. Therefore, in order not 
to lose content, such coding must be stored unchanged. It changes them.

At first I was ready to blame your email program. Now I realize it is deeper 
than that. We are dealing with incompatible systems, some of which have been 
around for some time. The earliest non-MIME complient standard was developed 
to show Asian characters, and one of the earliest MIME challenges was how to 
transmit those messages unchanged. (I’d forgotten about uuencoding, a way to 
transmit non-ASCII data as ASCII strings.)

You blame the mail program I am using. How it sends messages, i.e. the 
encoding it uses, I am not sure. However it appears that it sends ASCII 
strings. For non-ASCII characters, that would mean character entities. For 
receiving messages, it is designed to be read through a browser such as 
Firefox, Camino and Mozilla, so it passes on whatever message it receives 
unchanged: if Windows-1255, I can tell the browser to read it; if UTF-8, a 
different setting; if character entities, sent to the browser unchanged so 
that the content could be displayed unlost. Of course, I speculate because I 
don’t have enough data.

Then we have the ibiblio archiving program that takes only us-ascii. When it 
receives data, it loses content if that data is already encoded in character 
entities to be compatible with ASCII, because it changes it so that the data 
is no longer character entities. That losing content is certainly a bug.

The question is, what are we as a forum going to do about these incompatible 
standards? Shall we make it a rule that in order to participate, we need to 
use a certain standard? If so, which? Should the technically savvy among us 
draw up a list of approved email clients, recommending that only those be 
used by list members?    ??

Karl W. Randolph.

----- Original Message -----
From: "Peter Kirk" <peterkirk AT qaya.org>

> 
> On 12/07/2005 20:05, Karl Randolph wrote:
> 
>...
> 
> As a message passes through a series of computer systems, typically 
> from a human writer to a human reader, what needs to be preserved 
> is the CONTENT of the message, which consists (for a text message) 
> of a series of characters (sometimes known as abstract characters). 
> These characters may be encoded in several different ways, and the 
> encodings used may change several times on the way. But this does 
> not matter, because the content is unchanged. The underlying 
> encoding is generally hidden from the actual human user.
> 
> But programs do often provide ways for expert users to peek behind 
> the encoding and manipulate the rules. This is done so that people 
> like you and me can find out what is happening when something goes 
> wrong, as has happened in this case. But we should really look at 
> what happens when we are not peeking behind the encoding. When you 
> look at the b-hebrew archives in the normal way, do you see 
> anything wrong? I think you will see nothing wrong with messages 
> sent by me and most other list members. You may see things wrong 
> with messages you have sent, but the bug which causes this is in 
> your mail program, not in the archiving software.
> 
>...
> 
> The software is changing the encoding of the messages, which is 
> permissible, but it is not changing the content.
> 
> > What I expect now is that if I go back in the archives to one of 
> > your messages with some Hebrew in it, and quote it, that the 
> > Hebrew will come out as unintelligeable html text strings. Is 
> > that not a bug?
> >
> >
> >
> Can you show me an example of this, with one of my messages? Try 
> with this one, into which I will now insert some Hebrew in UTF-8 
> encoding:
> 
> אבגד
> 
> And you can look back to 
> https://lists.ibiblio.org/sympa/arc/b-hebrew/2005-July/023703.html, 
> my posting on 5th July, for Hebrew in Windows-1255 encoding - look 
> only at the "properly encoded Hebrew characters", the next lines 
> are deliberately not properly encoded.
> 
> By the way, there is another clear bug in your mail software that 
> it does not send information about which message is being replied 
> to, which is why the archiving software takes each of your messages 
> as the start of a new thread.
> 
> 
> -- Peter Kirk
> peter AT qaya.org (personal)
> peterkirk AT qaya.org (work)
> http://www.qaya.org/