Biblical Hebrew Forum

[Peter Kirk <peterkirk AT qaya.org>]

On 13/07/2005 16:17, Karl Randolph wrote:

> Peter:
>
> Content is exactly what is being lost when character entities are changed to 
> html text strings.
>
> That non-ASCII characters are stored in the archives as character entities 
> means that they can not be quoted without losing content.
>
> The encoding is defined by <META http-equiv="Content-Type" content="text/html; 
> charset=us-ascii"> means that the only way to show non-ASCII characters is to have them in 
> ASCII coding, such as character entities. There is no Windows-1255 or UTF-8 encoding. Therefore, in 
> order not to lose content, such coding must be stored unchanged. It changes them.
>  

Karl, you have failed to understand the distinction between content and 
encoding. As your definition "<META..." is an HTML tag, I assume that 
you are referring here to an HTML page in the archives. The whole point 
of character entities is that (in HTML, but not in plain text) they 
allow non-ASCII characters to be included in a document which is 
otherwise ASCII-only, or in general that they allow any Unicode 
character to be included in a document whatever the declared charset. 
But this is not a change of logical content, it is only a change in how 
that content is stored. It should be no more relevant to the user than 
knowing whether the content is stored in computer memory, on a magnetic 
disk or tape, or on a CD.

> At first I was ready to blame your email program. Now I realize it is deeper 
> than that. We are dealing with incompatible systems, some of which have been 
> around for some time. The earliest non-MIME complient standard was developed 
> to show Asian characters, and one of the earliest MIME challenges was how to 
> transmit those messages unchanged. (I’d forgotten about uuencoding, a way to 
> transmit non-ASCII data as ASCII strings.)
>
> You blame the mail program I am using. How it sends messages, i.e. the 
> encoding it uses, I am not sure. However it appears that it sends ASCII 
> strings. For non-ASCII characters, that would mean character entities. ...

No, if we are talking about plain text rather than HTML, it is not 
sending character entities, but strings of ASCII characters which it may 
intend as a representation of non-ASCII characters but are not 
representations according to any standard.

> ... For receiving messages, it is designed to be read through a browser such 
> as Firefox, Camino and Mozilla, so it passes on whatever message it receives 
> unchanged: if Windows-1255, I can tell the browser to read it; if UTF-8, a 
> different setting; if character entities, sent to the browser unchanged so 
> that the content could be displayed unlost. Of course, I speculate because I 
> don’t have enough data.
>  

But it doesn't appear to do so. For if it receives in plain text a 
string which in HTML would represent a character entity, it does not 
send this through to the browser unchanged in content, but misinterprets 
it as a character entity and transforms the string to a non-ASCII character.

> Then we have the ibiblio archiving program that takes only us-ascii. When it 
> receives data, it loses content if that data is already encoded in character 
> entities to be compatible with ASCII, because it changes it so that the data 
> is no longer character entities. That losing content is certainly a bug.
>  

No, it does not do this. If it receives HTML e-mail (which it cannot on 
this list), it presumably deals with character entities correctly. But 
in plain text e-mail it is IMPOSSIBLE for the archiving program to 
receive character entities, so it is impossible that it can lose content 
from them. This seems to be the point that you are stubbornly refusing 
to accept: that in plain text there is NO SUCH THING as a character 
entity. You will find no mention of them in the MIME spec I referred you 
to. Character entities are a concept applicable to HTML e-mail only, and 
so not applicable to this list. Thus there is no possibility of anyone's 
program processing character entities incorrectly, in plain text e-mail.

> The question is, what are we as a forum going to do about these incompatible 
> standards? Shall we make it a rule that in order to participate, we need to 
> use a certain standard? If so, which? Should the technically savvy among us 
> draw up a list of approved email clients, recommending that only those be 
> used by list members?    ??
>  

Karl, the standard which we should follow is certainly the MIME spec I 
referred you to, and its companions, which apply to all Internet e-mail, 
and which make no reference to character entities. A decision has been 
made, at least for the moment, not to use HTML e-mail, and that means no 
character entities as well. If you really want to suggest a list of 
approved e-mail clients, then yours will not be on it because it does 
not conform to the MIME spec. But I don't suggest banning anyone, only 
that it is made clear that anyone who, or whose mail program, attempts 
to send character entities to this list will find that what they send is 
neither received nor archived as character entities, but as ASCII 
strings - and that is in accordance with the standard.

If you want to send Hebrew to this list, you need to find a way to do so 
which is valid according to MIME, which means that the message must be 
encoded in UTF-8 or one of the Hebrew encodings.

--
Peter Kirk
peter AT qaya.org (personal)
peterkirk AT qaya.org (work)
http://www.qaya.org/



--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.323 / Virus Database: 267.8.13/47 - Release Date: 12/07/2005