[b-hebrew] Hebrew in messages
peterkirk at qaya.org
Wed Jul 13 12:27:09 EDT 2005
On 13/07/2005 16:17, Karl Randolph wrote:
>Content is exactly what is being lost when character entities are changed to html text strings.
>That non-ASCII characters are stored in the archives as character entities means that they can not be quoted without losing content.
>The encoding is defined by <META http-equiv="Content-Type" content="text/html; charset=us-ascii"> means that the only way to show non-ASCII characters is to have them in ASCII coding, such as character entities. There is no Windows-1255 or UTF-8 encoding. Therefore, in order not to lose content, such coding must be stored unchanged. It changes them.
Karl, you have failed to understand the distinction between content and
encoding. As your definition "<META..." is an HTML tag, I assume that
you are referring here to an HTML page in the archives. The whole point
of character entities is that (in HTML, but not in plain text) they
allow non-ASCII characters to be included in a document which is
otherwise ASCII-only, or in general that they allow any Unicode
character to be included in a document whatever the declared charset.
But this is not a change of logical content, it is only a change in how
that content is stored. It should be no more relevant to the user than
knowing whether the content is stored in computer memory, on a magnetic
disk or tape, or on a CD.
>At first I was ready to blame your email program. Now I realize it is deeper than that. We are dealing with incompatible systems, some of which have been around for some time. The earliest non-MIME complient standard was developed to show Asian characters, and one of the earliest MIME challenges was how to transmit those messages unchanged. (I’d forgotten about uuencoding, a way to transmit non-ASCII data as ASCII strings.)
>You blame the mail program I am using. How it sends messages, i.e. the encoding it uses, I am not sure. However it appears that it sends ASCII strings. For non-ASCII characters, that would mean character entities. ...
No, if we are talking about plain text rather than HTML, it is not
sending character entities, but strings of ASCII characters which it may
intend as a representation of non-ASCII characters but are not
representations according to any standard.
>... For receiving messages, it is designed to be read through a browser such as Firefox, Camino and Mozilla, so it passes on whatever message it receives unchanged: if Windows-1255, I can tell the browser to read it; if UTF-8, a different setting; if character entities, sent to the browser unchanged so that the content could be displayed unlost. Of course, I speculate because I don’t have enough data.
But it doesn't appear to do so. For if it receives in plain text a
string which in HTML would represent a character entity, it does not
send this through to the browser unchanged in content, but misinterprets
it as a character entity and transforms the string to a non-ASCII character.
>Then we have the ibiblio archiving program that takes only us-ascii. When it receives data, it loses content if that data is already encoded in character entities to be compatible with ASCII, because it changes it so that the data is no longer character entities. That losing content is certainly a bug.
No, it does not do this. If it receives HTML e-mail (which it cannot on
this list), it presumably deals with character entities correctly. But
in plain text e-mail it is IMPOSSIBLE for the archiving program to
receive character entities, so it is impossible that it can lose content
from them. This seems to be the point that you are stubbornly refusing
to accept: that in plain text there is NO SUCH THING as a character
entity. You will find no mention of them in the MIME spec I referred you
to. Character entities are a concept applicable to HTML e-mail only, and
so not applicable to this list. Thus there is no possibility of anyone's
program processing character entities incorrectly, in plain text e-mail.
>The question is, what are we as a forum going to do about these incompatible standards? Shall we make it a rule that in order to participate, we need to use a certain standard? If so, which? Should the technically savvy among us draw up a list of approved email clients, recommending that only those be used by list members? ??
Karl, the standard which we should follow is certainly the MIME spec I
referred you to, and its companions, which apply to all Internet e-mail,
and which make no reference to character entities. A decision has been
made, at least for the moment, not to use HTML e-mail, and that means no
character entities as well. If you really want to suggest a list of
approved e-mail clients, then yours will not be on it because it does
not conform to the MIME spec. But I don't suggest banning anyone, only
that it is made clear that anyone who, or whose mail program, attempts
to send character entities to this list will find that what they send is
neither received nor archived as character entities, but as ASCII
strings - and that is in accordance with the standard.
If you want to send Hebrew to this list, you need to find a way to do so
which is valid according to MIME, which means that the message must be
encoded in UTF-8 or one of the Hebrew encodings.
peter at qaya.org (personal)
peterkirk at qaya.org (work)
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.323 / Virus Database: 267.8.13/47 - Release Date: 12/07/2005
More information about the b-hebrew