microid AT lists.ibiblio.org
Subject: Microid mailing list
List archive
[Microid] MicroID hashing algorithm(s) and normalization
- From: "Eran Sandler" <eran AT yedda.com>
- To: <microid AT lists.ibiblio.org>
- Subject: [Microid] MicroID hashing algorithm(s) and normalization
- Date: Tue, 28 Nov 2006 19:25:52 +0200
Hello all, Continuing Fred's talk about drafting a spec and Peter's willingness
to commit to the end of next week, there are a few issues that are very rough
on the edges. Two such issues, which popped to my attention while
implementing microid support at Yedda was the hashing algorithm and the normalization. There was a bit of a discussion about it here: http://microid.org/blog/?p=4 In Yedda, instead of normalizing the URL used, we are using the
URL from the request itself (as it is passed in the HTTP header) and use it to
create the microid. It does raises a few issues, one of which is the fact that
we get two different microids if we access a page with or without a trailing
slash. The other is the unnecessary re-computation of the microid every
request. This was done as means to overcome the lack of formal normalization
and to be as compatible as we can at this stage in life of microid. Regarding algorithms, while SHA1 is very common, the issues about
it being not unique in certain cases might require the use of a different hashing
algorithm (or maybe more than one algorithm). In our implementation I had to use SHA1 to make sure that
sites like claimid.com will be able to claim Yedda profiles. These issues are something that every microid implementation
cannot be complete and compatible without and must be covered in the spec (of
course it will help companies like Yahoo to adopt such a thing faster if the
spec is indeed complete J ) My suggestion regarding normalization is relatively easy: -
Make sure the URL is lower cased -
If the hostname start's with "www."
(i.e. http://www.microid.org/somewhere/), remove the "www." part (i.e.
http://microid.org/somewhere/) -
If the URL ends with a file
(including a file extension, i.e. http://microid.org/somewhere/this.html),
leave it as it is -
If the URL ends with a directory
or path (not a file, i.e. http://microid.org/somewhere),
make sure it has a trailing slash at the end (i.e. http://microid.org/somewhere/) The only problem I have is with URLs that contains query
parameters (i.e. http://microid.org/somewhere/test.html?a=1&b=2),
how to handle the order of these parameters and make sure the microid is
generate with these parameters at the same order. The easy path is to simply use them as they are shown and
hope that users won't mix them J Regarding hashing algorithm, I liked the suggestion specified
in the blog post's comments (http://microid.org/blog/?p=4)of
specifying the exact algorithm near the hash value. The main problem with this
approach is that the various systems like claimid.com will have to handle quite
a few of these hashing algorithms (unless the spec specifically gives a couple
of options and that's it). What do you say? Eran |
-
[Microid] MicroID hashing algorithm(s) and normalization,
Eran Sandler, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Fred Stutzman, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Peter Saint-Andre, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Fred Stutzman, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Peter Saint-Andre, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Fred Stutzman, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Peter Saint-Andre, 11/28/2006
- Re: [Microid] MicroID hashing algorithm(s) and normalization, Evan Prodromou, 11/28/2006
- Re: [Microid] MicroID hashing algorithm(s) and normalization, Peter Saint-Andre, 11/28/2006
- Re: [Microid] MicroID hashing algorithm(s) and normalization, Peter Saint-Andre, 11/28/2006
- Re: [Microid] MicroID hashing algorithm(s) and normalization, Eran Sandler, 11/29/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Peter Saint-Andre, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Fred Stutzman, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Peter Saint-Andre, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Fred Stutzman, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Peter Saint-Andre, 11/28/2006
-
Re: [Microid] MicroID hashing algorithm(s) and normalization,
Fred Stutzman, 11/28/2006
Archive powered by MHonArc 2.6.24.