[Microid] MicroID hashing algorithm(s) and normalization
Eran Sandler
eran at yedda.com
Tue Nov 28 12:25:52 EST 2006
Hello all,
Continuing Fred's talk about drafting a spec and Peter's willingness to
commit to the end of next week, there are a few issues that are very rough
on the edges.
Two such issues, which popped to my attention while implementing microid
support at Yedda was the hashing algorithm and the normalization.
There was a bit of a discussion about it here: http://microid.org/blog/?p=4
In Yedda, instead of normalizing the URL used, we are using the URL from the
request itself (as it is passed in the HTTP header) and use it to create the
microid.
It does raises a few issues, one of which is the fact that we get two
different microids if we access a page with or without a trailing slash.
The other is the unnecessary re-computation of the microid every request.
This was done as means to overcome the lack of formal normalization and to
be as compatible as we can at this stage in life of microid.
Regarding algorithms, while SHA1 is very common, the issues about it being
not unique in certain cases might require the use of a different hashing
algorithm (or maybe more than one algorithm).
In our implementation I had to use SHA1 to make sure that sites like
claimid.com will be able to claim Yedda profiles.
These issues are something that every microid implementation cannot be
complete and compatible without and must be covered in the spec (of course
it will help companies like Yahoo to adopt such a thing faster if the spec
is indeed complete J )
My suggestion regarding normalization is relatively easy:
- Make sure the URL is lower cased
- If the hostname start's with "www." (i.e.
http://www.microid.org/somewhere/), remove the "www." part (i.e.
http://microid.org/somewhere/)
- If the URL ends with a file (including a file extension, i.e.
http://microid.org/somewhere/this.html), leave it as it is
- If the URL ends with a directory or path (not a file, i.e.
http://microid.org/somewhere), make sure it has a trailing slash at the end
(i.e. http://microid.org/somewhere/)
The only problem I have is with URLs that contains query parameters (i.e.
http://microid.org/somewhere/test.html?a=1
<http://microid.org/somewhere/test.html?a=1&b=2> &b=2), how to handle the
order of these parameters and make sure the microid is generate with these
parameters at the same order.
The easy path is to simply use them as they are shown and hope that users
won't mix them J
Regarding hashing algorithm, I liked the suggestion specified in the blog
post's comments (http://microid.org/blog/?p=4)of specifying the exact
algorithm near the hash value. The main problem with this approach is that
the various systems like claimid.com will have to handle quite a few of
these hashing algorithms (unless the spec specifically gives a couple of
options and that's it).
What do you say?
Eran
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ibiblio.org/pipermail/microid/attachments/20061128/fbf64f0e/attachment.htm
More information about the Microid
mailing list