Skip to Content.
Sympa Menu

microid - [Microid] MicroID hashing algorithm(s) and normalization

microid AT lists.ibiblio.org

Subject: Microid mailing list

List archive

Chronological Thread  
  • From: "Eran Sandler" <eran AT yedda.com>
  • To: <microid AT lists.ibiblio.org>
  • Subject: [Microid] MicroID hashing algorithm(s) and normalization
  • Date: Tue, 28 Nov 2006 19:25:52 +0200

Hello all,

 

Continuing Fred's talk about drafting a spec and Peter's willingness to commit to the end of next week, there are a few issues that are very rough on the edges.

Two such issues, which popped to my attention while implementing microid support at Yedda was the hashing algorithm and the normalization.

 

There was a bit of a discussion about it here: http://microid.org/blog/?p=4

 

In Yedda, instead of normalizing the URL used, we are using the URL from the request itself (as it is passed in the HTTP header) and use it to create the microid.

It does raises a few issues, one of which is the fact that we get two different microids if we access a page with or without a trailing slash.

The other is the unnecessary re-computation of the microid every request.

 

This was done as means to overcome the lack of formal normalization and to be as compatible as we can at this stage in life of microid.

 

Regarding algorithms, while SHA1 is very common, the issues about it being not unique in certain cases might require the use of a different hashing algorithm (or maybe more than one algorithm).

In our implementation I had to use SHA1 to make sure that sites like claimid.com will be able to claim Yedda profiles.

 

These issues are something that every microid implementation cannot be complete and compatible without and must be covered in the spec (of course it will help companies like Yahoo to adopt such a thing faster if the spec is indeed complete J )

 

 

My suggestion regarding normalization is relatively easy:

-          Make sure the URL is lower cased

-          If the hostname start's with "www." (i.e. http://www.microid.org/somewhere/), remove the "www." part (i.e. http://microid.org/somewhere/)

-          If the URL ends with a file (including a file extension, i.e. http://microid.org/somewhere/this.html), leave it as it is

-          If the URL ends with a directory or path (not a file, i.e. http://microid.org/somewhere), make sure it has a trailing slash at the end (i.e. http://microid.org/somewhere/)

 

The only problem I have is with URLs that contains query parameters (i.e. http://microid.org/somewhere/test.html?a=1&b=2), how to handle the order of these parameters and make sure the microid is generate with these parameters at the same order.

The easy path is to simply use them as they are shown and hope that users won't mix them J

 

 

Regarding hashing algorithm, I liked the suggestion specified in the blog post's comments (http://microid.org/blog/?p=4)of specifying the exact algorithm near the hash value. The main problem with this approach is that the various systems like claimid.com will have to handle quite a few of these hashing algorithms (unless the spec specifically gives a couple of options and that's it).

 

What do you say?

 

Eran

 




Archive powered by MHonArc 2.6.24.

Top of Page