Towards standard methods and metrics for evaluating IA

["Livia Labate" <liv AT livlab.com>]

Hello Susan, thanks! :)
I'm sending this back to the metric list so everyone can discuss it

I didn't understand why asking submitters for the raw format. I think PDF
*ONLY* is the best possibility as it means we won't have to format anything
or have the author worry about us editing it.

I read the wiki doc and added some suggestion for the submission
requirements. Context: I'm thinking about it visualizing a submission form,
so this is how it looks in my mind:

1. Submitter information
- name
- email
- institution

2. Submission information:

a) Authors
- name
- email
- institution
(repeat for more authors, if applicable)

b) Category
- Checklist
- Heuristics
- Do's and dont's
- (add more)

c) Contents
- Abstract (brief explaination)
- Full submission (pdf format)

d) Legacy
- This submission is:
[ ] Original work
    > example: a brand new metric
[ ] Adapted from an original for a specific context
    > example: original 'ROI evaluation for UX',
               adaption 'ROI evaluation for IA'
[ ] Adapted metric from original for similar context
    > example: original 'ROI evaluation for UX',
               adaption 'checklists for UX'

e) Comments about the submission

3. Metric Details

a) Use case (context):

- Goals of metric
> measure
 [ ] ROI
 [ ] Performance
 [ ] Appeal
 [ ] Other ________________
> validate
 [ ] Requirements
 [ ] Navigation
 [ ] Labels
 [ ] Design
 [ ] Other ________________
> Refine
 [ ] Prototype
 [ ] Task flows
 [ ] Final product
 [ ] Other ________________

- Application
 [ ] post requirement gathering
 [ ] post content organization
 [ ] post structure creation
 [ ] post layout
 [ ] post prototype
 [ ] post launch

- Type
 [ ] Qualitative
 [ ] Quantitative
 [ ] Hybrid


b) Characterization of the Metric/Method

- Case Study
> What application(s)/system(s) it was used for?
> Was it successful for that purpose?
> Other aspects of usage

- Outcome and conclusion
> Pros (Advantages)
> Cons (Limitations)

c) Resources involved
- People
 > Professionals
 > Test subjects
- Time
 > Preparation
 > Implementation
 > Evaluation
 > Reporting
- Technology
 > Hardware
 > Operational System
 > Software
 > Accessories
 > Other materials

d) Audience
- who would use this metric?
[ ] Senior IA's
[ ] Junior IA's
[ ] Business Executives
[ ] Project Managers
[ ] Statisticians
[ ] Developers
[ ] Designers
[ ] Others _______________

- who would sign-off/approve this metric's results?
(who you would share this information with & the level of understanding or
experience required to "get" it)

- - - - - - - -

[ ] I agree with <link>AIFIA's metric submission terms</link>.


The subitems for all questions need to be improved, I just added some to
contextualize, but I ran out of time to work on this further. I hope it
helps.

Cheers,

Livia Labate
http://livlab.com



----- Original Message ----- 
From: "Susan Campbell" <SusanC AT ZAAZ.com>
To: "'Livia Labate'" <liv AT livlab.com>
Sent: Tuesday, September 02, 2003 9:13 PM
Subject: RE: [Aifia-metrics] metric submission reqs


I agree, context is especially important for metrics. If we were to make
some required/some optional, which of the draft questions should we require?

BTW, congrats on your position on the board!
Susan

> -----Original Message-----
> From: Livia Labate [mailto:liv AT livlab.com]
> Sent: Tuesday, September 02, 2003 4:56 PM
> To: Susan Campbell
> Subject: Re: [Aifia-metrics] metric submission reqs
>
>
> It seems to me that an abstract should be mandatory -- any
> submission, be it
> for a congress, a university or work, requires an introductory note to
> provide context for the reader.
>
> Livia
>