Skip to Content.
Sympa Menu

commons-research - Re: [Commons-research] open review model for upcoming workshop

commons-research AT lists.ibiblio.org

Subject: Commons-research mailing list

List archive

Chronological Thread  
  • From: Giorgos Cheliotis <gcheliotis.lists AT gmail.com>
  • To: commons-research AT lists.ibiblio.org
  • Subject: Re: [Commons-research] open review model for upcoming workshop
  • Date: Wed, 7 Jan 2009 18:47:16 +0800

Thanks Philipp for the contribution to what has been a very quiet mailing list lately (but this will change as we should soon shift gears with planning for the next workshop/conference).

Personally I don't find much benefit in such a move, but it is something to consider, if there is enough support for getting rid of the anonymity in reviews and enough strong arguments to back up such a decision.

My thoughts are in the exactly opposite direction: if anything, I think this year we could try to go with double-blind reviews, which is the standard for many of the more serious events and journals in several academic fields, and for good reason. There are many such good reasons why anonymity is valuable in such situations where one is asked to judge the fruits of another's often long and painstaking labor, i.e. when we judge works that people invest a lot of time, effort, money and ego in. I think I do not need to list out all the possible scenarios where lack of anonymity can go horribly wrong and compromise the integrity of reviews, or, destroy relationships which are badly needed for any academic community to sustain itself.

As for accountability, let's remember that the reviewers are never entirely anonymous. They are known to the chairs and to the PC members. I believe a certain degree of accountability is preserved under double-blind reviews, as PC members and chairs can control for each other's integrity as well as for the quality of submitted reviews. Both myself and Tyng-Ruey for example prompted many reviewers last year to improve the quality of their reviews, or to reconsider cases where we felt their reviews were not entirely fair. There was significant juggling involved in achieving anonymity as many PC members were also authors of submitted papers, but I think we did a good job, especially Tyng-Ruey who was in charge of coordinating reviews (as he had not submitted a paper himself, so had no direct conflict of interest).

Now if an author feels that all of a PC is colluding or is otherwise corrupt, then that author may just as well avoid submitting to that venue and/or write a letter of complaint, or take whatever other action with respect to any specific claims that he/she wants to make. In special cases I'm sure a process can be followed to investigate any claims of unfair treatment of a paper. But I for one want to be able to call a poor paper "poor" and still be friends with the author. If my assessment is not sufficiently supported by arguments and I just call it "poor" for no good reason, the opinions of the other reviewers and PC members should weigh in and I should be called upon by them to provide justification for my assessment. But making the reviews public will not make me more accountable, it will make me more timid and perhaps in some cases less honest.

Ok, well, these are my thoughts, sorry for the long post... let's keep talking. In short, publishing reviews for accepted papers can still be embarassing to all, as a paper can be accepted even with a very critical review. Anonymity aims not only at limiting the potential for public embarassment, it is more about safeguarding the relationships that we build with each other, as well as our ability to honestly critique each other's work. I have yet to see convincing arguments to the contrary.


On Jan 7, 2009, at 3:52 PM, Philipp Schmidt wrote:

I am participating in an open peer review process for the Human
Resources for Health journal. I think something like this would be a
good model for the upcoming commons research workshop/meeting as it
would address many of the concerns about being too open that were
raised prior to the iSummit 2008. I am forwarding the instructions for
review, but have removed information that allows identification of the
article.

The review state is closed to the public (I suppose to avoid
embarrassment of researchers whose papers might be rejected) but open
insofar as authors receive reviews of their paper along with the names
of the reviewers. Once a paper has been accepted all reviews (with
names of reviewers) and the paper are published online.

Best - P


---------- Forwarded message ----------
From: HRH Editorial <hrhjournal AT who.int>
Date: Wed, Jan 7, 2009 at 9:43 AM
Subject: Review of Human Resources for Health manuscript
To: Philipp Schmidt <phi.schmidt AT gmail.com>


Human Resources for Health
Methodology

Dear Dr Schmidt,

Many thanks for agreeing to review the above manuscript, which has
been submitted to Human Resources for Health. Please let us know
whether there is likely to be any difficulty in meeting the deadline
for your report, which is 28 January 2009.

Below you will find instructions for retrieving the manuscript and
returning your report via our website. Please remember that the
information in this manuscript is confidential.

As we mentioned in our previous message, we operate an open peer
review process for this journal in order to give peer reviewers credit
for their work and to ensure accountability. This means that we ask
reviewers to agree to their signed report being passed on to the
authors. If the manuscript is accepted for publication, your signed
report will be posted on the website along with the article and the
other reviewers' reports. We ask you to confirm that you consent to
this at the end of the peer review report.

Many thanks for all your help with this manuscript.

With best wishes,


The HRH Editorial Team

Tel: +44 (0)20 7323 0323
Facsimile: +44 (0)20 7631 9923
e-mail: hrhjournal AT who.int
Web: http://www.human-resources-health.com/
_______________________________________________
Commons-research mailing list
Commons-research AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/commons-research





Archive powered by MHonArc 2.6.24.

Top of Page