Skip to Content.
Sympa Menu

commons-research - Re: [Commons-research] Reviews

commons-research AT lists.ibiblio.org

Subject: Commons-research mailing list

List archive

Chronological Thread  
  • From: "Giorgos Cheliotis" <giorgos AT smu.edu.sg>
  • To: <commons-research AT lists.ibiblio.org>
  • Subject: Re: [Commons-research] Reviews
  • Date: Thu, 28 Feb 2008 16:16:49 +0800

My concern, as already stated, is that (a) there are good reasons why reviews are communicated only to the authors, and (b) allowing "the public" to vote on favorite abstracts may sound more "democratic", and would  be an interesting experiment as some have suggested, but could turn our review process into a reality tv show, where the public can vote on their favorites and then researchers will be vying for popularity rather than focusing on producing solid work.
 
Then why not do it as an experiment? Well, once we officially allow everyone to vote on their favorites, it will be difficult to ignore these votes and say "oh, it was just an experiment, your vote doesn't really count".
 
I would therefore favor a solution where we post all accepted abstracts online and then allow for people to comment on those (instead of vote), to start a conversation with the authors or with other people interested in the same topic. I think this will be much more fruitful as a first approach towards engaging the public. 
 
Then, depending on the success of the workshop and the level of engagement of the broader community, we can discuss during the summit whether we should change the format or try something more "risky" next year.
 
Giorgos


From: commons-research-bounces AT lists.ibiblio.org [mailto:commons-research-bounces AT lists.ibiblio.org] On Behalf Of Anas TAWILEH
Sent: Tuesday, February 26, 2008 6:46 AM
To: commons-research AT lists.ibiblio.org
Subject: Re: [Commons-research] Reviews

Thanks Melanie,

 

Yes, you are right, I was suggesting a hybrid model where we have two processes running in parallel, with some intersection in what is made public.

 

Cheers,

Anas

 


From: commons-research-bounces AT lists.ibiblio.org [mailto:commons-research-bounces AT lists.ibiblio.org] On Behalf Of melanie dulong de rosnay
Sent: 24 February 2008 19:19
To: commons-research AT lists.ibiblio.org
Subject: Re: [Commons-research] Reviews

 

Dear all,

 

Le 24 févr. 08 à 13:35, Anas TAWILEH a écrit :



Dear all,

 

I would vote for the extended abstracts, as this usually leaves some room for the development of the research, and would invite more researchers to submit their work.

I agree

 

As for the discussion around the open vs. closed review process, I would like to see an open review, but not sure what is a possible mechanism that would facilitate this without the drawbacks mentioned by Giorgos.

Could we, for example, have an open voting process, where the number of votes given to each paper is made available online, while reviews and critics are sent to each author individually?

Anas, do you mean having both process in parallel:

- a closed review by the programme committee,

- and an open voting process on icommons website, which may designate the 1, 2 or 3 favorite abstracts by the public community?

 

It could be interesting to compare both results.

 

Best,

Melanie

 

What do you think?

Anas

 


From: commons-research-bounces AT lists.ibiblio.org [mailto:commons-research-bounces AT lists.ibiblio.org] On Behalf Of Giorgos Cheliotis
Sent: 24 February 2008 16:58
To: commons-research AT lists.ibiblio.org
Subject: [Commons-research] Reviews

 

Hi to all again,

 

some of us (mostly Philipp and myself) have been discussing how to best organize the reviews for the academic program of the workshop being planned for the iSummit.

 

Fist of all, what I have proposed on the draft CFP is to ask for the submission of only extended abstracts, not full papers. Naturally it's better to have full papers than just abstracts, but requesting for abstracts only would help focus the workshop on the presentation and critique of work in progress and would leave the presenters more room with respect to how they wish to present their work.

 

If we stick with the above plan, then it follows that reviews will not be as thorough as they would be when reviewing full papers. Reviews would have to be based on whether the topic and method of investigation are original and relevant, and whether the work looks promising sand is likely to instigate fruitful discussions at the summit. These are to some extent subjective criteria (let's not forget that all academic reviews are relatively subjective), but I feel that this is not a bad thing. We will need to use some judgment in order to ensure a good mix of presentations.

 

Is everyone on this list fine with the above, or do you feel that we should request for full papers (even if these are "working" papers)?

 

Also, the review process I have suggested would be closed, like all academic reviews that I am aware of. The purpose of keeping reviews closed is to allow reviewers total freedom to express their views, without the risk of damaging the author's public reputation in case these reviews are negative. In the case of reviewing abstracts, reviews will likely be quite short anyhow and they will be communicated to each respective author individually, but they could still range from very positive to very negative.

 

Philipp has been suggesting an open review process. This can be understood in two ways:

 

(a) The general public can vote for favorite abstracts. This is reminiscent of reality tv shows and might be an interesting experiment, but I understand that this is not what Philipp has in mind).

 

(b) The reviews are made by the program committee, but are published online along with the abstracts. This might be doable, but I wonder what would be the benefit of such an approach. I see the potential danger of either reviewers holding back on their comments because they would not want to embarrass anyone, or of authors being publicly humiliated by the publication of potentially very negative reviews.

 

I would now join Philipp in inviting others to join in and participate in the discussion.

 

Giorgos

 

_______________________________________________

Commons-research mailing list

 




Archive powered by MHonArc 2.6.24.

Top of Page