Eric Ringger | 1 Oct 01:31 2011

Re: why LREC2012 NOT blind-reviewed?

Thanks to all for the open discussion.

 

Graeme’s reason (1)(a) – the impact on merit review – is for me the strongest reason to encourage LREC to move away from reviewing extended abstracts and toward reviewing full papers.

 

Best,

--Eric

 

From: Graeme Hirst [mailto:gh <at> cs.toronto.edu]
Sent: Friday, September 30, 2011 11:02 AM
To: Yorick Wilks
Cc: Eric Ringger; corpora <at> uib.no
Subject: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Yorick,

 

(1)  Whether a conference is reviewed by abstract or by full paper makes an enormous difference:

 

   (a) to merit, as perceived by tenure committees, granting agencies, and others, who count only fully peer-reviewed papers.

   (b) to funding for travel.  Right now, one of my colleagues has the problem that he cannot be funded to travel to give a paper at LREC because it isn't a fully-reviewed conference, so he doesn't even bother submitting.

 

You might say that these situations aren't desirable, but they are nonetheless reality right now.

 

(2)  I wonder how you are so sure that you almost invariably identify the author of an anonymous paper correctly.  If the paper is not ultimately accepted at the conference, which is 60 to 80% of them at ACL and COLING conferences, you will never find out who the authors actually are.  I've certainly guessed wrongly in the past.  And in my own papers, I often throw in "hidden signals" to deceive the reviewers.

 

(3)  I think Eric Ringger is 100% right about LREC.  As you say, LREC's reputation and quality have grown, and for that reason it has to start acting like a grown-up conference.

 

Regards,

Graeme

 

--
::::  Graeme Hirst
::::  University of Toronto * Department of Computer Science

 

On 2011-09-30, at 11:27, Yorick Wilks wrote:



I disagree strongly. I dont see why all conferences should be exactly like all others. Extended abstracts are less of a burden on busy academics --both as writers and reviewers----and there is no evidence they lower the final quality; COLING used to do this and I am sorry it changed. The whole blind-review business is a huge nonsense: I rarely meet a paper to review where i cannot identify the authors from a careful trawl of hidden signals and the references. Trying to make a paper genuinely anonymous is almost impossible if one has a body of past work and publication to link it to---the mental gymnastics required are undignified and best avoided. LRECs reputation has grown steadily and it will be the quality of its papers that sustain it--there is no evidence at all anonymity would improve matters in the least. if it ain't broke........

Yorick Wilks

 

 

On 30 Sep 2011, at 16:02, Eric Ringger wrote:



Greetings.

 

LREC has been operated in this manner since its inception.  Personally and for the sake of LREC’s reputation, I would like to see the reviewing process for LREC upgraded to double-blind review.

 

I believe that LREC fills a couple of important niches: its focus on language resources and evaluation/validation is important and not well served elsewhere, and it does a good job of bringing a large, diverse group together.  (I should add that it does a good job of selecting attractive venues as well!)  If implemented well, I believe that double-blind review would not detract from the primary objectives of the conference but would refine the quality of the program and improve the reputation of the venue.  I have said as much in private feedback after past LRECs.

 

I also think it is time for LREC to move up from reviewing extended abstracts to reviewing full papers.

 

Regards,

--Eric

 

 

From: corpora-bounces <at> uib.no [mailto:corpora-bounces <at> uib.no] On Behalf Of Isabella Chiari
Sent: Friday, September 30, 2011 8:45 AM
To: corpora <at> uib.no
Subject: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Dear Corpora members,

I just noticed that the LREC2012 call specifies that submissions are NOT anonymous and there will not be blind-reviewing.

 

Does anyone know why? Which is the policy under this decision?

Best regards,

Isabella Chiari

 

 

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora
Yassine Benajiba | 1 Oct 02:58 2011
Picon

Re: why LREC2012 NOT blind-reviewed?

Hi everyone,


I say let's judge the conference by the results. LREC is an awesome conference constantly improving year after year. Even though it would be great if somebody from the organizing committee could join this conversation and tell us a bit more about the reasons.

Best,

--Yassine.

On Fri, Sep 30, 2011 at 11:31 PM, Eric Ringger <ringger <at> cs.byu.edu> wrote:

Thanks to all for the open discussion.

 

Graeme’s reason (1)(a) – the impact on merit review – is for me the strongest reason to encourage LREC to move away from reviewing extended abstracts and toward reviewing full papers.

 

Best,

--Eric

 

From: Graeme Hirst [mailto:gh <at> cs.toronto.edu]
Sent: Friday, September 30, 2011 11:02 AM
To: Yorick Wilks
Cc: Eric Ringger; corpora <at> uib.no
Subject: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Yorick,

 

(1)  Whether a conference is reviewed by abstract or by full paper makes an enormous difference:

 

   (a) to merit, as perceived by tenure committees, granting agencies, and others, who count only fully peer-reviewed papers.

   (b) to funding for travel.  Right now, one of my colleagues has the problem that he cannot be funded to travel to give a paper at LREC because it isn't a fully-reviewed conference, so he doesn't even bother submitting.

 

You might say that these situations aren't desirable, but they are nonetheless reality right now.

 

(2)  I wonder how you are so sure that you almost invariably identify the author of an anonymous paper correctly.  If the paper is not ultimately accepted at the conference, which is 60 to 80% of them at ACL and COLING conferences, you will never find out who the authors actually are.  I've certainly guessed wrongly in the past.  And in my own papers, I often throw in "hidden signals" to deceive the reviewers.

 

(3)  I think Eric Ringger is 100% right about LREC.  As you say, LREC's reputation and quality have grown, and for that reason it has to start acting like a grown-up conference.

 

Regards,

Graeme

 

--
::::  Graeme Hirst
::::  University of Toronto * Department of Computer Science

 

On 2011-09-30, at 11:27, Yorick Wilks wrote:



I disagree strongly. I dont see why all conferences should be exactly like all others. Extended abstracts are less of a burden on busy academics --both as writers and reviewers----and there is no evidence they lower the final quality; COLING used to do this and I am sorry it changed. The whole blind-review business is a huge nonsense: I rarely meet a paper to review where i cannot identify the authors from a careful trawl of hidden signals and the references. Trying to make a paper genuinely anonymous is almost impossible if one has a body of past work and publication to link it to---the mental gymnastics required are undignified and best avoided. LRECs reputation has grown steadily and it will be the quality of its papers that sustain it--there is no evidence at all anonymity would improve matters in the least. if it ain't broke........

Yorick Wilks

 

 

On 30 Sep 2011, at 16:02, Eric Ringger wrote:



Greetings.

 

LREC has been operated in this manner since its inception.  Personally and for the sake of LREC’s reputation, I would like to see the reviewing process for LREC upgraded to double-blind review.

 

I believe that LREC fills a couple of important niches: its focus on language resources and evaluation/validation is important and not well served elsewhere, and it does a good job of bringing a large, diverse group together.  (I should add that it does a good job of selecting attractive venues as well!)  If implemented well, I believe that double-blind review would not detract from the primary objectives of the conference but would refine the quality of the program and improve the reputation of the venue.  I have said as much in private feedback after past LRECs.

 

I also think it is time for LREC to move up from reviewing extended abstracts to reviewing full papers.

 

Regards,

--Eric

 

 

From: corpora-bounces <at> uib.no [mailto:corpora-bounces <at> uib.no] On Behalf Of Isabella Chiari
Sent: Friday, September 30, 2011 8:45 AM
To: corpora <at> uib.no
Subject: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Dear Corpora members,

I just noticed that the LREC2012 call specifies that submissions are NOT anonymous and there will not be blind-reviewing.

 

Does anyone know why? Which is the policy under this decision?

Best regards,

Isabella Chiari

 

 


_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora


_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora
Yuri Tambovtsev | 1 Oct 11:20 2011
Picon

Reviewers

Surely, it is good to abolish any reviewers. Usually, the reviewers write what they were taught long ago. Besides, they have no time to penetrate into the logic construction of the report or article. So, only "grey" products are aproved. Brilliant reports or articles are thrown away. So, the reviewers make linguistics not brilliant. The other thing is, why the reviewers think that they know linguistics better? Why do they think that any other linguist cannot distinguish a good article from a bad article? I noticed that usually grey articles with nothing to discuss are published. Reviewers gladly recommend the articles in which the common and stale ideas are repeated. Please, share your ideas on abolishing the reviewers as the hindrence of the progress in linguistics to my e-mail addtess yutamb <at> mail.ru  Be well, Yuri Tambovtsev, a reviewer
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora
Declerck | 1 Oct 11:46 2011
Picon

Re: Reviewers

On 01.10.2011 11:20, Yuri Tambovtsev wrote:
Surely, it is good to abolish any reviewers. Usually, the reviewers write what they were taught long ago. Besides, they have no time to penetrate into the logic construction of the report or article. So, only "grey" products are aproved. Brilliant reports or articles are thrown away. So, the reviewers make linguistics not brilliant. The other thing is, why the reviewers think that they know linguistics better? Why do they think that any other linguist cannot distinguish a good article from a bad article? I noticed that usually grey articles with nothing to discuss are published. Reviewers gladly recommend the articles in which the common and stale ideas are repeated. Please, share your ideas on abolishing the reviewers as the hindrence of the progress in linguistics to my e-mail addtess yutamb <at> mail.ru  Be well, Yuri Tambovtsev, a reviewer

Dear All,

Quite interesting discussion going on on the basis of an initial question on submission and reviewing procedures at LREC.

Yuri's statement about reviewers is for sure a provocative one, but it gives the opportunity to talk about the role of reviewers.

Reviewers are in a Programme/Editorial  Committee and does do not have to decide (only) on the quality of a submission, but (also) if the submission can contribute to the topics of a workshop (or symposium or conference or journal etc.). So that a brilliant paper submission might not be adequate for a specific event.

Reviewers also should make comments and suggestions to the authors of the submission, this potentially  leading to reactions by the author(s) of the submission.
Reviewers are also not alone in taking decisions, and sometimes various reviewers are discussing on one and the same submission.  This together with  comments/suggestions on a submission are a kind of "hidden" scientific discourse. Maybe something to exploit in the future?

And personally I learn a lot about what is going on in the field on the basis of reviewing activity., also reading (interesting) submissions that do not make it onto official publications.
So that I am "voting" against Yuri's suggestion:-)

Thierry
-- --- Thierry Declerck, Senior Consultant at DFKI GmbH, Language Technology Lab Stuhlsatzenhausweg, 3 D-66123 Saarbruecken Phone: +49 681 / 857 75-53 58 Fax: +49 681 / 857 75-53 38 email: declerck <at> dfki.de ------------------------------------------------------------- Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Amtsgericht Kaiserslautern, HRB 2313 -------------------------------------------------------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora
Anil Singh | 1 Oct 12:56 2011
Picon

Re: Reviewers

I am in favour of continuing with a peer review process (a non-blind one, as I wrote in an earlier mail), but only as the first step in in a longer community review process, not as the last step and the final judgement (something that it is not, can't be, should not pretend to be and should not even try to be). The provocation for stating that last part is the fact that we now even have some very influential reviewers and conference organizers trying to set up a vigilante kind of process that would ensure that if a paper gets rejected at one place, it should get rejected everywhere else: by sharing of information (as if about terrorists!) across these events and their organizers. It sounded very scary to me when it was first proposed a few years ago (as far as I know).


And it has turned out be scary.

On Sat, Oct 1, 2011 at 3:16 PM, Declerck <declerck <at> dfki.de> wrote:
On 01.10.2011 11:20, Yuri Tambovtsev wrote:
Surely, it is good to abolish any reviewers. Usually, the reviewers write what they were taught long ago. Besides, they have no time to penetrate into the logic construction of the report or article. So, only "grey" products are aproved. Brilliant reports or articles are thrown away. So, the reviewers make linguistics not brilliant. The other thing is, why the reviewers think that they know linguistics better? Why do they think that any other linguist cannot distinguish a good article from a bad article? I noticed that usually grey articles with nothing to discuss are published. Reviewers gladly recommend the articles in which the common and stale ideas are repeated. Please, share your ideas on abolishing the reviewers as the hindrence of the progress in linguistics to my e-mail addtess yutamb <at> mail.ru  Be well, Yuri Tambovtsev, a reviewer

Dear All,

Quite interesting discussion going on on the basis of an initial question on submission and reviewing procedures at LREC.

Yuri's statement about reviewers is for sure a provocative one, but it gives the opportunity to talk about the role of reviewers.

Reviewers are in a Programme/Editorial  Committee and does do not have to decide (only) on the quality of a submission, but (also) if the submission can contribute to the topics of a workshop (or symposium or conference or journal etc.). So that a brilliant paper submission might not be adequate for a specific event.

Reviewers also should make comments and suggestions to the authors of the submission, this potentially  leading to reactions by the author(s) of the submission.
Reviewers are also not alone in taking decisions, and sometimes various reviewers are discussing on one and the same submission.  This together with  comments/suggestions on a submission are a kind of "hidden" scientific discourse. Maybe something to exploit in the future?

And personally I learn a lot about what is going on in the field on the basis of reviewing activity., also reading (interesting) submissions that do not make it onto official publications.
So that I am "voting" against Yuri's suggestion:-)

Thierry
-- --- Thierry Declerck, Senior Consultant at DFKI GmbH, Language Technology Lab Stuhlsatzenhausweg, 3 D-66123 Saarbruecken Phone: +49 681 / 857 75-53 58 Fax: +49 681 / 857 75-53 38 email: declerck <at> dfki.de ------------------------------------------------------------- Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Amtsgericht Kaiserslautern, HRB 2313 -------------------------------------------------------------

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora


_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora
Stan Szpakowicz | 1 Oct 12:40 2011
Picon

why not double-NOT-blind-reviewed? (was: why LREC2012 NOT blind-reviewed?)

Alberto Simões wrote:

 > In fact, I am concerned by the fact that LREC is blind reviewed in 
the reviewer side.
 >
 > If somebody is commenting a paper, he/she should be responsible for 
what he/she says, and therefore, sign his/her review.

Yes. Yes. Yes. For a much more eloquent argument, see Inderjeet Mani's 
Last Words in CL 37(1) at

http://www.mitpressjournals.org/doi/pdf/10.1162/coli_a_00046

I recognize the validity of Graeme's argument, but that is a separate 
(sad) story.

Cheers,

Stan

--
Stan Szpakowicz, PhD, Professor
EECS, Computer Science, University of Ottawa

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora

Anil Singh | 1 Oct 12:32 2011
Picon

Re: why LREC2012 NOT blind-reviewed?

As some academicians/researchers I have respect for have expressed opinions which I believe to be right and which I have been writing about on my blog and elsewhere (apart from arguing for them in conversations), I dare to chip in and say my bit.

When I had entered this area and was thinking of submitting my first paper (around 2003-2004) and I found out that reviewing will be blind, I was delighted. I was a nobody (a graduate student) from a developing country (India, but not even from one of the IITs and not with a very good pedigree), I thought double blind reviewing will be definitely more fair for people like me.

My experience since then has completely disabused me of that naive idea. While it may not be possible to exactly identify the author(s) of the paper, one does get enough information (and meta-information) that is more than enough to trigger all the prejudices, biases etc. that blind reviewing is supposed to be an antidote against. This happens in almost all the cases. Needless to add that there can be exceptions.

You can, of course, give numerous counter-examples from cases where no bias or prejudice is likely anyway or is very unlikely. But those examples are not the ones that matter here.

As far as I am concerned, if you can just identify the fact that the author is from India, that alone removes at least half of the supposed effectiveness of the idea of double blind reviewing. And if you work on Indian languages and do certain kind of work, it's a no-brainer.

Then there can be things like whether the author is just a student or an established researcher, whether the project is funded or non-funded, whether the language is that of a native speaker or not etc. These are the very things that double blind reviewing is supposed to guard against, but it simply can't. It just can't and I am sorry that it can't. Theoretically the idea still appeals to me, but may be like many other theoretically good things, it is not practically implementable.

I especially like Yorick's comment about undignified gymnastics that one is required to perform to hide one's identity. It even lowers the academic quality of the paper quite often because you can't add information that is very relevant. And I am totally in favour of the reviewer taking responsibility for his comments. I have a corpus of reviews and some of the comments simply make one embarrassed that academicians (which one is too) can behave like that -- and that too in writing.

One of the things that has always left me wondering (to put it lightly) is the fact that the conduct of academicians during the actual meetings, i.e., paper presentations, panel discussions etc. is so exceedingly civilized (for want of a better word) that I sometimes feel out of place there (coming from a chaotic third world country and being disordered personally). But a lot of the same academicians, when they blind-review a paper, behave like bullies, vigilantes or just plain hooligans. Fortunately, their number is still a minority.

Of course, like everyone else, I have received wonderful (even if very critical) reviews. But that can happen even with non-blind reviewing. Just read literary supplements of papers that take literature seriously.

To conclude, I would just say that if for nothing else, at least to maintain the basic dignity of the academic community and of individual academicians, it would be best if we switch to a reviewing process that does not pretend to be blind and where reviewers take responsibility for their comments.

I am agnostic about whether extended abstracts should be reviewed or full papers. Both seem to have their merits. For a conference like LREC, extended abstracts do seem better to me, though I won't fight for that (borrowing a phrase from review forms).

I hope am not doing anything wrong by adding this link here:




On Sat, Oct 1, 2011 at 6:28 AM, Yassine Benajiba <benajibayassine <at> gmail.com> wrote:
Hi everyone,

I say let's judge the conference by the results. LREC is an awesome conference constantly improving year after year. Even though it would be great if somebody from the organizing committee could join this conversation and tell us a bit more about the reasons.

Best,

--Yassine.

On Fri, Sep 30, 2011 at 11:31 PM, Eric Ringger <ringger <at> cs.byu.edu> wrote:

Thanks to all for the open discussion.

 

Graeme’s reason (1)(a) – the impact on merit review – is for me the strongest reason to encourage LREC to move away from reviewing extended abstracts and toward reviewing full papers.

 

Best,

--Eric

 

From: Graeme Hirst [mailto:gh <at> cs.toronto.edu]
Sent: Friday, September 30, 2011 11:02 AM
To: Yorick Wilks
Cc: Eric Ringger; corpora <at> uib.no
Subject: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Yorick,

 

(1)  Whether a conference is reviewed by abstract or by full paper makes an enormous difference:

 

   (a) to merit, as perceived by tenure committees, granting agencies, and others, who count only fully peer-reviewed papers.

   (b) to funding for travel.  Right now, one of my colleagues has the problem that he cannot be funded to travel to give a paper at LREC because it isn't a fully-reviewed conference, so he doesn't even bother submitting.

 

You might say that these situations aren't desirable, but they are nonetheless reality right now.

 

(2)  I wonder how you are so sure that you almost invariably identify the author of an anonymous paper correctly.  If the paper is not ultimately accepted at the conference, which is 60 to 80% of them at ACL and COLING conferences, you will never find out who the authors actually are.  I've certainly guessed wrongly in the past.  And in my own papers, I often throw in "hidden signals" to deceive the reviewers.

 

(3)  I think Eric Ringger is 100% right about LREC.  As you say, LREC's reputation and quality have grown, and for that reason it has to start acting like a grown-up conference.

 

Regards,

Graeme

 

--
::::  Graeme Hirst
::::  University of Toronto * Department of Computer Science

 

On 2011-09-30, at 11:27, Yorick Wilks wrote:



I disagree strongly. I dont see why all conferences should be exactly like all others. Extended abstracts are less of a burden on busy academics --both as writers and reviewers----and there is no evidence they lower the final quality; COLING used to do this and I am sorry it changed. The whole blind-review business is a huge nonsense: I rarely meet a paper to review where i cannot identify the authors from a careful trawl of hidden signals and the references. Trying to make a paper genuinely anonymous is almost impossible if one has a body of past work and publication to link it to---the mental gymnastics required are undignified and best avoided. LRECs reputation has grown steadily and it will be the quality of its papers that sustain it--there is no evidence at all anonymity would improve matters in the least. if it ain't broke........

Yorick Wilks

 

 

On 30 Sep 2011, at 16:02, Eric Ringger wrote:



Greetings.

 

LREC has been operated in this manner since its inception.  Personally and for the sake of LREC’s reputation, I would like to see the reviewing process for LREC upgraded to double-blind review.

 

I believe that LREC fills a couple of important niches: its focus on language resources and evaluation/validation is important and not well served elsewhere, and it does a good job of bringing a large, diverse group together.  (I should add that it does a good job of selecting attractive venues as well!)  If implemented well, I believe that double-blind review would not detract from the primary objectives of the conference but would refine the quality of the program and improve the reputation of the venue.  I have said as much in private feedback after past LRECs.

 

I also think it is time for LREC to move up from reviewing extended abstracts to reviewing full papers.

 

Regards,

--Eric

 

 

From: corpora-bounces <at> uib.no [mailto:corpora-bounces <at> uib.no] On Behalf Of Isabella Chiari
Sent: Friday, September 30, 2011 8:45 AM
To: corpora <at> uib.no
Subject: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Dear Corpora members,

I just noticed that the LREC2012 call specifies that submissions are NOT anonymous and there will not be blind-reviewing.

 

Does anyone know why? Which is the policy under this decision?

Best regards,

Isabella Chiari

 

 


_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora



_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora


_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora
Rayson, Paul | 1 Oct 17:01 2011
Picon

ICAME Journal call for papers and subscriptions

The ICAME Journal is published annually in both electronic format and paper copy.

 

CALL FOR SUBMISSIONS

====================

 

Deadline for submissions: 1 December 2011

Deadline for reviews: 31 December 2011

 

The ICAME Journal invites submissions for proposed contributions in the field of English Corpus Linguistics for immediate consideration for the next issue in 2012 or the following issue in 2013. Manuscripts for articles, progress reports and shorter notices can be sent to one of the editors:

 

Merja Kytö                          Anna-Brita Stenström

Department of English               Nedanvägen 7

Uppsala University                  SE-291 35 Kristianstad

PO Box 527                          Sweden

SE-751 20 Uppsala                        

Sweden                             

e-mail: merja.kyto <at> engelska.uu.se   e-mail: ab.stenstrom <at> telia.com

 

Books for review and correspondence on reviews and abstracts should be sent to:

 

Ilka Mindt

Universität Potsdam

Institut für Anglistik und Amerikanistik

Am Neuen Palais 10

14469 Potsdam

e-mail: imindt <at> uni-potsdam.de

 

Date of publication of issue 36: May 2012

 

CALL FOR SUBSCRIPTIONS

======================

 

More information and previous issues (in PDF) are available on-line at the ICAME Journal website: http://icame.uib.no/journal.html

 

To subscribe to the ICAME Journal for the next issue to be published in May 2012 please visit our secure on-line order form:

 

https://online-payments.lancaster-university.co.uk/browse/product.asp?catid=98&modid=1&compid=1

 

If this longer URL is broken by your mail reader, please use the following shortcut:

 

http://tinyurl.com/3xyswlu

 

Previous issues are still on sale via the same order form.

 

Please note that a subscription to issue 36 (May 2012) is included in the registration for the ICAME 2012 conference in Leuven. Those who register for ICAME 2012 will automatically receive a copy at the conference.

 

Queries on subscriptions and distribution should be directed to:

 

Paul Rayson

School of Computing and Communications

Infolab21

Lancaster University

Lancaster

LA1 4WA

UK.

e-mail: p.rayson <at> lancaster.ac.uk

 

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora
lbarron | 2 Oct 00:54 2011
Picon

Registration is open - Cross-Language !ndian Text Re-Use


               CL!TR - Cross-Language !ndian Text Re-Use

                 held in conjunction with the FIRE 2011
               Forum for Information Retrieval Evaluation
                  in Bombay, India, 2-4 December 2011
       http://www.dsic.upv.es/grupos/nle/fire-workshop-clitr.html
----------------------------------------------------------------------

The registration for the task on Cross-Language !ndian Text Re-Use at
FIRE is now open. A mailing list has been created as well in order to
let discussion among participants.

Both registration and mailing list subscription forms are available
through the task website (Registration/Discussion section).

Note that participants are allowed to submit up to three runs in order
to experimenting with different settings.

The call for participation follows.

----------------------------------------------------------------------
In this first edition of the evaluation task the focus is exclusively
on the cross-language text re-use detection task on Indian languages.
Text re-use detection across languages represents an interesting
retrieval task, which is even more challenging when considering
languages distantly related, yet in permanent contact (this year we
start with the English-Hindi).

As Wikipedia has been identified as a preferred source for students,
some of whom fall into the temptation of re-using its contents over
their own writing, the collection of texts provided includes several
Wikipedia articles (in English), some of which could have been
considered when answering to a set of questions (in Hindi). The
proposed task is to retrieve, if it is the case, the Wikipedia
article that has been used to generate a dubious piece of text.

We invite researchers and practitioners from all fields to participate.

-----------------------------------------------------------------------
Important Dates
-----------------------------------------------------------------------

Aug 25				Corpus release (training corpus)
Sep 13				Query release  (test corpus)
Oct 21				Run submission
Nov 15 				Qrel release   (result notification)
Nov 28				Working notes due
Dec 2-4 			Conference

-----------------------------------------------------------------------
Contact
-----------------------------------------------------------------------

E-mail: clitr <at> dsic.upv.es
Web page: http://www.dsic.upv.es/grupos/nle/fire-workshop-clitr.html

-----------------------------------------------------------------------
Organization
-----------------------------------------------------------------------

Alberto Barrón-Cedeño, Paolo Rosso
NLE Lab  <at>  Universidad Politécnica de Valencia, Spain

Sobha Lalitha Devi
CLR Group  <at>  AU-KBC Research Centre, Chennai, India

Paul Clough, Mark Stevenson
IR & NLP Groups  <at>  University of Sheffield, UK
-----------------------------------------------------------------------

Best regards,
CL!TR 2011 Organising Committee

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora

Michal Ptaszynski | 2 Oct 16:23 2011
Picon

why LREC2012 NOT blind-reviewed?

Just two humble cents from me.
I'm sure most of us, even those strongly against double blind reviewing,
remember or at least realize why this process was introduced.
The fact is many people DO send many crappy papers, often re-sending
exactly the same crap simultaneously to many conferences. If blind
reviewing can flush at least some of this, and improve the world of
science even in 1/100 of a percent, I wouldn't vote against it.

Although what I would really like to have (and I think it is feasible) is
a system telling me more-less if the paper is a candidate for a crap or a
reliable piece of work.

Also a word about the reviewers being hidden behind the curtain. This is
not exactly true, since the list of all reviewers is always available on
the conference homepage (and the list of sub-reviewers is sometimes also
added in the proceedings). If you spent some time in the field you can
more less guess who does things similar to you and can narrow down the
list of your potential reviewers (so its very much like with guessing the
paper authors).

Best,

Michal

-----------------------------------
Od: Anil Singh <anil.phdcl <at> gmail.com>
Kopia dla: corpora <at> uib.no, Yorick Wilks <Y.Wilks <at> dcs.shef.ac.uk>
Do: Yassine Benajiba <benajibayassine <at> gmail.com>
Data: Sat, 1 Oct 2011 16:02:56 +0530
Temat: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

As some academicians/researchers I have respect for have expressed
opinions which I believe to be right and which I have been writing about
on my blog and elsewhere (apart from arguing for them in conversations), I
dare to chip in and say my bit.

When I had entered this area and was thinking of submitting my first paper
(around 2003-2004) and I found out that reviewing will be blind, I was
delighted. I was a nobody (a graduate student) from a developing country
(India, but not even from one of the IITs and not with a very good
pedigree), I thought double blind reviewing will be definitely more fair
for people like me.

My experience since then has completely disabused me of that naive idea.
While it may not be possible to exactly identify the author(s) of the
paper, one does get enough information (and meta-information) that is more
than enough to trigger all the prejudices, biases etc. that blind
reviewing is supposed to be an antidote against. This happens in almost
all the cases. Needless to add that there can be exceptions.

You can, of course, give numerous counter-examples from cases where no
bias or prejudice is likely anyway or is very unlikely. But those examples
are not the ones that matter here.

As far as I am concerned, if you can just identify the fact that the
author is from India, that alone removes at least half of the supposed
effectiveness of the idea of double blind reviewing. And if you work on
Indian languages and do certain kind of work, it's a no-brainer.

Then there can be things like whether the author is just a student or an
established researcher, whether the project is funded or non-funded,
whether the language is that of a native speaker or not etc. These are the
very things that double blind reviewing is supposed to guard against, but
it simply can't. It just can't and I am sorry that it can't. Theoretically
the idea still appeals to me, but may be like many other theoretically
good things, it is not practically implementable.

I especially like Yorick's comment about undignified gymnastics that one
is required to perform to hide one's identity. It even lowers the academic
quality of the paper quite often because you can't add information that is
very relevant. And I am totally in favour of the reviewer taking
responsibility for his comments. I have a corpus of reviews and some of
the comments simply make one embarrassed that academicians (which one is
too) can behave like that -- and that too in writing.

One of the things that has always left me wondering (to put it lightly) is
the fact that the conduct of academicians during the actual meetings,
i.e., paper presentations, panel discussions etc. is so exceedingly
civilized (for want of a better word) that I sometimes feel out of place
there (coming from a chaotic third world country and being disordered
personally). But a lot of the same academicians, when they blind-review a
paper, behave like bullies, vigilantes or just plain hooligans.
Fortunately, their number is still a minority.

Of course, like everyone else, I have received wonderful (even if very
critical) reviews. But that can happen even with non-blind reviewing. Just
read literary supplements of papers that take literature seriously.

To conclude, I would just say that if for nothing else, at least to
maintain the basic dignity of the academic community and of individual
academicians, it would be best if we switch to a reviewing process that
does not pretend to be blind and where reviewers take responsibility for
their comments.

I am agnostic about whether extended abstracts should be reviewed or full
papers. Both seem to have their merits. For a conference like LREC,
extended abstracts do seem better to me, though I won't fight for that
(borrowing a phrase from review forms).

I hope am not doing anything wrong by adding this link here:

http://reviewscontd.org/

    On Sat, Oct 1, 2011 at 6:28 AM, Yassine Benajiba
<benajibayassine <at> gmail.com> wrote:
    Hi everyone,

I say let's judge the conference by the results. LREC is an awesome
conference constantly improving year after year. Even though it would be
great if somebody from the organizing committee could join this
conversation and tell us a bit more about the reasons.

Best,

--Yassine.

On Fri, Sep 30, 2011 at 11:31 PM, Eric Ringger <ringger <at> cs.byu.edu> wrote:
Thanks to all for the open discussion.

Graeme’s reason (1)(a) – the impact on merit review – is for me the
strongest reason to encourage LREC to move away from reviewing extended
abstracts and toward reviewing full papers.

Best,
--Eric

From: Graeme Hirst [mailto:gh <at> cs.toronto.edu]
Sent: Friday, September 30, 2011 11:02 AM
To: Yorick Wilks
Cc: Eric Ringger; corpora <at> uib.no
Subject: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

Yorick,

(1)  Whether a conference is reviewed by abstract or by full paper makes
an enormous difference:

      (a) to merit, as perceived by tenure committees, granting agencies,  
and
others, who count only fully peer-reviewed papers.
      (b) to funding for travel.  Right now, one of my colleagues has the
problem that he cannot be funded to travel to give a paper at LREC because
it isn't a fully-reviewed conference, so he doesn't even bother submitting.

You might say that these situations aren't desirable, but they are
nonetheless reality right now.

(2)  I wonder how you are so sure that you almost invariably identify the
author of an anonymous paper correctly.  If the paper is not ultimately
accepted at the conference, which is 60 to 80% of them at ACL and COLING
conferences, you will never find out who the authors actually are.  I've
certainly guessed wrongly in the past.  And in my own papers, I often
throw in "hidden signals" to deceive the reviewers.

(3)  I think Eric Ringger is 100% right about LREC.  As you say, LREC's
reputation and quality have grown, and for that reason it has to start
acting like a grown-up conference.

Regards,
Graeme

--
::::  Graeme Hirst
::::  University of Toronto * Department of Computer Science

    On 2011-09-30, at 11:27, Yorick Wilks wrote:

I disagree strongly. I dont see why all conferences should be exactly like
all others. Extended abstracts are less of a burden on busy academics
--both as writers and reviewers----and there is no evidence they lower the
final quality; COLING used to do this and I am sorry it changed. The whole
blind-review business is a huge nonsense: I rarely meet a paper to review
where i cannot identify the authors from a careful trawl of hidden signals
and the references. Trying to make a paper genuinely anonymous is almost
impossible if one has a body of past work and publication to link it
to---the mental gymnastics required are undignified and best avoided.
LRECs reputation has grown steadily and it will be the quality of its
papers that sustain it--there is no evidence at all anonymity would
improve matters in the least. if it ain't broke........
Yorick Wilks

On 30 Sep 2011, at 16:02, Eric Ringger wrote:

Greetings.

LREC has been operated in this manner since its inception.  Personally and
for the sake of LREC’s reputation, I would like to see the reviewing
process for LREC upgraded to double-blind review.

I believe that LREC fills a couple of important niches: its focus on
language resources and evaluation/validation is important and not well
served elsewhere, and it does a good job of bringing a large, diverse
group together.  (I should add that it does a good job of selecting
attractive venues as well!)  If implemented well, I believe that
double-blind review would not detract from the primary objectives of the
conference but would refine the quality of the program and improve the
reputation of the venue.  I have said as much in private feedback after
past LRECs.

I also think it is time for LREC to move up from reviewing extended
abstracts to reviewing full papers.

Regards,
--Eric

From: corpora-bounces <at> uib.no [mailto:corpora-bounces <at> uib.no] On Behalf Of
Isabella Chiari
Sent: Friday, September 30, 2011 8:45 AM
To: corpora <at> uib.no
Subject: [Corpora-List] why LREC2012 NOT blind-reviewed?

Dear Corpora members,
I just noticed that the LREC2012 call specifies that submissions are NOT
anonymous and there will not be blind-reviewing.

Does anyone know why? Which is the policy under this decision?
Best regards,
Isabella Chiari

    _______________________________________________
    UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
    Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora

_______________________________________________
    UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
    Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora <at> uib.no
http://mailman.uib.no/listinfo/corpora

Gmane