Slashdot Mirror

← Back to Stories (view on slashdot.org)

Experiment On Public Pre-reviewing and Discussion of Workshop Paper Submissions (reddit.com)

Posted by timothy on from the wisdom-of-crowds dept.

An anonymous reader writes: The ADAPT workshop (6th international workshop on adaptive, self-tuning computing systems) is trying a new publication model: all papers have been submitted via Arxiv, are now publicly discussed via Reddit, and will then be selected by a Program Committee for a presentation at the workshop. The idea is to speed up dissemination of novel ideas while making reviews more fair and letting the authors actively engage in discussions, defend their techniques, fix mistakes and eventually improve their open articles.

19 of 41 comments (clear)

  1. Re:Not new by guises · · Score: 2

    The Reddit thing is the notable part, and bizarre. A very odd choice. It seems likely they're doing it for publicity - no different from Facebook integration. It's just a workshop after all, and doing something weird like this can draw people in.

  2. Re:So now the SJWs... by Beck_Neard · · Score: 1

    Figures that this would be the first response on slashdot.

    Your two-bit brain is not capable of understanding the real world. Ask your parents for a genetic refund.

    --
    A fool and his hard drive are soon parted.
  3. Ah, arXiv by mbone · · Score: 4, Interesting

    You do know that some people are blocked from arXiv, and at least in some cases there is no obvious reason why (and no real appeal)? (Opponents of string theory, for example, seem to get this, or at least complain about this, fairly often.) I have seen this in action, it is real and it is capricious.

    I do not think that arXiv is suitable for a filter for a public meeting as long as its internal filtering is opaque in this fashion.

    1. Re:Ah, arXiv by fph+il+quozientatore · · Score: 3, Interesting

      I have never heard of this, and I am interested. Can you name an example of a respectable scientist (not a "fringe" controversial person, I mean) who has been banned?

      --
      My first program:

      Hell Segmentation fault

    2. Re:Ah, arXiv by Anonymous Coward · · Score: 2, Informative

      I have never heard of this, and I am interested. Can you name an example of a respectable scientist (not a "fringe" controversial person, I mean) who has been banned?

      Nobel Laureate Brian Josephson had this happen to him. He posted a record of his interactions with the arXiv moderators at

      http://www.tcm.phy.cam.ac.uk/~bdj10/articles/arxiv_correspondence.html

      Often the people who get their papers rejected by arXiv are just ordinary researchers who are trying to post ordinary papers. One such example is the following.

      http://blog.tanyakhovanova.com/2013/12/arxivs-police

    3. Re:Ah, arXiv by Anonymous Coward · · Score: 1

      Has Josephson been banned, or is he just upset that his paper, which seems to be more on the philosophy of biology and quantum mechanics, was deemed not appropriate for being listed as quantum physics? The paper is still on arxiv. His communications with the moderators and higher ups could maybe have been a little more professional. Not that it should matter. But the actual complaint is very questionable, and it is not a clear case of the moderators blatantly disregarding what they should be doing, as it is pretty easy to argue the the paper shouldn't have been cross-posted, even if the benefit of the doubt could have let it slip by...

    4. Re:Ah, arXiv by Anonymous Coward · · Score: 1

      Has Josephson been banned, or is he just upset that his paper, which seems to be more on the philosophy of biology and quantum mechanics, was deemed not appropriate for being listed as quantum physics?

      Josephson claims that he was temporarily "blacklisted" from posting to arXiv. See

      http://www.tcm.phy.cam.ac.uk/~bdj10/archivefreedom/main.html

    5. Re:Ah, arXiv by mbone · · Score: 2

      I have never heard of this, and I am interested. Can you name an example of a respectable scientist (not a "fringe" controversial person, I mean) who has been banned?

      Marni Sheppeard.
      Peter Woit.

      Note that they are not (as far as I can tell) banned, just blocked. Nothing is made public, it's just that certain things seem to happen consistently. And, in my experience, moderated papers are not available to the public.

      Note that the real problem here is not that papers are moderated. I understand the desire for moderation. It's the way it's being done that is problematic.

    6. Re:Ah, arXiv by Sibko · · Score: 1

      Can you explain to me why a "fringe" scientist (err... controversial person...?) shouldn't be allowed to speak on Arxiv? I just find it really curious that you immediately imply it's okay to censor a certain kind of speech you don't personally like. I mean, you're pretty much using the No True Scotsman fallacy right here; if the OP comes up with a name you can just declare him to not be a "real" scientist and you'll never be proven wrong.

      You either have an open forum and the idiots that come with that, or you don't. Arxiv would, frankly, do better with natural meritocratic filtering options - like science is meant to be based upon! - than to allow petty bureaucrats to control what is and is not considered science. We already HAVE methods to discard junk science: It's fundamental to how the scientific method works. Censoring people just because they present "controversial" ideas - that isn't science. That isn't just not science, that is pretty much the exact opposite of how science is supposed to work.

    7. Re:Ah, arXiv by serviscope_minor · · Score: 1

      Can you explain to me why a "fringe" scientist (err... controversial person...?) shouldn't be allowed to speak on Arxiv?

      Because the actual scientists using arXiv don't want to have to manually filter out a crapflood of spam from whackadoodles. Make no mistake, if the normal channels get too much "fringe" science, then the real scientists will leave and it will be worthless for anything but timecube theories.

      I just find it really curious that you immediately imply it's okay to censor a certain kind of speech you don't personally like

      Aw jeez not this shit again. Yeah arXiv has great power and great responsibility blah blah but part of that is not crapflooding working scientists inboxes with loopy spam. The right to speak is not the same as the right to force others to listen. You are demanding the latter.

      No matter where you set the filters someone will complain. And if you have no filters, you'll get nothing but a massive flood of penis enlargement pill advertisements.

      Arxiv would, frankly, do better with natural meritocratic filtering options - like science is meant to be based upon!

      Hahah it's funny you believe that could actually happen.

      And, who gets to judge merit? Bonus points with coming up for a method which (a) is immune to criticism about cliques and (b) immune to spam.

      We already HAVE methods to discard junk science: It's fundamental to how the scientific method works.

      It's a trap! Our methods cannot repel spam floods of this magnitude! (you know if you just open the door and let anything in)

      --
      SJW n. One who posts facts.
    8. Re:Ah, arXiv by mbone · · Score: 1

      If you are unable to contact the organizer of a workshop when something like that comes up and work out an alternative when needed, then you will likely have much bigger problems.

      I was actually thinking more about the organizers than the submitters (I have organized scientific meetings). One reason why organizers like this is that the arxiv paper submission is part of their automated submission process (i.e., they don't have to set up a paper hosting service, arXiv does it for them). This means that there is a real risk that arXiv is involved in their Editorial process, and that they might not even know it, and I think most conference organizers would find that unacceptable.

    9. Re:Ah, arXiv by eric_harris_76 · · Score: 1

      Not banned, and not from arXiv, but a recent Nobel laureate was co-author of a paper that recently didn't survive peer-review at a couple of supposedly prestigious medical journals (JAMA and NEJM, as I recall). (Link below.)

      They took their paper elsewhere, where it was better received.

      How recent? The latest round of awards. How recently? The rejections happened before the awarding of the prize.

      Feather in the cap of the publishing journal, and a black eye for the two that rejected it.

      https://duckduckgo.com/?q=angus+deaton+paper+rejected&t=ffcm

      --
      There's no time like the present. Well, the past used to be.
  4. Re:Integrated Face System by Anomalyst · · Score: 1

    the integrated face system is a system that reviews faces and villafies them for submission.

    FTFY

    --
    There is no right to feel safe thru security vaudeville at the expense of everyone's freedom, privacy and tax money.
  5. Re:What happens to the rejections? by fph+il+quozientatore · · Score: 1

    The same reason why you can't steal an already-published idea: your paper is publicly available and timestamped on arxiv, so everyone provably knows that it was your idea first. Moreover, a google search will likely reveal your previous contribution.

    --
    My first program:

    Hell Segmentation fault

  6. Just Havta GNow by Anomalyst · · Score: 1

    is the program to be in Java and interpreted, C and compiled or specific to a CPU family and assembled, possibly in binary and toggled in via the front panel (don't reviews require a panel of ancients)?

    --
    There is no right to feel safe thru security vaudeville at the expense of everyone's freedom, privacy and tax money.
  7. Re:Not new by the_povinator · · Score: 1

    Agreed. The ICLR conference, which is a machine learning conference organized by Yann LeCun (who now heads Facebook's machine learning group), is a bit like this. They use their own site for discussions, not Reddit, though.

    --
    The .sig is dead, and I believe I had a hand in killing it.
  8. Re:What happens to the rejections? by techno-vampire · · Score: 1

    One other thing I like about this form of reviewing articles is the fact that it can also prevent a small group of reviewers with an axe to grind rejecting everything that doesn't fit their world-view. I won't say that it's being done right now, but I do know that it's been claimed at least once.

    --
    Good, inexpensive web hosting
  9. Re:Not new by serviscope_minor · · Score: 1

    The Reddit thing is the notable part, and bizarre.

    I interpret it differently.

    Reddit is basically a discussion forum and allows voting on stuff. I was assuming they they were going to reddit because that meant they don't have to set up their own forum, secondly, don't have to persuade everyone to sign up to their own forum (a probably nontrivial fraction of people will already have reddit logins).

    --
    SJW n. One who posts facts.
  10. Re:Not new by enriquevagu · · Score: 1

    No, they are not doing this for publicity. Their goal is to improve the review process by attracting more people interested in the topic to the review process and make it more open.

    I happened to attend the workshop last year and there was a very interesting discussion at the end about how to modify a review process to make it more open. While I didn't take part in the discussion, there were many aspects considered about the open peer-review process, both positive and negative. For example, some authors might be frightened to submit a paper when sending preliminary versions of their work. The selection of Reddit and ArXiv didn't have any publicity (or political) objective, they were just tools familiar to the people involved in the organization and the discussion.

    I am some skeptical to this model, but still is a very interesting experiment so it will be nice to see how it compares to the reviews from previous editions.