Slashdot Mirror
Quitting the Graphics Field Over SIGGRAPH
An anonymous reader writes "A Professor at Stony Brook university has quit the field of computer graphics. He claims too much importance is given to one particular conference (SIGGRAPH) and that acceptance of papers in this conference has too much importance in terms of the careers (tenure, grants etc) of a researcher. Furthermore he claims the paper reviewing for SIGGRAPH is not fair and bright and novel papers are summarily rejected because they are either not from a 'hot' field or because the reviewer does not understand the concept and is not willing to spend time understanding it. He has started a discussion forum which has comments from several big names in the field including the papers chair of SIGGRAPH 2007."
And this guy is changing to which academic field where things are different?
A-Bomb
In replying to this comment, I know that I'm going to sound like a bitter grad student; but, for some reason, I feel inclined to burn karma and make this statement:
I sympathize with this professor, and the trouble that he has faced. Although I work in the field of computer security (instead of computer graphics), I have seen many novel and ingenious papers rejected from conferences precisely because they are not from the current 'fad' field. Usually, I require large amounts of caffeine (and alcohol) just to make it through the conferences I attend, because they are filled with uninteresting papers written by hack academics attempting to ride the latest trend.
Perhaps it is this experience that has influenced the way in which I do academic reviews for conferences, when I am called upon to do so. I have no patience for papers that have nothing meaningful to say. Whenever I give an 'accept' rating to a paper, it is because I feel that the authors have something genuinely interesting to say. Whenever I give a 'reject' rating to a paper, I do my best to give as many constructive comments as I can -- I try to point out what insightful or meaningful things the author has done, as well as things that are genuine technical flaws and should be addressed. But, the thing I am never scared to do? I have never backed down from stating in a review, blatently, that the author's work seems novel and useful, and that some of the details are way over my head and should be subject to further review.
Given all the (meaningless) talk about reforming the academic review process, I often wonder: how much of the problem described by this professor would be solved if more reviewers had the balls to admit that some of the most novel ideas were over their heads?
This has been recognized for years. See "How to get your SIGGRAPH paper rejected, from 1993.
Some years ago, I stopped submitting papers to SIGGRAPH and started filing patents. It's been much more profitable.
Anyway, SIGGRAPH seems to have shrunk. I think the show floor peaked in size around 1997. Today, the Game Developer's Conference is where the real technical action is.
SIGGRAPH is mostly a rendering convention now; there's a little animation, a little behavior, and a tiny bit of physics in the papers this year, but other than that, it's rendering and compression. Which are relatively mature technologies.
I'm not one to dis peer review, but in this case screw them. Put your money where your mouth is and show them who's boss by showing them the money.
--- Grow a pair, liberals... stop letting the Republicans bully you!
There once was a boy who was hired to work on a construction crew. He was proud to be doing something useful and productive. The forman set him to tossing building stone over a wall.
The boy labored hard and was proud to have moved the pile of stone in record time. Surely this would show his usefullness and move up in the crew heirarchy in time.
The boy went to the forman and asked what task he should perform next.
"Throw 'em back over the wall," said the forman.
"What?" yelled the boy. "Why did you have me throw them over the wall in the first place if you were just going to have me throw them back?"
"Well," said the foreman. "You seemed a fine lad to me and I was proud to be able to offer you something to do in order that could learn to earn a wage. Perhaps someday I'll actually have something useful for you to do."
"To hell with this," the boy muttered under his breath and wandered off to find something useful he could do right now, whether it earned him a wage or not.
The moral of the story is: Fuck 'em. Fuck 'em all. Sideways.
KFG
I for one usually finding SIGGRAPH paper interesting and sometimes useful. I've read TFA and found it mostly non-informative ranting.
There is one example of the unfair editor behavior in the article - surely not enough to condemn all the conference.
Auther of the article don't like preferred treatment of the "hot subjects". But that is quite natural - "hot subjects" is what most people interested in this moment. If other researcher/practitioners in the field are not interested in what auther doing, they can not be blamed for it.
From the other hand establishing a rival conference would only improve things - more paper, more possibly overlooked approaches, more ideas.
I believe the best solution is to have authors of papers that have
amounted to something important and were rejected by the peer review
processes (not only be siggraph but also other important conferences),
to register somewhere, and basically have the person or persons which
peer-reviewed the paper and then rejected it noted, and to also carry
out an examination of their past rejections and acceptances and attempt
to establish a form of behavior with regards to them.
If said behavior is deemed unacceptable then that person or persons
will very simply not be asked to peer-review papers anymore. Add to that
a question mark will be placed on any future academic contributions they
make.
I would like to see people with the above hanging over their heads try
not to take the time and effort to understand what they are reviewing.
Arash
Arash Partow's Philosophy: Be a person who knows what they don't know, and not a person who doesn't know.
I don't know if SIGGRAPH has shrunk or not (I wasn't in graphics in '97), but I wouldn't say that the GDC has taken its place. I sympathize with Ashikhmin's frustration at the conference (but not his reaction), having been on the receiving end of a few cryptic SIGGRAPH rejections.
First of all, I don't agree that it's "mostly a rendering convention now". I'd say there were about 20 papers on rendering and compression out of 80 or 90 papers (unofficial page of papers). I also think that there's lots of "technical action" going on there.
The real problem is that SIGGRAPH hasn't grown with its field. One major conference was fine for the first 20 years or so, but graphics has grown in size and diversity so much in the last 15 years that it's ridiculous that there's still only one "top-shelf" conference. Look at the proceedings for this year's conference; there are papers on rendering, compression, ray-tracing, image processing, vision, data-driven modelling, GPGPU, procedural modelling, HDR, graphics APIs, fluid simulation, photography, mocap, light fields, pcrt, computational geometry, crowd sim, animation, and npr.
EACH of these things that are getting lumped into "GRAPHICS" is enough of a field in its own right that it deserves several journals and conferences of its own.
That's not even the meat of the problem; there ARE conferences for each of these topics, but people generally only submit SIGGRAPH rejects to them! The problem is that everyone wants the prestige that goes with a SIGGRAPH publication, and it's a vicious cycle; there are reviewers who shoot down every paper they feel is a threat to their own work and get away with it, and this forces anyone else who wants to survive there to do the same.
What needs to happen, in my bull-headed opinion, is for all of those people who write good papers that never make it to SIGGRAPH start submitting the first time around to the other conferences - I3D, Pacific Graphics, SCA, IEEE VIS, Eurographics, et cetera. These are all perfectly viable venues that will become as prestigious as people would like, if only people would take them seriously.
I say, let the small-minded dweebs have SIGGRAPH; we shouldn't gauge the quality of our work solely based on SIGGRAPH's rejection policy - even if it were a totally fair process, not every good paper can make it in. Submit your awesome paper to the other conferences, and once these other conferences are packed with impressive work, it'll mean as much as SIGGRAPH.
Just wishful (and a little bitter) thinking.
I don't think "hardware" was the right category for this...
Salon des Refusés:
In the 1860s, artists of the nascent realist and impressionist movements submitted works to the Salon de Paris, the official exhibition sponsored by the Académie des beaux-arts, selection committee only to be rejected. The resultant complaints of bias led French emperor Napoleon III to allow the rejected works to be displayed in a separate exhibition.
The first Salon des Refusés in 1863 invited art-works rejected for display at the Salon de Paris.
Most were poor quality, leading to ridicule in the press. However, the exhibition included several important paintings including Édouard Manet's Le déjeuner sur l'herbe (The Luncheon on the Grass) and James McNeill Whistler's The White Girl. Other artists who showed at the Salon des Refusés include Henri Fantin-Latour, Paul Cézanne, Armand Guillaumin, Johan Jongkind, and Camille Pissarro.
There are different kinds of "I don't understand it".
If the reviewer doesn't understand the importance of the claims or conclusion of the paper, then that's the author's problem. It's the responsibility of the author to make those clear and accessible to everybody.
If the reviewer doesn't understand the methods of the paper, that's the reviewer's problem. Methods sections need to be detailed, accurate, and take as little room as possible, which makes them intrinsically hard to understand. But that's not a problem because they are meant for reproducing the work, not for understanding it.
You know, up until a couple of years ago, I worked my entire adult life (about 20 years - or so :-)) in IT. Call it mid-life crisis, whatever. I needed a change. I was disgusted with corporate idiocy, among other career-specific reasons.
I completely changed careers; although I had some studies in my new field (translation), I got another degree to "re-establish" myself, and set out to work for myself. I can honestly say I've never been happier. Is it because I changed careers, or because I now work for myself? I don't know. All I know is I'm a much happier person, and (I'm told) more pleasant to be around.
I'm one of those people that firmly believes that humans are not meant to do just one thing in life.
I'm quite certain he'll find something that gives him more satisfaction, if he hasn't already.
As far as I can tell, most of academia is like this. Paper reviews (and conference reviews in particular) are really a bit of a lottery. Since academics don't get paid to review, they will often palm the reviews off to grad students who may or may not have the first clue about the field. And there is generally no rejoinder process for conferences, so you just have to wear it, improve the paper and resubmit it somewhere else. Journals and grant applications generally allow you a right of response, but you are still subject to the lottery of whether the reviewers:
a) know anything about the field,
b) actually read the paper
c) are open minded enough to consider new ideas or
d) have brains at all.
A colleague of mine recently had a brief paper (restricted to a maximum of two pages) rejected because it was too short - at exactly two pages. I kid you not.
I had a tenured position at a university as well, but I left the system anyway, and it was partly due to issues of the kind described.
Academia hinges almost entirely on your research karma, your success at obtaining grants, and the funds you can bring in to your department. In computing, it has very little to do with how effectively your work extends understanding in your area, even less to do with using honest scientific methods, and absolutely nothing to do with teaching.
And since your research karma is in the hands of the high priests in the field and has relatively little to do with your own technical abilities, I can fully understand the frustrations of other research academics. It's a dead-man's-shoes area, and not a good field to be in unless you're good at cultivating your profile through social engineering.
Fortunately I left early because of the compelling attraction of fat paycheques in freelance contracting, an order of magnitude better than academic payscales. But even without that, I think the social problems within academia might have made me leave in disgust at some point too.
I don't know anything specific to SIGGRAPH, but that kind of malaise is quite widespread in the academic sector.
PS. The current publication/conference-based approach in peer review needs change. The author of TFA actually gave one possible avenue, arXiv, which fits in well with today's greater interest in open systems. I support that.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Reminds me of a story I've read sometime ago. It went something like this:
A guy was employed in a factory. He'd sit there with a screwdriver, and two cups on a chain would come down from a hole in the ceiling, one of them containing two weird shape pieces of metal and a screw. So he'd take them, fasten them together with the supplied screw, place them in the other cup, and both would go back up to the next floor. Presumably to the next step in the assembly line.
So the guy does his job well for years, and fastens such metal pieces by the thousands a day. Which earns him a promotion to the next floor. Not much of a promotion, really, since it's at the same assembly line, but it pays a little better.
So he's shown to his seat at the next floor, given a screwdriver and shown what he has to do. Two cups would be raised on a chain through a hole in the floor, one containing two pieces fastened with a screw. He'd have to unscrew it and place the pieces and the screw in the other cup, at which point they'd descend back through the hole in the floor.
A polar bear is a cartesian bear after a coordinate transform.
Well, there were studies around saying that the average job satisfaction and happiness in IT was IIRC lower than anywhere else, including the garbage truck people. So it kinda makes me wonder if you're happy just generically because you're doing something else, or, perchance, it's just moving out of _IT_ that does the trick.
A polar bear is a cartesian bear after a coordinate transform.
If there are truly such systematic problems with SIGGRAPH, then there are probably sufficient other researchers interested in a new, improved computer graphics conference.
Sebastian Thrun and a few others were fed up with the quality of ICRA and IROS, so they started a wholey new conference last year, Robotics Science and Systems. It was successful, and IEEE is now even helping to organize future sessions.
Also, this kind of competition works. ICRA was noticeably better this year, as conferences will make changes in order to stay relevant.
Having done my UG dissertation on image processing, i feel somewhat inclined to agree with the assertion papers are complex! Having read my fair share of SIGGRAPH papers... the simpler ones waffle on and on above novel uses for convolution filters, the more complex ones take you to a realm of mathematical uncertainty - they ask for great leaps of faith (specially those that over generalise the pseudo code and dont link to working programs!)
Computer imagary is a very large and wide ranging subject, and because of that a conference MUST specialize in trends to generate the worthwhile feedback and peer review we all crave (if you read paper after paper on unrelated domains, it doesnt make you an expert, nor is such an atmosphere likely to attract experts!)
Matt
The purpose of publishing a paper is not to boost the authors' egos. It's to convey ideas to other people. A paper which does not communicate concepts clearly does not deserve to be published.
... why couldn't he just spell this part out stepwise instead of being so verbose" and also complained that if you make the proof too easy to understand, it won't get accepted.
Bingo. That's exactly the problem. When I first started in grad school (mechanical engineering), I found the papers very difficult to understand, and I thought it was a problem in my knowledge. But then when I had someone else explain it to me, I was like, "uh, couldn't they have just said [simpler version]" and my adviser politely explained how something that looks too easy won't look novel and notable enough to publish.
In a lecture from a math professor (Erdos number 1), I heard exactly the same thing. He said it takes him a long time to review a submission, because he has to say, "er, okay, how did he get from here to here
You really have to wonder what this is supposed to accomplish. Are you less smart because you got more people to understand your idea? (I've always thought that if you can't explain what you did to a reasonably intelligent layman, given enough time, you don't understand it yourself.)
Apology to Ubuntu forum.
Can I have your stuff?
Coding with assembly is like playing with Legos. Coding an application in assembly is like building a car with Legos.
Don't agree that global warming is man-made, severe, and long-term? You lose. Maybe the oil industry will throw you a few bucks for the paper, making your research look tainted.
The reviewers have their own careers at stake. If you don't support their little club, you're the enemy.
Wow,
I am glad that it isn't just management types that stuck on faddy subjects. this reminds me of all of the sales meetings, training courses, emails, and presentations that I've had to attend/read/view about the latest "breakthrtough" in business philosophy. It seems like every year I have to sit through some presentation about some repackaged, unoriginal bollocks that some arse came up with. Meanwhile, the guy with the real idea is on the phone arranging for some venture capital.
Glad to hear the academics are stuck with the same problem.
No. That's just how you interpreted it based on your own belief system.
And these people are, sadly, correct. If others can understand their work, then the Indians can likely do it cheaper. Consequently they get fired and their job outsourced.
The lesson here is that people want and need job security, and will take steps to create it themselves if it won't be provided to them, no matter how much harm it causes to the field or firm they work in.
I'm sure that some people will now post how you can switch jobs like socks and pick the best when you're amongst the best in your field, but the sad truth is that most people will never be amongst the best (by the definition of "best"). They still have children and need to feed them and feed them, and need a reliable source of income for that. So if such a person gets a chance of making himself irreplacable, at the expense of making his workplace less efficient... Well, what would you pick, your children or your job ?
If you want to get rid of this kind of behavior, give people other ways of insuring their continued employment. And if you insist on cutthroat competition, don't be surprised when people start using armored collars.
Forget magic. Any technology distinguishable from divine power is insufficiently advanced.
Such an absolute exclusion seems objectively very unlikely. One's academic reputation is ordinarily comprised of the totality of all the usual professional academic activities including reviewed publications, editorships, chairmanships, admin. duties, peer reviewing, scholarly society memberships, prizes, awards, grants, etc. Every selection committee in my experience considers all aspects of this totality when assessing candidates' curriculum vitae for promotions and new appointments.
It depends what you mean by "problem for me later". Feedback on the reviewer would not need to be made public, and assuming it is done double-blind, it would not directly affect career prospects. However, if a person refuses in principle to accept any feedback from peers on the quality of their reviewing -- whether it be praise or constructive criticism -- I think it must put their suitability as a reviewer in question. Directly accepting full responsibility for our reviewing is essential to the integrity of the peer-review process.
Why the disjunction? Whether or not you are a fair reviewer is a logically separate issue from whether or not some of the papers you review are of very poor quality. Let's stay with the original issue of judging the fairness of individual reviewers. Firstly, systematic cross-forum monitoring of the fairness of reviewers' reviews simply does not happen because nobody has been given the resources to do it across all the many different journals and conferences in which each reviewer participates as a reviewer. Secondly, a reviewer, whether that be you, me or anybody else, cannot possibly be expected to judge absolutely impartially whether or not their reviewing is fair. Such judgments must involve an independent anonymous (thus excluding well-known journal editors) peer; the whole process of meta review should also be double blind.
They might think that about your reviews, but whether or not they are correct is something that you, the reviewer, cannot judge impartially.
I think what you meant, to be more precise, is that reviews are done anonymously only to prevent authors being able to influence the reviewers during the review process or in other ways outside the review process, not to stop authors from having opinions about the fairness of the reviews themselves.
Not all good authors become reviewers in any particular peer community, and not all reviewers are good authors in that peer community. Good authors who are never reviewers are effectively excluded as peers. I think we would all accept that most reviewers do a good job and the quality of their reviews is generally high, but at the same time it has to be admitted that there is variation and this is something we should be trying to improve. A system of double-blind meta review would provide for systematic monitoring of and feedback on the fairness of reviews. As both an author and a reviewer, I think this would be a good thing for authors and reviewers.
I disagree. Good editors and review committees by themselves are not enough because systematically monitoring the fair
Scroogle