Slashdot Mirror
Quitting the Graphics Field Over SIGGRAPH
An anonymous reader writes "A Professor at Stony Brook university has quit the field of computer graphics. He claims too much importance is given to one particular conference (SIGGRAPH) and that acceptance of papers in this conference has too much importance in terms of the careers (tenure, grants etc) of a researcher. Furthermore he claims the paper reviewing for SIGGRAPH is not fair and bright and novel papers are summarily rejected because they are either not from a 'hot' field or because the reviewer does not understand the concept and is not willing to spend time understanding it. He has started a discussion forum which has comments from several big names in the field including the papers chair of SIGGRAPH 2007."
And this guy is changing to which academic field where things are different?
A-Bomb
In replying to this comment, I know that I'm going to sound like a bitter grad student; but, for some reason, I feel inclined to burn karma and make this statement:
I sympathize with this professor, and the trouble that he has faced. Although I work in the field of computer security (instead of computer graphics), I have seen many novel and ingenious papers rejected from conferences precisely because they are not from the current 'fad' field. Usually, I require large amounts of caffeine (and alcohol) just to make it through the conferences I attend, because they are filled with uninteresting papers written by hack academics attempting to ride the latest trend.
Perhaps it is this experience that has influenced the way in which I do academic reviews for conferences, when I am called upon to do so. I have no patience for papers that have nothing meaningful to say. Whenever I give an 'accept' rating to a paper, it is because I feel that the authors have something genuinely interesting to say. Whenever I give a 'reject' rating to a paper, I do my best to give as many constructive comments as I can -- I try to point out what insightful or meaningful things the author has done, as well as things that are genuine technical flaws and should be addressed. But, the thing I am never scared to do? I have never backed down from stating in a review, blatently, that the author's work seems novel and useful, and that some of the details are way over my head and should be subject to further review.
Given all the (meaningless) talk about reforming the academic review process, I often wonder: how much of the problem described by this professor would be solved if more reviewers had the balls to admit that some of the most novel ideas were over their heads?
This has been recognized for years. See "How to get your SIGGRAPH paper rejected, from 1993.
Some years ago, I stopped submitting papers to SIGGRAPH and started filing patents. It's been much more profitable.
Anyway, SIGGRAPH seems to have shrunk. I think the show floor peaked in size around 1997. Today, the Game Developer's Conference is where the real technical action is.
SIGGRAPH is mostly a rendering convention now; there's a little animation, a little behavior, and a tiny bit of physics in the papers this year, but other than that, it's rendering and compression. Which are relatively mature technologies.
I'm not one to dis peer review, but in this case screw them. Put your money where your mouth is and show them who's boss by showing them the money.
--- Grow a pair, liberals... stop letting the Republicans bully you!
There once was a boy who was hired to work on a construction crew. He was proud to be doing something useful and productive. The forman set him to tossing building stone over a wall.
The boy labored hard and was proud to have moved the pile of stone in record time. Surely this would show his usefullness and move up in the crew heirarchy in time.
The boy went to the forman and asked what task he should perform next.
"Throw 'em back over the wall," said the forman.
"What?" yelled the boy. "Why did you have me throw them over the wall in the first place if you were just going to have me throw them back?"
"Well," said the foreman. "You seemed a fine lad to me and I was proud to be able to offer you something to do in order that could learn to earn a wage. Perhaps someday I'll actually have something useful for you to do."
"To hell with this," the boy muttered under his breath and wandered off to find something useful he could do right now, whether it earned him a wage or not.
The moral of the story is: Fuck 'em. Fuck 'em all. Sideways.
KFG
I don't know if SIGGRAPH has shrunk or not (I wasn't in graphics in '97), but I wouldn't say that the GDC has taken its place. I sympathize with Ashikhmin's frustration at the conference (but not his reaction), having been on the receiving end of a few cryptic SIGGRAPH rejections.
First of all, I don't agree that it's "mostly a rendering convention now". I'd say there were about 20 papers on rendering and compression out of 80 or 90 papers (unofficial page of papers). I also think that there's lots of "technical action" going on there.
The real problem is that SIGGRAPH hasn't grown with its field. One major conference was fine for the first 20 years or so, but graphics has grown in size and diversity so much in the last 15 years that it's ridiculous that there's still only one "top-shelf" conference. Look at the proceedings for this year's conference; there are papers on rendering, compression, ray-tracing, image processing, vision, data-driven modelling, GPGPU, procedural modelling, HDR, graphics APIs, fluid simulation, photography, mocap, light fields, pcrt, computational geometry, crowd sim, animation, and npr.
EACH of these things that are getting lumped into "GRAPHICS" is enough of a field in its own right that it deserves several journals and conferences of its own.
That's not even the meat of the problem; there ARE conferences for each of these topics, but people generally only submit SIGGRAPH rejects to them! The problem is that everyone wants the prestige that goes with a SIGGRAPH publication, and it's a vicious cycle; there are reviewers who shoot down every paper they feel is a threat to their own work and get away with it, and this forces anyone else who wants to survive there to do the same.
What needs to happen, in my bull-headed opinion, is for all of those people who write good papers that never make it to SIGGRAPH start submitting the first time around to the other conferences - I3D, Pacific Graphics, SCA, IEEE VIS, Eurographics, et cetera. These are all perfectly viable venues that will become as prestigious as people would like, if only people would take them seriously.
I say, let the small-minded dweebs have SIGGRAPH; we shouldn't gauge the quality of our work solely based on SIGGRAPH's rejection policy - even if it were a totally fair process, not every good paper can make it in. Submit your awesome paper to the other conferences, and once these other conferences are packed with impressive work, it'll mean as much as SIGGRAPH.
Just wishful (and a little bitter) thinking.
I don't think "hardware" was the right category for this...
Salon des Refusés:
In the 1860s, artists of the nascent realist and impressionist movements submitted works to the Salon de Paris, the official exhibition sponsored by the Académie des beaux-arts, selection committee only to be rejected. The resultant complaints of bias led French emperor Napoleon III to allow the rejected works to be displayed in a separate exhibition.
The first Salon des Refusés in 1863 invited art-works rejected for display at the Salon de Paris.
Most were poor quality, leading to ridicule in the press. However, the exhibition included several important paintings including Édouard Manet's Le déjeuner sur l'herbe (The Luncheon on the Grass) and James McNeill Whistler's The White Girl. Other artists who showed at the Salon des Refusés include Henri Fantin-Latour, Paul Cézanne, Armand Guillaumin, Johan Jongkind, and Camille Pissarro.
There are different kinds of "I don't understand it".
If the reviewer doesn't understand the importance of the claims or conclusion of the paper, then that's the author's problem. It's the responsibility of the author to make those clear and accessible to everybody.
If the reviewer doesn't understand the methods of the paper, that's the reviewer's problem. Methods sections need to be detailed, accurate, and take as little room as possible, which makes them intrinsically hard to understand. But that's not a problem because they are meant for reproducing the work, not for understanding it.
You know, up until a couple of years ago, I worked my entire adult life (about 20 years - or so :-)) in IT. Call it mid-life crisis, whatever. I needed a change. I was disgusted with corporate idiocy, among other career-specific reasons.
I completely changed careers; although I had some studies in my new field (translation), I got another degree to "re-establish" myself, and set out to work for myself. I can honestly say I've never been happier. Is it because I changed careers, or because I now work for myself? I don't know. All I know is I'm a much happier person, and (I'm told) more pleasant to be around.
I'm one of those people that firmly believes that humans are not meant to do just one thing in life.
I'm quite certain he'll find something that gives him more satisfaction, if he hasn't already.
As far as I can tell, most of academia is like this. Paper reviews (and conference reviews in particular) are really a bit of a lottery. Since academics don't get paid to review, they will often palm the reviews off to grad students who may or may not have the first clue about the field. And there is generally no rejoinder process for conferences, so you just have to wear it, improve the paper and resubmit it somewhere else. Journals and grant applications generally allow you a right of response, but you are still subject to the lottery of whether the reviewers:
a) know anything about the field,
b) actually read the paper
c) are open minded enough to consider new ideas or
d) have brains at all.
A colleague of mine recently had a brief paper (restricted to a maximum of two pages) rejected because it was too short - at exactly two pages. I kid you not.
People don't get paid to review papers for conferences or journals. They only review papers because of a genuine interest, and because someone asked them nicely. If potential harm (to your career) could come from going out of your way to be helpful, would you still be helpful? It's already difficult to find qualified people who are willing to review papers; if they could be penalized later, I'm sure it will be much harder.
A good editor will know if the reviewer(s) are fair.
It is a flawed system, but I don't see how it can be fixed without some fundamental changes. One such change would be to have less emphasis on publications, but I don't see that happening anytime soon.
I had a tenured position at a university as well, but I left the system anyway, and it was partly due to issues of the kind described.
Academia hinges almost entirely on your research karma, your success at obtaining grants, and the funds you can bring in to your department. In computing, it has very little to do with how effectively your work extends understanding in your area, even less to do with using honest scientific methods, and absolutely nothing to do with teaching.
And since your research karma is in the hands of the high priests in the field and has relatively little to do with your own technical abilities, I can fully understand the frustrations of other research academics. It's a dead-man's-shoes area, and not a good field to be in unless you're good at cultivating your profile through social engineering.
Fortunately I left early because of the compelling attraction of fat paycheques in freelance contracting, an order of magnitude better than academic payscales. But even without that, I think the social problems within academia might have made me leave in disgust at some point too.
I don't know anything specific to SIGGRAPH, but that kind of malaise is quite widespread in the academic sector.
PS. The current publication/conference-based approach in peer review needs change. The author of TFA actually gave one possible avenue, arXiv, which fits in well with today's greater interest in open systems. I support that.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
If there are truly such systematic problems with SIGGRAPH, then there are probably sufficient other researchers interested in a new, improved computer graphics conference.
Sebastian Thrun and a few others were fed up with the quality of ICRA and IROS, so they started a wholey new conference last year, Robotics Science and Systems. It was successful, and IEEE is now even helping to organize future sessions.
Also, this kind of competition works. ICRA was noticeably better this year, as conferences will make changes in order to stay relevant.
Having done my UG dissertation on image processing, i feel somewhat inclined to agree with the assertion papers are complex! Having read my fair share of SIGGRAPH papers... the simpler ones waffle on and on above novel uses for convolution filters, the more complex ones take you to a realm of mathematical uncertainty - they ask for great leaps of faith (specially those that over generalise the pseudo code and dont link to working programs!)
Computer imagary is a very large and wide ranging subject, and because of that a conference MUST specialize in trends to generate the worthwhile feedback and peer review we all crave (if you read paper after paper on unrelated domains, it doesnt make you an expert, nor is such an atmosphere likely to attract experts!)
Matt
The purpose of publishing a paper is not to boost the authors' egos. It's to convey ideas to other people. A paper which does not communicate concepts clearly does not deserve to be published.
... why couldn't he just spell this part out stepwise instead of being so verbose" and also complained that if you make the proof too easy to understand, it won't get accepted.
Bingo. That's exactly the problem. When I first started in grad school (mechanical engineering), I found the papers very difficult to understand, and I thought it was a problem in my knowledge. But then when I had someone else explain it to me, I was like, "uh, couldn't they have just said [simpler version]" and my adviser politely explained how something that looks too easy won't look novel and notable enough to publish.
In a lecture from a math professor (Erdos number 1), I heard exactly the same thing. He said it takes him a long time to review a submission, because he has to say, "er, okay, how did he get from here to here
You really have to wonder what this is supposed to accomplish. Are you less smart because you got more people to understand your idea? (I've always thought that if you can't explain what you did to a reasonably intelligent layman, given enough time, you don't understand it yourself.)
Apology to Ubuntu forum.
Problem is, that strips away the objective response that's possible in *anonymous* review. At least half of the papers I've reviewed in the past few years (since I started doing it) have been seriously flawed in one way or another. I've felt no hesitation in saying so. If there's an open link along the lines of "George thought your paper was utter crap, Bob wanted major revisions, Tim said he didn't know so he passed it to his newest grad student who said it must be great because he didn't understand it"... then that freedom goes away.
Journal editors (or conference paper committees) are the ones who need to know this sort of thing, not individual authors. And they already do, and already use that information to decide who to send a manuscript to -- "Hmm... this one should go to ____, but he takes six months to read anything, so he's out... Jill's fast but she's dead against the approach this takes so she'll be an automatic 'no', better send it to John too except that the grammar's going to need work and he'll get too caught up in that..."
"A question mark will be placed..." on their own work? Just because someone's a lousy reviewer doesn't mean their research is bad. In the same way that being a good (or bad) researcher doesn't automatically mean one's a good (or bad) teacher.
So, the good part of your idea is already done, and the rest of it shouldn't be.
Can I have your stuff?
Coding with assembly is like playing with Legos. Coding an application in assembly is like building a car with Legos.