Slashdot Mirror
Quitting the Graphics Field Over SIGGRAPH
An anonymous reader writes "A Professor at Stony Brook university has quit the field of computer graphics. He claims too much importance is given to one particular conference (SIGGRAPH) and that acceptance of papers in this conference has too much importance in terms of the careers (tenure, grants etc) of a researcher. Furthermore he claims the paper reviewing for SIGGRAPH is not fair and bright and novel papers are summarily rejected because they are either not from a 'hot' field or because the reviewer does not understand the concept and is not willing to spend time understanding it. He has started a discussion forum which has comments from several big names in the field including the papers chair of SIGGRAPH 2007."
And this guy is changing to which academic field where things are different?
A-Bomb
There once was a boy who was in charge of watching sheep. He did his job so well that no wolves ever came around. It was very boring, day in and day out watching the sheep, making sure they didn't get eaten by wolves.
One day he decided that in order to make things more interesting and himself more important, he would fake a wolf attack. He yelled at the top of his lungs "Wolf! Wolf!" and the villagers came running to help chase away the wolves. But when the villagers arrived, no wolves were to be seen.
"It's a good thing I was able to chase those wolves away!" the boy exclaimed. The villagers agreed and patted the boy on the head and congratulated him on a job well-done.
The next week the boy did the same thing. "Wolf! Wolf!" The villagers came running, but no wolf was to be found. They thanked the boy and went back to their homes.
A week later, the boy tried his little ruse again. This time only half the villagers came, and when they saw there was no wolf, they went home.
The next week a real wolf came. The boy cried "Wolf! Wolf!" But this time no one came. They just assumed that the boy was lying again.
The wolf helped himself to the sheep and the poor little boy.
The moral of the story is: Don't be a fucking dumbass and end your career because you've got a bee in your fucking bonnet. Don't wear bonnets.
In replying to this comment, I know that I'm going to sound like a bitter grad student; but, for some reason, I feel inclined to burn karma and make this statement:
I sympathize with this professor, and the trouble that he has faced. Although I work in the field of computer security (instead of computer graphics), I have seen many novel and ingenious papers rejected from conferences precisely because they are not from the current 'fad' field. Usually, I require large amounts of caffeine (and alcohol) just to make it through the conferences I attend, because they are filled with uninteresting papers written by hack academics attempting to ride the latest trend.
Perhaps it is this experience that has influenced the way in which I do academic reviews for conferences, when I am called upon to do so. I have no patience for papers that have nothing meaningful to say. Whenever I give an 'accept' rating to a paper, it is because I feel that the authors have something genuinely interesting to say. Whenever I give a 'reject' rating to a paper, I do my best to give as many constructive comments as I can -- I try to point out what insightful or meaningful things the author has done, as well as things that are genuine technical flaws and should be addressed. But, the thing I am never scared to do? I have never backed down from stating in a review, blatently, that the author's work seems novel and useful, and that some of the details are way over my head and should be subject to further review.
Given all the (meaningless) talk about reforming the academic review process, I often wonder: how much of the problem described by this professor would be solved if more reviewers had the balls to admit that some of the most novel ideas were over their heads?
to increase the chances of this document being treated as a bit more than whining of a looser
Given that various links are 404ing as well, I'm wondering, is this a scam?
This has been recognized for years. See "How to get your SIGGRAPH paper rejected, from 1993.
Some years ago, I stopped submitting papers to SIGGRAPH and started filing patents. It's been much more profitable.
Anyway, SIGGRAPH seems to have shrunk. I think the show floor peaked in size around 1997. Today, the Game Developer's Conference is where the real technical action is.
SIGGRAPH is mostly a rendering convention now; there's a little animation, a little behavior, and a tiny bit of physics in the papers this year, but other than that, it's rendering and compression. Which are relatively mature technologies.
I'm not one to dis peer review, but in this case screw them. Put your money where your mouth is and show them who's boss by showing them the money.
--- Grow a pair, liberals... stop letting the Republicans bully you!
I for one usually finding SIGGRAPH paper interesting and sometimes useful. I've read TFA and found it mostly non-informative ranting.
There is one example of the unfair editor behavior in the article - surely not enough to condemn all the conference.
Auther of the article don't like preferred treatment of the "hot subjects". But that is quite natural - "hot subjects" is what most people interested in this moment. If other researcher/practitioners in the field are not interested in what auther doing, they can not be blamed for it.
From the other hand establishing a rival conference would only improve things - more paper, more possibly overlooked approaches, more ideas.
I believe the best solution is to have authors of papers that have
amounted to something important and were rejected by the peer review
processes (not only be siggraph but also other important conferences),
to register somewhere, and basically have the person or persons which
peer-reviewed the paper and then rejected it noted, and to also carry
out an examination of their past rejections and acceptances and attempt
to establish a form of behavior with regards to them.
If said behavior is deemed unacceptable then that person or persons
will very simply not be asked to peer-review papers anymore. Add to that
a question mark will be placed on any future academic contributions they
make.
I would like to see people with the above hanging over their heads try
not to take the time and effort to understand what they are reviewing.
Arash
Arash Partow's Philosophy: Be a person who knows what they don't know, and not a person who doesn't know.
I don't know if SIGGRAPH has shrunk or not (I wasn't in graphics in '97), but I wouldn't say that the GDC has taken its place. I sympathize with Ashikhmin's frustration at the conference (but not his reaction), having been on the receiving end of a few cryptic SIGGRAPH rejections.
First of all, I don't agree that it's "mostly a rendering convention now". I'd say there were about 20 papers on rendering and compression out of 80 or 90 papers (unofficial page of papers). I also think that there's lots of "technical action" going on there.
The real problem is that SIGGRAPH hasn't grown with its field. One major conference was fine for the first 20 years or so, but graphics has grown in size and diversity so much in the last 15 years that it's ridiculous that there's still only one "top-shelf" conference. Look at the proceedings for this year's conference; there are papers on rendering, compression, ray-tracing, image processing, vision, data-driven modelling, GPGPU, procedural modelling, HDR, graphics APIs, fluid simulation, photography, mocap, light fields, pcrt, computational geometry, crowd sim, animation, and npr.
EACH of these things that are getting lumped into "GRAPHICS" is enough of a field in its own right that it deserves several journals and conferences of its own.
That's not even the meat of the problem; there ARE conferences for each of these topics, but people generally only submit SIGGRAPH rejects to them! The problem is that everyone wants the prestige that goes with a SIGGRAPH publication, and it's a vicious cycle; there are reviewers who shoot down every paper they feel is a threat to their own work and get away with it, and this forces anyone else who wants to survive there to do the same.
What needs to happen, in my bull-headed opinion, is for all of those people who write good papers that never make it to SIGGRAPH start submitting the first time around to the other conferences - I3D, Pacific Graphics, SCA, IEEE VIS, Eurographics, et cetera. These are all perfectly viable venues that will become as prestigious as people would like, if only people would take them seriously.
I say, let the small-minded dweebs have SIGGRAPH; we shouldn't gauge the quality of our work solely based on SIGGRAPH's rejection policy - even if it were a totally fair process, not every good paper can make it in. Submit your awesome paper to the other conferences, and once these other conferences are packed with impressive work, it'll mean as much as SIGGRAPH.
Just wishful (and a little bitter) thinking.
I don't think "hardware" was the right category for this...
Salon des Refusés:
In the 1860s, artists of the nascent realist and impressionist movements submitted works to the Salon de Paris, the official exhibition sponsored by the Académie des beaux-arts, selection committee only to be rejected. The resultant complaints of bias led French emperor Napoleon III to allow the rejected works to be displayed in a separate exhibition.
The first Salon des Refusés in 1863 invited art-works rejected for display at the Salon de Paris.
Most were poor quality, leading to ridicule in the press. However, the exhibition included several important paintings including Édouard Manet's Le déjeuner sur l'herbe (The Luncheon on the Grass) and James McNeill Whistler's The White Girl. Other artists who showed at the Salon des Refusés include Henri Fantin-Latour, Paul Cézanne, Armand Guillaumin, Johan Jongkind, and Camille Pissarro.
because the reviewer does not understand the concept and is not willing to spend time understanding it.
The SIGGRAPH reviewers are highly competent, and within their time constraints, thorough (the process is described here). If they don't understand the concept in your paper, maybe you didn't explain it clearly enough.
The purpose of publishing a paper is not to boost the authors' egos. It's to convey ideas to other people. A paper which does not communicate concepts clearly does not deserve to be published.
There are different kinds of "I don't understand it".
If the reviewer doesn't understand the importance of the claims or conclusion of the paper, then that's the author's problem. It's the responsibility of the author to make those clear and accessible to everybody.
If the reviewer doesn't understand the methods of the paper, that's the reviewer's problem. Methods sections need to be detailed, accurate, and take as little room as possible, which makes them intrinsically hard to understand. But that's not a problem because they are meant for reproducing the work, not for understanding it.
You know, up until a couple of years ago, I worked my entire adult life (about 20 years - or so :-)) in IT. Call it mid-life crisis, whatever. I needed a change. I was disgusted with corporate idiocy, among other career-specific reasons.
I completely changed careers; although I had some studies in my new field (translation), I got another degree to "re-establish" myself, and set out to work for myself. I can honestly say I've never been happier. Is it because I changed careers, or because I now work for myself? I don't know. All I know is I'm a much happier person, and (I'm told) more pleasant to be around.
I'm one of those people that firmly believes that humans are not meant to do just one thing in life.
I'm quite certain he'll find something that gives him more satisfaction, if he hasn't already.
As far as I can tell, most of academia is like this. Paper reviews (and conference reviews in particular) are really a bit of a lottery. Since academics don't get paid to review, they will often palm the reviews off to grad students who may or may not have the first clue about the field. And there is generally no rejoinder process for conferences, so you just have to wear it, improve the paper and resubmit it somewhere else. Journals and grant applications generally allow you a right of response, but you are still subject to the lottery of whether the reviewers:
a) know anything about the field,
b) actually read the paper
c) are open minded enough to consider new ideas or
d) have brains at all.
A colleague of mine recently had a brief paper (restricted to a maximum of two pages) rejected because it was too short - at exactly two pages. I kid you not.
I had a tenured position at a university as well, but I left the system anyway, and it was partly due to issues of the kind described.
Academia hinges almost entirely on your research karma, your success at obtaining grants, and the funds you can bring in to your department. In computing, it has very little to do with how effectively your work extends understanding in your area, even less to do with using honest scientific methods, and absolutely nothing to do with teaching.
And since your research karma is in the hands of the high priests in the field and has relatively little to do with your own technical abilities, I can fully understand the frustrations of other research academics. It's a dead-man's-shoes area, and not a good field to be in unless you're good at cultivating your profile through social engineering.
Fortunately I left early because of the compelling attraction of fat paycheques in freelance contracting, an order of magnitude better than academic payscales. But even without that, I think the social problems within academia might have made me leave in disgust at some point too.
I don't know anything specific to SIGGRAPH, but that kind of malaise is quite widespread in the academic sector.
PS. The current publication/conference-based approach in peer review needs change. The author of TFA actually gave one possible avenue, arXiv, which fits in well with today's greater interest in open systems. I support that.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Reminds me of a story I've read sometime ago. It went something like this:
A guy was employed in a factory. He'd sit there with a screwdriver, and two cups on a chain would come down from a hole in the ceiling, one of them containing two weird shape pieces of metal and a screw. So he'd take them, fasten them together with the supplied screw, place them in the other cup, and both would go back up to the next floor. Presumably to the next step in the assembly line.
So the guy does his job well for years, and fastens such metal pieces by the thousands a day. Which earns him a promotion to the next floor. Not much of a promotion, really, since it's at the same assembly line, but it pays a little better.
So he's shown to his seat at the next floor, given a screwdriver and shown what he has to do. Two cups would be raised on a chain through a hole in the floor, one containing two pieces fastened with a screw. He'd have to unscrew it and place the pieces and the screw in the other cup, at which point they'd descend back through the hole in the floor.
A polar bear is a cartesian bear after a coordinate transform.
Well, there were studies around saying that the average job satisfaction and happiness in IT was IIRC lower than anywhere else, including the garbage truck people. So it kinda makes me wonder if you're happy just generically because you're doing something else, or, perchance, it's just moving out of _IT_ that does the trick.
A polar bear is a cartesian bear after a coordinate transform.
If there are truly such systematic problems with SIGGRAPH, then there are probably sufficient other researchers interested in a new, improved computer graphics conference.
Sebastian Thrun and a few others were fed up with the quality of ICRA and IROS, so they started a wholey new conference last year, Robotics Science and Systems. It was successful, and IEEE is now even helping to organize future sessions.
Also, this kind of competition works. ICRA was noticeably better this year, as conferences will make changes in order to stay relevant.
How much technical action did you see on the show floor of SIGGRAPH in 1998? And how much viscous fluid modelling gets done at GDC, exactly? GDC replaces SIGGRAPH like a cheatsheet replaces mathematics. SIGGRAPH papers from 2006 will be GDC material in 2012.
...
SIGGRAPH accepted more papers this year than were submitted in 1998. There is no sense in which SIGGRAPH is lacking technical content. It is fiercely competitive and that's why good papers get rejected.
Still, if you can't take the heat
The patent office is less demanding than any peer review board, this is certainly true. You can impress generalists and lawyers a lot more easily than you can impress actual technical people.
Of course, the problem with patents is they encourage people to engage in FUD mongering such as:
"This broad patent covers most spring/damper character simulation systems. If it falls, it has joints, it looks right, and it works right, it's probably covered by our patent."
(From your website.)
It's not really the high road, though, is it?
Having done my UG dissertation on image processing, i feel somewhat inclined to agree with the assertion papers are complex! Having read my fair share of SIGGRAPH papers... the simpler ones waffle on and on above novel uses for convolution filters, the more complex ones take you to a realm of mathematical uncertainty - they ask for great leaps of faith (specially those that over generalise the pseudo code and dont link to working programs!)
Computer imagary is a very large and wide ranging subject, and because of that a conference MUST specialize in trends to generate the worthwhile feedback and peer review we all crave (if you read paper after paper on unrelated domains, it doesnt make you an expert, nor is such an atmosphere likely to attract experts!)
Matt
Can I have your stuff?
Coding with assembly is like playing with Legos. Coding an application in assembly is like building a car with Legos.
Don't agree that global warming is man-made, severe, and long-term? You lose. Maybe the oil industry will throw you a few bucks for the paper, making your research look tainted.
The reviewers have their own careers at stake. If you don't support their little club, you're the enemy.
Wow,
I am glad that it isn't just management types that stuck on faddy subjects. this reminds me of all of the sales meetings, training courses, emails, and presentations that I've had to attend/read/view about the latest "breakthrtough" in business philosophy. It seems like every year I have to sit through some presentation about some repackaged, unoriginal bollocks that some arse came up with. Meanwhile, the guy with the real idea is on the phone arranging for some venture capital.
Glad to hear the academics are stuck with the same problem.
Ever read a patent? All those claims written in legalese just to describe what is often a trivial idea. Makes it look novel.
Ever read code? Even some open-source stuff - sendmail comes to mind - is hard to understand.
Job security and ego is everywhere. Some people think if others can understand their work, they become less important/useful.
No. That's just how you interpreted it based on your own belief system.
Such an absolute exclusion seems objectively very unlikely. One's academic reputation is ordinarily comprised of the totality of all the usual professional academic activities including reviewed publications, editorships, chairmanships, admin. duties, peer reviewing, scholarly society memberships, prizes, awards, grants, etc. Every selection committee in my experience considers all aspects of this totality when assessing candidates' curriculum vitae for promotions and new appointments.
It depends what you mean by "problem for me later". Feedback on the reviewer would not need to be made public, and assuming it is done double-blind, it would not directly affect career prospects. However, if a person refuses in principle to accept any feedback from peers on the quality of their reviewing -- whether it be praise or constructive criticism -- I think it must put their suitability as a reviewer in question. Directly accepting full responsibility for our reviewing is essential to the integrity of the peer-review process.
Why the disjunction? Whether or not you are a fair reviewer is a logically separate issue from whether or not some of the papers you review are of very poor quality. Let's stay with the original issue of judging the fairness of individual reviewers. Firstly, systematic cross-forum monitoring of the fairness of reviewers' reviews simply does not happen because nobody has been given the resources to do it across all the many different journals and conferences in which each reviewer participates as a reviewer. Secondly, a reviewer, whether that be you, me or anybody else, cannot possibly be expected to judge absolutely impartially whether or not their reviewing is fair. Such judgments must involve an independent anonymous (thus excluding well-known journal editors) peer; the whole process of meta review should also be double blind.
They might think that about your reviews, but whether or not they are correct is something that you, the reviewer, cannot judge impartially.
I think what you meant, to be more precise, is that reviews are done anonymously only to prevent authors being able to influence the reviewers during the review process or in other ways outside the review process, not to stop authors from having opinions about the fairness of the reviews themselves.
Not all good authors become reviewers in any particular peer community, and not all reviewers are good authors in that peer community. Good authors who are never reviewers are effectively excluded as peers. I think we would all accept that most reviewers do a good job and the quality of their reviews is generally high, but at the same time it has to be admitted that there is variation and this is something we should be trying to improve. A system of double-blind meta review would provide for systematic monitoring of and feedback on the fairness of reviews. As both an author and a reviewer, I think this would be a good thing for authors and reviewers.
I disagree. Good editors and review committees by themselves are not enough because systematically monitoring the fair
Scroogle
Mathematician Grigory Perelman just doesn't give a damn about public recognition. In the internet era you can make your research know by everybody without the need to be published. This story is more about politics but also how you can find more ways to make public your work:
0 828fa_fact2
http://www.newyorker.com/fact/content/articles/06