Ask Slashdot: How Could You Statistically Identify The Best Sci-Fi Books?

jimharris writes: Over at SF Signal I wrote a piece "How Well-Read Are You in Science Fiction?" There are three databases that collect lists of popular science fiction books that try to statistically identify the best books of the genre, [offering] combined list that shows which books were cited the most. They use different sets of best-of lists, but their results are often similar. The final lists are, Classics of Science Fiction, Worlds Without End Top Listed, and Premiosylista Comparativas: Comparativas: Ciencia ficcion (Spain).
Interestingly, each list has a different book in its #1 position (though both "Dune" and "Frankenstein" make the top four on at least two of the three lists). But is this really a good methodology for determining the classic canon? What would be the best way to statistically identify the greatest sci-fi books? (And have you read any good science fiction novels lately?)

Comments

define your terms first

[Anonymous Coward]

You need to define best, greatest, and classic before you can go further in your quest.

Re:define your terms first

[Anonymous Coward]

And not in the least decide what the definition of the term science fiction is (it's not as easy as it sounds).

You can't

[110010001000]

Art/literature/music is subjective. You can't rank them, except personally. Next question.

Re:You can't

[sunderland56]

Exactly. If you want to know how well statistics fares at this, turn on any top-40 radio station.

Re:You can't

[mwvdlee]

Bullshit.

You can most certainly rank literature by the number of pages, art by the size of the canvas or weight of the sculpture and music by the duration.
Whether these rankings are useful is debatable, but they can definitely be ranked.

Re:You can't

[jellomizer]

Even with the greats, there is a lot of subjective views on it. Sometimes such flicks are popular due to lowbrow reasons. Frankenstein popularity while may be a classic story, its success carries on from the movie adaptation. Also some genius Science Fiction has passed away threw the years, because modern sensibilities had made such stories obsolete. While other unpopular stories of the time, my get renewed attention as such stories may be ahead of its time.

There are a lot of factors that can become a classic. If it were so mechanical that a computer could figure it out, you would think writers would figure out the formula and use it in their stories, if they did, then the stories they produce would be too formulaic and relegated as cheap knockoff.

Re:You can't

[Sique]

Frankenstein was famous for 100 years without any movie adaption. I would say, it's exactly the other way around. It was adapted early and often for movies because it was famous already. And it is definitely quoted everywhere. Frankenstein's monster is an iconic figure by itself, and Frankenstein's laboratory is how the workspace of a crazy genius is portrayed since then.

On the other hand, many of those lists are very U.S. centric. If it wasn't for Jules Verne, you would think that there wasn't any science fiction outside the anglophonic world, and if you substract H.G.Wells, science fiction seems to be an U.S. only phenomenon. For me, any science fiction list has at least to include Stanislaw Lem, but as far as I know, most of his works are not translated into English yet, and if, then sometimes they are translated from other languages than his native Polish (Solaris for instance was first translated into English from the French edition). And does anyone has ever read Italo Calvino's Cosmicomics or Franz Fühmann's Sajäns Fiktschen?

Most of the U.S. science fiction I've read so far I would put into the 'meh' category. It's mostly wellknown plotlines, just with lasers and star ships. What I am missing in most U.S. science fiction are plots that are genuinely based on some scientific ideas and concepts and won't work without them. But the only plot vehicle that seems to be regularly used is Time Travel, and then it's mainly a deus ex machina, some additional ingredient to get the story to the desired end. But what about Italo Calvino's short story where QfwfQ (his eternal main character) tells how it was before the Big Bang, how it was completely irrelevant that the woman most of the guys felt in love with went in bed with her boyfriend, as before the Big Bang everyone was in the same geometric point and thus in the same bed anyway?

Google it.

[bennebw]

Google's Page Rank algorithm probably has as good a shot as anything else.

Re: Looking backwards, not forward...

[johnsmithperson123]

Uh oh.... Attack of the Puppies.

Re: Works knowledge of which is required for geek

[hackwrench]

Your comment makes no sense. I started writing this post without visiting the link but stopped myself and your comment still was nonsense, like a bunch of English words randomly strung together.

Re:Maybe something like a points race?

[tepples]

You appear to suggest something similar to a Borda count to turn a ranking into a range vote. Doesn't a Condorcet count produce a less gameable result than a Borda count?

Re: Works knowledge of which is required for geek

[tepples]

I'll rephrase with less embedding.

Alice makes an inside joke alluding to a particular work of fiction. The joke goes over Bob's head because Bob has never read it. Alice notices this and tells Bob to turn in his geek card. With which works should one be familiar in order to get jokes that geeks are expected to get?

Re:Not Dune

[__aaclcg7560]

Dune has faded badly.

What do expect from a novel written over 50 years ago? Go buy yourself a new paperback from Amazon. I usually replace my copy every ten years or so.

How many have been read by today's readers?

[mykepredko]

Looking at the "Classics of Science Fiction", I have to question how many of these books have been actually read.

Going through the first 100 list, I have read and attempted to read 90 of them (which is why I would put the challenge on the Subject line).

"Frankenstein", while undoubtedly having a huge impact on modern society is basically unreadable by modern readers - the copy I have has a forward by Stephen King saying it took him four attempts over decades to get through the book. The same comments for "The Lost World", "Gulliver's Travels" and "Brave New World".

I have never seen a copy of "R.U.R." (either as the original play, translated into English, or as a novelization), my copy of "Cities in Flight" is from 1966 and there are a number of books that are listed pre-1950 that I have heard about but never seen copies of. Now, as a counter-point, I just looked for many books I haven't seen on Amazon.com and did find them (the two cited in this paragraph are easily found and ordered) but I haven't seen them in new & used book stores even though I have looked for them.

So, to get this list, how many were actually read and how many were checked off because the reviewer/person being surveyed has actually read them? So, going back to the original question how can you statistically identify the best science fiction books if not all of the "classics" haven been read?

Re:How many have been read by today's readers?

[Ecuador]

"Frankenstein", while undoubtedly having a huge impact on modern society is basically unreadable by modern readers - the copy I have has a forward by Stephen King saying it took him four attempts over decades to get through the book.

That's strange, my wife recently finished Frankenstein and she told me the exact opposite, while urging me to read it. While she is a humanities PhD she actually puts down books that many people would say are "unreadable" like Ulysses (the James Joyce one), or Paradise Lost and actually English is not even her native language, so it is not like she reads unusually hard/unreadable etc books. Anyway, she told me it was an amazing book, that the narration method is so modern you'd think it was written recently. Out of curiosity I checked the good reads page and scrolled down to the first negative review and the guy actually says he hasn't hated a book that much since he read Kafka's "Metamorphosis"...
Perhaps Stephen King is allergic to reading successful works of the genre? :)

Re:How many have been read by today's readers?

[werepants]

"Frankenstein", while undoubtedly having a huge impact on modern society is basically unreadable by modern readers

What? Seriously? Frankenstein, along with many similarly classic works like 20,000 Leagues Under the Sea, War of the Worlds, are often placed in the children's section of book stores and libraries. I know that I read many classics growing up for just that reason - I saw some cool covers with aliens and monsters and submarines and got my start in sci-fi that way. If I was able to read that stuff as a kid, it's frankly absurd to call it "unreadable" for current adults.

Re:Why Limit This Contrived Gimmick to Just SF?

[__aaclcg7560]

I used to find new stuff through the Amazon recommendation list all the time until about ten years ago. Something changed in the algorithm. Now it pushes more of the same for everything. That might be the fate for an algorithm that picks the best of anything, drowning in a white noise as the picks for the best becomes worse and worse.

Re:Why Limit This Contrived Gimmick to Just SF?

[mwvdlee]

The problem is that you can't really state your preference.
If they offered a choice to say "thanks for the recommendation, but I won't be buying this. Ever", the recommendations could be improved.
On a site like Goodreads, you can state which books you like, and it uses that information to recommend others; worked quite well for me in the past.

Re:Looking backwards, not forward...

[Z80a]

The thing is much more complex than just "men and women" or even "person of ethnicity X or Y", at a point that if you try to boil down everything that might make a group better than others by such labels, you would take years, get insane and still get two or three people crumbling your system apart like a castle of cards near some pesky hyperactive kids.

So, the best you can do is just judge people by their individual merits instead of worrying about group X or Y.

Re:Why Limit This Contrived Gimmick to Just SF?

[smooth wombat]

Unfortunately, the local bookstores have gone out of business in my area. I can't spend an afternoon browsing the shelves to find something new.

Are you sure? Neil Gaiman just celebrated Independent Bookstore Day and we know he's an authority on such things.

Re:Looking backwards, not forward...

[__aaclcg7560]

So, the best you can do is just judge people by their individual merits instead of worrying about group X or Y.

Unfortunately, people will look at a book and decide not to read it because a women wrote it. As several commentators on Slashdot has already mentioned: "the best science fiction is written by men." In fact, some women writers wrote under a pen name because of this obvious bias.

http://io9.gizmodo.com/5077952/women-who-pretended-to-be-men-to-publish-scifi-books

Measure the subjective responses

[Bruce66423]

If a lot of people are wowed by product A and bored silly by product B, it is irrational to argue that the two can't be ranked as to which is the one most worth investing time in to read, given that we have a finite amount of time to spend doing so. Therefore to get people to vote for their 'favourite' seems a rational way forward, despite its subjective foundation.

Otherwise I know a wall with some paint drying that you can watch this evening...

Re:Measure the subjective responses

[Dutch Gun]

I think the trick in any sort of subjective ranking is to find a person or group of persons that tends you match your own personal tastes. Subjective ranking certainly isn't worthless when I recommend a book, movie, or TV show to my parents. That's because I can more or less accurately judge how well they'll like something since I know their tastes.

I think this is where most algorithmic approaches tend to fail (like on Amazon), at least from what I've been able to see. I think they tend to find general correlation - that is, "those who like book x would also like book y", but I think a much more effective approach would instead be to search for other customers who's general ranking patterns tend to match your own most closely, and build a personalized recommendation group from which to mine predictive data. In this way, the predictions would be more or less tailored for each individual customers based on similar likes and dislikes, rather than being based on general popularity trends.

For better accuracy, the algorithm should pay even more attention to statistical outliers. For instance, I generally love space-opera-y science fiction (Honorverse, Lost Fleet), as well as "harder" works, like The Martian. But I didn't care for the Heinlein I've read, so would wish to avoid more of him, and more important, more books in that general style. So, a clever algorithm would notice that trend of mine and find sci-fi fans who *also* didn't care for Heinlein, and give their recommendations a slightly higher bias based on that data point.

Generally speaking, any sort of non-personalized "ranking" is going to simply be a popularity contest within the target audience you select. There's nothing wrong with that, so long as people understand that's all it can ever really be. Which, of course, they won't.

Re: Not Dune

[PopeRatzo]

I also assume you are a progressive fag, so dune isn't hip enough for you anyhow.

Not enough space vaginas kicking ass and killing all the mens.

Clearly, you never read the Dune saga.

What Would the Movies Do?

[BarryHaworth]

In answering this question for books it might be instructive to look at what happens in another artistic field, that of the movies. Although there are some major differences (Movies cost a lot more to make and therefore there aren't so many made each year for a start) the comparison might shed a little light.

With rating movies statistically there are a number of methods:

- Box office takings, such as Box Office Mojo

- DVD and Video sales

- Movie audience figures (when broadcast on television or similar)

- Industry awards, such as the Academy Awards or the Baftas

- Ratings from critics, such as Rotten Tomatoes or Metacritic - Ratings from general users, such as IMDB

- and finally, "Best of" listings voted on by critics or interest groups.

I include the last not because it is really a very good statistical comparison as compared to any of the other methods, but because it is the only one analogous to the sorts of lists being considered in the Worlds Without End rankings.

To get a good statistical ranking for books or movies we need to get a comprehensive set of data that covers all (or most) of the entries, and which applies the same rankings to each. None of the rankings for Movies which I have listed really does that, but some do better than others in some ways at least. For example, ticket and unit sales cover all movies, though they have the problem that the number of people going to movies, and the price they pay per ticket, have increased over time so that the ranking metric isn't the same for all movies. It also has the disadvantage that ticket sales are not necessarily related to how good a movie is. Industry awards can probably be assumed to cover all movies released in a given year and therefore cover the whole population, but have the problem that the award givers may not cover all entries equally, and may be subject to bias. Critical judgement, whether from professional critics or members of the public, also have the problem of coverage - I personally cannot expect to be able to see every movie made, and the ones I do see will be affected by by things like advertising budgets which are not necessarily related to how good the movie actually is.

With books we do have some similar data sets. Figures for number of books printed, or sales on the likes of Amazon can be compiled, though these have the same problem of not being related to quality. I don't know of any compilation sites for professional book critics (anybody?), but there are sites such as Goodreads where members of the public can give their subjective rankings. Industry awards also exist, such as the Hugo or Nebula awards, but these have the disadvantage of being subject to politics (*cough* Puppies vs SJW anyone?). Finally, there are "Best of" lists, such as the ones cited by Worlds Without End.

Books have a problem compared to movies in that far more books get published than movies get made. While a good critic can expect to see all the movies that come out in a year (at least all those released theatrically), reading every book that is published is impossible. This eats into the quality of critical rankings out there, or even into Industry awards. Any "Best of the Year" list can't really hope to be definitive, because a book - especially a ground breaking, iconoclastic new classic - will take time to find a wide audience and be widely recognised.

For my money, I think the likes of Goodreads are probably the best bet as an objective, comprehensive and timely statistical source for