Slashdot Mirror

olsmeister writes "Several weeks after having the PlayStation Network hacked, and apologizing to users for the breach, Sony is offering $1 million in identity theft protection for users who sign up before June 18th. The protection is being offered through Debix and is called AllClear ID Plus. This appears to be some kind of custom plan especially for Sony, as their normal offerings are called AllClear ID Free and AllClear ID Pro."

An anonymous reader writes "Yesterday, in a letter to Congress, Sony detailed the steps they were taking to resolve the issues that have been plaguing them since the PlayStation Network and SOE online components were hacked, claiming to have found evidence linking the crime to Anonymous. Now, Anonymous has responded."

angry tapir writes "Oracle has subpoenaed the Apache Software Foundation in connection with its ongoing intellectual property suit against Google. Oracle filed suit against Google in August, alleging that its Android mobile operating system infringes on seven of Oracle's Java patents. Google has denied any wrongdoing. The subpoena, which was received by ASF on Monday, seeks 'the production of documents related to the use of Apache Harmony code in the Android software platform, and the unsuccessful attempt by Apache to secure an acceptable license to the Java SE Technology Compatibility Kit.'"

An anonymous reader writes "Remember when we learned that Facebook had resumed talks with Skype? Well, it turns out that Facebook is considering buying Skype outright. 'Skype is reportedly talking to Facebook about some sort of deal. Facebook CEO Mark Zuckerberg has been involved in internal discussions about buying Skype, while Facebook also reached out to the Luxembourg-based company about forming a joint venture.'"

MassDosage writes "Apache JMeter written by Emily H. Halili is very much an introductory guide to using Apache's open source JMeter testing tool. Unfortunately a book that should have been good fodder to whet the appetites of testers is spoiled by shoddy editing, poor writing and very little content that isn't already included in JMeter's own user manual." Read below for the rest of MassDosage's review. Apache JMeter author Emily H.Halili pages 129 publisher Packt Publishing rating 3/10 reviewer Mass Dosage ISBN 978-1-847192-95-0 summary A practical beginner's guide to automated testing and performance measurement for your websites. I am one of those (fairly) rare software developers with a genuine interest in testing and first tried using JMeter many years ago after its initial release as a tool primarily focused on testing the Apache Tomcat web/application server. It was a bit rough around the edges and poorly documented in those days so I was looking forward to a book that would re-familiarize me with JMeter in its current form. This book clearly targets itself at the testing community and doesn't require any programming (or even much testing) knowledge.

Apache JMeter starts off with a not particularly convincing overview of why one would want to automate testing and goes off on a tangent trying to prove the monetary savings this leads too. Any testing will depend very much on the type of software being tested, the skills of the tester, technology stack in use etc. so trying to come up with a general formula proving that testing will save money is probably impossible and pointless. It would have been far more useful to include some practical examples where testing saved time, effort, caught bugs early etc. A very brief history of JMeter is provided as well as an overview of its GUI and the the various elements available. For those who haven't used it before — JMeter acts as a client which sends requests to the application being tested and can then act on the responses returned. Overall this isn't the smoothest of introductions with lots of grammatical errors, fragmented text and repeated concepts, which doesn't bode well for the rest of the book.

Installing and setting up JMeter is covered next and this is straightforward and simple. A good overview of what a JMeter test plan consists of is provided and this covers controlling how many simultaneous users a test will use, what logic and timing will be used to issue requests, the type of requests (e.g. HTTP for web sites, FTP for file retrieval etc.), various configuration options, how to assert the contents of responses and so on. The book is a bit dated and the version of JMeter I was using didn't always match up with the text and diagrams but most of the differences were self explanatory so this wasn't really an issue.

The chapter on load and performance testing is the best on offer and provides sensible guidelines on this type of testing. These include tips such as ensuring network bandwidth between the machines under test, running the tests on physically separate machines from the software being tested, running tests for long periods of time and what to monitor. This is mainly common sense but it's good to see them summarized and the book would have benefited from more sections that appear to be informed by the author's practical experiences as a tester. The example given in the book is the load testing of a web site over HTTP and it's left up to the reader to translate the various types of request being made to an example website of their own. The screen shots of setting the various GUI values are clear and useful. Remote testing (i.e. not using a single JMeter GUI but multiple command-line instances of JMeter) is mentioned but sadly not covered which is a shame as any serious load or stress testing usually requires this. Using JMeter as a proxy to quickly generate a test plan by recording HTTP requests from a browser is both neat and useful and explained well.

JMeter is not just limited to load testing but can also be used to do simple functional tests as it can inspect the values returned by requests and perform assertions on these responses (e.g. checking that the returned HTML contains some text). Unfortunately functional testing is covered very poorly with far too little detail and no explanation of how to run the examples yourself. The author should have stuck to one simple use case (ideally that the reader could follow along with) which would show how to add one's own custom values to requests and to assert the results returned. In the end I could figure it out for myself, but I'd expect a book to guide me through this.

Advanced features are up next and the reader is pointed towards the sample code included with the book (a .rar file inside a .zip file, why?) but absolutely no explanation is given as to what one should actually do with the bunch of PHP and SQL files included in it. Using regular expressions and configuring JMeter to run tests using loops is covered but this doesn't make much sense as there is no context for the example used in setting this all up. Fortunately the sections on using JMeter to test a database and an FTP server are introduced with clear setup steps that are easy to follow. I wouldn't classify most of this material as advanced, it just gives the reader the knowledge that JMeter can be used to test more than just HTTP sites and provides a few very simple examples which they can build on.

Finally the book wraps up by stressing once again that it's just an introduction to JMeter and a concluding table clearly showing that more is not covered than actually covered. The appendices consist of definitions of nearly every JMeter element that can be accessed in the GUI (a good quick reference), some pointers to material for readers interested in learning more, and a glossary of testing terms, the vast majority of which are never used in the book itself. Unfortunately the latter is yet more evidence of the biggest weakness of this book — the appalling editing that doesn't even remove sentences which are completely incorrect. 'You can too contribute.' (the closing sentence of Chapter 1) is just one of many examples of poor writing that somehow survived into print.

On the whole the book does give one an overview of how to get started with JMeter and the section on load testing a website is a useful introduction to various JMeter concepts. This is the only printed book on the topic out there so if you learn best reading from print then you don't have many options. This is a shame as a niche topic like this could do with an interesting, well-informed and authoritative guide. For everyone else I would suggest reading the User Manual on the JMeter website as it covers all the material here (and much more) and is far more readable.

You can purchase Apache JMeter from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

MassDosage writes "Apache JMeter written by Emily H. Halili is very much an introductory guide to using Apache's open source JMeter testing tool. Unfortunately a book that should have been good fodder to whet the appetites of testers is spoiled by shoddy editing, poor writing and very little content that isn't already included in JMeter's own user manual." Read below for the rest of MassDosage's review. Apache JMeter author Emily H.Halili pages 129 publisher Packt Publishing rating 3/10 reviewer Mass Dosage ISBN 978-1-847192-95-0 summary A practical beginner's guide to automated testing and performance measurement for your websites. I am one of those (fairly) rare software developers with a genuine interest in testing and first tried using JMeter many years ago after its initial release as a tool primarily focused on testing the Apache Tomcat web/application server. It was a bit rough around the edges and poorly documented in those days so I was looking forward to a book that would re-familiarize me with JMeter in its current form. This book clearly targets itself at the testing community and doesn't require any programming (or even much testing) knowledge.

Apache JMeter starts off with a not particularly convincing overview of why one would want to automate testing and goes off on a tangent trying to prove the monetary savings this leads too. Any testing will depend very much on the type of software being tested, the skills of the tester, technology stack in use etc. so trying to come up with a general formula proving that testing will save money is probably impossible and pointless. It would have been far more useful to include some practical examples where testing saved time, effort, caught bugs early etc. A very brief history of JMeter is provided as well as an overview of its GUI and the the various elements available. For those who haven't used it before — JMeter acts as a client which sends requests to the application being tested and can then act on the responses returned. Overall this isn't the smoothest of introductions with lots of grammatical errors, fragmented text and repeated concepts, which doesn't bode well for the rest of the book.

Installing and setting up JMeter is covered next and this is straightforward and simple. A good overview of what a JMeter test plan consists of is provided and this covers controlling how many simultaneous users a test will use, what logic and timing will be used to issue requests, the type of requests (e.g. HTTP for web sites, FTP for file retrieval etc.), various configuration options, how to assert the contents of responses and so on. The book is a bit dated and the version of JMeter I was using didn't always match up with the text and diagrams but most of the differences were self explanatory so this wasn't really an issue.

The chapter on load and performance testing is the best on offer and provides sensible guidelines on this type of testing. These include tips such as ensuring network bandwidth between the machines under test, running the tests on physically separate machines from the software being tested, running tests for long periods of time and what to monitor. This is mainly common sense but it's good to see them summarized and the book would have benefited from more sections that appear to be informed by the author's practical experiences as a tester. The example given in the book is the load testing of a web site over HTTP and it's left up to the reader to translate the various types of request being made to an example website of their own. The screen shots of setting the various GUI values are clear and useful. Remote testing (i.e. not using a single JMeter GUI but multiple command-line instances of JMeter) is mentioned but sadly not covered which is a shame as any serious load or stress testing usually requires this. Using JMeter as a proxy to quickly generate a test plan by recording HTTP requests from a browser is both neat and useful and explained well.

JMeter is not just limited to load testing but can also be used to do simple functional tests as it can inspect the values returned by requests and perform assertions on these responses (e.g. checking that the returned HTML contains some text). Unfortunately functional testing is covered very poorly with far too little detail and no explanation of how to run the examples yourself. The author should have stuck to one simple use case (ideally that the reader could follow along with) which would show how to add one's own custom values to requests and to assert the results returned. In the end I could figure it out for myself, but I'd expect a book to guide me through this.

Advanced features are up next and the reader is pointed towards the sample code included with the book (a .rar file inside a .zip file, why?) but absolutely no explanation is given as to what one should actually do with the bunch of PHP and SQL files included in it. Using regular expressions and configuring JMeter to run tests using loops is covered but this doesn't make much sense as there is no context for the example used in setting this all up. Fortunately the sections on using JMeter to test a database and an FTP server are introduced with clear setup steps that are easy to follow. I wouldn't classify most of this material as advanced, it just gives the reader the knowledge that JMeter can be used to test more than just HTTP sites and provides a few very simple examples which they can build on.

Finally the book wraps up by stressing once again that it's just an introduction to JMeter and a concluding table clearly showing that more is not covered than actually covered. The appendices consist of definitions of nearly every JMeter element that can be accessed in the GUI (a good quick reference), some pointers to material for readers interested in learning more, and a glossary of testing terms, the vast majority of which are never used in the book itself. Unfortunately the latter is yet more evidence of the biggest weakness of this book — the appalling editing that doesn't even remove sentences which are completely incorrect. 'You can too contribute.' (the closing sentence of Chapter 1) is just one of many examples of poor writing that somehow survived into print.

On the whole the book does give one an overview of how to get started with JMeter and the section on load testing a website is a useful introduction to various JMeter concepts. This is the only printed book on the topic out there so if you learn best reading from print then you don't have many options. This is a shame as a niche topic like this could do with an interesting, well-informed and authoritative guide. For everyone else I would suggest reading the User Manual on the JMeter website as it covers all the material here (and much more) and is far more readable.

You can purchase Apache JMeter from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

An anonymous reader writes "With the release of the dual-GPU AMD Radeon HD 6990, closely followed by the competing Nvidia GeForce GTX 590, we saw graphics card performance reach new heights. With bandwidth throughput in excess of 300GB/s, these cards can consume more power than entire computer systems. By utilizing three monitors, games can become roughly 3x more demanding, as the graphics card is required to render an overwhelmingly higher number of pixels. Whereas graphics card reviews and benchmarks usually test GPUs at single monitor resolutions, TechSpot has added two more LCD monitors and tested eleven games running at resolutions of 5040x1050, 5760x1200 and 7680x1600."

mask.of.sanity writes "The first dynamic Android firewall, dubbed WhisperMonitor, has been released by respected security researcher Moxie Marlinspike. The firewall will allow users to stop location-tracking apps and restrict connection attempts by applications. Marlinspike, whose company created the application, designed WhisperMonitor in response to the incidence of location tracking and malware on Android platforms. It monitors all outbound connection attempts by applications and the operating system, and asks users to permit or block any URLs and port numbers that are accessed."

iluvcapra writes "Google, in its continuing struggle to provide phone carriers (if not its end users) with an open platform, is now banning tethering apps from the Android market. These apps haven't disappeared and can still be sideloaded, insofar as your carrier doesn't lock this functionality or snoop on your packets."

cf18 writes "Wikileaks released a set of leaked Guantanamo prisoner files to the public last week. Among them is a document dated from 2008, which mentioned both Osama's trusted courier's name and Abbottabad, the city in which Osama had been hiding. There are speculations that, fearing al-Qaida realized their courier may have been tracked and move Osama, the US administration accelerated their plan and attacked the target site over the weekend. This link highlights the relevant section of the document."

Twinbee writes "Gamers often find 'input lag' annoying, but over the years, delay has crept into many other gadgets with equally painful results. Something as simple as mobile communication or changing TV channels can suffer. Software too is far from innocent (Java or Visual Studio 2010 anyone?), and even the desktop itself is riddled with 'invisible' latencies which can frustrate users (take the new Launcher bar in Ubuntu 11 for example). More worryingly, Bufferbloat is a problem that plagues the internet, but has only recently hit the news. Half of the problem is that it's often difficult to pin down unless you look out for it. As Mick West pointed out: 'Players, and sometimes even designers, cannot always put into words what they feel is wrong with a particular game's controls ... Or they might not be able to tell you anything, and simply say the game sucked, without really understanding why it sucked.'"

AffidavitDonda writes with this excerpt from Torrentfreak: "A possible landmark ruling in one of the mass-BitTorrent lawsuits in the US may spell the end of the 'pay-up-or-else-schemes' that have targeted over 100,000 Internet users in the last year. District Court Judge Harold Baker has denied a copyright holder the right to subpoena the ISPs of alleged copyright infringers, because an IP-address does not equal a person. Among other things, Judge Baker cited a recent child porn case where the US authorities raided the wrong people, because the real offenders were piggybacking on their Wi-Fi connections. Using this example, the judge claims that several of the defendants in VPR's case may have nothing to do with the alleged offense either. ... Baker concludes by saying that his Court is not supporting a 'fishing expedition' for subscribers' details if there is no evidence that it has jurisdiction over the defendants."

An anonymous reader writes with an update to yesterday morning's news that Sony Online Entertainment's game service was taken offline to investigate a potential data breach related to the PSN intrusion. SOE has now said that they too suffered a major theft of user data. "... personal information from approximately 24.6 million SOE accounts may have been stolen, as well as certain information from an outdated database from 2007. The information from the outdated database that may have been stolen includes approximately 12,700 non-US credit or debit card numbers and expiration dates (but not credit card security codes), and about 10,700 direct debit records of certain customers in Austria, Germany, Netherlands and Spain."

decora writes "Last year ProPublica won the first Pulitzer for an online news site. This year, they have been awarded the first Pulitzer for a series that did not appear in print. The series was Eisinger and Bernstein's 'The Wall Street Money Machine,' which described how hedge funds and financiers profited from the collapse of the economy. ProPublica publishes under a Creative Commons license and hosts a Nerd Blog where they write about journalism-related hacking and publish open source tools they have developed."

RedEaredSlider writes "Two Android phone users are suing Google for $50 million in the wake of revelations that their phones might be tracking their locations. The lawsuit, filed in the US District Court for the Eastern District of Michigan on April 27, is seeking class-action status. The plaintiffs, Julie Brown and Kayla Molaski, are residents of Oakland County. The two say in the suit that Google's privacy policy did not say that the phones broadcast their location information. Further, they say Google knew that most users would not understand that the privacy policy would allow for Google to track users' locations." Apple was sued for their location tracking last week. According to Boy Genius Report, iOS tracking will be addressed in version 4.3.3, which is due out within a couple weeks.

wiredmikey writes "Following the successful operation by US forces to kill Osama bin Laden, Internet users are searching in the masses for any details about the incident they can find. Cyber-criminals know this and have already been at work to 'poison' common search results hoping to gain access to people's computers and infect them with malware."

iSimon19 writes with word that after last week's unscheduled service disruption and security breach, "On their blog last night, Playstation representatives announced they were restoring services throughout the week. This also included giving all users a month of Playstation Plus free, as well as select downloads for free with their 'Complimentary Offering and "Welcome Back" Appreciation Program.'"

iSimon19 writes with word that after last week's unscheduled service disruption and security breach, "On their blog last night, Playstation representatives announced they were restoring services throughout the week. This also included giving all users a month of Playstation Plus free, as well as select downloads for free with their 'Complimentary Offering and "Welcome Back" Appreciation Program.'"

An anonymous reader writes "The 32-bit OpenRISC CPU has been available for many FPGAs and was turned into a commercial ASIC in 2003. Now, the OpenCores community is asking for donations to create a new ASIC with the OpenRISC CPU, ethernet, PCI, UART, USB and other peripherals. The goal is to be able to sell these ASICs at a low price to anyone who wants to build a cheap embedded system built completely on open source. The OpenRISC currently runs on Linux 2.6.37 and has ports of gcc 4.5.1 among other things."

itwbennett writes "Google and Oracle each submitted proposals on Friday to reduce the number of claims in their Java patent infringement lawsuit, which could help bring the case to a speedier conclusion. Earlier this month, lawyers for the two companies gave Judge William Alsup of the US District Court in San Francisco a crash course in Java to prepare him for a claim construction conference."

eldavojohn writes "It's known that home chemistry sets are in danger of going extinct, which has spurred set makers to add the label 'Chemical Free!' on modern chemistry sets (NSFW warning — JAYFK stands for Journal of Are You *expletive* Kidding). The kit for ages 10+ provides 60 chemistry activities that are mind-bogglingly chemical free. The pedantic blog entry points out the many questions that arise when the set promises 'fun activities' like growing plants and crystals — sans chemicals! That would be quite the feat to accomplish without the evilest of chemicals: dihydrogen monoxide. While this rebuttal is done in jest, this set's intentions do highlight the chilling growth of a new mentality: Chemicals are bad. Despite their omnipresence from the beginning of time, they are no longer safe. Even real researchers are starting to notice the possible voluntary stunting of science education that is occurring in the name of overreaching safety."

rudy_wayne writes "News Corp. is reportedly trying to sell MySpace for $100 million, a fraction of the $580 million it originally paid for the social network in 2005. Parties interested in acquiring MySpace include private equity firm THL Partners, Redscout Ventures and Criterion Capital, owner of social network Bebo (the company AOL bought for $850 million and then sold for $10 million). Chinese Internet holding company Tencent is also reportedly interested, and so is MySpace co-founder Chris De Wolfe. What's not yet clear is what any of these companies plan to do with MySpace if a sale goes through." This follows news of massive layoffs and a rapidly shrinking userbase in recent months.

rudy_wayne writes "News Corp. is reportedly trying to sell MySpace for $100 million, a fraction of the $580 million it originally paid for the social network in 2005. Parties interested in acquiring MySpace include private equity firm THL Partners, Redscout Ventures and Criterion Capital, owner of social network Bebo (the company AOL bought for $850 million and then sold for $10 million). Chinese Internet holding company Tencent is also reportedly interested, and so is MySpace co-founder Chris De Wolfe. What's not yet clear is what any of these companies plan to do with MySpace if a sale goes through." This follows news of massive layoffs and a rapidly shrinking userbase in recent months.

angry tapir writes "Nokia will outsource its Symbian software activities to Accenture, transferring 3,000 employees to the company in the process, as it moves its focus to making phones running on Microsoft's Windows Phone operating system. The Finnish phone manufacturer will also close some of its research and development sites and eliminate a further 4,000 jobs by the end of next year. Last week Nokia announced the signing of a definitive agreement regarding their global mobile ecosystem partnership."

generalhavok writes "I read the story on Slashdot earlier about the EFF encouraging people to leave their WiFi open to share the internet. I would like to do this! I don't mind sharing my connection and letting my neighbors check their email or browse the web. However, when I used to leave it open, I quickly found my limited bandwidth dissappearing, as my neighbors started using it heavily by streaming videos, downloading large files, and torrenting. What is an easy way I can share my internet, while enforcing some limits so there is enough bandwidth left for me? What about separating the neighbors from my internal home network? Can this be done with consumer-grade routers? If the average consumer wants to share, what's the easiest and safest way to do it?"

1sockchuck writes "The US government has closed 39 data centers this year, and expects to shut down 98 more by the end of 2011, federal CIO Vivek Kundra said Wednesday. The 137 closures are a step towards the long-term goal of consolidating 800 of the government's 2,094 data centers by 2015. Government agencies have identified 100 email systems and 950,000 mailboxes to migrate to a cloud computing model as part of Kundra's 'Cloud First' initiative."

1sockchuck writes "The US government has closed 39 data centers this year, and expects to shut down 98 more by the end of 2011, federal CIO Vivek Kundra said Wednesday. The 137 closures are a step towards the long-term goal of consolidating 800 of the government's 2,094 data centers by 2015. Government agencies have identified 100 email systems and 950,000 mailboxes to migrate to a cloud computing model as part of Kundra's 'Cloud First' initiative."

Slashdot regular contributor Bennett Haselton writes "In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would make it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 for the best reader suggestions on how to improve the idea, or why they thought it wouldn't work. Read their suggestions and decide what value I got for my infotainment dollar."

In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would scale to a large number of users while also making it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 to readers sending in the best suggestions for improvements, or alternative algorithms, or fatal flaws in the whole idea that would require starting from scratch. As the suggestions were coming in, Facebook obligingly kept the issue in the news by removing a photo of two men kissing from a user's profile, sending a form letter to the user that they had violated Facebook's prohibition on "nudity, or any kind of graphic or sexually suggestive content". (It would be a cheap shot to say that a photo of a man and a woman kissing probably would not have been removed; in truth, probably just about anything will get removed from Facebook automatically if enough users file complaints against it, which is the problem for unpopular but legal content.)

How would these complaints have been handled under my proposed algorithm? The gist of my idea was that any users could sign up to be voluntary reviewers of "abuse complaints" filed against public content on Facebook. Once Facebook had built up a roster of tens of thousands of reviewers, new abuse complaints would be handled as follows. When a complaint (or some threshold of complaints) is filed against a piece of content, a random group of, say, 100 users could be selected from the entire population of eligible reviewers, and Facebook would send them a request to "vote" on whether that content violated the Terms of Service. If the number of "Yes" votes exceeded some threshold, the content would be removed (or at least, put in a high-priority queue for a Facebook employee to determine if the content really did warrant removal). The main benefit of this algorithm is that would be much harder for co-ordinated mobs to "game the system", because in order to swing the vote, they would have to comprise a significant fraction of the 100 randomly selected reviewers, and to achieve that, the mob members would have to comprise a significant fraction of the entire reviewer population. This would be prohibitively difficult if hundreds of thousands of users signed up as content reviewers.

All of the emails I received -- not just "almost" all of them, but really all of them -- contained some insightful suggestions worth mentioning, although there was some duplication between the ideas. If you didn't see the last article, you might consider it worth while to stop reading before proceeding further, and mull over the description of the algorithm above to see how you would improve it. Then read the suggestions that came in to see how well your ideas matched up with the submissions I received.

The upshot is that nobody found what I believed to be completely fatal flaws, although one reader brought something to my attention that might cause trouble for the algorithm after a few more years. Beyond that, reader suggestions could be divided essentially into two categories. The first category of suggestions related to ensuring that the basic premise would actually work -- that the votes cast by a random sample would be representative of general user opinion, and could not be gamed by a coordinated mob or a very resourceful cabal trying to game the system. The second category of suggestions started by assuming that the voting system would work, and suggested other features that could be added to the algorithm -- or, in one case, an entire alternative algorithm to replace it.

To begin with the attacks and counter-attacks against the basic voting algorithm. Walter Freeman and Haydn Huntley independently suggested monitoring for users who vote in a small minority in a significant portion of vote-offs, and reducing their influence in future votes (by either not inviting them to vote on future juries, or sending them the future invites but then ignoring their votes anyway). The assumption is that if a user is frequently among the 10% who vote "Yes [this is abuse]" when the other 90% of respondents are voting "No [this is not abuse]", or vice versa, then that user is voting randomly, or their point of view is so skewed that their votes could safely be ignored even if they are sincere. I like the idea of eliminating deadweight voters, but this might also incentivize voters to vote the way they think the crowd would vote, instead of voting their true opinions -- for example, if they were called to vote on an anti-Obama page that showed Obama wearing a Hitler mustache. Some people's knee-jerk reaction would be to call the page "racist" or "hate speech" or "a threat of violence", even though comparing Obama to Hitler is not, strictly speaking, any of those things. If I were voting my honest opinion, I would count that page as "not abuse". But if I knew that I were voting along with dozens of other people, and my future voting rights might be revoked if I didn't vote with the majority, I might be tempted to vote "abuse".

Similarly, Walter Freeman and reader "mjrosenbaum" both suggested setting deliberate traps for deadweight users, by creating artificial cases where the answer was pre-determined to be obviously yes or obviously no, calling for votes, and revoking privileges for users who gave the wrong answer. This would eliminate the problem of borderline cases like the one above, where smart users think, "I suspect the majority will give the wrong answer, so I'm just going to go with the crowd, to keep my voting rights." On the other hand, it's more labor for Facebook to create the cases, and any public content authored by them -- especially content that is deliberately crafted to be "questionable" -- would probably have to run a gauntlet of being reviewed by lawyers and PR mavens before being released. My suggestion would be to use these artificial scenarios periodically to make sure that the system is working (i.e. that juries are giving the right answers), but it would be too inefficient to use it to try and weed out problem voters.

In fact, these and several other suggestions fell into a category of ideas that could possibly improve the efficiency of the algorithm by reducing voter shenanigans (where "efficient" means that fewer users have to be invited to each vote-off in order to get statistically valid results), but might not be worth the effort. As long as most of the votes cast by users are sane and sincere, all you have to do is invite enough voters to a vote-off, and the majority will still get the correct answer most of the time, even if you have problem voters in the system. That's the simplest possible algorithm. The more complicated an algorithm you come up with, the more likely that Facebook (or any other site you recommended this to) would just throw up their hands and say, "Sounds too hard", and leave the idea dead in the water. That's why I like the algorithm as lean and tight as possible.

So it's not quite like designing an algorithm for your own use, where you could feel free to introduce more complications as long as you're responsible for keeping track of them. In recommending an algorithm for widespread adoption, the basic form of the algorithm should be as simple as possible. In the case of the voting algorithm some interesting wrinkles may come up if you don't eliminate problem voters, but this is not fatal to the idea as long as it's still true that, given a large enough random sample of voters, the majority will tend to vote the correct answer.

For example, James Renken pointed out that as voters dropped out due to boredom, the remaining users casting votes would tend to be either (1) weirdos who just wanted to view questionable material; and (2) prudes bent on removing as much material from Facebook as possible. But that's OK, as long as those two groups vote sanely enough (or as long as there are enough sane users outside those two groups) that material which does violate the TOS, tends to get more "Yes [this is abuse]" votes than material that doesn't. Then all you have to do is make the jury size large enough to make a statistically significant distinction between those two cases.

Similarly, Joshua Megerman suggested surveying users for their religious, political, and other beliefs when they sign up as volunteer reviewers (they could of course decline the survey). This makes it possible, insofar as people answer truthfully, to make sure that a jury is composed of a group with diverse belief sets. (On the other hand, users could game the system by reporting beliefs that are the opposite of what they truly feel. For example, if you're a leftist, register as a right-winger. Then when an abuse case comes before you, if it's a piece of content more offensive to leftists, then the real leftists on the jury will tend to vote against it -- but as a registered right-winger, you'll be able to cast a vote against it as well, and you'll be displacing a real right-wing voter who probably wouldn't have voted that way, so your vote will be worth more!) Again, it's fine if Facebook wants to do this, but even without collecting this data and simply selecting jurors at random, it should still be true that genuinely abusive pages get more "Yes" votes in a jury vote, than non-abusive pages.

Lastly in the "keep the jurors honest" category, Paul Ellsworth suggested allowing jurors to anonymously review each other -- when a given juror is chosen for the "hot seat" (perhaps randomly, perhaps as a result of a history of skewed voting), other jurors are randomly selected from the voting pool, to review that juror's voting record and decide whether that juror has been voting honestly and judiciously, or not. When I first read this idea, I instinctively thought that because a contaminated jury pool would be reviewing itself, it would not be able to reduce the percentage of problem voters, but a little more thought revealed that this isn't true. Suppose initially your jury pool consists of 80% "honest voters" and 20% "dishonest voters", that honest voters who review the voting record of another voter will always vote correctly whether that person is "honest" or "dishonest", and that dishonest voters will always vote incorrectly. It's still the case that when a voter's record is reviewed by a panel of, say, 20 other voters, virtually 100% of the time the majority will get the right answer. If you strip voting rights from a voter whenever a jury of other voters determines them to be a "dishonest voter", then over time, the percentage of honest voters in the system will creep from 80% to 100%. So again, this might work, and again, it might just be adding unnecessary complexity if the basic algorithm could work without it.

Note that none of these precautions would address the case of a "sleeper" voter -- a voter who joins the system with the sole intention of voting incorrectly on particular types of cases (perhaps planning on voting "yes" to shut down pages made by a particular organization, or pages advocating a particular view on a single issue), while still planning to vote correctly on everything else. By voting honestly in all other cases, they prevent themselves from being flagged by the system for casting too many minority votes, or from being blacklisted by other jurors for having a questionable overall voting record. The only real way I can see to address this problem is to hope that such users are outnumbered by the honest users in the system, and that juries are large enough that the chances of "rogue voters" gaining a majority on any one jury are nearly zero.

Which brings us to the one potentially fatal weakness in the system that I'm aware of: reader George Lawton referred me to a program run by the U.S. government to create armies of fake accounts to infiltrate social media, named, apparently without irony, Earnest Voice:

The project aims to enable military personnel to control multiple 'sock puppets' located at a range of geographically diverse IP addresses, with the aim of spreading pro-US propaganda.

An entity with the resources of the U.S. military could potentially create enough remote-controlled voters to overwhelm the system. I'm not sure if there is a way to deal with a system if the majority of voters are compromised. Presumably by making all decisions appealable to a core group of trusted Facebook employees at the top (although this then creates a bottleneck and limits scalability, especially if filing an appeal is free and all the parties who lose abuse cases are constantly filing appeals to the next level up).

Now. On to the second category of suggestions: Assuming the majority of voters are honest, what other features would be desirable to build into the system?

Walter Freeman, on the subject of filing appeals, suggested putting appealed pages in a special queue where they could be publicly viewed and users could comment on the ongoing appeals process, in addition to reading arguments posted by either side; this also negates the censorship itself due to the to the Streisand effect. I agree, but it's not obvious why this is a desirable feature. This does create perverse incentives, since some users could get extra traffic for their content by creating a page that makes whatever argument you're trying to promote, spiking it with some TOS-violating content, waiting for the page to get shut down, appealing the decision, and enjoying all the extra Streisand attention that it gets while on public display during the "appeal".

Meanwhile, James Renken pointed out that the system would work best for content that was originally public anyway, like a controversial Facebook page or event. If someone filed a complaint regarding a private message that they received, and they wanted a "jury vote" about whether the content of the message constituted abuse, then either the sender or the recipient would have to waive their right to privacy regarding the message before it could be shared with jurors. If the message really was abusive, then in some cases the recipient might waive their privacy rights -- reasoning that they didn't mind sharing the nasty message that someone sent them, in order to get the sender's account penalized. The problem arises if the message also contains sensitive personal facts about the recipient, which they wouldn't want to share with anonymous jurors. The system could allow them to black out any personal information before submitting the message for review, but that creates a recursive problem of abuse within the abuse system -- how do you know that someone didn't alter the content (and thus the offensiveness) of the message through their selective blacking-out? So it's not obvious whether this idea could be applied to non-public content at all.

Reader George Lawton suggested allowing content reviewers to vote on the funniest or weirdest content they had to review, to be posted in a public "Hall of Infamy". I love the thought of this, but I think Facebook's lawyers would be uncomfortable glamorizing anything questionable even if it were ultimately voted to be non-abusive (and certainly if it was voted to be abusive). Besides, this also has the perverse-incentives problem -- tie your message to something that you know will not only get an abuse complaint, but will hopefully end up in the Hall of Weird. (Even without the abuse jury system, there are already plenty of incentives for people to make a political point and hope that it will go viral.)

David Piepgrass suggested that new content reviewers should be allowed to specify certain types of content that they don't want to be asked to review -- nudity, graphic violence, etc. This sounds like a good idea. He adds that users probably shouldn't be able to opt-in only to review certain categories of content (or jurors might sign up only to review nudity, and then who would be left to review the death threats?).

Finally, in the other corner: Jerome Shaver suggested bypassing the jury voting system altogether and working on a heuristic algorithm to determine when abuse reports were being submitted by organized mobs of users, based on the patterns shown by mutual friendships between the users filing the abuse reports. The difficulties in designing such an algorithm, are too complicated to summarize quickly, and could fill an entire separate article. (Convince yourself that it's not an easy problem to solve. You can't just ignore abuse complaints from clusters of users that have many mutual friendships, because it can happen that real tight-knit communities of users might file abuse complaints against a piece of content, where the complaints are actually genuine.) But again, there is the problem that if a proposed solution is too complicated or too nebulous, Facebook has the excuse that they are "weighing several options", that they're "already working on something similar internally", etc. The jury vote system has the advantage that it can be described in just a few sentences, and the general public always knows whether it has been implemented or not -- which means that as long as abuses of the complaint system continue, people can ask, "Why doesn't Facebook try this?"

You'll notice this is just a laundry list of the ideas I received, without any definitive conclusions about which ones are good or bad, but that's all I was going for. The original algorithm, I could argue with the force of mathematical proof that, under certain reasonable assumptions, it would work. There's no such proof or disproof for any of the suggested modifications, so I don't feel as strongly about any of them. But at the top of the article I suggested for readers to stop reading and see how many of these ideas they could come up with on their own. How did you do?

The final honor roll of readers who were each the first, or only, person to submit an original idea: Walter Freeman (bonus points for getting in several good ones), James Renken, Joshua Megerman, Paul Ellsworth, George Lawton, Jerome Shaver, and David Piepgrass. Most of them volunteered to donate their winnings to charity, and agreed to let me donate their share to Vittana, which arranges microloans to college students in developing countries. One preferred a charity of their choosing, and only one actually kept the money. To be clear, for future contests, it's awesome if you want to donate the money to charity, but it's not dickish to keep it. That was the original deal after all.

So, all very clever and interesting suggestions, some of which might inspire readers to keep coming up with their own further variations. I said which ideas I probably would have incorporated and which ones I wouldn't, and I'm sure many of you would tell me that I'm wrong on some of those points. Although from here on out you're doing it for free.

Slashdot regular contributor Bennett Haselton writes "In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would make it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 for the best reader suggestions on how to improve the idea, or why they thought it wouldn't work. Read their suggestions and decide what value I got for my infotainment dollar."

In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would scale to a large number of users while also making it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 to readers sending in the best suggestions for improvements, or alternative algorithms, or fatal flaws in the whole idea that would require starting from scratch. As the suggestions were coming in, Facebook obligingly kept the issue in the news by removing a photo of two men kissing from a user's profile, sending a form letter to the user that they had violated Facebook's prohibition on "nudity, or any kind of graphic or sexually suggestive content". (It would be a cheap shot to say that a photo of a man and a woman kissing probably would not have been removed; in truth, probably just about anything will get removed from Facebook automatically if enough users file complaints against it, which is the problem for unpopular but legal content.)

How would these complaints have been handled under my proposed algorithm? The gist of my idea was that any users could sign up to be voluntary reviewers of "abuse complaints" filed against public content on Facebook. Once Facebook had built up a roster of tens of thousands of reviewers, new abuse complaints would be handled as follows. When a complaint (or some threshold of complaints) is filed against a piece of content, a random group of, say, 100 users could be selected from the entire population of eligible reviewers, and Facebook would send them a request to "vote" on whether that content violated the Terms of Service. If the number of "Yes" votes exceeded some threshold, the content would be removed (or at least, put in a high-priority queue for a Facebook employee to determine if the content really did warrant removal). The main benefit of this algorithm is that would be much harder for co-ordinated mobs to "game the system", because in order to swing the vote, they would have to comprise a significant fraction of the 100 randomly selected reviewers, and to achieve that, the mob members would have to comprise a significant fraction of the entire reviewer population. This would be prohibitively difficult if hundreds of thousands of users signed up as content reviewers.

All of the emails I received -- not just "almost" all of them, but really all of them -- contained some insightful suggestions worth mentioning, although there was some duplication between the ideas. If you didn't see the last article, you might consider it worth while to stop reading before proceeding further, and mull over the description of the algorithm above to see how you would improve it. Then read the suggestions that came in to see how well your ideas matched up with the submissions I received.

The upshot is that nobody found what I believed to be completely fatal flaws, although one reader brought something to my attention that might cause trouble for the algorithm after a few more years. Beyond that, reader suggestions could be divided essentially into two categories. The first category of suggestions related to ensuring that the basic premise would actually work -- that the votes cast by a random sample would be representative of general user opinion, and could not be gamed by a coordinated mob or a very resourceful cabal trying to game the system. The second category of suggestions started by assuming that the voting system would work, and suggested other features that could be added to the algorithm -- or, in one case, an entire alternative algorithm to replace it.

To begin with the attacks and counter-attacks against the basic voting algorithm. Walter Freeman and Haydn Huntley independently suggested monitoring for users who vote in a small minority in a significant portion of vote-offs, and reducing their influence in future votes (by either not inviting them to vote on future juries, or sending them the future invites but then ignoring their votes anyway). The assumption is that if a user is frequently among the 10% who vote "Yes [this is abuse]" when the other 90% of respondents are voting "No [this is not abuse]", or vice versa, then that user is voting randomly, or their point of view is so skewed that their votes could safely be ignored even if they are sincere. I like the idea of eliminating deadweight voters, but this might also incentivize voters to vote the way they think the crowd would vote, instead of voting their true opinions -- for example, if they were called to vote on an anti-Obama page that showed Obama wearing a Hitler mustache. Some people's knee-jerk reaction would be to call the page "racist" or "hate speech" or "a threat of violence", even though comparing Obama to Hitler is not, strictly speaking, any of those things. If I were voting my honest opinion, I would count that page as "not abuse". But if I knew that I were voting along with dozens of other people, and my future voting rights might be revoked if I didn't vote with the majority, I might be tempted to vote "abuse".

Similarly, Walter Freeman and reader "mjrosenbaum" both suggested setting deliberate traps for deadweight users, by creating artificial cases where the answer was pre-determined to be obviously yes or obviously no, calling for votes, and revoking privileges for users who gave the wrong answer. This would eliminate the problem of borderline cases like the one above, where smart users think, "I suspect the majority will give the wrong answer, so I'm just going to go with the crowd, to keep my voting rights." On the other hand, it's more labor for Facebook to create the cases, and any public content authored by them -- especially content that is deliberately crafted to be "questionable" -- would probably have to run a gauntlet of being reviewed by lawyers and PR mavens before being released. My suggestion would be to use these artificial scenarios periodically to make sure that the system is working (i.e. that juries are giving the right answers), but it would be too inefficient to use it to try and weed out problem voters.

In fact, these and several other suggestions fell into a category of ideas that could possibly improve the efficiency of the algorithm by reducing voter shenanigans (where "efficient" means that fewer users have to be invited to each vote-off in order to get statistically valid results), but might not be worth the effort. As long as most of the votes cast by users are sane and sincere, all you have to do is invite enough voters to a vote-off, and the majority will still get the correct answer most of the time, even if you have problem voters in the system. That's the simplest possible algorithm. The more complicated an algorithm you come up with, the more likely that Facebook (or any other site you recommended this to) would just throw up their hands and say, "Sounds too hard", and leave the idea dead in the water. That's why I like the algorithm as lean and tight as possible.

So it's not quite like designing an algorithm for your own use, where you could feel free to introduce more complications as long as you're responsible for keeping track of them. In recommending an algorithm for widespread adoption, the basic form of the algorithm should be as simple as possible. In the case of the voting algorithm some interesting wrinkles may come up if you don't eliminate problem voters, but this is not fatal to the idea as long as it's still true that, given a large enough random sample of voters, the majority will tend to vote the correct answer.

For example, James Renken pointed out that as voters dropped out due to boredom, the remaining users casting votes would tend to be either (1) weirdos who just wanted to view questionable material; and (2) prudes bent on removing as much material from Facebook as possible. But that's OK, as long as those two groups vote sanely enough (or as long as there are enough sane users outside those two groups) that material which does violate the TOS, tends to get more "Yes [this is abuse]" votes than material that doesn't. Then all you have to do is make the jury size large enough to make a statistically significant distinction between those two cases.

Similarly, Joshua Megerman suggested surveying users for their religious, political, and other beliefs when they sign up as volunteer reviewers (they could of course decline the survey). This makes it possible, insofar as people answer truthfully, to make sure that a jury is composed of a group with diverse belief sets. (On the other hand, users could game the system by reporting beliefs that are the opposite of what they truly feel. For example, if you're a leftist, register as a right-winger. Then when an abuse case comes before you, if it's a piece of content more offensive to leftists, then the real leftists on the jury will tend to vote against it -- but as a registered right-winger, you'll be able to cast a vote against it as well, and you'll be displacing a real right-wing voter who probably wouldn't have voted that way, so your vote will be worth more!) Again, it's fine if Facebook wants to do this, but even without collecting this data and simply selecting jurors at random, it should still be true that genuinely abusive pages get more "Yes" votes in a jury vote, than non-abusive pages.

Lastly in the "keep the jurors honest" category, Paul Ellsworth suggested allowing jurors to anonymously review each other -- when a given juror is chosen for the "hot seat" (perhaps randomly, perhaps as a result of a history of skewed voting), other jurors are randomly selected from the voting pool, to review that juror's voting record and decide whether that juror has been voting honestly and judiciously, or not. When I first read this idea, I instinctively thought that because a contaminated jury pool would be reviewing itself, it would not be able to reduce the percentage of problem voters, but a little more thought revealed that this isn't true. Suppose initially your jury pool consists of 80% "honest voters" and 20% "dishonest voters", that honest voters who review the voting record of another voter will always vote correctly whether that person is "honest" or "dishonest", and that dishonest voters will always vote incorrectly. It's still the case that when a voter's record is reviewed by a panel of, say, 20 other voters, virtually 100% of the time the majority will get the right answer. If you strip voting rights from a voter whenever a jury of other voters determines them to be a "dishonest voter", then over time, the percentage of honest voters in the system will creep from 80% to 100%. So again, this might work, and again, it might just be adding unnecessary complexity if the basic algorithm could work without it.

Note that none of these precautions would address the case of a "sleeper" voter -- a voter who joins the system with the sole intention of voting incorrectly on particular types of cases (perhaps planning on voting "yes" to shut down pages made by a particular organization, or pages advocating a particular view on a single issue), while still planning to vote correctly on everything else. By voting honestly in all other cases, they prevent themselves from being flagged by the system for casting too many minority votes, or from being blacklisted by other jurors for having a questionable overall voting record. The only real way I can see to address this problem is to hope that such users are outnumbered by the honest users in the system, and that juries are large enough that the chances of "rogue voters" gaining a majority on any one jury are nearly zero.

Which brings us to the one potentially fatal weakness in the system that I'm aware of: reader George Lawton referred me to a program run by the U.S. government to create armies of fake accounts to infiltrate social media, named, apparently without irony, Earnest Voice:

The project aims to enable military personnel to control multiple 'sock puppets' located at a range of geographically diverse IP addresses, with the aim of spreading pro-US propaganda.

An entity with the resources of the U.S. military could potentially create enough remote-controlled voters to overwhelm the system. I'm not sure if there is a way to deal with a system if the majority of voters are compromised. Presumably by making all decisions appealable to a core group of trusted Facebook employees at the top (although this then creates a bottleneck and limits scalability, especially if filing an appeal is free and all the parties who lose abuse cases are constantly filing appeals to the next level up).

Now. On to the second category of suggestions: Assuming the majority of voters are honest, what other features would be desirable to build into the system?

Walter Freeman, on the subject of filing appeals, suggested putting appealed pages in a special queue where they could be publicly viewed and users could comment on the ongoing appeals process, in addition to reading arguments posted by either side; this also negates the censorship itself due to the to the Streisand effect. I agree, but it's not obvious why this is a desirable feature. This does create perverse incentives, since some users could get extra traffic for their content by creating a page that makes whatever argument you're trying to promote, spiking it with some TOS-violating content, waiting for the page to get shut down, appealing the decision, and enjoying all the extra Streisand attention that it gets while on public display during the "appeal".

Meanwhile, James Renken pointed out that the system would work best for content that was originally public anyway, like a controversial Facebook page or event. If someone filed a complaint regarding a private message that they received, and they wanted a "jury vote" about whether the content of the message constituted abuse, then either the sender or the recipient would have to waive their right to privacy regarding the message before it could be shared with jurors. If the message really was abusive, then in some cases the recipient might waive their privacy rights -- reasoning that they didn't mind sharing the nasty message that someone sent them, in order to get the sender's account penalized. The problem arises if the message also contains sensitive personal facts about the recipient, which they wouldn't want to share with anonymous jurors. The system could allow them to black out any personal information before submitting the message for review, but that creates a recursive problem of abuse within the abuse system -- how do you know that someone didn't alter the content (and thus the offensiveness) of the message through their selective blacking-out? So it's not obvious whether this idea could be applied to non-public content at all.

Reader George Lawton suggested allowing content reviewers to vote on the funniest or weirdest content they had to review, to be posted in a public "Hall of Infamy". I love the thought of this, but I think Facebook's lawyers would be uncomfortable glamorizing anything questionable even if it were ultimately voted to be non-abusive (and certainly if it was voted to be abusive). Besides, this also has the perverse-incentives problem -- tie your message to something that you know will not only get an abuse complaint, but will hopefully end up in the Hall of Weird. (Even without the abuse jury system, there are already plenty of incentives for people to make a political point and hope that it will go viral.)

David Piepgrass suggested that new content reviewers should be allowed to specify certain types of content that they don't want to be asked to review -- nudity, graphic violence, etc. This sounds like a good idea. He adds that users probably shouldn't be able to opt-in only to review certain categories of content (or jurors might sign up only to review nudity, and then who would be left to review the death threats?).

Finally, in the other corner: Jerome Shaver suggested bypassing the jury voting system altogether and working on a heuristic algorithm to determine when abuse reports were being submitted by organized mobs of users, based on the patterns shown by mutual friendships between the users filing the abuse reports. The difficulties in designing such an algorithm, are too complicated to summarize quickly, and could fill an entire separate article. (Convince yourself that it's not an easy problem to solve. You can't just ignore abuse complaints from clusters of users that have many mutual friendships, because it can happen that real tight-knit communities of users might file abuse complaints against a piece of content, where the complaints are actually genuine.) But again, there is the problem that if a proposed solution is too complicated or too nebulous, Facebook has the excuse that they are "weighing several options", that they're "already working on something similar internally", etc. The jury vote system has the advantage that it can be described in just a few sentences, and the general public always knows whether it has been implemented or not -- which means that as long as abuses of the complaint system continue, people can ask, "Why doesn't Facebook try this?"

You'll notice this is just a laundry list of the ideas I received, without any definitive conclusions about which ones are good or bad, but that's all I was going for. The original algorithm, I could argue with the force of mathematical proof that, under certain reasonable assumptions, it would work. There's no such proof or disproof for any of the suggested modifications, so I don't feel as strongly about any of them. But at the top of the article I suggested for readers to stop reading and see how many of these ideas they could come up with on their own. How did you do?

The final honor roll of readers who were each the first, or only, person to submit an original idea: Walter Freeman (bonus points for getting in several good ones), James Renken, Joshua Megerman, Paul Ellsworth, George Lawton, Jerome Shaver, and David Piepgrass. Most of them volunteered to donate their winnings to charity, and agreed to let me donate their share to Vittana, which arranges microloans to college students in developing countries. One preferred a charity of their choosing, and only one actually kept the money. To be clear, for future contests, it's awesome if you want to donate the money to charity, but it's not dickish to keep it. That was the original deal after all.

So, all very clever and interesting suggestions, some of which might inspire readers to keep coming up with their own further variations. I said which ideas I probably would have incorporated and which ones I wouldn't, and I'm sure many of you would tell me that I'm wrong on some of those points. Although from here on out you're doing it for free.

Slashdot regular contributor Bennett Haselton writes "In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would make it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 for the best reader suggestions on how to improve the idea, or why they thought it wouldn't work. Read their suggestions and decide what value I got for my infotainment dollar."

In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would scale to a large number of users while also making it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 to readers sending in the best suggestions for improvements, or alternative algorithms, or fatal flaws in the whole idea that would require starting from scratch. As the suggestions were coming in, Facebook obligingly kept the issue in the news by removing a photo of two men kissing from a user's profile, sending a form letter to the user that they had violated Facebook's prohibition on "nudity, or any kind of graphic or sexually suggestive content". (It would be a cheap shot to say that a photo of a man and a woman kissing probably would not have been removed; in truth, probably just about anything will get removed from Facebook automatically if enough users file complaints against it, which is the problem for unpopular but legal content.)

How would these complaints have been handled under my proposed algorithm? The gist of my idea was that any users could sign up to be voluntary reviewers of "abuse complaints" filed against public content on Facebook. Once Facebook had built up a roster of tens of thousands of reviewers, new abuse complaints would be handled as follows. When a complaint (or some threshold of complaints) is filed against a piece of content, a random group of, say, 100 users could be selected from the entire population of eligible reviewers, and Facebook would send them a request to "vote" on whether that content violated the Terms of Service. If the number of "Yes" votes exceeded some threshold, the content would be removed (or at least, put in a high-priority queue for a Facebook employee to determine if the content really did warrant removal). The main benefit of this algorithm is that would be much harder for co-ordinated mobs to "game the system", because in order to swing the vote, they would have to comprise a significant fraction of the 100 randomly selected reviewers, and to achieve that, the mob members would have to comprise a significant fraction of the entire reviewer population. This would be prohibitively difficult if hundreds of thousands of users signed up as content reviewers.

All of the emails I received -- not just "almost" all of them, but really all of them -- contained some insightful suggestions worth mentioning, although there was some duplication between the ideas. If you didn't see the last article, you might consider it worth while to stop reading before proceeding further, and mull over the description of the algorithm above to see how you would improve it. Then read the suggestions that came in to see how well your ideas matched up with the submissions I received.

The upshot is that nobody found what I believed to be completely fatal flaws, although one reader brought something to my attention that might cause trouble for the algorithm after a few more years. Beyond that, reader suggestions could be divided essentially into two categories. The first category of suggestions related to ensuring that the basic premise would actually work -- that the votes cast by a random sample would be representative of general user opinion, and could not be gamed by a coordinated mob or a very resourceful cabal trying to game the system. The second category of suggestions started by assuming that the voting system would work, and suggested other features that could be added to the algorithm -- or, in one case, an entire alternative algorithm to replace it.

To begin with the attacks and counter-attacks against the basic voting algorithm. Walter Freeman and Haydn Huntley independently suggested monitoring for users who vote in a small minority in a significant portion of vote-offs, and reducing their influence in future votes (by either not inviting them to vote on future juries, or sending them the future invites but then ignoring their votes anyway). The assumption is that if a user is frequently among the 10% who vote "Yes [this is abuse]" when the other 90% of respondents are voting "No [this is not abuse]", or vice versa, then that user is voting randomly, or their point of view is so skewed that their votes could safely be ignored even if they are sincere. I like the idea of eliminating deadweight voters, but this might also incentivize voters to vote the way they think the crowd would vote, instead of voting their true opinions -- for example, if they were called to vote on an anti-Obama page that showed Obama wearing a Hitler mustache. Some people's knee-jerk reaction would be to call the page "racist" or "hate speech" or "a threat of violence", even though comparing Obama to Hitler is not, strictly speaking, any of those things. If I were voting my honest opinion, I would count that page as "not abuse". But if I knew that I were voting along with dozens of other people, and my future voting rights might be revoked if I didn't vote with the majority, I might be tempted to vote "abuse".

Similarly, Walter Freeman and reader "mjrosenbaum" both suggested setting deliberate traps for deadweight users, by creating artificial cases where the answer was pre-determined to be obviously yes or obviously no, calling for votes, and revoking privileges for users who gave the wrong answer. This would eliminate the problem of borderline cases like the one above, where smart users think, "I suspect the majority will give the wrong answer, so I'm just going to go with the crowd, to keep my voting rights." On the other hand, it's more labor for Facebook to create the cases, and any public content authored by them -- especially content that is deliberately crafted to be "questionable" -- would probably have to run a gauntlet of being reviewed by lawyers and PR mavens before being released. My suggestion would be to use these artificial scenarios periodically to make sure that the system is working (i.e. that juries are giving the right answers), but it would be too inefficient to use it to try and weed out problem voters.

In fact, these and several other suggestions fell into a category of ideas that could possibly improve the efficiency of the algorithm by reducing voter shenanigans (where "efficient" means that fewer users have to be invited to each vote-off in order to get statistically valid results), but might not be worth the effort. As long as most of the votes cast by users are sane and sincere, all you have to do is invite enough voters to a vote-off, and the majority will still get the correct answer most of the time, even if you have problem voters in the system. That's the simplest possible algorithm. The more complicated an algorithm you come up with, the more likely that Facebook (or any other site you recommended this to) would just throw up their hands and say, "Sounds too hard", and leave the idea dead in the water. That's why I like the algorithm as lean and tight as possible.

So it's not quite like designing an algorithm for your own use, where you could feel free to introduce more complications as long as you're responsible for keeping track of them. In recommending an algorithm for widespread adoption, the basic form of the algorithm should be as simple as possible. In the case of the voting algorithm some interesting wrinkles may come up if you don't eliminate problem voters, but this is not fatal to the idea as long as it's still true that, given a large enough random sample of voters, the majority will tend to vote the correct answer.

For example, James Renken pointed out that as voters dropped out due to boredom, the remaining users casting votes would tend to be either (1) weirdos who just wanted to view questionable material; and (2) prudes bent on removing as much material from Facebook as possible. But that's OK, as long as those two groups vote sanely enough (or as long as there are enough sane users outside those two groups) that material which does violate the TOS, tends to get more "Yes [this is abuse]" votes than material that doesn't. Then all you have to do is make the jury size large enough to make a statistically significant distinction between those two cases.

Similarly, Joshua Megerman suggested surveying users for their religious, political, and other beliefs when they sign up as volunteer reviewers (they could of course decline the survey). This makes it possible, insofar as people answer truthfully, to make sure that a jury is composed of a group with diverse belief sets. (On the other hand, users could game the system by reporting beliefs that are the opposite of what they truly feel. For example, if you're a leftist, register as a right-winger. Then when an abuse case comes before you, if it's a piece of content more offensive to leftists, then the real leftists on the jury will tend to vote against it -- but as a registered right-winger, you'll be able to cast a vote against it as well, and you'll be displacing a real right-wing voter who probably wouldn't have voted that way, so your vote will be worth more!) Again, it's fine if Facebook wants to do this, but even without collecting this data and simply selecting jurors at random, it should still be true that genuinely abusive pages get more "Yes" votes in a jury vote, than non-abusive pages.

Lastly in the "keep the jurors honest" category, Paul Ellsworth suggested allowing jurors to anonymously review each other -- when a given juror is chosen for the "hot seat" (perhaps randomly, perhaps as a result of a history of skewed voting), other jurors are randomly selected from the voting pool, to review that juror's voting record and decide whether that juror has been voting honestly and judiciously, or not. When I first read this idea, I instinctively thought that because a contaminated jury pool would be reviewing itself, it would not be able to reduce the percentage of problem voters, but a little more thought revealed that this isn't true. Suppose initially your jury pool consists of 80% "honest voters" and 20% "dishonest voters", that honest voters who review the voting record of another voter will always vote correctly whether that person is "honest" or "dishonest", and that dishonest voters will always vote incorrectly. It's still the case that when a voter's record is reviewed by a panel of, say, 20 other voters, virtually 100% of the time the majority will get the right answer. If you strip voting rights from a voter whenever a jury of other voters determines them to be a "dishonest voter", then over time, the percentage of honest voters in the system will creep from 80% to 100%. So again, this might work, and again, it might just be adding unnecessary complexity if the basic algorithm could work without it.

Note that none of these precautions would address the case of a "sleeper" voter -- a voter who joins the system with the sole intention of voting incorrectly on particular types of cases (perhaps planning on voting "yes" to shut down pages made by a particular organization, or pages advocating a particular view on a single issue), while still planning to vote correctly on everything else. By voting honestly in all other cases, they prevent themselves from being flagged by the system for casting too many minority votes, or from being blacklisted by other jurors for having a questionable overall voting record. The only real way I can see to address this problem is to hope that such users are outnumbered by the honest users in the system, and that juries are large enough that the chances of "rogue voters" gaining a majority on any one jury are nearly zero.

Which brings us to the one potentially fatal weakness in the system that I'm aware of: reader George Lawton referred me to a program run by the U.S. government to create armies of fake accounts to infiltrate social media, named, apparently without irony, Earnest Voice:

The project aims to enable military personnel to control multiple 'sock puppets' located at a range of geographically diverse IP addresses, with the aim of spreading pro-US propaganda.

An entity with the resources of the U.S. military could potentially create enough remote-controlled voters to overwhelm the system. I'm not sure if there is a way to deal with a system if the majority of voters are compromised. Presumably by making all decisions appealable to a core group of trusted Facebook employees at the top (although this then creates a bottleneck and limits scalability, especially if filing an appeal is free and all the parties who lose abuse cases are constantly filing appeals to the next level up).

Now. On to the second category of suggestions: Assuming the majority of voters are honest, what other features would be desirable to build into the system?

Walter Freeman, on the subject of filing appeals, suggested putting appealed pages in a special queue where they could be publicly viewed and users could comment on the ongoing appeals process, in addition to reading arguments posted by either side; this also negates the censorship itself due to the to the Streisand effect. I agree, but it's not obvious why this is a desirable feature. This does create perverse incentives, since some users could get extra traffic for their content by creating a page that makes whatever argument you're trying to promote, spiking it with some TOS-violating content, waiting for the page to get shut down, appealing the decision, and enjoying all the extra Streisand attention that it gets while on public display during the "appeal".

Meanwhile, James Renken pointed out that the system would work best for content that was originally public anyway, like a controversial Facebook page or event. If someone filed a complaint regarding a private message that they received, and they wanted a "jury vote" about whether the content of the message constituted abuse, then either the sender or the recipient would have to waive their right to privacy regarding the message before it could be shared with jurors. If the message really was abusive, then in some cases the recipient might waive their privacy rights -- reasoning that they didn't mind sharing the nasty message that someone sent them, in order to get the sender's account penalized. The problem arises if the message also contains sensitive personal facts about the recipient, which they wouldn't want to share with anonymous jurors. The system could allow them to black out any personal information before submitting the message for review, but that creates a recursive problem of abuse within the abuse system -- how do you know that someone didn't alter the content (and thus the offensiveness) of the message through their selective blacking-out? So it's not obvious whether this idea could be applied to non-public content at all.

Reader George Lawton suggested allowing content reviewers to vote on the funniest or weirdest content they had to review, to be posted in a public "Hall of Infamy". I love the thought of this, but I think Facebook's lawyers would be uncomfortable glamorizing anything questionable even if it were ultimately voted to be non-abusive (and certainly if it was voted to be abusive). Besides, this also has the perverse-incentives problem -- tie your message to something that you know will not only get an abuse complaint, but will hopefully end up in the Hall of Weird. (Even without the abuse jury system, there are already plenty of incentives for people to make a political point and hope that it will go viral.)

David Piepgrass suggested that new content reviewers should be allowed to specify certain types of content that they don't want to be asked to review -- nudity, graphic violence, etc. This sounds like a good idea. He adds that users probably shouldn't be able to opt-in only to review certain categories of content (or jurors might sign up only to review nudity, and then who would be left to review the death threats?).

Finally, in the other corner: Jerome Shaver suggested bypassing the jury voting system altogether and working on a heuristic algorithm to determine when abuse reports were being submitted by organized mobs of users, based on the patterns shown by mutual friendships between the users filing the abuse reports. The difficulties in designing such an algorithm, are too complicated to summarize quickly, and could fill an entire separate article. (Convince yourself that it's not an easy problem to solve. You can't just ignore abuse complaints from clusters of users that have many mutual friendships, because it can happen that real tight-knit communities of users might file abuse complaints against a piece of content, where the complaints are actually genuine.) But again, there is the problem that if a proposed solution is too complicated or too nebulous, Facebook has the excuse that they are "weighing several options", that they're "already working on something similar internally", etc. The jury vote system has the advantage that it can be described in just a few sentences, and the general public always knows whether it has been implemented or not -- which means that as long as abuses of the complaint system continue, people can ask, "Why doesn't Facebook try this?"

You'll notice this is just a laundry list of the ideas I received, without any definitive conclusions about which ones are good or bad, but that's all I was going for. The original algorithm, I could argue with the force of mathematical proof that, under certain reasonable assumptions, it would work. There's no such proof or disproof for any of the suggested modifications, so I don't feel as strongly about any of them. But at the top of the article I suggested for readers to stop reading and see how many of these ideas they could come up with on their own. How did you do?

The final honor roll of readers who were each the first, or only, person to submit an original idea: Walter Freeman (bonus points for getting in several good ones), James Renken, Joshua Megerman, Paul Ellsworth, George Lawton, Jerome Shaver, and David Piepgrass. Most of them volunteered to donate their winnings to charity, and agreed to let me donate their share to Vittana, which arranges microloans to college students in developing countries. One preferred a charity of their choosing, and only one actually kept the money. To be clear, for future contests, it's awesome if you want to donate the money to charity, but it's not dickish to keep it. That was the original deal after all.

So, all very clever and interesting suggestions, some of which might inspire readers to keep coming up with their own further variations. I said which ideas I probably would have incorporated and which ones I wouldn't, and I'm sure many of you would tell me that I'm wrong on some of those points. Although from here on out you're doing it for free.

Slashdot regular contributor Bennett Haselton writes "In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would make it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 for the best reader suggestions on how to improve the idea, or why they thought it wouldn't work. Read their suggestions and decide what value I got for my infotainment dollar."

In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would scale to a large number of users while also making it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 to readers sending in the best suggestions for improvements, or alternative algorithms, or fatal flaws in the whole idea that would require starting from scratch. As the suggestions were coming in, Facebook obligingly kept the issue in the news by removing a photo of two men kissing from a user's profile, sending a form letter to the user that they had violated Facebook's prohibition on "nudity, or any kind of graphic or sexually suggestive content". (It would be a cheap shot to say that a photo of a man and a woman kissing probably would not have been removed; in truth, probably just about anything will get removed from Facebook automatically if enough users file complaints against it, which is the problem for unpopular but legal content.)

How would these complaints have been handled under my proposed algorithm? The gist of my idea was that any users could sign up to be voluntary reviewers of "abuse complaints" filed against public content on Facebook. Once Facebook had built up a roster of tens of thousands of reviewers, new abuse complaints would be handled as follows. When a complaint (or some threshold of complaints) is filed against a piece of content, a random group of, say, 100 users could be selected from the entire population of eligible reviewers, and Facebook would send them a request to "vote" on whether that content violated the Terms of Service. If the number of "Yes" votes exceeded some threshold, the content would be removed (or at least, put in a high-priority queue for a Facebook employee to determine if the content really did warrant removal). The main benefit of this algorithm is that would be much harder for co-ordinated mobs to "game the system", because in order to swing the vote, they would have to comprise a significant fraction of the 100 randomly selected reviewers, and to achieve that, the mob members would have to comprise a significant fraction of the entire reviewer population. This would be prohibitively difficult if hundreds of thousands of users signed up as content reviewers.

All of the emails I received -- not just "almost" all of them, but really all of them -- contained some insightful suggestions worth mentioning, although there was some duplication between the ideas. If you didn't see the last article, you might consider it worth while to stop reading before proceeding further, and mull over the description of the algorithm above to see how you would improve it. Then read the suggestions that came in to see how well your ideas matched up with the submissions I received.

The upshot is that nobody found what I believed to be completely fatal flaws, although one reader brought something to my attention that might cause trouble for the algorithm after a few more years. Beyond that, reader suggestions could be divided essentially into two categories. The first category of suggestions related to ensuring that the basic premise would actually work -- that the votes cast by a random sample would be representative of general user opinion, and could not be gamed by a coordinated mob or a very resourceful cabal trying to game the system. The second category of suggestions started by assuming that the voting system would work, and suggested other features that could be added to the algorithm -- or, in one case, an entire alternative algorithm to replace it.

To begin with the attacks and counter-attacks against the basic voting algorithm. Walter Freeman and Haydn Huntley independently suggested monitoring for users who vote in a small minority in a significant portion of vote-offs, and reducing their influence in future votes (by either not inviting them to vote on future juries, or sending them the future invites but then ignoring their votes anyway). The assumption is that if a user is frequently among the 10% who vote "Yes [this is abuse]" when the other 90% of respondents are voting "No [this is not abuse]", or vice versa, then that user is voting randomly, or their point of view is so skewed that their votes could safely be ignored even if they are sincere. I like the idea of eliminating deadweight voters, but this might also incentivize voters to vote the way they think the crowd would vote, instead of voting their true opinions -- for example, if they were called to vote on an anti-Obama page that showed Obama wearing a Hitler mustache. Some people's knee-jerk reaction would be to call the page "racist" or "hate speech" or "a threat of violence", even though comparing Obama to Hitler is not, strictly speaking, any of those things. If I were voting my honest opinion, I would count that page as "not abuse". But if I knew that I were voting along with dozens of other people, and my future voting rights might be revoked if I didn't vote with the majority, I might be tempted to vote "abuse".

Similarly, Walter Freeman and reader "mjrosenbaum" both suggested setting deliberate traps for deadweight users, by creating artificial cases where the answer was pre-determined to be obviously yes or obviously no, calling for votes, and revoking privileges for users who gave the wrong answer. This would eliminate the problem of borderline cases like the one above, where smart users think, "I suspect the majority will give the wrong answer, so I'm just going to go with the crowd, to keep my voting rights." On the other hand, it's more labor for Facebook to create the cases, and any public content authored by them -- especially content that is deliberately crafted to be "questionable" -- would probably have to run a gauntlet of being reviewed by lawyers and PR mavens before being released. My suggestion would be to use these artificial scenarios periodically to make sure that the system is working (i.e. that juries are giving the right answers), but it would be too inefficient to use it to try and weed out problem voters.

In fact, these and several other suggestions fell into a category of ideas that could possibly improve the efficiency of the algorithm by reducing voter shenanigans (where "efficient" means that fewer users have to be invited to each vote-off in order to get statistically valid results), but might not be worth the effort. As long as most of the votes cast by users are sane and sincere, all you have to do is invite enough voters to a vote-off, and the majority will still get the correct answer most of the time, even if you have problem voters in the system. That's the simplest possible algorithm. The more complicated an algorithm you come up with, the more likely that Facebook (or any other site you recommended this to) would just throw up their hands and say, "Sounds too hard", and leave the idea dead in the water. That's why I like the algorithm as lean and tight as possible.

So it's not quite like designing an algorithm for your own use, where you could feel free to introduce more complications as long as you're responsible for keeping track of them. In recommending an algorithm for widespread adoption, the basic form of the algorithm should be as simple as possible. In the case of the voting algorithm some interesting wrinkles may come up if you don't eliminate problem voters, but this is not fatal to the idea as long as it's still true that, given a large enough random sample of voters, the majority will tend to vote the correct answer.

For example, James Renken pointed out that as voters dropped out due to boredom, the remaining users casting votes would tend to be either (1) weirdos who just wanted to view questionable material; and (2) prudes bent on removing as much material from Facebook as possible. But that's OK, as long as those two groups vote sanely enough (or as long as there are enough sane users outside those two groups) that material which does violate the TOS, tends to get more "Yes [this is abuse]" votes than material that doesn't. Then all you have to do is make the jury size large enough to make a statistically significant distinction between those two cases.

Similarly, Joshua Megerman suggested surveying users for their religious, political, and other beliefs when they sign up as volunteer reviewers (they could of course decline the survey). This makes it possible, insofar as people answer truthfully, to make sure that a jury is composed of a group with diverse belief sets. (On the other hand, users could game the system by reporting beliefs that are the opposite of what they truly feel. For example, if you're a leftist, register as a right-winger. Then when an abuse case comes before you, if it's a piece of content more offensive to leftists, then the real leftists on the jury will tend to vote against it -- but as a registered right-winger, you'll be able to cast a vote against it as well, and you'll be displacing a real right-wing voter who probably wouldn't have voted that way, so your vote will be worth more!) Again, it's fine if Facebook wants to do this, but even without collecting this data and simply selecting jurors at random, it should still be true that genuinely abusive pages get more "Yes" votes in a jury vote, than non-abusive pages.

Lastly in the "keep the jurors honest" category, Paul Ellsworth suggested allowing jurors to anonymously review each other -- when a given juror is chosen for the "hot seat" (perhaps randomly, perhaps as a result of a history of skewed voting), other jurors are randomly selected from the voting pool, to review that juror's voting record and decide whether that juror has been voting honestly and judiciously, or not. When I first read this idea, I instinctively thought that because a contaminated jury pool would be reviewing itself, it would not be able to reduce the percentage of problem voters, but a little more thought revealed that this isn't true. Suppose initially your jury pool consists of 80% "honest voters" and 20% "dishonest voters", that honest voters who review the voting record of another voter will always vote correctly whether that person is "honest" or "dishonest", and that dishonest voters will always vote incorrectly. It's still the case that when a voter's record is reviewed by a panel of, say, 20 other voters, virtually 100% of the time the majority will get the right answer. If you strip voting rights from a voter whenever a jury of other voters determines them to be a "dishonest voter", then over time, the percentage of honest voters in the system will creep from 80% to 100%. So again, this might work, and again, it might just be adding unnecessary complexity if the basic algorithm could work without it.

Note that none of these precautions would address the case of a "sleeper" voter -- a voter who joins the system with the sole intention of voting incorrectly on particular types of cases (perhaps planning on voting "yes" to shut down pages made by a particular organization, or pages advocating a particular view on a single issue), while still planning to vote correctly on everything else. By voting honestly in all other cases, they prevent themselves from being flagged by the system for casting too many minority votes, or from being blacklisted by other jurors for having a questionable overall voting record. The only real way I can see to address this problem is to hope that such users are outnumbered by the honest users in the system, and that juries are large enough that the chances of "rogue voters" gaining a majority on any one jury are nearly zero.

Which brings us to the one potentially fatal weakness in the system that I'm aware of: reader George Lawton referred me to a program run by the U.S. government to create armies of fake accounts to infiltrate social media, named, apparently without irony, Earnest Voice:

The project aims to enable military personnel to control multiple 'sock puppets' located at a range of geographically diverse IP addresses, with the aim of spreading pro-US propaganda.

An entity with the resources of the U.S. military could potentially create enough remote-controlled voters to overwhelm the system. I'm not sure if there is a way to deal with a system if the majority of voters are compromised. Presumably by making all decisions appealable to a core group of trusted Facebook employees at the top (although this then creates a bottleneck and limits scalability, especially if filing an appeal is free and all the parties who lose abuse cases are constantly filing appeals to the next level up).

Now. On to the second category of suggestions: Assuming the majority of voters are honest, what other features would be desirable to build into the system?

Walter Freeman, on the subject of filing appeals, suggested putting appealed pages in a special queue where they could be publicly viewed and users could comment on the ongoing appeals process, in addition to reading arguments posted by either side; this also negates the censorship itself due to the to the Streisand effect. I agree, but it's not obvious why this is a desirable feature. This does create perverse incentives, since some users could get extra traffic for their content by creating a page that makes whatever argument you're trying to promote, spiking it with some TOS-violating content, waiting for the page to get shut down, appealing the decision, and enjoying all the extra Streisand attention that it gets while on public display during the "appeal".

Meanwhile, James Renken pointed out that the system would work best for content that was originally public anyway, like a controversial Facebook page or event. If someone filed a complaint regarding a private message that they received, and they wanted a "jury vote" about whether the content of the message constituted abuse, then either the sender or the recipient would have to waive their right to privacy regarding the message before it could be shared with jurors. If the message really was abusive, then in some cases the recipient might waive their privacy rights -- reasoning that they didn't mind sharing the nasty message that someone sent them, in order to get the sender's account penalized. The problem arises if the message also contains sensitive personal facts about the recipient, which they wouldn't want to share with anonymous jurors. The system could allow them to black out any personal information before submitting the message for review, but that creates a recursive problem of abuse within the abuse system -- how do you know that someone didn't alter the content (and thus the offensiveness) of the message through their selective blacking-out? So it's not obvious whether this idea could be applied to non-public content at all.

Reader George Lawton suggested allowing content reviewers to vote on the funniest or weirdest content they had to review, to be posted in a public "Hall of Infamy". I love the thought of this, but I think Facebook's lawyers would be uncomfortable glamorizing anything questionable even if it were ultimately voted to be non-abusive (and certainly if it was voted to be abusive). Besides, this also has the perverse-incentives problem -- tie your message to something that you know will not only get an abuse complaint, but will hopefully end up in the Hall of Weird. (Even without the abuse jury system, there are already plenty of incentives for people to make a political point and hope that it will go viral.)

David Piepgrass suggested that new content reviewers should be allowed to specify certain types of content that they don't want to be asked to review -- nudity, graphic violence, etc. This sounds like a good idea. He adds that users probably shouldn't be able to opt-in only to review certain categories of content (or jurors might sign up only to review nudity, and then who would be left to review the death threats?).

Finally, in the other corner: Jerome Shaver suggested bypassing the jury voting system altogether and working on a heuristic algorithm to determine when abuse reports were being submitted by organized mobs of users, based on the patterns shown by mutual friendships between the users filing the abuse reports. The difficulties in designing such an algorithm, are too complicated to summarize quickly, and could fill an entire separate article. (Convince yourself that it's not an easy problem to solve. You can't just ignore abuse complaints from clusters of users that have many mutual friendships, because it can happen that real tight-knit communities of users might file abuse complaints against a piece of content, where the complaints are actually genuine.) But again, there is the problem that if a proposed solution is too complicated or too nebulous, Facebook has the excuse that they are "weighing several options", that they're "already working on something similar internally", etc. The jury vote system has the advantage that it can be described in just a few sentences, and the general public always knows whether it has been implemented or not -- which means that as long as abuses of the complaint system continue, people can ask, "Why doesn't Facebook try this?"

You'll notice this is just a laundry list of the ideas I received, without any definitive conclusions about which ones are good or bad, but that's all I was going for. The original algorithm, I could argue with the force of mathematical proof that, under certain reasonable assumptions, it would work. There's no such proof or disproof for any of the suggested modifications, so I don't feel as strongly about any of them. But at the top of the article I suggested for readers to stop reading and see how many of these ideas they could come up with on their own. How did you do?

The final honor roll of readers who were each the first, or only, person to submit an original idea: Walter Freeman (bonus points for getting in several good ones), James Renken, Joshua Megerman, Paul Ellsworth, George Lawton, Jerome Shaver, and David Piepgrass. Most of them volunteered to donate their winnings to charity, and agreed to let me donate their share to Vittana, which arranges microloans to college students in developing countries. One preferred a charity of their choosing, and only one actually kept the money. To be clear, for future contests, it's awesome if you want to donate the money to charity, but it's not dickish to keep it. That was the original deal after all.

So, all very clever and interesting suggestions, some of which might inspire readers to keep coming up with their own further variations. I said which ideas I probably would have incorporated and which ones I wouldn't, and I'm sure many of you would tell me that I'm wrong on some of those points. Although from here on out you're doing it for free.

eldavojohn writes "You're probably familiar with Microsoft's long running assault on Android but, as noticed by Groklaw, Barnes and Noble has fired back saying, 'Microsoft has asserted patents that extend only to arbitrary, outmoded, or non-essential design features, but uses these patents to demand that every manufacturer of an Android-based mobile device take a license from Microsoft and pay exorbitant licensing fees or face protracted and expensive patent infringement litigation.' Barnes and Noble goes on to assert that Microsoft violates 'antitrust laws, threatens competition for mobile device operating systems and is further evidence of Microsoft's efforts to dominate and control Android and other open source operating systems.' The PDF of the filing from two days ago is rife with accusations including, 'Microsoft intends to utilize its patents to control the activities of and extract fees from the designers, developers, and manufacturers of devices, including tablets, eReaders, and other mobile devices, that employ the Android Operating System.' and 'Microsoft has falsely and without justification asserted that its patents somehow provide it with the right to prohibit device manufacturers from employing new versions of the Android Operating System, or third party software.' Barnes and Noble does not mince words when explaining Microsoft's FUD campaign to both the public and developers in its attempts to suppress Android. It's good to see PJ still digging through massive court briefs to bring us the details on IP court battles."

eldavojohn writes "You're probably familiar with Microsoft's long running assault on Android but, as noticed by Groklaw, Barnes and Noble has fired back saying, 'Microsoft has asserted patents that extend only to arbitrary, outmoded, or non-essential design features, but uses these patents to demand that every manufacturer of an Android-based mobile device take a license from Microsoft and pay exorbitant licensing fees or face protracted and expensive patent infringement litigation.' Barnes and Noble goes on to assert that Microsoft violates 'antitrust laws, threatens competition for mobile device operating systems and is further evidence of Microsoft's efforts to dominate and control Android and other open source operating systems.' The PDF of the filing from two days ago is rife with accusations including, 'Microsoft intends to utilize its patents to control the activities of and extract fees from the designers, developers, and manufacturers of devices, including tablets, eReaders, and other mobile devices, that employ the Android Operating System.' and 'Microsoft has falsely and without justification asserted that its patents somehow provide it with the right to prohibit device manufacturers from employing new versions of the Android Operating System, or third party software.' Barnes and Noble does not mince words when explaining Microsoft's FUD campaign to both the public and developers in its attempts to suppress Android. It's good to see PJ still digging through massive court briefs to bring us the details on IP court battles."

suraj.sun writes "Like clockwork, the first lawsuit resulting from the security breach of the personal data of more than 75 million Sony PlayStation Network customers has been filed. The suit was filed today on behalf of Kristopher Johns, 36, of Birmingham, Ala., in the US District Court for the Northern District of California. Johns accuses Sony of not taking 'reasonable care to protect, encrypt, and secure the private and sensitive data of its users.' He also believes Sony took too long to notify him and other customers that their personal information had been exposed. Because of that, the complaint alleges, Sony did not allow its customers 'to make an informed decision as to whether to change credit card numbers, close the exposed accounts, check their credit reports, or take other mitigating actions.'"

KuanH writes "Amazon SimpleDB Developer Guide is billed as a complete guide to using Amazon's SimpleDB database API. It's most detailed for PHP. It's helpful for Python. But the Java code and explanations aren't up to the standard of the others. It includes a primer on using Amazon S3 with SimpleDB: files stored on S3, file metadata stored in SimpleDB — again, less good for Java. It also covers tuning to reduce usage costs, caching using memcached, and ways to batch-update and make serial or parallel requests to SimpleDB. However, it's missing some information that beginners might need, and it's perhaps not quite advanced enough for the more experienced. Downloadable example code is available only for PHP." Keep reading for the rest of Kuan's review. Amazon SimpleDB Developer Guide author Prabhakar Chaganti, Rich Helms pages 252 publisher Packt Publishing rating 6 reviewer Kuan Hon ISBN 1847197345 summary "Getting started" guide to using Amazon's SimpleDB cloud database Say "cloud" to get the attention of CIOs seeking to cut costs in these recessionary times. One well known "database in the cloud" option is Amazon Web Services' SimpleDB, which Amazon describes as "a highly available, flexible, and scalable non-relational data store that offloads the work of database administration."

Those who prefer traditional relational databases could try eg Amazon RDS. This book only covers SimpleDB, a NoSQL or non-relational database. As is well known, NoSQL databases grew in popularity with the growth of large distributed systems and cloud computing, and their proponents tout their scalability and speed.

For anyone wanting a quick primer on NoSQL databases, this book includes a chapter on NoSQL which isn't limited to SimpleDB. It outlines some key conceptual differences between NoSQL and relational database management systems, with pros and cons, using the analogy of "a spreadsheet with some XML characteristics", and illustrating with some concrete examples. That chapter's been made available as a free sample chapter (SimpleDB versus RDBMS), so you can get a flavour of the book.

The contents list for this book is online, I won't recite it here. As well as an overview of SimpleDB, its terminology and advantages, the book goes through signing up with AWS and SimpleDB, and the account access keys. That chapter is also online, as a tutorial.

You may ask, how does this book differ from Amazon's free SimpleDB documentation, which includes a developer guide and a "getting started" guide? Amazon's own "getting started" is certainly helpful, and it's worth downloading and trying their web app scratchpad. But Amazon's detailed developer guide concentrates on REST and SOAP requests, which most people wouldn't want to deal with direct at that low level.

This book's focus is on using the SimpleDB web services API through certain specific languages and libraries — namely Java (JDK6 — using the typical 1.6 library plus several dependencies), Python (2.5 — you need boto), and PHP (with curl). It recommends the SDBtool Firefox extension (SDBizo), which is excellent for checking the results of running the code.

I've tried the book's Java and Python examples, on Windows. Not PHP, as I've not got round to learning PHP yet, though I skimmed the PHP explanations. Similarly, I've not had time to try it all over again on Linux. Generally, the book's coverage seems fuller and better for PHP than for Java or Python. Perhaps it was originally written for PHP, and the rest was bolted on — the stuff for Java more hurriedly than for Python?

The downloadable code samples, as mentioned, are PHP only. They really should have provided downloadable code for all 3 languages, plus some fake MP3 files (see later). If you get the e-book (available in PDF and epub), you can copy and paste the Java or Python code. But that's a tad tedious, especially when the code runs onto a new page, and there are stray end of lines etc that you have to delete manually. Furthermore, the Python code provided is for the interpreter in interactive mode (not for .py files, except a couple towards the end). So, for the Python, you also have to copy/paste each line one at a time. But that still beats having to re-type pages of code in full.

In other words, if you want this book and you're only interested in PHP, you can get away with just buying the hard copy and downloading the code from the Packt site. But if you prefer Python or Java, to save your fingers and blood pressure you should buy just the e-version, or get both paper and e books together. I really hope Packt will in future provide downloadable code samples for all the languages covered.

I have more issues with the sample code given in this book. The typical imports should have been spelled out in the example Java code. Eclipse offers more than one possible import in some cases. It was "try everything till it works", at least until I found this tutorial. I've included the initial required typical imports (though not the standard java.util etc ones) in my own list of points, which I'll say more about at the end of this review. Surely it wouldn't have been difficult to include just those few lines of imports, which could have saved readers a lot of time trying to work out the correct imports. There are also errors in the Python code, and on one page the code that should have been included is missing altogether.

Now, more on the book proper. After the overview described above, this book walks you through the basic SimpleDB operations: how to create a SimpleDB "domain" (equivalent to a worksheet in a spreadsheet), list domains, create/retrieve items (like spreadsheet rows), and delete domains.

Items have attributes (spreadsheet column headings), as key:value pairs — the key is the attribute name, the value is its value, eg address:1 Acacia Avenue. An attribute can have more than one value, eg the same item can have both address:1 Acacia Avenue and address:2 Broadway. The book also lists the SimpleDB constraints on domains, items and attributes — maximum number or size, etc — but it's best to check the AWS site for the latest info.

Code examples are given for each of the 3 languages mentioned. The examples are similar, but don't always cover the same ground. If they'd done that, where possible, it would have been more helpful to those of us trying examples in more than one language. One advantage of a book with associated website is that electronic updates can be published, and it would have been great if that had been done for this book. For instance, the book gave conditional put/delete code examples only for PHP. At the date of this review, boto now supports those features, but sample supplemental Python code for that still hadn't been made available.

SimpleDB stores attribute values as UTF-8 strings. This means that comparisons for sorting or searching are done lexicographically (character by character, left to right, numbers take precedence over uppercase over lowercase), and to handle numbers or dates you have to encode and decode them yourself. So, the book has a chapter explaining lexicographical comparison, data types, and how to encode and decode data to enable proper sorting and comparison of numbers, dates, Boolean values and XML-restricted characters. In the case of numbers this means zero padding and offsets, and there's example code for decoding and encoding numbers. Unlike with PHP and Python, oddly the Java code given was for the body of the typical method that carries out the encoding etc. This could have been omitted, and they should have given example code illustrating the method's usage instead. Similarly for the date formats code.

The SimpleDB query syntax is generally covered well, in a chapter which takes readers through first creating a sample database of song metadata to run queries against. It's not too painful copy/pasting the Java code (3+ pages), but with Python in interactive mode I drew the line at creating every song item and attributes using individual statements, even with pasting, so I just tried adding a couple of random ones to test that the code worked. I say again, full downloadable code please...!

That chapter then gives helpful examples of queries against the sample database and their results, including for more complex combined queries ("and", "or" type queries, "not" etc), and querying for multiple-value attributes. It also provides code examples for sorting and counting query results. But the Java code for retrieving an item's attributes wouldn't run, and I couldn't find the method used (getItemsAttributes()) detailed in the typical documentation; perhaps the book is out of date here?

The book starts going beyond the basics from Chapter 7 onwards, with a chapter on Amazon's S3 storage service — another well known component of Amazon Web Services, where "objects" (files) may be stored in "buckets" (directories), with "keys" used to retrieve objects.

For S3, the book uses JetS3t for Java. However, the Java code given for uploading files to S3 didn't demonstrate any integration with SimpleDB at all — the files were just uploaded with their filenames as the S3 keys, and the code didn't seem to deal with the creation of your own custom S3 keys for uploaded objects. In contrast, the Python code generated the S3 keys for the files from hashes previously produced and stored in the SimpleDB database, as well as dealing with their uploading. In addition, for me the Java code for downloading files from S3 just wouldn't run, and also it wasn't clear where the files were supposed to be downloaded to locally, unlike with the Python example. Inexplicably, there was no info on how to delete objects from S3 buckets, or indeed how to delete buckets. So, while the S3 chapter is of help, it could definitely do with being expanded, especially the Java sections.

Next, money money money. AWS charges are based on usage, so the chapter on tuning and usage costs has some practical value in explaining how SimpleDB is charged for, the "BoxUsage" value returned by requests to SimpleDB, using BoxUsage to optimize queries and compute costs, and how to get BoxUsage values back with your queries using Java, Python etc. There are code examples that, when run, illustrate the different BoxUsage values you get when you use different operators or expressions in queries (eg, using LIKE costs more).

However, partitioning your data into multiple domains is covered in only a few paragraphs, with no code given. I'd have liked to see more info on that, and some sample code for the partitioning process.

To further save money, you can use a cache to store data locally, trying your local cache first; and, only if the data is not there, would your app go out to SimpleDB and incur costs for querying it. This book accordingly has a chapter on how to install and use the popular open source caching system memcached to cache your query results locally. (CacheLite for PHP is also covered.) Again, the Java sections caused me some frustration. The Java test code showed that the memcached server was running properly on my machine, but the Java code for using the cache just didn't work; it ran, but continued to query SimpleDB direct. The Python code, however, worked perfectly — except that, if you're using memcached in Windows, you'll need to use port 11211 instead of what's shown in the book. (I didn't try it in Linux.)

Finally, the book deals with running parallel operations against SimpleDB, using its BatchPutAttributes. The section on updating SimpleDB in Python by making serial consecutive calls to SimpleDB is completely missing the code for the script, but the book does then cover inserting multiple items concurrently into SimpleDB using a threadpool in Java. It also gives sample Python code for alternative ways of parallelising requests: using Python's built-in threading module, threading and queues combined, then threading using the open source workerpool module.

To conclude, in substance the book has a fair amount of useful information on the basics of getting started with SimpleDB, particularly for Python (and probably PHP). But not providing downloadable code samples in Java and Python, or "fake" MP3 files to try S3 uploading/downloading, is a minus.

Some errors, inconsistencies and missing information from the department of "I-wish-they'd-included-this-even-if-they-thought-it-was-basic-as-it's-too-easily-missed-if-it's-not-spelled-out", mean that the book is not really "complete", and not as suitable as it should be for relative beginners — especially for Java and (in whatever language) Windows. It wouldn't take much extra work to get it up to scratch on that front. Perhaps the next edition, or better still an online update/supplement?

For the more experienced, the book doesn't take readers to as advanced a stage as it could have, in my view. In particular, it would have been good to have more info and example code on partitioning data between different domains, and also how to migrate data from an existing database to SimpleDB — their code for "importing" the sample database literally just adds each item and attribute individually.

Fix the errors, add the missing info for beginners, provide downloads of code in all relevant languages and "fake files", and I'd have given it a 7. Provide working sample Java code with more explanation, plus proper integration with S3, an 8. Add fuller info on partitioning, migration, and perhaps even integration with yet more AWS services, a 9.

All opinions are personal to me: half geek, half lawyer, mostly harmless. I'm researching legal issues in cloud computing.

You can purchase Amazon SimpleDB Developer Guide from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

KuanH writes "Amazon SimpleDB Developer Guide is billed as a complete guide to using Amazon's SimpleDB database API. It's most detailed for PHP. It's helpful for Python. But the Java code and explanations aren't up to the standard of the others. It includes a primer on using Amazon S3 with SimpleDB: files stored on S3, file metadata stored in SimpleDB — again, less good for Java. It also covers tuning to reduce usage costs, caching using memcached, and ways to batch-update and make serial or parallel requests to SimpleDB. However, it's missing some information that beginners might need, and it's perhaps not quite advanced enough for the more experienced. Downloadable example code is available only for PHP." Keep reading for the rest of Kuan's review. Amazon SimpleDB Developer Guide author Prabhakar Chaganti, Rich Helms pages 252 publisher Packt Publishing rating 6 reviewer Kuan Hon ISBN 1847197345 summary "Getting started" guide to using Amazon's SimpleDB cloud database Say "cloud" to get the attention of CIOs seeking to cut costs in these recessionary times. One well known "database in the cloud" option is Amazon Web Services' SimpleDB, which Amazon describes as "a highly available, flexible, and scalable non-relational data store that offloads the work of database administration."

Those who prefer traditional relational databases could try eg Amazon RDS. This book only covers SimpleDB, a NoSQL or non-relational database. As is well known, NoSQL databases grew in popularity with the growth of large distributed systems and cloud computing, and their proponents tout their scalability and speed.

For anyone wanting a quick primer on NoSQL databases, this book includes a chapter on NoSQL which isn't limited to SimpleDB. It outlines some key conceptual differences between NoSQL and relational database management systems, with pros and cons, using the analogy of "a spreadsheet with some XML characteristics", and illustrating with some concrete examples. That chapter's been made available as a free sample chapter (SimpleDB versus RDBMS), so you can get a flavour of the book.

The contents list for this book is online, I won't recite it here. As well as an overview of SimpleDB, its terminology and advantages, the book goes through signing up with AWS and SimpleDB, and the account access keys. That chapter is also online, as a tutorial.

You may ask, how does this book differ from Amazon's free SimpleDB documentation, which includes a developer guide and a "getting started" guide? Amazon's own "getting started" is certainly helpful, and it's worth downloading and trying their web app scratchpad. But Amazon's detailed developer guide concentrates on REST and SOAP requests, which most people wouldn't want to deal with direct at that low level.

This book's focus is on using the SimpleDB web services API through certain specific languages and libraries — namely Java (JDK6 — using the typical 1.6 library plus several dependencies), Python (2.5 — you need boto), and PHP (with curl). It recommends the SDBtool Firefox extension (SDBizo), which is excellent for checking the results of running the code.

I've tried the book's Java and Python examples, on Windows. Not PHP, as I've not got round to learning PHP yet, though I skimmed the PHP explanations. Similarly, I've not had time to try it all over again on Linux. Generally, the book's coverage seems fuller and better for PHP than for Java or Python. Perhaps it was originally written for PHP, and the rest was bolted on — the stuff for Java more hurriedly than for Python?

The downloadable code samples, as mentioned, are PHP only. They really should have provided downloadable code for all 3 languages, plus some fake MP3 files (see later). If you get the e-book (available in PDF and epub), you can copy and paste the Java or Python code. But that's a tad tedious, especially when the code runs onto a new page, and there are stray end of lines etc that you have to delete manually. Furthermore, the Python code provided is for the interpreter in interactive mode (not for .py files, except a couple towards the end). So, for the Python, you also have to copy/paste each line one at a time. But that still beats having to re-type pages of code in full.

In other words, if you want this book and you're only interested in PHP, you can get away with just buying the hard copy and downloading the code from the Packt site. But if you prefer Python or Java, to save your fingers and blood pressure you should buy just the e-version, or get both paper and e books together. I really hope Packt will in future provide downloadable code samples for all the languages covered.

I have more issues with the sample code given in this book. The typical imports should have been spelled out in the example Java code. Eclipse offers more than one possible import in some cases. It was "try everything till it works", at least until I found this tutorial. I've included the initial required typical imports (though not the standard java.util etc ones) in my own list of points, which I'll say more about at the end of this review. Surely it wouldn't have been difficult to include just those few lines of imports, which could have saved readers a lot of time trying to work out the correct imports. There are also errors in the Python code, and on one page the code that should have been included is missing altogether.

Now, more on the book proper. After the overview described above, this book walks you through the basic SimpleDB operations: how to create a SimpleDB "domain" (equivalent to a worksheet in a spreadsheet), list domains, create/retrieve items (like spreadsheet rows), and delete domains.

Items have attributes (spreadsheet column headings), as key:value pairs — the key is the attribute name, the value is its value, eg address:1 Acacia Avenue. An attribute can have more than one value, eg the same item can have both address:1 Acacia Avenue and address:2 Broadway. The book also lists the SimpleDB constraints on domains, items and attributes — maximum number or size, etc — but it's best to check the AWS site for the latest info.

Code examples are given for each of the 3 languages mentioned. The examples are similar, but don't always cover the same ground. If they'd done that, where possible, it would have been more helpful to those of us trying examples in more than one language. One advantage of a book with associated website is that electronic updates can be published, and it would have been great if that had been done for this book. For instance, the book gave conditional put/delete code examples only for PHP. At the date of this review, boto now supports those features, but sample supplemental Python code for that still hadn't been made available.

SimpleDB stores attribute values as UTF-8 strings. This means that comparisons for sorting or searching are done lexicographically (character by character, left to right, numbers take precedence over uppercase over lowercase), and to handle numbers or dates you have to encode and decode them yourself. So, the book has a chapter explaining lexicographical comparison, data types, and how to encode and decode data to enable proper sorting and comparison of numbers, dates, Boolean values and XML-restricted characters. In the case of numbers this means zero padding and offsets, and there's example code for decoding and encoding numbers. Unlike with PHP and Python, oddly the Java code given was for the body of the typical method that carries out the encoding etc. This could have been omitted, and they should have given example code illustrating the method's usage instead. Similarly for the date formats code.

The SimpleDB query syntax is generally covered well, in a chapter which takes readers through first creating a sample database of song metadata to run queries against. It's not too painful copy/pasting the Java code (3+ pages), but with Python in interactive mode I drew the line at creating every song item and attributes using individual statements, even with pasting, so I just tried adding a couple of random ones to test that the code worked. I say again, full downloadable code please...!

That chapter then gives helpful examples of queries against the sample database and their results, including for more complex combined queries ("and", "or" type queries, "not" etc), and querying for multiple-value attributes. It also provides code examples for sorting and counting query results. But the Java code for retrieving an item's attributes wouldn't run, and I couldn't find the method used (getItemsAttributes()) detailed in the typical documentation; perhaps the book is out of date here?

The book starts going beyond the basics from Chapter 7 onwards, with a chapter on Amazon's S3 storage service — another well known component of Amazon Web Services, where "objects" (files) may be stored in "buckets" (directories), with "keys" used to retrieve objects.

For S3, the book uses JetS3t for Java. However, the Java code given for uploading files to S3 didn't demonstrate any integration with SimpleDB at all — the files were just uploaded with their filenames as the S3 keys, and the code didn't seem to deal with the creation of your own custom S3 keys for uploaded objects. In contrast, the Python code generated the S3 keys for the files from hashes previously produced and stored in the SimpleDB database, as well as dealing with their uploading. In addition, for me the Java code for downloading files from S3 just wouldn't run, and also it wasn't clear where the files were supposed to be downloaded to locally, unlike with the Python example. Inexplicably, there was no info on how to delete objects from S3 buckets, or indeed how to delete buckets. So, while the S3 chapter is of help, it could definitely do with being expanded, especially the Java sections.

Next, money money money. AWS charges are based on usage, so the chapter on tuning and usage costs has some practical value in explaining how SimpleDB is charged for, the "BoxUsage" value returned by requests to SimpleDB, using BoxUsage to optimize queries and compute costs, and how to get BoxUsage values back with your queries using Java, Python etc. There are code examples that, when run, illustrate the different BoxUsage values you get when you use different operators or expressions in queries (eg, using LIKE costs more).

However, partitioning your data into multiple domains is covered in only a few paragraphs, with no code given. I'd have liked to see more info on that, and some sample code for the partitioning process.

To further save money, you can use a cache to store data locally, trying your local cache first; and, only if the data is not there, would your app go out to SimpleDB and incur costs for querying it. This book accordingly has a chapter on how to install and use the popular open source caching system memcached to cache your query results locally. (CacheLite for PHP is also covered.) Again, the Java sections caused me some frustration. The Java test code showed that the memcached server was running properly on my machine, but the Java code for using the cache just didn't work; it ran, but continued to query SimpleDB direct. The Python code, however, worked perfectly — except that, if you're using memcached in Windows, you'll need to use port 11211 instead of what's shown in the book. (I didn't try it in Linux.)

Finally, the book deals with running parallel operations against SimpleDB, using its BatchPutAttributes. The section on updating SimpleDB in Python by making serial consecutive calls to SimpleDB is completely missing the code for the script, but the book does then cover inserting multiple items concurrently into SimpleDB using a threadpool in Java. It also gives sample Python code for alternative ways of parallelising requests: using Python's built-in threading module, threading and queues combined, then threading using the open source workerpool module.

To conclude, in substance the book has a fair amount of useful information on the basics of getting started with SimpleDB, particularly for Python (and probably PHP). But not providing downloadable code samples in Java and Python, or "fake" MP3 files to try S3 uploading/downloading, is a minus.

Some errors, inconsistencies and missing information from the department of "I-wish-they'd-included-this-even-if-they-thought-it-was-basic-as-it's-too-easily-missed-if-it's-not-spelled-out", mean that the book is not really "complete", and not as suitable as it should be for relative beginners — especially for Java and (in whatever language) Windows. It wouldn't take much extra work to get it up to scratch on that front. Perhaps the next edition, or better still an online update/supplement?

For the more experienced, the book doesn't take readers to as advanced a stage as it could have, in my view. In particular, it would have been good to have more info and example code on partitioning data between different domains, and also how to migrate data from an existing database to SimpleDB — their code for "importing" the sample database literally just adds each item and attribute individually.

Fix the errors, add the missing info for beginners, provide downloads of code in all relevant languages and "fake files", and I'd have given it a 7. Provide working sample Java code with more explanation, plus proper integration with S3, an 8. Add fuller info on partitioning, migration, and perhaps even integration with yet more AWS services, a 9.

All opinions are personal to me: half geek, half lawyer, mostly harmless. I'm researching legal issues in cloud computing.

You can purchase Amazon SimpleDB Developer Guide from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Runaway1956 was one of many users to continue to update us about the intrusion we've been following this week. "Sony is warning its millions of PlayStation Network users to watch out for identity-theft scams after hackers breached its security and plundered the user names, passwords, addresses, birth dates, and other information used to register accounts. Sony's stunning admission came six days after the PlayStation Network was taken down following what the company described as an 'external intrusion'. The stolen information may also include payment-card data, purchase history, billing addresses, and security answers used to change passwords, Sony said on Tuesday. The company plans to keep the hacked system offline for the time being, and to restore services gradually. The advisory also applies to users of Sony's related Qriocity network."

Runaway1956 was one of many users to continue to update us about the intrusion we've been following this week. "Sony is warning its millions of PlayStation Network users to watch out for identity-theft scams after hackers breached its security and plundered the user names, passwords, addresses, birth dates, and other information used to register accounts. Sony's stunning admission came six days after the PlayStation Network was taken down following what the company described as an 'external intrusion'. The stolen information may also include payment-card data, purchase history, billing addresses, and security answers used to change passwords, Sony said on Tuesday. The company plans to keep the hacked system offline for the time being, and to restore services gradually. The advisory also applies to users of Sony's related Qriocity network."

Over the weekend, we discussed news that the PlayStation Network had been down for days, with Sony saying little other than that it was caused by an "external intrusion" and that they were "rebuilding their network." Many of you have written to point out that the outage continues, with Sony saying they "don't have an update or timeframe to share at this point." One theory about the cause behind the network's downtime was recently espoused on Reddit by 'chesh,' a moderator at PlayStation-modding enthusiast site PSX-Scene.com. According to him, recently released custom firmware called Rebug allowed people to essentially turn their PS3s into dev consoles, though some features were missing. A different group supposedly used this firmware to get on PSN through the developer networks, and also found that fake credit card numbers were not being validated for game purchases, leading to what chesh called "extreme piracy." He acknowledges that this theory is speculation. Sony's handling of this outage is starting to draw attention from the government. Update: 04/26 20:47 GMT by S : Sony just posted more details, saying that a massive data breach occurred: An "unauthorized person" has PSN users' "name, address (city, state, zip), country, email address, birthdate, PlayStation Network/Qriocity password and login, and handle/PSN online ID." Billing address, password questions, and credit card info may also have been taken.

Over the weekend, we discussed news that the PlayStation Network had been down for days, with Sony saying little other than that it was caused by an "external intrusion" and that they were "rebuilding their network." Many of you have written to point out that the outage continues, with Sony saying they "don't have an update or timeframe to share at this point." One theory about the cause behind the network's downtime was recently espoused on Reddit by 'chesh,' a moderator at PlayStation-modding enthusiast site PSX-Scene.com. According to him, recently released custom firmware called Rebug allowed people to essentially turn their PS3s into dev consoles, though some features were missing. A different group supposedly used this firmware to get on PSN through the developer networks, and also found that fake credit card numbers were not being validated for game purchases, leading to what chesh called "extreme piracy." He acknowledges that this theory is speculation. Sony's handling of this outage is starting to draw attention from the government. Update: 04/26 20:47 GMT by S : Sony just posted more details, saying that a massive data breach occurred: An "unauthorized person" has PSN users' "name, address (city, state, zip), country, email address, birthdate, PlayStation Network/Qriocity password and login, and handle/PSN online ID." Billing address, password questions, and credit card info may also have been taken.

fysdt writes "There has obviously been a lot of discussion about last week's disclosure that iOS devices are maintaining an easily-accessible database tracking the movements of users dating back to the introduction of iOS 4 a year ago. The issue has garnered the attention of US elected officials and has played fairly heavily in the mainstream press. One MacRumors reader emailed Apple CEO Steve Jobs asking for clarification on the issue while hinting about a switch to Android if adequate explanations are not forthcoming. Jobs reportedly responded, turning the tables by claiming both that Apple does not track users and that Android does, while referring to the information about iOS shared in the media as 'false.'" Apple has now been hit with a class-action lawsuit over the location-tracking issue.

brothke writes "Network Security Auditing is touted as the complete guide to auditing security, measuring risk, and promoting compliance. The book lives up to its promise and is a comprehensive reference to all things network security audit related." Read below for the rest of Ben's review. Network Security Auditing author Chris Jackson pages 528 publisher Cisco Press rating 9/10 reviewer Ben Rothke ISBN 1587053527 summary Excellent highly technical and detailed reference At almost 450 pages, the book covers all of the key areas around network security that is of relevance to those working in information security. As a Cisco Press title, written by a Cisco technical solutions architect, the book naturally has a heavy Cisco slant to it. Nonetheless, it is still an excellence reference even for those not working in a Cisco environment. While the first 3 chapters of the book provide an overview that is great even for a security newbie, the overall style of the book is highly technical and comprehensive.

Chapters 1-3 provide an introduction to the principles of auditing, information security and the law, and governance, frameworks and standards. Each chapter is backed with a significant amount of information and the reader is presented with a thorough overview of the concepts.

Chapter 3 does a good job of providing the reader with the details of current frameworks and standards, including PCI DSS, ITIL, ISO 17799/27001 and others. Author Chris Jackson does a good job of explaining the differences between them and where they are best used. Given this is a Cisco-centric book, he also shows how the various Cisco security products can be integrated for such regulatory and standards support.

Throughout the book, the author makes excellent use of many auditing checklists for each area that can be used to quickly ascertain the level of security audit compliance.

Chapter 6 is perhaps the best chapter in the book on the topic of Policy, Compliance and Management, and the author provides an exceptionally good overview of the need for auditing security policies. This is a critical area as far too many organizations create an initial set of information security policies, but subsequently never take the time to go back and see if they are indeed effective and providing the necessary levels of data protection.

Jackson notes that accessing the effectiveness of a policy requires the auditor to look at the policy from the viewpoint of those who will interpreting its meaning. A well intentioned policy might recommend a particular course of action, but unless specific actions are required, there is little an organization can expect the policy to actually accomplish to help the organization protect its data assets if it is misinterpreted.

The chapter suggests that the auditor ask questions such as: is the policy implementable, enforceable, easy to understand, based on risk, in line with business objectives, cost effective, effectively communicated and more. If these criteria are not well-defined and delineated, then the policies will exist in text only, offering little information security protection to the organization.

Jackson also writes of the need to measure how well policies are implemented as part of a security assessment. He suggested using a maturity model as a way to gauge if the organization is in its evolution towards fully integrating security into its business process or if it already has a formal integration process in place.

In chapter 8 on Perimeter Intrusion Prevention, Jackson writes that protecting a network perimeter used to be a relatively easy task. All an organization would have to do is stick a firewall on its Internet connection, lock down the unused ports and monitor activity. But in most corporate networks today, the perimeter has been significantly collapsed. If you compound that with increased connectivity, third-party access, and more; and then bring in advanced persistent threats into the equation, it is no longer a simple endeavor to protect a network.

Chapter 8 provides detailed framework on how to perform a perimeter design review and assessment. As part of the overall review, the chapter details other aspects of the assessment including the need for reviews of the logical and physical architectures, in addition to a review of the firewall. Jackson also lists a large number of security tools that can be used to during an audit.

Chapter 11 covers endpoint protection with a focus on the end-user. Jackson notes that users never cease to amaze with their abilities to disappoint by opening suspicious file attachments, running untrusted Facebook applications, and much more. The book notes that organizations today face significantly higher levels of risk from endpoint security breaches than ever before due to our highly mobile and connected workforce.

The chapter details an endpoint protection operational control review that can be used to assess the organizations processes for identifying threats and performing proactive management of endpoint devices. While the chapter is quite Cisco-centric, with references to the Cisco SIO (Security Intelligence Operations) and a number of other Cisco products, the chapter does provide a good overview of the fundamentals of endpoint protection and how to do it the right way.

Overall, Network Security Auditing is highly technical and detailed reference that makes for an excellent primary reference on the fundamental of information security. With ample amounts of checklist, coding references, detailed diagrams and just the right amount of screen shots, it makes an excellent guide that any member of an IT or security group should find quite informative.

Ben Rothke is the author of Computer Security: 20 Things Every Employee Should Know

You can purchase Network Security Auditing from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

brothke writes "Network Security Auditing is touted as the complete guide to auditing security, measuring risk, and promoting compliance. The book lives up to its promise and is a comprehensive reference to all things network security audit related." Read below for the rest of Ben's review. Network Security Auditing author Chris Jackson pages 528 publisher Cisco Press rating 9/10 reviewer Ben Rothke ISBN 1587053527 summary Excellent highly technical and detailed reference At almost 450 pages, the book covers all of the key areas around network security that is of relevance to those working in information security. As a Cisco Press title, written by a Cisco technical solutions architect, the book naturally has a heavy Cisco slant to it. Nonetheless, it is still an excellence reference even for those not working in a Cisco environment. While the first 3 chapters of the book provide an overview that is great even for a security newbie, the overall style of the book is highly technical and comprehensive.

Chapters 1-3 provide an introduction to the principles of auditing, information security and the law, and governance, frameworks and standards. Each chapter is backed with a significant amount of information and the reader is presented with a thorough overview of the concepts.

Chapter 3 does a good job of providing the reader with the details of current frameworks and standards, including PCI DSS, ITIL, ISO 17799/27001 and others. Author Chris Jackson does a good job of explaining the differences between them and where they are best used. Given this is a Cisco-centric book, he also shows how the various Cisco security products can be integrated for such regulatory and standards support.

Throughout the book, the author makes excellent use of many auditing checklists for each area that can be used to quickly ascertain the level of security audit compliance.

Chapter 6 is perhaps the best chapter in the book on the topic of Policy, Compliance and Management, and the author provides an exceptionally good overview of the need for auditing security policies. This is a critical area as far too many organizations create an initial set of information security policies, but subsequently never take the time to go back and see if they are indeed effective and providing the necessary levels of data protection.

Jackson notes that accessing the effectiveness of a policy requires the auditor to look at the policy from the viewpoint of those who will interpreting its meaning. A well intentioned policy might recommend a particular course of action, but unless specific actions are required, there is little an organization can expect the policy to actually accomplish to help the organization protect its data assets if it is misinterpreted.

The chapter suggests that the auditor ask questions such as: is the policy implementable, enforceable, easy to understand, based on risk, in line with business objectives, cost effective, effectively communicated and more. If these criteria are not well-defined and delineated, then the policies will exist in text only, offering little information security protection to the organization.

Jackson also writes of the need to measure how well policies are implemented as part of a security assessment. He suggested using a maturity model as a way to gauge if the organization is in its evolution towards fully integrating security into its business process or if it already has a formal integration process in place.

In chapter 8 on Perimeter Intrusion Prevention, Jackson writes that protecting a network perimeter used to be a relatively easy task. All an organization would have to do is stick a firewall on its Internet connection, lock down the unused ports and monitor activity. But in most corporate networks today, the perimeter has been significantly collapsed. If you compound that with increased connectivity, third-party access, and more; and then bring in advanced persistent threats into the equation, it is no longer a simple endeavor to protect a network.

Chapter 8 provides detailed framework on how to perform a perimeter design review and assessment. As part of the overall review, the chapter details other aspects of the assessment including the need for reviews of the logical and physical architectures, in addition to a review of the firewall. Jackson also lists a large number of security tools that can be used to during an audit.

Chapter 11 covers endpoint protection with a focus on the end-user. Jackson notes that users never cease to amaze with their abilities to disappoint by opening suspicious file attachments, running untrusted Facebook applications, and much more. The book notes that organizations today face significantly higher levels of risk from endpoint security breaches than ever before due to our highly mobile and connected workforce.

The chapter details an endpoint protection operational control review that can be used to assess the organizations processes for identifying threats and performing proactive management of endpoint devices. While the chapter is quite Cisco-centric, with references to the Cisco SIO (Security Intelligence Operations) and a number of other Cisco products, the chapter does provide a good overview of the fundamentals of endpoint protection and how to do it the right way.

Overall, Network Security Auditing is highly technical and detailed reference that makes for an excellent primary reference on the fundamental of information security. With ample amounts of checklist, coding references, detailed diagrams and just the right amount of screen shots, it makes an excellent guide that any member of an IT or security group should find quite informative.

Ben Rothke is the author of Computer Security: 20 Things Every Employee Should Know

You can purchase Network Security Auditing from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Hugh Pickens writes "Copyright troll Righthaven makes their money by coercing defendants of alleged copyright infringement into settling with them with threats of $150,000 in damages and forfeiture of the defendants' website domain names. Now EFF reports that Chief Judge Hunt of the federal court in Nevada, which is overseeing more than 200 Righthaven copyright cases, has dismissed Righthaven's merit-less claim to seize its victim's domain names. Righthaven contended that the mere hosting of any infringing material meant that the entire domain name was forfeit but the judge rejected that claim, explaining that the 'Court finds that Righthaven's request for such relief fails as a matter of law and is dismissed.' But now Righthaven has filed a new copyright case in Nevada federal court that not only demands forfeiture of the domain name but has asked the Court to 'order the surrender to Righthaven of all hardware, software, electronic media and domains, including the Domain used to store, disseminate and display the unauthorized versions of any and all copyrighted works.' The new complaint also asserts that Righthaven holds the 'exclusive rights' to Stephens Media news articles, despite the Strategic Alliance Agreement showing that Stephens Media retains these rights."

alphadogg writes "The outage of Sony's PlayStation Network and Qriocity service, now in its fourth day, looks set to continue after the company said on Sunday that it is 'rebuilding' its system to better guard against attacks. Sony said on Saturday that the outage was caused by an 'external intrusion' into the network, but has yet to detail the problem. The PlayStation Network is used for PlayStation 3 online gaming and sales of software to consoles and the PlayStation Portable. The Qriocity service runs on the same network infrastructure and provides audio and video to Sony consumer electronics products."

An anonymous reader writes "Apple's iPhone 3G, iPhone 3GS, the iPhone 4, and iPad models are keeping track of consumers whereabouts. Mac computers running Snow Leopard and even Windows computers running Safari 5 are being watched. But the question is why? 'To provide the high quality products and services that its customers demand, Apple must have access to the comprehensive location-based information,' Apple says."

Several readers have noted that outages on Sony's PlayStation Network have prevented online play for the past few days. The company has now blamed an 'external intrusion' for the trouble, saying they took down the network to "conduct a thorough investigation and to verify the smooth and secure operation of our network services going forward." Some suspect an attack by Anonymous, who declared war on Sony earlier this month, but Anonymous has disavowed knowledge of such an attack. Meanwhile, others are asking whether Sony should compensate users for the inability to play PS3 multiplayer modes, and even single-player modes on a few downloadable games.

An anonymous reader writes "Much has been written about the recent EC2/EBS outage, but Keir Thomas at PC World has a different take: it's shown how much cutting-edge Internet infrastructure relies on Amazon, and we should be grateful. Quoting: 'Amazon is a personification of the spirit of the Internet, which is one of true democracy, access to the means of distribution, and rapid evolution.'" An article at O'Reilly comes to a similarly positive conclusion from a different angle.

doperative writes with this quote from a speculative piece at Business Insider about Apple's real motive behind its recent lawsuit against Samsung's Galaxy devices: "Android is free. In some cases, it's even cheaper than free, with Google sharing some revenue from Google searches on Android phones with partners. This is hugely disruptive to both Microsoft and Apple's business models; Microsoft because they make money on software licenses, and Apple on hardware. And this disruptive approach is winning: Android is surging past iOS in marketshare. A lawsuit from a big company, even if doomed, still takes a lot of time, energy and money to fight off. So Samsung or someone else might settle, accepting to pay some form of license. If that happens, Apple can go around to the other manufacturers asking for the same license and have a much stronger claim. And now OEMs have to factor that cost into the decision to choose Android. And all of a sudden, Android has a price." Samsung has fired back with a lawsuit of its own.

don9030582 writes "After Google announced it would permanently shutter its Google Videos collection, dozens of volunteers from around the world sprung into action in a massive effort to make a copy of the entire site. It was originally slated to go dark on April 29th, but now they have eliminated any such deadline and furthermore they will be migrating the collection to YouTube. We wish Google would have planned to do that from the beginning, but ultimately this is a victory for the preservation of user-generated content on the Internet."