Slashdot Mirror
Big Talk About Small Samples
The smallest sample I've ever used to make an argument was when I submitted some legal briefs, each no longer than five pages, in the anti-spam cases that I'd been filing in Washington State small claims court. Since I suspected the judges were not taking the cases seriously, I filed the briefs with the third and fourth pages stuck together in the center, by a tiny thread of paper joining the back of the third page to the front of the fourth page. (If someone were to turn the pages and actually readthe brief, the thread would break.) I did something similar in six different cases, and when the motions were all rejected, I went to the courthouse to look at the paper motions still in the file. In three out of six cases, the judge had rejected the motion without reading it first.
Now, the point was not to make any accurate estimation of the actual proportion, in the total population of small claims court judges, who would reject a brief in an anti-spam case without reading it. There's no basis for saying that the proportion of such judges is close to 50%. But we can still probably reject any contention that the proportion of such judges is very low. If only 10% of judges were rejecting motions without reading them, then there is only about a 1.4% chance of taking a random sample of six rejected motions and finding that in three or more cases, the judge did not read the motion. Even if 20% of judges were doing so, for an event with a probability of p=0.20 you would still only see it occur in three out of six cases, about 8.2% of the time. (If an event has probability p, the exact probability of that event occuring three or more times in six trials is given by 20*(p^3)*((1-p)^3) + 15*(p^4)*((1-p)^2) + 6*(p^5)*((1-p)^1) + 1*(p^6)*((1-p)^0).) So we can say that the proportion of such judges is quite probably more than 20%. I did this repeatedly because even after I had "caught" the first judge, I wanted to head off any objection that this was just an isolated case of rare behavior.
And, as always, it's important not to generalize too much about the behavior whose probability we're estimating. I don't think that 20% or more of judges, even in small claims court, are throwing most types of cases without reading or listening to the arguments. My impression was that most judges see view small claims court as a place to redress injustices, and that they see anti-spam and anti-telemarketer plaintiffs as just trying to "make money" at it, so they take those suits less seriously. I disagreed with this stance because (1) anti-spam plaintiffs usually really have been harmed and are not just "whining about one email" which they are trying to "cash in" (I still get so much spam that it interferes somewhat with the operation of my server and with my ability to get through my daily email); and (2) the law is intended after all as a deterrent, with disproportionate damages in order to discourage spammers from spamming in the first place. However, the charitable reading of the results is to assume that judges are merely biased against anti-spam plaintiffs -- but at least they probably don't treat all cases as casually as they treat anti-spam suits!
Back to the issue of small samples. My previous article was prompted by an editorial about the online response that had been elicited by two different photos -- one showing a black woman breastfeeding, and a nearly identical photo showing a white woman breastfeeding. The author asserted that the photos had received vastly different responses, which she attributed to racism. I presented a survey to a sample if users recruited from Amazon's Mechanical Turk, randomly showed each survey-taker one of the two photos, and asked:
Our academic department has asked everyone to submit a "fun" photo of themselves, so that our photos can be displayed together on the department home page. One of our employees submitted a photo that has caused some internal debate about whether the photo is inappropriate. I wanted to do a poll to get the opinion of a random sample of Internet users of different backgrounds.
Do you think this is an appropriate picture to be used in a photo collection on our academic department home page?
Out of 47 respondents who saw the black woman's photo, 36 of them (77%) said it was inappropriate. Out of 54 respondents who saw the white woman's photo, 38 of them (70%) said it was inappropriate.
As before, these samples are to small to say precisely what the relevant proportions in the background populations are, but we can probably reject certain statements about the populations -- for example, that the percentage of users offended by the black woman's photo is 20 percentage points higher than the percentage of users offended by the white woman's photo. This is where the counterintuitive part comes in. Suppose that in the background population, 81% of respondents would find the black woman's photo offensive, but only 61% would be offended by the white woman's photo. What are the odds of getting 77% or less "yes that's offensive" responses from a sample of 47 users shown the black woman's photo, and getting 70% or more "yes that's offensive" responses from a sample of 54 users shown the white woman's photo? It doesn't sound unlikely at all, because the percentages are quite close to the originals -- but you can verify, either with statistical calculations or with a quickly written computer program, that the odds are only about 2.5%.
Two main factors contribute to this counterintuitive result. First, even with a sample size of a few dozen, the frequency of an event starts to tend very closely to the frequency in the background population (if 80% of your population has some trait, and you take a sample of size 50, there's about a 95% chance that the number with that trait in your population will be between 34 and 46). Second, to find the odds of seeing both of these deviations at the same time (deviating from an assumed 81% in the background population down to 77% in the first sample, and deviating from an assumed 61% in the background population up to 70% in the second sample), you have to mutiply the probabilities of these two unlikely events. The probability of the first deviation is about 19%, the probability of the second is about 13%, and so the probability of them both occurring is about 2.5%.
The reason I calculated the odds of getting 77% or less "offended" responses for the black woman's photo while also getting 70% or more "offended" responses for the white woman's photo, is that in calculating the "unlikeliness" of a statistical result, it's customary to calculate the odds of getting "this result or a more extreme one". For example, suppose you want to know if a company's hiring process is gender-balanced (assuming a 50/50 gender split in the population), and you notice that in a random sample of 100 recent hires, 61 were men. You wouldn't ask "What are the odds of there being exactly 61 men in this sample?", because the odds of getting any particular number, are small. You'd ask, "What are the odds of getting this result or a more extreme one -- i.e. the odds of getting 61 or more men out of a random sample of 100, if the population were truly gender-balanced? As this calculation tool shows, the odds are only about 1.7%.
Similarly, in the case of the two populations being measured, the author of the original editorial hypothesized that there was some significant gap between the percentages of the population that were offended by the two photos, which I arbitrarily assumed to be 20 percentage points. Under that assumption, showing the two pictures to two different groups and having them be offended at similar rates, is the unexpected, "extreme" result, and the closer the rates are to each other, the more extreme the result is. That's why I calculated "77% of less" for the first group vs. "70% or more" for the second group.
And out of the pairs of numbers that I tested which were separated by 20 percentage points, 81% and 61% were the numbers which made the given result the least unlikely. 80/60 and 79/59 give odds of about 2.5% and 2.4%; 82/62 and 83/63 give odds of 2.4% and 2.2%.
You can do the statistical calculations directly, but in case you won't believe it unless you see the results unfold with your own eyes, you can run this perl script, which iterates through a million trials of the experiment, counting the number of times that the unexpected result occurs.
Why did I assume a 20-point gap? That was the most subjective leap that I made. Looking through the original editorial, I figured that on the basis of inflammatory statements like
"Only one woman was called 'adorable' by the media and portrayed with girlish innocence, and it wasn't the black one. It never is."
and
"The contrast in headlines is so stark, it deserves to be examined" [I assume here she meant the contrast in responses]
the author meant to imply a difference in people's attitudes that was at least that large. But the results suggest that it isn't.
For all of this effort, of course, I could have just expanded the original experiment to a sample of several hundred and mollified some people's concerns. But I wanted to argue for what you can show, even with small samples, because I would like to try (and would like others to try) similar experiments in the future, and do not think people should be discouraged if they can't afford to pay a thousand Amazon Mechanical Turk workers to take their survey. I paid my 100 respondents $0.25 each; naturally, one experiment I'd like to do soon is to figure out what's the lowest I can get away with paying them.
Fried post.
Slashdot is trying to move their user base from news for nerds and geeks to news for normals.
Seriously, I've noticed the Register getting more active as people move over there.
We, geeks, view this entire article as a bunch of shenanigans that waste our time. Please stop spitting in my face.
Give me an article about Intel latest and greatest chipset plans or how AMD screwed the poorch or about how one can modify a blackberry to run android applications. Those things are Useful.
\
Infotainment designed to incite does not nor should enter my world, it makes my world more stressful and wastes my time.
In a recent poll conducted randomly via the Internet among people who are girl gamers, we found that 99 percent think breastfeeding images are none of your business.
Equally sound on a statistical basis, and just as randomized, with a t value of 42.
-- Tigger warning: This post may contain tiggers! --
.
He already wasted ten minutes of my life with his last episode of keyboard effluent, why should I waste my time with him anymore?
Oh no it's Bennett. Go get a blogger account. This tripe is worthless.
I guess it's kinda cool that you took over what use to be a major tech-news website and turned it into your personal blog.
That way the rest of us don't have to hear about his bullshit.
General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
Is sounds like Haselton is missing the point, which is why people oppose to see women breastfeeding to begin with.
Linux is for people who don't mind RTFM.
If you like to use sample sizes that are too small, then I would like to interest you in another useful technique.
Correlation is causation.
For example:
The tides cause the moon. The correlation proves it.
Similarly, murder rates are higher in the summer, and ice cream sales are higher in summer months. Therefore ice cream causes murder.
I hope that was helpful.
I'll see your senator, and I'll raise you two judges.
There's a really good book that talks about brevity and how to communicate your ideas more concisely with fewer words. I suggest Bennett read it.
"First they came for the slanderers and i said nothing."
I'm sorry, but this is getting absurd.
If Slashdot is going to be Bennett "aint I smug and pointless" Haselton's personal blog ....
Give us a STORY EXCLSUION for this clown.
I do not see value in Bennett and hit shit, and I don't care.
But apparently at least samzenpus and timothy with post any of the shit this idiot writes.
Seriously, just fucking make it stop. Nobody here gives a shit about Bennett Haselton. So give us a fucking way to stop reading his crap.
Lost at C:>. Found at C.
It might be different if Bennet were a frequent poster and would be actively engaged in discussions, but he's not. He's just some guy who once heard that brevity is the soul of wit and went off to write ten thousand words explaining what it meant.
I am TheRaven on Soylent News
Poisson statistics. I have to wonder if Mr. Haselton has ever heard of the term.
If by some weird alignment of forces I were to become a Judge, and Mr. Haselton presented this to me in a brief, I would try and have him disbarred for abuse of statistical process. I know that the actual legal profession is soft about such abuses, but by God they wouldn't be in my courtroom.
I don't read his long articles, generally speaking, but he has been an advocate against censorship and I respect that much.
No one makes anyone read the articles, and without even checking, I'd guess you can configure /. not to even show them.
The Haselton hate reminds me of the Jon Katz days, which is kind of amusing ;)
...complain about another useless Bennett Hasselton post.
FTFY
Come on, it'd be easy.
Slashcode is Open Source. Start a new blog. I liked your old blog. I don't like Bennett Haselton's blog.
What is this bizarre Slashdot alternate universe, where uninteresting shitposting becomes the headline article? This troll couldn't even reply to his own stupid post, had to make a new one to explain himself.
Make it stop, dear God make it stop!
Letter To Iran
Your sample size is the least of my problems.
FOR FUCK'S SAKE SLASHDOT, MAKE IT STOP!
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
This is the last straw. Someone needs to slash Bennett's tires. Who's with me?
Know your audience. Armchair social science on Slashdot? /facepalm
Slashdot by now has OBVIOUSLY seen how much we don't like this guy. The fact that they keep posting him means they're just trolling us, or going for pageviews, or both. Or maybe Bennett has some kind of deal with the site, or has something on one of the editors. Whatever. I don't care. From now on, NO ONE post any comments on one of his stories. Not even to say how much you hate his stories. This will be my last comment on one of his stories. Hope this takes!
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
But we can still probably reject any contention that the proportion of such judges is very low. If only 10% of judges were rejecting motions without reading them, then there is only about a 1.4% chance of taking a random sample of six rejected motions and finding that in three or more cases, the judge did not read the motion.
But you DIDN'T HAVE A RANDOM SAMPLE. In particular, you had a sample from Washington State small claims court. So you can ONLY draw conclusions about Washington State small claims court. You have no idea what happens in New York, or in England. But that's only one example of how non-random your sample was. The problem is, ANY small sample is going to have non-random attributes, because it's a small sample. You can roll a dice three times and the results will appear highly non-random - no instances at all of some values - you have to roll it a hundred times to get a good distribution and the dice is random. If you start with a non-random dice - like your "sampling only from one court" or your "using Mechanical Turk" - your small sample size gives you results that are simply MEANINGLESS.
Go and study stats and stop posting this drivel on Slashdot where people might believe it.
This used to be a place for nerds to read things related to tech and science. In those glory days of old one could simply provide a "confidence interval" rather than spew out four paragraphs that badly describes what a confidence interval is. Those of us not familiar with the idea had the meager brain power to look it up.
By the way, when I say it was described "badly" what I mean is that it is a completely hopeless, tangled mess that even I, a statistics geek just gave up part way into that contorted mess of rotted simpleminded tripe. If you are going to treat us like morons at least use language that resembles something closer to the language we learned in school that has some basic structure including subjects and verbs. They do not even have to agree which tells you how low our standards are.
Bennett, try this experiment.
Make a program that flips 54 coins and notes the number of heads and the number of tails at each round. Then run this program for one million rounds.
When you're done, note the number of rounds the random generator saw 38 or more heads and frame this as a proportion; ie - "the random generator reached this level X% of the time".
Then compare your results with the random generator. If your results are unlikely to come from the random generator, then perhaps you have something.
Now, " unlikely" is an arbitrary measure with no compelling foundation (it's the wrong measure to determine the significance of a result(*)), but in scientific circles we use a "rule of thumb": results are considered significant when they are less likely than 95% of the random results.
Even at this level, we expect 1-in-20 studies to be due to random chance, but then follow-on studies should confirm or deny the findings (and 1-in-20x20 of *those* will be due to random chance as well).
If the results might lead to potentially catastrophic decisions we might use a higher level of significance; for example, 99% confidence when deciding whether a drug is safe. Physics uses an insanely high level of confidence.
Try that and get back to us - we await your next post with baited breath.
(*) The correct measure is the number of bits saved by compressing the original data by factoring out the result (glossing over some details).
Bennett, what the hell are you doing? This is a straightforward difference between proportions test. In R it's simply
> prop.test(c(36,38),c(47,54))
which gives a 95% confidence interval for the difference as (-0.13 to 0.25), meaning loosely that we wouldn't be terribly surprised if the white women were thought to be inappropriate at 13 percentage points higher or if the black women were thought to be inappropriate at 25 percentage points higher.
Hassle ton of Slashdot readers, get flamed in the comments. Seems to be a pattern, huh editors?
If tylervigen.com has tought me anything it's that you are 100% correct.
For example, did you know?
Motorcycle riders killed in collision with stationary object correlates with Corporate Political Action Committees (US)
Obviously PACs are bad for motorcyclists!
Creationist Textbook Stickers Declared Unconstitutional by CowboyNeal
Bennett Haselton (born November 20, 1978) is a frequent commenter on the website Slashdot.org, where he is widely disliked by readers.
Haselton has gone meta.
Two photos only.
Differences in the small number (2) of photos that might be just as relevant:
1) One is very public (breastfeeding in a crowd), the other not.
2) One is standing, the other is sitting.
3) One looks at the child, the other at the camera.
4) One looks like it was scanned the other is crisp.
5) There is a difference in weight.
6) One is married.
There are probably dozens more.
I mean, a picture of a black woman or a white woman breast-feeding her baby wouldn't interest me.
A picture of a black woman or a white woman simultaneously breast-feeding both George W. Bush and Bill Clinton would interest me. That would be a hoot and a half.
Quotes from the romp . . .
"I did NOT suckle on that woman!"
"Who said anything about breast milk costing $4 a gallon?"
I've tried to offend both major political parties in the US with this post. I could try to also offend the Libertarians, Greens, or Tea Party folks . . . but there don't seem to be enough breasts to go around!
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
More PACs -> more government contracts to place stationary objects (eg construction sites) near roads
To me, the difference that stood out between the two pictures was that the black woman was apparently breast feeding in a public place with people around whereas the white woman was not. I think this contributes to the result, making the black woman appear more brazen.
my gripe about the first story wasn't the small sample size. It's the source of the sample. The scope of the question seemed framed in terms of US society. Who knows where you are sampling on mechanical turk?
Even a large sample can be bad is the sample doesn't represent the population as a whole. Sample size only goes so far when you have a skewed sample based on the demographics of the population as a whole.
Slashdot is not a your blog. Go away.
See? Even AC agrees. And, just like Bennett Haselton, he's always right!
Creationist Textbook Stickers Declared Unconstitutional by CowboyNeal
seriously, like there's a whole field called statistics that figured this shit out millenia ago.
You want to see a more meaningful sample size? Look at the number of comments in Bennett's "submissions" that are complaining about this waste of time. Compare that to the number that actually gives a shit. /. now needs to post a whiny follow-up piece???
It was bad enough that the first sorry the other day had NOTHING whatsoever to do with news for nerds, nor was it well written, nor was it well conducted.
But
Few people care about this Miley Cyrus' opinion on things that do matter, and fewer still care about his opinion on all the crap that doesn't matter.
Breastfeeding pictures? Burning Man parking? Burning Man Ice distribution? How come 5th Ammendment?
Fuck this clown.
"and a nearly identical photo showing a white woman breastfeeding. "
No, they are not.
There are at least 3 HUGE differences.
In the first image (the "black" lady) the woman is CLEARLY out doors in the middle of a crowd, and more importantly is staring DIRECTLY at the camera AND is clearly engaged in doing something else/in addition to feeding her (one presumes) baby.
In the second image (the "white" lady) the woman is *NOT* clearly in the outdoors or in the middle of the crowd, and is looking down at her (one presumes) baby, and appears to be doing nothing else other than feeding the baby.
Both of these differences are INCREDIBLY important. The first one picture draws strong associations to NatGeo type photographs of aboriginals/tribal africans. The second shows a woman working against stereotypes (getting a degree/diploma after having a child).
It would be interesting to stage 4 of these photos putting the mothers in identical *backgrounds* in addition to the black/white color difference.
Wikipedia is not a blog, but Slashdot is not Wikipedia. Plenty of newspapers and the like have in-house opinion columnists and other writers producing exclusive original content that distinguishes each publication from other AP/Reuters aggregators.
... because every time he posts, it's the great American novel, about the page count of War and Peace, and I read one once.
ONCE!
Sure, his stuff is fucked up, but it's become an iconic meme and I love to see it appear.
Scanning through the comments is a pop-corn and Dr. Pepper moment and humor abounds nd I m greatly amused.
To Bennett Haselton: I want you to have my babies and stuff.
It little behooves the best of us to comment on the rest of us.
Dude, it isn't that you had a small sample size, it's that you're extrapolating from independent contexts. There's a large difference between what people will say on posts IN facebook among friends (i.e. a semblance of privacy) versus what some warm body clicks for a quarter on mechanical turk (no semblance of privacy and perhaps an intent to please the questioner and adjust to the bias of the question).
... and still coming up short.
....
Your "fun picture" at a "department party" or whatever (I RTFirstA but not this justification drivel) attempts to locate the issue in the image, but fails to account for the different contexts and motivations. It also fails to differentiate between 'do you like/are you offended by this' versus 'would you share this and what would you say about it'.
Your "experiment" is akin to trying to generalize that ballet shoes (liking/offensiveness of these images) aren't very good (no bias) because you played basketball in them (paid someone to answer a survey). And the metaphor in that last sentence was deliberately contorted as an example of how you structured your experiment incorrectly and extrapolated the results from an improper experiment.
Throwing stats about sample sizes, percentages and previous glory mean nothing if you're measuring the wrong thing
This is coming from a husband of a two breastfed kids who only says in public that I agreed with breastfeeding so I'd see more nipplage around the house
The outcome of any serious research can only be to make two questions grow where only one grew before. - Thorstein
For lowering the editorial quality of the /. website.
Bennett-- you're an idiot. Get over it. If I ever meet you in real life, and I'm sure I'll probably get the chance, I'm going to call you an idiot to your face.
Also, it looks like it's time to move on-- fuck you, dice. I've loved Slashdot for over a decade and now I have to go. Between Hackernews being a bunch of VC loving twats and Slashdot being bent over by those same twats (more or less), being a nerd and finding stuff that matters is becoming impossible.
But the posts so far obviously show that people don't understand statistics in general much less the confidence interval of a stat. Our thinking as devolved into cliches like "Correlation is not causation" and "Sample size is too small" without understanding what those really mean.
Personally I think it would be better if he ditched the Judge part at the beginning and explained the statistical parts first. Once people's emotions get set because of the subject matter there is no more room to deal with the logical parts of this.
Personally I would actually like to see a defense of a Mechanical Turk Survey as being random inside it's population and that it's population is a good representation of internet users as a whole or better yet some country or countries.
Add this to your favorite userscript-handling add-on:
window.onload = function () {
$("a[href='/tag/bennett'],a[href='/tag/haselton']").parents("article").hide()
}
And if an un-tagged article slips through, make sure you tag it for the rest of us!
Good riddance assholes.
I will never visit slashdot again. I've deleted it from my web history on all my devices, from all of my bookmarks.
Even though this course has "public health" in the title, it is really quite generic. The methods used and very(!) well explained by the very likable John McGready (Johns Hopkins University) are exactly the same as what is relevant to understand for what is being discussed here.
Statistical Reasoning for Public Health 1: Estimation, Inference, & Interpretation
A conceptual and interpretive public health approach to some of the most commonly used methods from basic statistics.
https://www.coursera.org/cours...
This post is a prime example of statistical sophistry while ignoring uncontrolled factors: the backgrounds (settings) of the two photos are drastically different, not just the color of the two mothers. Not that nursing a baby should be inappropriate in any situation, but social crazies about such is very situation dependent...
now i know how the onion does it... they had an exclusive deal with bennett... they basically transcribed his view of the world.
voila, humor... unless you're the one that needs to deal with the crazy fucker.
I'm not sure why /. is posting the ramblings of a non-researcher, non-statistician as though he knew what he was talking about.
Has it gotten to this? Really?
Bennett took 1,800+ words to describe what a normal research would take under 100 to say. This is what happens when someone thinks they know what they're talking about, and need to rationalize the heck out of it in order to make sense.
I'd like to know what percentage of people at Burning Man are offended by various women breastfeeding, and also how we can optimize the queues so that everyone can see it happen without waiting too long.
William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
I did a Bayesian analysis, using independent uniform priors for the proportion offended in each case. Let B be the proportion offended by the photo of the black woman, and W be the proportion offended by the photo of the white woman. I find
* a 75% probability that B > W;
* a 32% probability that B > W + 0.10 (i.e., 10% or more higher);
* a 5% probability that B > W + 0.20 (i.e., 20% or more higher).
Technical details: The posterior distribution for B is a beta distribution with shape parameters 37 and 12; the posterior distribution for W is a beta distribution with shape parameters 39 and 17. I computed Pr(B > W) by taking a million draws from each distribution, and computing how often the draw from the distribution for B was larger than the draw from the distribution for W. Likewise for the other two questions.
There remains a problem: the sample cannot reasonably be assumed to be representative. You could correct for this by gathering demographic information on the survey respondents and using that information to correct the results, using regression with post-stratification.
Sir Humphrey Explains it All.
Seriously, his long post crashed my RSS app (OSX NewsBar). I've seen this once before. It won't refresh Slashdot again until the article falls out of range.
The sample was a set of legal briefs, but the conclusions were about judges. Small samples may work, but you can't sample population A to make an inference about unrelated population B.
By analogy, the fact that my ice cream truck only sells half as much ice cream as I expect doesn't tell me that there aren't many kids in the neighborhood. Maybe my prices are crazy. Maybe my only flavor is chocolate-chutney ripple. Maybe the scary clown on the top of my truck frightens children away. From looking at my inventory, there's probably not enough information to tell.
The fact that judges didn't read pages and 3 and 4 of the briefs could be because the documents were late, incorrectly presented, or manifestly incorrect on the first page.
No need to read the rest.
You can say that but you are wrong.
With a small, non-random sample you cannot say ANYTHING about anything.
Random is not the same as non-random.
A small sample size that is random is NOT THE SAME as a small sample size that is non-random.
Again, your sample was not random.
No matter how many times you try to imply/claim that it was random, it was not random.
Dude, go to college and take a statistics class.
Also, you say
My previous article was prompted by an editorial about the online response that had been elicited by two different photos -- one showing a black woman breastfeeding, and a nearly identical photo showing a white woman breastfeeding.
Just to be clear, the photos were very different. One woman was in a public place, the other was in a private place. One woman was looking at the camera, one woman was looking at the child. One woman's photo was widely seen in the United States, on woman's photo was widely seen in Australia. Those are all differences that can account for the different reaction.
Basically, you're just bloviating about things you don't understand using techniques that you made up to find conclusions that are meaningless.
And... half of your article is some butthurt about how some judge didn't bother to read all six of your amateur legal briefs. Nobody cares about your butthurt. Nobody. Leave it alone.
I pray to Jesus Christ, our Lord and Savior, that he takes you. You are the worst thing to ever happen to this site. The GNAA was better for us than you are.
News for People who don't understand statistics.
At least they appear to have had the good grace to get rid of "Stuff that matters" from the tagline.
When the mainstream Media have a slow news day, they dredge up some garbage, hopefully controversial, to fill the space. Samzenpus, stop it.
I give. Is it because...
...they don't like to be reminded that breasts *aren't* primarily sexual?
...they don't like to be reminded that children need to be fed?
...they are pathologically incapable of handling the sight of bare skin?
Please, do fill us in.
I absolutely agree with you: the submitter has no clue about statistics. This type of problem has been solved use simple Bayesian statistics where you calculate the probability distribution function for the fraction of people agreeing with X. This gives you all the information you need to calculate the probability that two datasets are consistent. We use this in particle physics for efficiency calculations all the time. There is even a detailed write up here.
Ice Cream Dispute
Really giving someone front page space to argue their bad math wasn't really bad ?
The whole site has been shifting it's focus from being for people who build things to people who want to would be tech hipsters.
Now we even have math being destroyed on the site in order to make a politically correct point ?
Might as well just change the motto to read "Clickbait for silly people nothing that really matters"
Wait, hold up a minute... I thought murder caused cake.
The sample sizes he used are not a problem from a statistical POV. He has two small samples so he can legitimately use the central limit theorem to find the mean. He also has more than the minimum sample size of 32 required to calculate variance and confidence levels. The only thing likely to show significant improvement with a larger sample sizes is the confidence level, since he admits his confidence is weak I don't see a problem.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
.. and he is my Marlon Brando. I want to suck the cum from Bennet Haselton's friction-wrought typing fingers. Every dew drop of jism that Bennet emits make me writhe with intellectual orgasm. God Bless you Bennet Hasleton, Frequent Contributor! America needs you! God needs you!
Where to start?
This is just as valid as a "poll" on poll taxes among KKK members.
-- Tigger warning: This post may contain tiggers! --
write a more concise, coherent, and far more correct article than anything that ever comes out of Bennett Hasselton's keyboard...
dd if=/dev/urandom of=/. bs=1024 count=10
For the love of the FSM, put in a filter so that we can click "Don't want to read any more of this author's drivel"...
Sign up here to get Slashdot to stop accepting blog posts from Bennett Haselton.
Will you sign this petition? Click here:
Petition to Drop Bennett Haselton
Thanks!
I read your long essay. You can only make inferences about the population you survey here ("Mechanical Turk"), not the general population. (Are people on "Mechanical Turk" representative of the general population?) So the best you can do here is draw conclusions about people using "Mechanical Turk", but alas, your sample wasn't random. It takes hard work to do a statistical study well, especially if taking random samples is too hard or too expensive. Unfortunately, nobody believes this.
No mas. Estoy cansado.
Basic stats Margin of Error for 95% confidence on the first stat is plus-or-minus 12.2%
.23*47 = 10.81 at least 10 successes and failures expected (just barely)
.3*54 = 16.2 at least 10 successes or failures
.57 standard deviations of eachother. There is a very good chance with sample sizes this small that both are recording the same population parameter with different sampling errors. Larger sample sizes would reduce the standard deviation (proportional to inverse square root of number of sample) and make it easier to say that there was a definitive difference between the two.
Same confidence is plus or minus 12.4% on the second.
Randomness of the sample is unknown. This could be a problem.
We have less than 10% of the population so we don't have to worry about a sample that's too large.
Conclusion: if the sampling method is reasonably random then assuming a normal model is reasonably valid.
Unfortunately both values are within
3 out of 6 judges also agree that you don't have to read past the first paragraph to figure out that Bennett is full of crap.
your last article garnered even more objections to STOP WITH THE FUCKING BLOG SPAM.
Slashdot, please add a [Hazelton] tag to the subject line like you did for [video] posts, so we don't have to waste our time. I thought this was about statistics and opened the article.
As some commenters pointed out, one big problem with the methodology is making sure that the methodology for choosing respondents is sound.
However, another big problem is what the scrip actually calculates. There are two events:
Event A: Observing that 36 out of 47 survey-takers, or 77%, said that a picture of a black woman breast-feeding was inappropriate; while in a different group, 38 out of 54 survey-takers, or 70%, said that a picture of a white woman breast-feeding was inappropriate in the same context.
Event B: Knowing that the true negative response rate towards the white woman is at least 20 percentage points less than that of the negative response rate of the black woman.
The script that the op links to gives us a bound on the probability of event A given even B (that is, of observing the sample given that people are much more negative towards the black woman).
However, what we are really interested in is the probability of event B given event A (that is, what is the probabiltiy that people are actually more negative towards the black woman, given that we observed what we did in the survey) , which the op does not calculate or bound.
If anyone really want's to know why they let this guy post such long messages, just look at how many comments are in this thread!! 8-P
You did it to yourselves...
This could be my first and only post since 2005 when I joined. I'm sending an email to feedback@slashdot.org now to close my account.
I'm sick of Bennett Haselton's long-winded drivel being posted as "News for Nerds". They obviously read the comments but keep posting his garbage anyway.
Yes, I could just stop getting the emails and not visit the site but I'd like them to know why they are losing an invaluable long-term reader (hahahahahaha, they don't care or Bennett wouldn't have this place as his vomitorium).
Hi, reddit, I heard I can choose what I see and, what's that, I can even downvote stuff I never, ever want to see? You mean to say that Bennett could never survive in an environment where the readers get a say as to what's posted?!?!?!? My oh my.
"What a senseless waste of human life"
Is the difference in the outcome for black and white women statistically significant?
No. The proper way of testing this is by using Fisher's exact test. Quite simple in one line of R:
fisher.test(matrix(c(36, 11, 38, 16), nrow=2))
Running this shows we obtain such a difference (77% vs. 70%) with about 50% probability just due to chance, given the sample size. The output of the R command above is:
data: matrix(c(36, 11, 38, 16), nrow = 2)
p-value = 0.5081
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
0.5175229 3.7540626
sample estimates:
odds ratio
1.373629
It's embarrassing, the child-like understanding of statistics on display in the OP.