I'm really curious about what people think about this comment and my attempt to defend p-values and statistical significance testing as a concept. I used to hate p-values like any respectable scientist, but in teaching intro college stats class (targeted to behavioral science), I've come to appreciate them, for one major reason.
1. We have to take uncertain science and make certain decisions about the conclusions. Science gets simplified to dichotomous decisions. You either approve the drug or not. You either eat eggs or don't eat eggs. The defendant is guilty or not guilty. In each of these cases, we take scientific and other evidence and have to make a decision: do we trust these data. Confidence intervals, odds ratios, etc, help give a picture but they don't give a clear guideline about what to accept.
2. It's really hard to understand (and teach) Bayesian and other approaches. I think that statistical significance is a decent proxy, as long as the limitations are well-understood. I am a big believer in teaching science research to people who have no desire to ever be "researchers", and in order to evaluate their studies, statistical significance is a good proxy. If you are doing an intro biology lab testing whether there are more bacteria on your hands after washing your hands versus hand sanitizer, a t-test with a p.05 criterion is a good approach. It won't get published in JAMA, but it's good for teaching research concepts.
3. Reviewers still want p-values. Each time I have submitted a manuscript without p-values, I get a nasty reviewer who requires p-values. Maybe I've had bad luck, but I'm guessing this is pretty common in the literature. Any time I try a statistical technique that goes beyond null hypothesis testing, there is at least one reviewer who doesn't understand the technique and gripes because there are no p-values or decision criteria. As long as this is required to publish, we need to do it.
So these aren't very good defenses, but it's why I'm still teaching p-values and null hypothesis testing. Maybe we will get rid of it, but like some other comments here, it leaves the question of what the alternative would be.
I don't know the answer to this because every evolutionary theory is some degree of speculation. Sleep is probably the biggest one, but here are two ideas: (great book is "Why We Sleep", by Walker)
1. REM and NREM sleep have brain adaptive purposes that could not have been fulfilled any other way. Terrestrial mammals all show REM sleep (almost all, at least), whereas some aquatic mammals don't have REM sleep. We don't know what REM sleep is for but we know we need it, and being deprived of REM sleep causes organisms to make up the deficit. Interestingly, seals will show high levels of REM sleep when on land but almost no REM sleep when in the water.
2. Sleep conserves energy and that conservation is important for survival. Predators sleep more than prey, partly because they have less to fear and partly because they don't need to eat as frequently. It takes more immediate energy to hunt down a gazelle than to graze.
I didn't RTFA (or WTFA), but did it include hours and hours of paid promotions, buying instagram followers, needless tweets, networking, posting to other blogs "oh, i love your work, here is my unrelated website with my art" or etc.
How do you get heard when there are literally millions of hours of videos on youtube each day, and hundreds of podcasts, and so forth? Especially when we're slaves to "recommended posts" or "recommended videos" algorithms which make it hard to find related content.
I agree with all these points, and think you're exactly right because without Firefox, Chrome will eventually become the new IE6. The main issue I have is with the browser moving forward. If I install a browser, I like it to stay the same. I really don't like my menus moving or getting an update that says "Pocket suggests these sites based on what I thought was secret browser history". I don't care if Firefox has a browser for with all the new feature sets.
I've given up on Firefox, so I shouldn't care, but please stop.
Last time I updated Firefox, it had "suggestions by pocket". To turn these off, it took ten minutes of googling to fix it, and then they still came back after another update.
Maybe I'm weird, but I thought the best browser is one that simply works, works fast, and then allows for extensions to do whatever extras that I want. This worked really well for firefox in the beginning, but now it is caught in the same trap of so many other programs. Power users want to be able to control things. Average users just want something that works. Do any of these features help either set of users?
This is going to be hard to suggest, but these results are actually good news. I used to be in the field of psychology research, so I know all the dirt on how these studies work and the techniques researchers have to do to get published. The fact that over 60% of the findings replicated is very surprising, for a lot of reasons.
1. Top journals publish flashy research. Rather than doing technical research on the mechanisms of empathy, a study showing "fiction reading increases empathy" is more likely to be published in Science, and for a grad student, get you a grant/job/life. Building up more incremental research with stronger theoretical foundations is a lot harder to get published.
2. There's a lot of competition. People want grants/jobs/life and so they have to publish. But anyone can steal your ideas. One of my greatest ideas was stolen as a grad student, which is why I pretty much left the field. It's easy for someone with a bigger lab to do the same study, find similar results, and then publish it. So you have to get your ideas out quickly and you really can't share many of the flashy ones, because this could happen. So this creates a perverse incentive not to replicate the most flashy findings, because they are the ones that give you all the glory.
3. People make it very hard to replicate their research, because once someone gets a program of research, that is their gravy train. If I have a great finding that gets me a great paper and a top-level job, I need to keep up that research to get tenure. All those replication studies are necessary for my job, so again there's an incentive not to replicate other labs' research.
4. The sample sizes are way too low for the type of research being done, which is why false positives are very likely. fMRI research uses sample sizes that are 20-40. My dissertation used 30 participants, because it was all I could afford. To make great generalizable conclusions, I'd probably need 100 or more. So should I stop doing research? No, I should publish it and others should replicate it, or not. By replicating it in slightly different conditions, it increases the external validity of the findings.
So long story short, 60% replication is a good number, and should be how science goes. It's not the researchers' fault or the fault of psychology as a science, since studying complex systems is very hard. It's just the fault of a broken system that equates a published paper with truth, rather than one piece of evidence.
This is just another example of the absurdity of prescription drugs. Obviously there's no reason for Epi Pens to be that expensive, since they weren't that expensive long ago.
The real reason for the expense was because a lot of places were required to buy them. Kids are allergic to everything these days (seriously), and so schools need to have epi pens. Would you send your toddler to a preschool without an epi pen? Think of the children! So that cost could be hidden in the budget of "safety" for schools.
If you were a consumer, you could avoid them. I had allergy shots and my provider gave me an epi pen prescription. I didn't really need it, but it was in the 1 in a 1000 chance I had a reaction when I wasn't around. But she also gave me a coupon to make it go down to a reasonable cost.
But now there's an outcry about this absurdity, and so we get another option. This fixes the tiny problem, allows the drug companies to still make handfuls of money, and no one talks about what is really needed in the US: the government as the only buyer of drugs. If drugs are such that there is only one supplier, with patent protections that give a monopoly on production, there should be a single buyer, that would negotiate an acceptable price, and then this buyer sells to all pharmacies at cost, which leads to an equitable distribution.
I had a shared family computer, way back in the 90s (not old enough to have cool hacking stories about 80s era tech). But what was protected? Everything that is nasty about the internet was true then, just in lesser form. It's just the case that everyone is online now, rather than a smaller subset of people who were more aware of technology.
The irony is that the more anonymous the internet was, the nicer and safer it was. My thought is 90% of what is nasty online is because everything is less anonymous. Back in the 90s, when you were cooldude69 and talking to a guy pretending to be coolchick98, what harm could come of it? It's only when it becomes real life and you start giving real addresses that problems start creeping in. Doxxing and harassment isn't relevant when you could disappear and start again. Like on slashdot, my UID is 9 billion, but I could have been UID 42, get in trouble, delete the account, and disappear.
Of course, a fully anonymous net will let a lot of fringe views, conspiracies, and nasty stuff fester, but that stuff will always be present.
If I had more guts, I think this is part of what I would call a get rich scheme on Tesla. I see this same pattern, on Slashdot, the idea that Tesla does something, stock goes up, the short sellers put more pressure on Musk, then he snaps, the stock falls, then something good happens, and it goes up, etc. I bought a few shares of Tesla, mostly because I believe in electric cars, and always waited after some "Musk is crazy" news to do so, and each time, they jumped right afterward.
This one might be a bit more dangerous though, because from what I've read, Musk might be in trouble here. He may be a genius, or may not, but smarter people than he have gotten too arrogant and slapped by laws, whether they're rational or not. The only saving grace I'm hoping for is that Musk is rational enough to know that any benefit he would get from a misleading or false announcement would soon disappear once it's obvious there is no financing. The stock would plummet, the shorts would be excited, and Musk may be in legal trouble.
There are a lot of good reasons to have these devices connect remotely for firmware updates. For instance, the ability to recognize arrhythmia using signal detection has improved dramatically in the last 5-10 years. For defibrillators, that can be the difference between appropriate and inappropriate shocks where the machine misreads the rhythm. Same is true with pacing and other treatments for a pacemaker.
I have a device like this, so I've read a lot about these hacks. I have a device from a different manufacturer, so I don't know if this applies, but the lack of security in many of these devices is scary. Most of the hacks I've read before involve hacking the device itself. It takes a few minutes with an RF wand to do a firmware update, so hacking the pacemaker/defibrillator itself is hard to do But if you can hack the device that does the updates, that is really scary. It's a lot easier to hack a device left in a closet rather than something physically embedded in a person.
Slashdot Mirror
I'm really curious about what people think about this comment and my attempt to defend p-values and statistical significance testing as a concept. I used to hate p-values like any respectable scientist, but in teaching intro college stats class (targeted to behavioral science), I've come to appreciate them, for one major reason.
1. We have to take uncertain science and make certain decisions about the conclusions. Science gets simplified to dichotomous decisions. You either approve the drug or not. You either eat eggs or don't eat eggs. The defendant is guilty or not guilty. In each of these cases, we take scientific and other evidence and have to make a decision: do we trust these data. Confidence intervals, odds ratios, etc, help give a picture but they don't give a clear guideline about what to accept.
2. It's really hard to understand (and teach) Bayesian and other approaches. I think that statistical significance is a decent proxy, as long as the limitations are well-understood. I am a big believer in teaching science research to people who have no desire to ever be "researchers", and in order to evaluate their studies, statistical significance is a good proxy. If you are doing an intro biology lab testing whether there are more bacteria on your hands after washing your hands versus hand sanitizer, a t-test with a p .05 criterion is a good approach. It won't get published in JAMA, but it's good for teaching research concepts.
3. Reviewers still want p-values. Each time I have submitted a manuscript without p-values, I get a nasty reviewer who requires p-values. Maybe I've had bad luck, but I'm guessing this is pretty common in the literature. Any time I try a statistical technique that goes beyond null hypothesis testing, there is at least one reviewer who doesn't understand the technique and gripes because there are no p-values or decision criteria. As long as this is required to publish, we need to do it.
So these aren't very good defenses, but it's why I'm still teaching p-values and null hypothesis testing. Maybe we will get rid of it, but like some other comments here, it leaves the question of what the alternative would be.
I don't know the answer to this because every evolutionary theory is some degree of speculation. Sleep is probably the biggest one, but here are two ideas: (great book is "Why We Sleep", by Walker)
1. REM and NREM sleep have brain adaptive purposes that could not have been fulfilled any other way. Terrestrial mammals all show REM sleep (almost all, at least), whereas some aquatic mammals don't have REM sleep. We don't know what REM sleep is for but we know we need it, and being deprived of REM sleep causes organisms to make up the deficit. Interestingly, seals will show high levels of REM sleep when on land but almost no REM sleep when in the water.
2. Sleep conserves energy and that conservation is important for survival. Predators sleep more than prey, partly because they have less to fear and partly because they don't need to eat as frequently. It takes more immediate energy to hunt down a gazelle than to graze.
I didn't RTFA (or WTFA), but did it include hours and hours of paid promotions, buying instagram followers, needless tweets, networking, posting to other blogs "oh, i love your work, here is my unrelated website with my art" or etc.
How do you get heard when there are literally millions of hours of videos on youtube each day, and hundreds of podcasts, and so forth? Especially when we're slaves to "recommended posts" or "recommended videos" algorithms which make it hard to find related content.
I agree with all these points, and think you're exactly right because without Firefox, Chrome will eventually become the new IE6. The main issue I have is with the browser moving forward. If I install a browser, I like it to stay the same. I really don't like my menus moving or getting an update that says "Pocket suggests these sites based on what I thought was secret browser history". I don't care if Firefox has a browser for with all the new feature sets.
I guess there's still Pale Moon.
I've given up on Firefox, so I shouldn't care, but please stop.
Last time I updated Firefox, it had "suggestions by pocket". To turn these off, it took ten minutes of googling to fix it, and then they still came back after another update.
Maybe I'm weird, but I thought the best browser is one that simply works, works fast, and then allows for extensions to do whatever extras that I want. This worked really well for firefox in the beginning, but now it is caught in the same trap of so many other programs. Power users want to be able to control things. Average users just want something that works. Do any of these features help either set of users?
7.3 million divided by 800,000 customers doesn't leave much room for attorneys' fees, right?
This is going to be hard to suggest, but these results are actually good news. I used to be in the field of psychology research, so I know all the dirt on how these studies work and the techniques researchers have to do to get published. The fact that over 60% of the findings replicated is very surprising, for a lot of reasons.
1. Top journals publish flashy research. Rather than doing technical research on the mechanisms of empathy, a study showing "fiction reading increases empathy" is more likely to be published in Science, and for a grad student, get you a grant/job/life. Building up more incremental research with stronger theoretical foundations is a lot harder to get published.
2. There's a lot of competition. People want grants/jobs/life and so they have to publish. But anyone can steal your ideas. One of my greatest ideas was stolen as a grad student, which is why I pretty much left the field. It's easy for someone with a bigger lab to do the same study, find similar results, and then publish it. So you have to get your ideas out quickly and you really can't share many of the flashy ones, because this could happen. So this creates a perverse incentive not to replicate the most flashy findings, because they are the ones that give you all the glory.
3. People make it very hard to replicate their research, because once someone gets a program of research, that is their gravy train. If I have a great finding that gets me a great paper and a top-level job, I need to keep up that research to get tenure. All those replication studies are necessary for my job, so again there's an incentive not to replicate other labs' research.
4. The sample sizes are way too low for the type of research being done, which is why false positives are very likely. fMRI research uses sample sizes that are 20-40. My dissertation used 30 participants, because it was all I could afford. To make great generalizable conclusions, I'd probably need 100 or more. So should I stop doing research? No, I should publish it and others should replicate it, or not. By replicating it in slightly different conditions, it increases the external validity of the findings.
So long story short, 60% replication is a good number, and should be how science goes. It's not the researchers' fault or the fault of psychology as a science, since studying complex systems is very hard. It's just the fault of a broken system that equates a published paper with truth, rather than one piece of evidence.
This is just another example of the absurdity of prescription drugs. Obviously there's no reason for Epi Pens to be that expensive, since they weren't that expensive long ago. The real reason for the expense was because a lot of places were required to buy them. Kids are allergic to everything these days (seriously), and so schools need to have epi pens. Would you send your toddler to a preschool without an epi pen? Think of the children! So that cost could be hidden in the budget of "safety" for schools. If you were a consumer, you could avoid them. I had allergy shots and my provider gave me an epi pen prescription. I didn't really need it, but it was in the 1 in a 1000 chance I had a reaction when I wasn't around. But she also gave me a coupon to make it go down to a reasonable cost. But now there's an outcry about this absurdity, and so we get another option. This fixes the tiny problem, allows the drug companies to still make handfuls of money, and no one talks about what is really needed in the US: the government as the only buyer of drugs. If drugs are such that there is only one supplier, with patent protections that give a monopoly on production, there should be a single buyer, that would negotiate an acceptable price, and then this buyer sells to all pharmacies at cost, which leads to an equitable distribution.
I had a shared family computer, way back in the 90s (not old enough to have cool hacking stories about 80s era tech). But what was protected? Everything that is nasty about the internet was true then, just in lesser form. It's just the case that everyone is online now, rather than a smaller subset of people who were more aware of technology. The irony is that the more anonymous the internet was, the nicer and safer it was. My thought is 90% of what is nasty online is because everything is less anonymous. Back in the 90s, when you were cooldude69 and talking to a guy pretending to be coolchick98, what harm could come of it? It's only when it becomes real life and you start giving real addresses that problems start creeping in. Doxxing and harassment isn't relevant when you could disappear and start again. Like on slashdot, my UID is 9 billion, but I could have been UID 42, get in trouble, delete the account, and disappear. Of course, a fully anonymous net will let a lot of fringe views, conspiracies, and nasty stuff fester, but that stuff will always be present.
If I had more guts, I think this is part of what I would call a get rich scheme on Tesla. I see this same pattern, on Slashdot, the idea that Tesla does something, stock goes up, the short sellers put more pressure on Musk, then he snaps, the stock falls, then something good happens, and it goes up, etc. I bought a few shares of Tesla, mostly because I believe in electric cars, and always waited after some "Musk is crazy" news to do so, and each time, they jumped right afterward. This one might be a bit more dangerous though, because from what I've read, Musk might be in trouble here. He may be a genius, or may not, but smarter people than he have gotten too arrogant and slapped by laws, whether they're rational or not. The only saving grace I'm hoping for is that Musk is rational enough to know that any benefit he would get from a misleading or false announcement would soon disappear once it's obvious there is no financing. The stock would plummet, the shorts would be excited, and Musk may be in legal trouble.
There are a lot of good reasons to have these devices connect remotely for firmware updates. For instance, the ability to recognize arrhythmia using signal detection has improved dramatically in the last 5-10 years. For defibrillators, that can be the difference between appropriate and inappropriate shocks where the machine misreads the rhythm. Same is true with pacing and other treatments for a pacemaker. I have a device like this, so I've read a lot about these hacks. I have a device from a different manufacturer, so I don't know if this applies, but the lack of security in many of these devices is scary. Most of the hacks I've read before involve hacking the device itself. It takes a few minutes with an RF wand to do a firmware update, so hacking the pacemaker/defibrillator itself is hard to do But if you can hack the device that does the updates, that is really scary. It's a lot easier to hack a device left in a closet rather than something physically embedded in a person.