To a first approximation, the size of the binary will increase in proportion to the number of architectures supported.
This is something you might decide to ignore if you are only supporting two architectures. Debian Lenny supports twelve architectures, and I've lost count of how many the Linux kernel itself has been ported to. I really don't think this idea makes sense.
(Besides, what's wrong with simply shipping two or more binaries in the same package or tarball?)
As mentioned by the other poster, data portions of the program are shared. In some cases, that means that data files are shared directly; multiple binaries, one data file. In other cases (libraries, etc) where the data is embedded in the binary, it simply means that the FatELF binary will compress to produce a combined file that's smaller than n * single architecture size. (The same is true for packing multiple binaries into one tarball, of course. Though in the case of several different files in one tarball, the FatELF version may have (slightly) better compression because it puts all the versions of one file right next to each other, rather than grouping all of one architecture together; that makes it more likely that the copies of the same data are within the same encoding block.)
The only thing wrong with shipping multiple binaries in one package or tarball is that, afaik, none of the major package managers support it. Sure, you could add support, but he decided this was a better approach.
In the case of something like Debian, it obviously doesn't make sense to have the package repositories use FatELF binaries, nor to include all possible architectures on one install CD / DVD. However, it might make sense to include a couple common architectures on a single iso that would work for most people, and have the obscure architectures get their own isos (or use jigdo) like they do now.
The three power grids are out of phase with each other. Are they doing a AC->DC->AC conversion? It was my understanding that the biggest technical hurdle to connecting the grids was the difficult problem of shifting the phase of one grid to another.
Yes, they are. The superconducting cables are running high-voltage DC.
A typical lightweight power cord is 16 AWG. 60 Watts (assuming good power factor correction) is 0.5 A. 16 AWG wire is ~ 4 mOhm/ft. So 4 ft of wire (2 ft cord, supply and return) is 16 mOhm. That means you're losing 8 mV of your supply voltage, or 4 mW of power. That's about 99.993% efficient.
You have to get significantly longer extension cord and put a lot more current through it before the power loss is relevant. Even if you used a 12 ft cord, and drew a rather significant 4A, that's still only 1.5W out of 480, or 99.7% efficient. And most extension cords are 14 AWG or thicker.
Trace minerals aren't a problem. These aren't trace amounts. You'll notice the choice of metals: calcium, magnesium, and zinc are all things your body needs in non-trace quantities, and is capable of regulating the level of. A few tens of grams of metal, dissolving over a month or two, is a couple hundred mg per day. That's roughly comparable to the FDA recommended daily intake. It would be a lot like taking a extra multivitamin or two a day.
Plastic still exists hundreds of thousands of years after usefulness of the object has expired.
Last I checked, every single example of 100,000-year-old plastic that was no longer useful had long since been dug up out of its landfill and recycled.
Alternately, consider the whole policy, rather than simply the tax half of it. The rebate half of it matters too, which is the point of the proposal and what I've been trying to say.
Consider a person making $1000/mo who pays $100/mo in taxes and receives a $100/mo rebate, for a net tax rate of 0%. Now consider someone making $8000/mo who pays $400/mo in taxes (it's unreasonable to assume they don't spend *any* more; feel free to adjust that $400 number up or down a bit, though) and receives the same $100/mo rebate. They pay a net $300/$8000 = 3.75% tax rate. The overall tax policy is progressive with respect to spending, and also income (though less so, because spending doesn't rise quite as rapidly as income). You can adjust the rate and rebate values to match your desired total revenue and level of progressiveness / regressiveness.
Unfortunately, with only two degrees of freedom, you can't also adjust the curvature of the system; that's the basis for the argument that FairTax disproportionately hurts the middle class. We can adjust the numbers to get the line in the right place at the ends, but what we really wanted was a curve. That's where the various credits and exemptions come in; choose them carefully, and they have a different impact variation with income, and the result curves about the way you want too. Of course, this step is much harder and prone to pet projects and political agendas.
I had actually been thinking of it not so much as Silver mining for a pollster to pick on and coming up with SV, as the community at large (aka AAPOR) deciding to pick on SV and Silver then running tests. The community at large obviously had all the old poll data, and the correction is required whether Silver did the mining himself or not.
However, on re-reading the section you point out, I'm inclined to think the original accusations by AAPOR, and the subsequent decision to investigate SV by Silver, were both unrelated to the detailed poll results, and based only on the lack of published methodology, in which case you're right that no correction is needed.
OTOH, as Silver puts it (in a later post):
Mr. Johnson may be right that the implication that his data may have been forged could be difficult to categorically disprove. Had the statistical evidence been only marginally compelling, I would not have made it. With that said, I would also tend to treat -- and would encourage those in the media to treat -- "alternate hypotheses" raised by Strategic Vision with some greater-than-usual amount of sympathy. So far, Johnson has not offered any. (Emphasis added.)
I'm inclined to agree with that view. It's not the same as making p-value corrections, but the idea is similar. It also captures the basic idea the underlying data are not trivial to analyze (SV isn't quite polling the same regions and the same questions as everyone else), which makes the p-values that result more approximate than is normal for statistical analysis.
And, of course, the numbers are so extreme that no plausible amount of p-value correction would make them look good for SV. It's possible there are hypotheses other than fraud that explain those numbers, though, and we need to be careful to realize that p-values don't distinguish between fraud and something else weird but entirely legitimate -- all they say is "this isn't normal." Of course, the threats of lawsuits and lack of explanations point rather strongly to fraud at this point.
The issue isn't the hypothesis he forms about the second digits. The issue is that he forms a hypothesis that SV is producing strange results, based on their poll data produced to date, and then tests that hypothesis using the same data. He could have performed multiple tests and eventually decided the second-digits test looked interesting, but we'll assume Silver is a more honest statistician than that, and that the second digit test is the first test he did. No correction is required there.
Where the correction is needed is a step earlier -- it would have been equally reasonable to form a hypothesis that any other pollster was behaving badly; he chose to pick on SV only after looking at their poll results. This is perfectly reasonable and normal, however it requires a multiple-tests correction. If he has 100 pollsters in his database, we expect one of them to have a second-digits distribution that is weird at the p=0.01 level. So he needs to correct his p-value upwards by at least the number of polling agencies with a significant number of polls in his database. However, the SV p-values are so tiny that they're still very highly significant even after any plausible correction.
The problem isn't precisely that he performed multiple tests. It's that he formed the hypothesis and then tested it using the same set of data. That amounts to having performed multiple tests, but isn't quite the same thing. That's a fairly subtle problem to correct, and I can't claim to be well versed enough in stats to know more than that it exists and can be dealt with to a degree through sufficient cleverness.
Tests like Fisher or KS are tricky to apply here — because they polled different sets of questions, we don't precisely expect that SV's results are drawn from the same distribution as the general set. What we expect is that they won't be "too different". That's a hard thing to test, and is the reason why Silver makes the argument from generalities rather than p-values.
Fortunately, there are corrections you can do for that. And he took a fairly normal statistical test on the numbers, which is equivalent to saying he didn't perform that many comparisons. To very rough approximation, you need to correct your p-value for all the less weird analyses you might have performed on the data instead. It's a bit hard to pin down an exact p-value for the analysis he did (the underlying data isn't expected to be flat; it's also not expected to be that bizarrely lumpy), but I promise that Nate Silver has an understanding of this issue (which you'd see, if you'd read the post).
Um, what? The sales tax portion is flat. Everyone who goes into the store is charged the same rate. The sales tax itself is neither progressive nor regressive. There are other pieces added on that make the system as a whole progressive (the rebates, specifically, in the case of FairTax).
Sales taxes (and other consumption taxes) are regressive taxes. However, not all policies involving sales taxes are regressive. The simplest (perhaps not the best...) example of such is the FairTax proposal. It uses a combination of a flat sales tax rate with a constant dollar rebate to each consumer. The combination means that with increasing spending, a larger net fraction of your spending is on taxes. That is, it's a progressive sales tax.
Of course, the Google proposal also talks about various incentive taxes. Whether these are good or bad seems to depend mostly on whether you're calling them sin taxes or a way to internalize externalities so that the market can actually optimize overall wealth. Markets optimize locally; external costs of production that are borne by people other than the producers (like pollution) will be undervalued in the optimization process. Transferring those costs back onto the producer through taxes internalizes that externality and lets the market optimize the thing it should actually be optimizing.
A tax system that was actually based on setting goals, and then looking at data and evidence about what tax systems would actually achieve those goals, would be perhaps the biggest advance in government technology in centuries. Of course, it's also spectacularly idealistic and difficult to make work. But then, so are all the other ideas they list, so...
(I haven't actually decided which to cast my vote for yet, but the taxes proposal is on the short list.)
If you subscribe to that model, you still can't say that the vaccine is 95% likely to work; you need a prior estimate for the likelihood the vaccine works, which you can then modify based on the results of the study.
The two-sided test is appropriate here; we can't completely ignore the idea that the vaccine might have made things worse.
We don't know for sure that the two groups are precisely evenly split; we expect that they're close, but not precise, because some people will have dropped out of the study for one reason or another.
I'd say that confirms that my estimate of 5% was reasonable;)
Specifically, you could say you were unlucky at the 5% chance level -- that's the (approximate) odds of getting results more extreme than this, given the number of people in each group that actually got infected, purely by dumb luck, if the vaccine did exactly nothing. (74 vs 51, out of a total of 16402, broken into two groups; that's just using a poisson approximation, since I don't have the precise group sizes, which gives 2.06 standard deviations, or significant at the 5% level.)
Of course there's something about the level of confidence - a 99% confidence means there's a 1% your observation is random fluctuations.
Explanations of confidence intervals are one of the most frequently mistaken pieces of stats comments. The correct explanation goes something like this: If your results are significant at the 5% level, then that means that if there was no difference between the two groups, you would observe results at least this unlikely less than 5% of the time by random chance. We cannot say that the hypothesis (the vaccine works) is 95% likely to be true; it either is or isn't. What we can say is how *unlikely* we would be to observe this result purely by luck, in the absence of any actual difference.
We're bitching about the slashdot editors, not you. It's their job to catch submitter mistakes. That is what an editor does. The really annoying thing is they're as likely to "edit" the summary to introduce mistakes as to remove them.
The "machine learning engine" is a "datacenter" (warehouse) full of cheap African laborers who are all watching the cameras.
(this is a joke, it just isn't funny, and it is meant to illustrate a point. See the next line):
God/nature/FSM/evolution/al gore/$deity has done a pretty damn good job at building our brains, why are we trying to reinvent that wheel in a computer?
Because the owners of those brains get all whiny when you try to stick them in jars and make them solve the problems you want to solve, rather than sitting around watching porn? Really, sticking a bunch of brains in a 19" rack is harder than you'd think.
Those few calculator hackers (there are a lot more than 100 of them; they're a minority, but not that small a one) aren't just a few users. They're busy writing games and other useful programs. Those programs appear on just about every TI calculator out there, and plenty of people who aren't even remotely enthusiasts or geeks are using them. The enthusiasts have a disproportionate influence on how popular the platform is, because they make it more useful for everyone.
The hard problems for such a network involve things like searching and routing. Freenet isn't exactly fast, but it's worlds more secure than anything else for this sort of thing (even so, it's far from perfect). It's also quite usable for things like browsing freesites (Freenet-hosted websites), and publishing controversial content (though large, unpopular files don't stay around forever, due to limits on disk space (and probably some bugs, but we're working on those)).
Of course, if the problem is the encryption itself, which Freenet makes rather heavy use of, the problem is rather harder.
Because they're not a healthcare provider, if they acquire HIPAA protected information, they're not actually required to do anything in particular. They could leak it without consequences. They could use it maliciously. They could sell it.
That, I don't know. If you're really curious, though, I suggest asking Carmack, either directly or on the amateur rocketry mailing list. He's quite open about answering technical questions.
Slashdot Mirror
To a first approximation, the size of the binary will increase in proportion to the number of architectures supported.
This is something you might decide to ignore if you are only supporting two architectures. Debian Lenny supports twelve architectures, and I've lost count of how many the Linux kernel itself has been ported to. I really don't think this idea makes sense.
(Besides, what's wrong with simply shipping two or more binaries in the same package or tarball?)
As mentioned by the other poster, data portions of the program are shared. In some cases, that means that data files are shared directly; multiple binaries, one data file. In other cases (libraries, etc) where the data is embedded in the binary, it simply means that the FatELF binary will compress to produce a combined file that's smaller than n * single architecture size. (The same is true for packing multiple binaries into one tarball, of course. Though in the case of several different files in one tarball, the FatELF version may have (slightly) better compression because it puts all the versions of one file right next to each other, rather than grouping all of one architecture together; that makes it more likely that the copies of the same data are within the same encoding block.)
The only thing wrong with shipping multiple binaries in one package or tarball is that, afaik, none of the major package managers support it. Sure, you could add support, but he decided this was a better approach.
In the case of something like Debian, it obviously doesn't make sense to have the package repositories use FatELF binaries, nor to include all possible architectures on one install CD / DVD. However, it might make sense to include a couple common architectures on a single iso that would work for most people, and have the obscure architectures get their own isos (or use jigdo) like they do now.
The three power grids are out of phase with each other. Are they doing a AC->DC->AC conversion? It was my understanding that the biggest technical hurdle to connecting the grids was the difficult problem of shifting the phase of one grid to another.
Yes, they are. The superconducting cables are running high-voltage DC.
Just tell me which UNIX command you would use to get instructions on a program and deny that this isn't intentional.
$info info
Nope, not sexist.
A typical lightweight power cord is 16 AWG. 60 Watts (assuming good power factor correction) is 0.5 A. 16 AWG wire is ~ 4 mOhm/ft. So 4 ft of wire (2 ft cord, supply and return) is 16 mOhm. That means you're losing 8 mV of your supply voltage, or 4 mW of power. That's about 99.993% efficient.
You have to get significantly longer extension cord and put a lot more current through it before the power loss is relevant. Even if you used a 12 ft cord, and drew a rather significant 4A, that's still only 1.5W out of 480, or 99.7% efficient. And most extension cords are 14 AWG or thicker.
Trace minerals aren't a problem. These aren't trace amounts. You'll notice the choice of metals: calcium, magnesium, and zinc are all things your body needs in non-trace quantities, and is capable of regulating the level of. A few tens of grams of metal, dissolving over a month or two, is a couple hundred mg per day. That's roughly comparable to the FDA recommended daily intake. It would be a lot like taking a extra multivitamin or two a day.
Plastic still exists hundreds of thousands of years after usefulness of the object has expired.
Last I checked, every single example of 100,000-year-old plastic that was no longer useful had long since been dug up out of its landfill and recycled.
Alternately, consider the whole policy, rather than simply the tax half of it. The rebate half of it matters too, which is the point of the proposal and what I've been trying to say.
Consider a person making $1000/mo who pays $100/mo in taxes and receives a $100/mo rebate, for a net tax rate of 0%. Now consider someone making $8000/mo who pays $400/mo in taxes (it's unreasonable to assume they don't spend *any* more; feel free to adjust that $400 number up or down a bit, though) and receives the same $100/mo rebate. They pay a net $300/$8000 = 3.75% tax rate. The overall tax policy is progressive with respect to spending, and also income (though less so, because spending doesn't rise quite as rapidly as income). You can adjust the rate and rebate values to match your desired total revenue and level of progressiveness / regressiveness.
Unfortunately, with only two degrees of freedom, you can't also adjust the curvature of the system; that's the basis for the argument that FairTax disproportionately hurts the middle class. We can adjust the numbers to get the line in the right place at the ends, but what we really wanted was a curve. That's where the various credits and exemptions come in; choose them carefully, and they have a different impact variation with income, and the result curves about the way you want too. Of course, this step is much harder and prone to pet projects and political agendas.
I had actually been thinking of it not so much as Silver mining for a pollster to pick on and coming up with SV, as the community at large (aka AAPOR) deciding to pick on SV and Silver then running tests. The community at large obviously had all the old poll data, and the correction is required whether Silver did the mining himself or not.
However, on re-reading the section you point out, I'm inclined to think the original accusations by AAPOR, and the subsequent decision to investigate SV by Silver, were both unrelated to the detailed poll results, and based only on the lack of published methodology, in which case you're right that no correction is needed.
OTOH, as Silver puts it (in a later post):
Mr. Johnson may be right that the implication that his data may have been forged could be difficult to categorically disprove. Had the statistical evidence been only marginally compelling, I would not have made it. With that said, I would also tend to treat -- and would encourage those in the media to treat -- "alternate hypotheses" raised by Strategic Vision with some greater-than-usual amount of sympathy. So far, Johnson has not offered any. (Emphasis added.)
I'm inclined to agree with that view. It's not the same as making p-value corrections, but the idea is similar. It also captures the basic idea the underlying data are not trivial to analyze (SV isn't quite polling the same regions and the same questions as everyone else), which makes the p-values that result more approximate than is normal for statistical analysis.
And, of course, the numbers are so extreme that no plausible amount of p-value correction would make them look good for SV. It's possible there are hypotheses other than fraud that explain those numbers, though, and we need to be careful to realize that p-values don't distinguish between fraud and something else weird but entirely legitimate -- all they say is "this isn't normal." Of course, the threats of lawsuits and lack of explanations point rather strongly to fraud at this point.
The issue isn't the hypothesis he forms about the second digits. The issue is that he forms a hypothesis that SV is producing strange results, based on their poll data produced to date, and then tests that hypothesis using the same data. He could have performed multiple tests and eventually decided the second-digits test looked interesting, but we'll assume Silver is a more honest statistician than that, and that the second digit test is the first test he did. No correction is required there.
Where the correction is needed is a step earlier -- it would have been equally reasonable to form a hypothesis that any other pollster was behaving badly; he chose to pick on SV only after looking at their poll results. This is perfectly reasonable and normal, however it requires a multiple-tests correction. If he has 100 pollsters in his database, we expect one of them to have a second-digits distribution that is weird at the p=0.01 level. So he needs to correct his p-value upwards by at least the number of polling agencies with a significant number of polls in his database. However, the SV p-values are so tiny that they're still very highly significant even after any plausible correction.
The problem isn't precisely that he performed multiple tests. It's that he formed the hypothesis and then tested it using the same set of data. That amounts to having performed multiple tests, but isn't quite the same thing. That's a fairly subtle problem to correct, and I can't claim to be well versed enough in stats to know more than that it exists and can be dealt with to a degree through sufficient cleverness.
Tests like Fisher or KS are tricky to apply here — because they polled different sets of questions, we don't precisely expect that SV's results are drawn from the same distribution as the general set. What we expect is that they won't be "too different". That's a hard thing to test, and is the reason why Silver makes the argument from generalities rather than p-values.
Fortunately, there are corrections you can do for that. And he took a fairly normal statistical test on the numbers, which is equivalent to saying he didn't perform that many comparisons. To very rough approximation, you need to correct your p-value for all the less weird analyses you might have performed on the data instead. It's a bit hard to pin down an exact p-value for the analysis he did (the underlying data isn't expected to be flat; it's also not expected to be that bizarrely lumpy), but I promise that Nate Silver has an understanding of this issue (which you'd see, if you'd read the post).
Um, what? The sales tax portion is flat. Everyone who goes into the store is charged the same rate. The sales tax itself is neither progressive nor regressive. There are other pieces added on that make the system as a whole progressive (the rebates, specifically, in the case of FairTax).
Sales taxes (and other consumption taxes) are regressive taxes. However, not all policies involving sales taxes are regressive. The simplest (perhaps not the best...) example of such is the FairTax proposal. It uses a combination of a flat sales tax rate with a constant dollar rebate to each consumer. The combination means that with increasing spending, a larger net fraction of your spending is on taxes. That is, it's a progressive sales tax.
Of course, the Google proposal also talks about various incentive taxes. Whether these are good or bad seems to depend mostly on whether you're calling them sin taxes or a way to internalize externalities so that the market can actually optimize overall wealth. Markets optimize locally; external costs of production that are borne by people other than the producers (like pollution) will be undervalued in the optimization process. Transferring those costs back onto the producer through taxes internalizes that externality and lets the market optimize the thing it should actually be optimizing.
A tax system that was actually based on setting goals, and then looking at data and evidence about what tax systems would actually achieve those goals, would be perhaps the biggest advance in government technology in centuries. Of course, it's also spectacularly idealistic and difficult to make work. But then, so are all the other ideas they list, so...
(I haven't actually decided which to cast my vote for yet, but the taxes proposal is on the short list.)
If you subscribe to that model, you still can't say that the vaccine is 95% likely to work; you need a prior estimate for the likelihood the vaccine works, which you can then modify based on the results of the study.
The two-sided test is appropriate here; we can't completely ignore the idea that the vaccine might have made things worse.
We don't know for sure that the two groups are precisely evenly split; we expect that they're close, but not precise, because some people will have dropped out of the study for one reason or another.
I'd say that confirms that my estimate of 5% was reasonable ;)
Specifically, you could say you were unlucky at the 5% chance level -- that's the (approximate) odds of getting results more extreme than this, given the number of people in each group that actually got infected, purely by dumb luck, if the vaccine did exactly nothing. (74 vs 51, out of a total of 16402, broken into two groups; that's just using a poisson approximation, since I don't have the precise group sizes, which gives 2.06 standard deviations, or significant at the 5% level.)
Of course there's something about the level of confidence - a 99% confidence means there's a 1% your observation is random fluctuations.
Explanations of confidence intervals are one of the most frequently mistaken pieces of stats comments. The correct explanation goes something like this: If your results are significant at the 5% level, then that means that if there was no difference between the two groups, you would observe results at least this unlikely less than 5% of the time by random chance. We cannot say that the hypothesis (the vaccine works) is 95% likely to be true; it either is or isn't. What we can say is how *unlikely* we would be to observe this result purely by luck, in the absence of any actual difference.
We're bitching about the slashdot editors, not you. It's their job to catch submitter mistakes. That is what an editor does. The really annoying thing is they're as likely to "edit" the summary to introduce mistakes as to remove them.
The "machine learning engine" is a "datacenter" (warehouse) full of cheap African laborers who are all watching the cameras.
(this is a joke, it just isn't funny, and it is meant to illustrate a point. See the next line): God/nature/FSM/evolution/al gore/$deity has done a pretty damn good job at building our brains, why are we trying to reinvent that wheel in a computer?
Because the owners of those brains get all whiny when you try to stick them in jars and make them solve the problems you want to solve, rather than sitting around watching porn? Really, sticking a bunch of brains in a 19" rack is harder than you'd think.
Those few calculator hackers (there are a lot more than 100 of them; they're a minority, but not that small a one) aren't just a few users. They're busy writing games and other useful programs. Those programs appear on just about every TI calculator out there, and plenty of people who aren't even remotely enthusiasts or geeks are using them. The enthusiasts have a disproportionate influence on how popular the platform is, because they make it more useful for everyone.
You mean something like Freenet?
The hard problems for such a network involve things like searching and routing. Freenet isn't exactly fast, but it's worlds more secure than anything else for this sort of thing (even so, it's far from perfect). It's also quite usable for things like browsing freesites (Freenet-hosted websites), and publishing controversial content (though large, unpopular files don't stay around forever, due to limits on disk space (and probably some bugs, but we're working on those)).
Of course, if the problem is the encryption itself, which Freenet makes rather heavy use of, the problem is rather harder.
http://my.boss.is/a/stupid/fucking/bitch/i/hope/she/eats/shit/and/dies
Is that ok or not?
That's not a link TO defamatory content, that's defamatory content in the form of a link.
Hmm. So it is. However, I can do better; this is both: http://glennbeckrapedandmurderedayounggirlin1990.com/
Because they're not a healthcare provider, if they acquire HIPAA protected information, they're not actually required to do anything in particular. They could leak it without consequences. They could use it maliciously. They could sell it.
That, I don't know. If you're really curious, though, I suggest asking Carmack, either directly or on the amateur rocketry mailing list. He's quite open about answering technical questions.
Nope, wrong. That's *exactly* what the pendulum fallacy article is trying to explain is a fallacy.