Alternate Baseball Universes

Jamie found a NYTimes op-ed by a grad student and a professor from Cornell, outlining some research they did into alternate baseball universes. The goal was to find out how unlikely in fact was Joe DiMaggio's 56-game hitting streak, played out in the 1941 season. No one since has even come close to that record. The math guys ran simulations of the entire history of baseball from 1885 on — 10,000 of them. For each simulation they put each player up to the plate for each at-bat in each game in each year, just like it happened; and they rolled the dice on him, based on his actual hitting stats for that season. (Their algorithm sounds far simpler than whatever the Strat-O-Matic guys use.) The result: Joltin' Joe's record is not merely likely, it's basically a sure thing. Every alternate universe produced a streak of 39 games or better; one reached 109 games. Joe DiMaggio was not the likeliest player in the history of the game to accomplish the record, not by a long shot.

Comments

If its so likely, they why hasn't it happened?

[quanticle]

I know the statisticians among you are going to bash me with a cluestick for such a naive question, but I'll ask anyway - if this event is so likely to occur, then why hasn't it happened again?

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

Clearly they aren't factoring in the stress and nerves the average ballplayer would be dealing with as they got closer to the mark.

Re:If its so likely, they why hasn't it happened?

[EdIII]

It won't occur again. That's not their point. What they are saying is that if you took those games and played them out with other player's performance stats, then it is actually highly likely that another player would do the same thing or better.

They are just re-running 1941 over and over and over again and putting different players in DiMaggio's spot. So Joe was not necessarily good, just in the right place at the right time according to the research.

Re:If its so likely, they why hasn't it happened?

[Martin Blank]

It was likely to occur early in the history of baseball, and fell off dramatically after the 1930s. The early years tended to be batting competitions (in some ways like today's) rather than pitching competitions, and a pitcher's repertoire was limited to about a half-dozen pitches, plus whatever grease, oil, jelly, file, sandpaper, thumbtack, or razor blade he could conceal.

Re:If its so likely, they why hasn't it happened?

[hedwards]

The most likely reason is that statistics isn't the appropriate method by which to study this problem.

This sort of a study is really more about curiosity, it doesn't deal with things like changes to the way in which the game is played. For instance early on, and for quite a while later, it was common for a pitcher to pitch 9 innings every game, and in many cases to pitch both games out of a double header. Meaning more opportunity for errors and since batters get time to rest up, there's a bit of an edge under that style of play to the batter which doesn't exist today.

That also doesn't include the variety of pitching which players see today or the fact that a player might get to see 3 different pitchers in a single game.

Even the length of the season has an effect on how players play. None of those things are easily quantified, much less analyzed by statisticians.

Re:If its so likely, they why hasn't it happened?

[Martin Blank]

No, they reran 1871-2005 through the simulator a total of 10,000 times. This is clear not only from the statement that says as much ("Using a comprehensive collection of baseball statistics from 1871 to 2005, we simulated the entire history of baseball 10,000 times in a computer"), but from the mention that the record was set most often in 1894.

Re:If its so likely, they why hasn't it happened?

[EdIII]

Okay fair enough and I stand corrected, but my point is still valid. The research is not about predicting future games that have not occurred, but about predicting different outcomes or possibilities or past ones.

At least that is what I understand from the article, it is mostly about how Joe DiMaggio could have been substituted by another player of similar stats and the same records would likely be achieved.

Re:If its so likely, they why hasn't it happened?

[ByteSlicer]

Because baseball players aren't dice?

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

While it might be true that statistics is not an appropriate technique to study baseball, none of the things you mention are evidence against it being useful. All of the factors you mention influence a player's batting average, and the hypothesis they are using is that once you know the batting average you can calculate a set of possible histories of hits for that player, with the right statistical weight. They are assuming that the probability of a batter getting a hit in any game is uncorrelated with his performance in previous games.

The answer to the question in the subject line is "It has happened." They're not claiming that in any given year there is likely to be a 56 game or longer hitting streak. What they calculated was the probability of the longest streak in the entire history of baseball having a particular value, and found that the most likely longest streak is 51 games, about what is observed in the universe we live in.

Re:If its so likely, they why hasn't it happened?

[hedwards]

While it might be true that statistics is not an appropriate technique to study baseball, none of the things you mention are evidence against it being useful. All of the factors you mention influence a player's batting average, and the hypothesis they are using is that once you know the batting average you can calculate a set of possible histories of hits for that player, with the right statistical weight. They are assuming that the probability of a batter getting a hit in any game is uncorrelated with his performance in previous games. Actually, they are reasons why statistical analysis is not appropriate in this instance.

Statistical analysis isn't inappropriate in terms of studying baseball, it is just inappropriate to use it in this manner.

What you are suggesting is a good example of the gambler's fallacy. And it breaks down in this case for the reasons that I mentioned, the underlying conditions in which those batting averages were collected has changed in such a way that they no longer accurately reflect the present conditions.

The GP was asking if the occurrence is that common, why hasn't it happened since, and the answer I gave was that there was a fundamental change in the way that the game is played which changed who has the advantage. It's similar to why nobody has had a .400+ season since 1930.

Re:If its so likely, they why hasn't it happened?

[Frequency Domain]

No bashing, it's not a bad question. The answer is because it still qualifies as a "rare event". The thing that's kind of counter-intuitive, but easy to demonstrate, is that having a particular rare event happen is rare, but having some rare event happen is common.

A good illustration of this is the so-called "birthday paradox", which asks what's the probability of having duplicate birthdays in a group of n people (whose birthdays are independent of each other). Think of adding the people to the room one by one. The first person doesn't have any chance of having a duplicate birthday, because there's nobody else in the room. The second person has 1/365 chances of duplicating, 364/365 of missing the first one. Let's follow up on the misses, they're easier to work with. In general, if we've got k people in the room without a duplicate, that means they've used up k of the 365 days in the year, and the next person we introduce to the room has to miss all of those days to avoid a duplication. So the probability of everybody missing everybody else, by the time we get up to n people in the room, is (365/365)*(364/365)*(363/365)*...*((365-n+1)/365), which starts diving towards zero really fast. The probability of having one or more duplicates is 1 - P(no duplicates), which correspondingly climbs to one really fast. If you write a short program to do the exact calculations, you'll find that by the time you have 23 people in the room the probability is greater than 0.5 of having a duplicate, and by the time you get 57 people it's greater than 0.99!

If you pick one particular person and ask what's the probability of duplicating that birthday it remains quite small. That's the difference between having a particular rare event rather than having some rare event. For a large enough group, some pair of people will almost surely share a birthday but the odds of it being you (or any other designated person) remain quite small.

Just to preserve my computing geek cred, this is why you need collision resolution for hashing algorithms. You don't know which entries will share hash values, but collisions are almost certain to happen by the time you've loaded 3 * sqrt(Hash Table Capacity) values, e.g., if your hash table has capacity 10000 you will almost surely see a duplicate within the first 300 entries.

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

The result given in the summary doesn't address the question "What is the probability that you will have a hitting streak of a given length in any given year?" but rather "What is the probability of having a longest hitting streak of a given length in the entire history of baseball?" I'm not falling for any gambler's fallacy: the say quite clearly that they use the statistics for each year to construct the history of hits for each player in that year.

It is true that they find that it was more likely that the streak would come early in the history of baseball, possibly for the reasons you mention, but that doesn't mean that the quoted result is in any way an inappropriate use of statistics.

Maybe we're not disagreeing, if what you're saying to GGP is that the result doesn't mean what he thinks it means.

Re:If its so likely, they why hasn't it happened?

[GoodbyeBlueSky1]

I wish my mod points hadn't just expired, because you just summed it up perfectly. Silly study with no basis in reality.

In other news, I've just started a fund of stocks that are held and traded based on historical data. If you invest in it, I guarantee a large return, because complex systems that rely heavily on myriad human variables are of course determined entirely by statistics.

Re:If its so likely, they why hasn't it happened?

[maxume]

The other poster was wondering why it didn't happen in 1929, and then with DiMaggio in 1941, and then again in 1942 and again in 1955, not if it was going to happen in 2008.

Re:If its so likely, they why hasn't it happened?

[Opie812]

...plus whatever grease, oil, jelly, file, sandpaper, thumbtack, or razor blade he could conceal.

Don't forget snot*. The snot ball.

*Major League, circa 1989.

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

It's likely to occur once in the 100 odds year history of baseball, the probability for it occurring twice is correspondingly lower.

The statement isn't "it should be happening all the time" but "if not him someone else would have done it by now."

Re:If its so likely, they why hasn't it happened?

[Colonel Korn]

The parent should be modded insightful, not just funny.

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

The early years tended to be batting competitions (in some ways like today's) rather than pitching competitions If by "early years", you mean 1920 and later, yeah.

Otherwise, buddy, you're way off base.

NL year-by-year stats.

Look at those ERAs pre-1920. Before 1920, the ERA on the NL never significantly exceeded 3.00. After 1920, it never dropped below 3.3 or so, with the exception of a 2.99 in 1968, after which MLB made changes to the rules, amongst them lowering the acceptable height of the pitcher's mound.

The time prior to 1920 was marked by pitchers such as Cy Young, Mordecai Brown, Walther Johnson, Ed Walsh, Christy Mathewson. You've probably heard of most of them.

Here are the single-season MLB ERA leaders. Outside of Bob Gibson in the aforementioned 1968, you have to go all the way to Greg Maddux in 1994 at #48 all time to find a season after 1920 on the list. Barely 10 of the 100 lowest single-season ERAs in MLB history occurred after 1920. And that's only because Pedro Martinez in 2000 and Ron Guidry in 1978 tied with 9 others for #100 on the list. So only 8 of the best single-season ERAs happened after 1920.

You need to research "dead ball era", and the response by baseball to "Black Sox". (Hint: just like the response to the 1994 strike, it involves the ball...)

The fact that you got a +5 out of such a demonstrably incorrect post is a major indictment of the baseball knowledge of the Slashdot faithful.

Re:If its so likely, they why hasn't it happened?

[Kamineko]

Like Einstein said: "God does not play baseball!"

I think.

Re:If its so likely, they why hasn't it happened?

[Reziac]

Having shamefully RTFA, ISTM that the study assumed that baseball players are rather more homogenous in their performance, and in who gets to see whom in the field, and how that affects individual players, than occurs in reality. I wonder what sort of numbers it would come up with if it accounted for how managers juggle players around?

Gene Mauch's strategy when he managed the Angels was so unvarying and consistent that he probably COULD be reduced to an algorithm.

Re:If its so likely, they why hasn't it happened?

[popmaker]

Indeed, when there are 23 people in the room, the probability is around 55% that some two of them will share a birthday. 23 is the smallest number such that the probability is more than 50%. Then you only need 40 people so that the probability is more then 90%. It is around 91% at n = 40.

Re:If its so likely, they why hasn't it happened?

[Vellmont]

I wish my mod points hadn't just expired, because you just summed it up perfectly.

Really? For the purposes of this article, why?

It seems perfectly reasonable to me to take a set of data and try to model how likely a particular outcome is. That's a very valid question to ask that a statistical model can answer. The model may be flawed, need improvement, or whatever, but that doesn't mean the question isn't one that can't be answered by science.

If you invest in it, I guarantee a large return, because complex systems that rely heavily on myriad human variables are of course determined entirely by statistics.

This is simply an invalid analogy. The article isn't saying it can predict the future (or even the past!) based on a statistical model. All it's saying is "just how likely was it for DiMagio to get his streak, given past performance".

Re:If its so likely, they why hasn't it happened?

[JimboFBX]

I'm curious as to why they find it likely. Was the pitching on every single team that poor and his hitting that good? For example, the odds of winning MLB.com's "beat the streak" are very very small, even when you stack the odds in your favor by exploiting the most lopsided team match-ups each day and picking the most statistically likely player. A player with an awesome .333 average has only an 81.8% chance of getting a hit if they had 4 at-bats. After 10 days of getting 4 at-bats each game, same average, there's only a 13.4% probability of getting a hit each day. At 20 days your looking at a 1.8% chance. At 56 days your looking at a .0013% chance. I just can't see how you can say there is any certainty unless you have some bug in your code that isn't doing the math right.

Re:If its so likely, they why hasn't it happened?

[CougMerrik]

The solution you've outlined for the birthday paradox is another case where statistics is not accurate in solving the problem at hand. However, in this case statistics acts as a lower bound for how many you could have. In reality, a "random sample" of people will almost always yield much higher birthday repetition because humans, in general, have certain times of the year when they are more likely to procreate. This study makes the same kind of mistake, but in the opposite direction. You can pretty much assume that they've established something close to a statistical upper bound on this, because humans do not have uniform performance, are effected by a large number of outside elements (such as the stress that comes with approaching a record).

Re:If its so likely, they why hasn't it happened?

[Frequency Domain]

Yeah, I left out leap years too. None of which has enough impact to negate the point I was trying to make.

Re:If its so likely, they why hasn't it happened?

[mudetroit]

a pitcher's repertoire was limited to about a half-dozen pitches
I hate to break it to you, but even in modern day baseball there are few pitchers that throw more then 4 pitches. The fact of the matter is that it is difficult to develop that many pitches to major league caliber. Additionally, there are only 2 major pitches that I am aware of, split finger and the slider, that have been developed since the 30's. Add in the fact, as another poster pointed out, that the era pre-1920 was the dead ball era where, in general, hits weren't as prevalent. Might want to try verify your facts just a bit first.

Re:If its so likely, they why hasn't it happened?

[khallow]

So what does that have to do with the study? Statistics applies to a lot more than dice. No offense, but your observation sounds like one of those cute but irrelevant observations that just add noise.

Re:If its so likely, they why hasn't it happened?

[Bandman]

I'm definitely not a statistician, but if you perform a randomization based around a pre-existing set, aren't you most likely to get a bell curve surrounding the historically recorded values? In other words, exactly what they found? They "seeded" the likelihood that a player would get a hit based on the number of hits the player got. Chicken and egg?

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

Wouldn't it be easier, and more efficient, to calculate the probabilities directly from the distribution? Why the simulations?

Re:If its so likely, they why hasn't it happened?

[EdIII]

The original poster said,
 

I know the statisticians among you are going to bash me with a cluestick for such a naive question, but I'll ask anyway - if this event is so likely to occur, then why hasn't it happened again?

Now I don't understand how that supports your statement of,
 

The other poster was wondering why it didn't happen in 1929, and then with DiMaggio in 1941, and then again in 1942 and again in 1955, not if it was going to happen in 2008.

He seems to be asking why such an event has not occurred outside the date range of the study. I know I got the date range of the study wrong initially in my post, but am I not right about the fact that the study is not attempting to predict future events?

Re:If its so likely, they why hasn't it happened?

[Paradise Pete]

What they calculated was the probability of the longest streak in the entire history of baseball having a particular value

That's what they claim, but it's hardly what they actually did. Assigning a single probability to every game is absurd. It ignores pitching, game conditions, injury, and the simple fact that some games players get more at-bats than in others. Both those are hard, and slapping on a probability is easy.

Re:If its so likely, they why hasn't it happened?

[Paradise Pete]

Wouldn't it be easier, and more efficient, to calculate the probabilities directly from the distribution? Why the simulations?

Because you have to deal with the clumping of at-bats into games, and of course factor in weather, pitching, game conditions, injury, official scoring, and the fact that the number of at-bats per game can vary.

(I realize that this 'study' ignored almost all of that.)

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

"There are three types of lies. Lies, damn lies and statistics." Computer models can be and are used to "predict" all kinds of crap. What happens happens. Nothing else matters. Anybody wanna talk about global warming?

Re:If its so likely, they why hasn't it happened?

[Goldberg's Pants]

Most geeks find baseball deathly boring. In fact some think themselves morally superior because they don't watch competitive sports, which is of course a very stupid attitude.

What's sad is the fact that baseball is the ULTIMATE geek sport due to sheer volume of stats it produces. I adore baseball. The SPORT of baseball. I don't actually watch MLB, but thanks to the magic of the internets I can watch Nippon League etc... Plus I enjoy looking at historic stuff, plus I have a variety of baseball sims. In fact I could go and replicate this experiment quite easily, and in engines that would be probably produce more statistically accurate data.

The fact that folk on here probably look like a dog that's been shown a card trick when you start talking about the dead ball era, and had modded up a completely WRONG post doesn't surprise me in the slightest.

Baseball gets very little respect unfortunately.

Re:If its so likely, they why hasn't it happened?

[Paradise Pete]

Outside of Bob Gibson in the aforementioned 1968, you have to go all the way to Greg Maddux in 1994 at #48 all time to find a season after 1920 on the list.

Dwight Gooden's 1.53 1985 season is in there. A remarkable list though.

Re:If its so likely, they why hasn't it happened?

[SL Baur]

It's not. People are not coins. And you deserve a +5 insightful.

All it takes is one stressful disagreement with one's spouse and there goes the concentration in the evening's game. You cannot simulate that with coin flips and I disagree with the conclusions in TFA.

It's a personal matter, but I'll bet just about anything, that Mr. and Mrs. Dimaggio were on unusually good terms during that period and he had little or no stress outside of his work plus a lot of luck for the duration of the streak.

Sports is about going beyond oneself and getting into a special "zone" where everything you do is correct. I did it once in my life when I bowled a sanctioned 3-game 800 series, including a perfect game - 32 strikes out of 36 balls I rolled. I saw Glenn Alison do it less than a week later (after he personally asked me if I was the guy who did 805 a few days previous in the same center, heck he's Mr. 900 and a robot in the consistency of his control of his body when he puts his mind to it no matter his age, and he bettered my score by a few pins to boot).

Maintaining the kind of consistency and luck Joe Dimaggio had over that time frame is something I regard as little short of miraculous. To reduce the accomplishment over a coin-flip type simulation strikes me as being irrelevant.

Re:If its so likely, they why hasn't it happened?

[JohnFluxx]

> The solution you've outlined for the birthday paradox is another case where statistics is not accurate in solving the problem at hand....because humans, in general, have certain times of the year when they are more likely to procreate.

You can always just factor that into the model. It's ignorant to say that statistics is not accurate for this problem.

Re:If its so likely, they why hasn't it happened?

[ByteSlicer]

Well, they modeled the batter using random numbers and their player stats. The problem is that real people don't behave deterministically. They might hit better on their birthday, or when it was a clear sky the night before the game, just because they believe that (baseball players are extremely superstitious). The model doesn't take into account that some player might get psyched out by a certain number (and always screw up on the 13th consecutive hit), or just by the pressure of wanting to break the record. It doesn't take into account the pitcher, weather conditions, and a lot of other things that matter to real people but not to computers.

You might be able to model some long term behavior that way, but never the short term stuff, because the model is too simplified (man versus dice).

Re:If its so likely, they why hasn't it happened?

[khallow]

You might be able to model some long term behavior that way Like the probability of winning streaks over the lifespan of baseball? Pretty much what they are doing.

Re:If its so likely, they why hasn't it happened?

[OldBus]

But the point they were trying to make is that the statistics aren't valid. The guys writing the paper were measuring independent random events. Baseball hits may or may not be random, but the key thing is that they are not independent.

Players suffer from pressure because of streaks. A player who goes several games without hitting is under a huge amount of pressure to hit, and their form may suffer. A player who hits for a few games is likely to have high confidence and keep hitting. However, if they do keep hitting for a few games more, everyone starts talking about it. They get asked questions at interviews etc. Many players crack under that sort of extra pressure. Di Maggio's great streak is great because he overcame that. Sportsmen in all sorts of sports are heard to make comments such as, "I'm just focussing on hitting/scoring" or "I'm taking one game at a time". This is because they're trying to avoid that sort of pressure.

I'm sure the guys who wrote the article know about gamblers' fallacy and would be quick to point out someone's mistake. Eg the gambler who bets extra money on a six being rolled because no six has come up on the last 20 rolls and 'a six must be due'. Of course, a six is no more likely than it was on any other roll because each roll is random and independent of any other roll. In the case of sporting streaks, each game is not independent, so the argument is just and flawed and invalid as the gamblers' fallacy.

Re:If its so likely, they why hasn't it happened?

[khallow]

Also, the presence of a hitting streak means the player is going to get more opportunity to extend that streak (more times at bat). Further, how does pressure build over X games? I just don't see it becoming worse after a certain point. You're not going to get shot for striking out on game 85.

Re:If its so likely, they why hasn't it happened?

[maxume]

I read the other comment as pointing out that the hitting streak only happened once in our universe, so the fact that it occurs easily in the study raises some questions about the study. Of course, in that case, one possible answer is that it has happened again:

http://www.baseball-almanac.com/feats/feats-streak.shtml

Re:If its so likely, they why hasn't it happened?

[Paradigma11]

unless the factors (birthday, clear sky..) are correlated to the explanatory variables they are just noise. something behaves deterministically if somebody can (predict)determine the result in advance. here is an interesting book about propability and (theory of) science: http://csli-publications.stanford.edu/site/1575863332.html .it is also viewable online. don't forget that if you are correct then it simply is not possible to make predictions in this context and this is emirically observable. not in this special case since they are talking about counterfactuals but the principle remains the same. for a treatment of counterfactuals see http://bayes.cs.ucla.edu/BOOK-2K/ .

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

Speaking of dice, the math blog "God Plays Dice" has an interesting post about this topic: http://godplaysdice.blogspot.com/2008/03/its-almost-baseball-season-again.html

In it, the author suggests that the odds of any (average) player in the history of baseball making a streak of at least 56 games to be 1 in 300, though he reaches this conclusion via an admittedly simple and straightforward biased-coin flipping analysis. Still, that's quite a bit different from the Times' statistical conclusion of over 50% odds of finding a streak of at least 53 games.

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

I realize I said this in an unclear manner: by "any (average) player," I mean a single player. That is to say, the 1 in 300 odds is for finding the streak anywhere in the history of baseball.

Re:If its so likely, they why hasn't it happened?

[Gospodin]

Baseball hits may or may not be random, but the key thing is that they are not independent.

Announcers and actual players to the contrary, they actually are remarkably independent. Studies have been done. The one to start with is: Albright, S. C. (1993), "A statistical analysis of hitting streaks in baseball," Journal of the American Statistical Association , 88, 1175-1183. He shows that there is almost no evidence that hitting streaks are anything other than statistical noise.

Re:If its so likely, they why hasn't it happened?

[Heian-794]

:::The early years tended to be batting competitions (in some ways like today's) rather than pitching competitions

::If by "early years", you mean 1920 and later, yeah.

:Otherwise, buddy, you're way off base.

The only one off base is yourself -- check your own link (baseball-reference.com is an amazing site and I recommend it to anyone) and pay extra attention to the 1890s. In the years immediately after the pitcher's mound was moved back to its current 60 feet 6 inches, offensive totals soared far beyond what we're used to seeing. Given that you're familiar with the lowering of the mound for 1969, I'm surprised that you're not familiar with when it was fixed at its current distance.

The article even mentions that the record was most likely to have been set in 1894, when the National League ERA was well over 5.00, and there were 11.6 hits per team per game, more than 20% more than we see now.

Look at those ERAs pre-1920. Before 1920, the ERA on the NL never significantly exceeded 3.00.

I'm looking at them. The "5.32" for 1894, which is somewhat more than three, is particularly striking.

After 1920, it never dropped below 3.3 or so, with the exception of a 2.99 in 1968, after which MLB made changes to the rules, amongst them lowering the acceptable height of the pitcher's mound.

...

You need to research "dead ball era", and the response by baseball to "Black Sox". (Hint: just like the response to the 1994 strike, it involves the ball...)

While he's doing this, perhaps you could research what came before the dead ball era: namely, the high-offense 1890s. Teams were taken off guard by the increase in the pitching distance and continued to play an 1880s game in a new environment. It took several seasons for adjustments, such as four-man pitching rotations and the occasional use of relief pitchers, to balance the sudden advantage that had been given to the batters. It is not surprising that 1894 would be the year in which a long hitting streak would have been most likely -- the single-season record for runs scored, 194 by Billy Hamilton, was set that year and still stands today.

The fact that you got a +5 out of such a demonstrably incorrect post is a major indictment of the baseball knowledge of the Slashdot faithful.

No, Martin is right -- the 1890s, while not as famous as Ruth and Gehrig's 1930s, were one of the most offensive eras in baseball. His simple analysis is much more forgivable than the insults you throw his way even while being completely ignorant of an entire decade of baseball history, the data from which are right on the web page you so callously direct him to visit.

Re:If its so likely, they why hasn't it happened?

[elwinc]

The season average ERA is a useful stat; I don't think the single-season MLB ERA leaders is useful in this case because the pre-1920 years had a wider variance in both pitching and batting quality. And looking at the first table you provide, it says that in 1894 the average ERA was 5.32. I didn't check exhaustively, but I believe it's the highest ERA in the table. Not so coincidentally, 1894 was the year that most simulations found the longest streak. For 1895, the average ERA was 4.78; for 1893 it was 4.66. Other high years: 1929: 4.71, 1930:4.97.

However, a high average ERA is only part of the story; for the simulation to be likely to produce a long streak, you'd need several high batting averages in the same year as the high ERA. I'm too lazy to seek out a list of peak batting averages year by year, but I'd bet there were several high batting averages in each year that NYT figure 2 shows a season with a peak in long streaks. And the key thing about the pre-1920 years of baseball is that they are years of high variance - both year to year in average stats, and, I suspect, within the year between the best quartile and worst quartile of pitchers and batters.

Other posters have complained that the method is wrong because it doesn't know about "dead ball years" and other changes to the way the game is played. Although that's superficially true, there is still plenty of information about "dead ball years" captured in the stats. For example, Babe Ruth, one of the greatest hitters, was often pitched wildly or intentionally walked because pitchers and managers were afraid of him. That fact is reflected, perhaps imperfectly, in the Babe's stats. Similarly, changes in mound height, mound to home distance, outfield size, etc are reflected in ERAs and batting averages.

Re:If its so likely, they why hasn't it happened?

[mooingyak]

humans, in general, have certain times of the year when they are more likely to procreate.

One might even say, oh, *statistically speaking* certain times of the year have more birthdays.

Re:If its so likely, they why hasn't it happened?

[mcmonkey]

It's similar to why nobody has had a .400+ season since 1930.

Do you even know what baseball is?

Re:If its so likely, they why hasn't it happened?

[servognome]

Neither is the universe, doesn't stop us from using statistics in physics

Re:If its so likely, they why hasn't it happened?

[bkr1_2k]

I'll play devil's advocate here. Why should baseball get some sort of respect? Yes, it's a reasonably fun game to play, a dead boring game to watch, and a somewhat interesting statistical pastime, but why does it earn respect?

I understand it's entertaining for a lot of people, worldwide. So are crochet and knitting, with arguably far more in their favor as far as benefitting the general populace, but no one asks that they be provided some respect.

So, why?

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

Take a look at the career batting averages for some of the deadball hitters, and factor in that they played in some gigantic parks. 400 hitters weren't uncommon unto the mid-1920s. Singles count towards hitting streaks as much as home runs, so although runs might have been down, this isn't necessarily true for hits.

Re:If its so likely, they why hasn't it happened?

[soulgreen]

i think the formula is flawed:
for example they say that dimaggio had an 81% chance to get a hit based off of his perfomance
from that year. so they adjusted the simulator to use an 81% of the time hit response.

in my opinion you would want to use his previous years data.
i know that still doesn't give you an accurate model but you can't base likelihood of what has already happened.

Re:If its so likely, they why hasn't it happened?

[way2slo]

Yes. In fact, there are a lot of things they likely do not account for. Hot hitters are pitched around and more likely to walk. The ease of a batter getting a hit will change from inning to inning and pitcher to pitcher. Weather conditions. Injury. Pitchers "setting up" the hitter.

Re:If its so likely, they why hasn't it happened?

[Gizzmonic]

I adore baseball. The SPORT of baseball. I don't actually watch MLB,

Then you must not adore baseball too much...sounds like you're more of a stat geek searching for a socially acceptable avenue for your fascinations. But don't kid yourself. You don't love baseball if you're not watching the game at its highest level.

Re:If its so likely, they why hasn't it happened?

[ByteSlicer]

Interesting, it seems not only does God play dice, He also likes playing baseball.

Re:If its so likely, they why hasn't it happened?

[Anonymous Coward]

but if He did, He'd win!

Re:If its so likely, they why hasn't it happened?

[zippthorne]

and I wonder if how managers juggle players around is really very helpful at all, beyond the obvious "get a couple guys on base before the slugger" plan.

Re:If its so likely, they why hasn't it happened?

[zippthorne]

1 in 300 over the whole history of baseball (which lengthens by one year every year) or 1 in 300 over any given year in the history of baseball?

Re:If its so likely, they why hasn't it happened?

[Reziac]

I've often wondered the same thing... instead of reinforcing the notion that only certain combinations work, why not just let 'em play through it and maybe learn something.

Admittedly I've seen it go both ways. But I think the player juggling can paint you into a corner more often than just leaving 'em in catalog order will set you back.

I'm not a fan of slugger-ball, tho... sometimes it's great, but I think it's wrongheaded to *depend* on it.

Re:If its so likely, they why hasn't it happened?

[Goldberg's Pants]

Thank you for proving my point.

Re:If its so likely, they why hasn't it happened?

[Goldberg's Pants]

Nope. Wrong. I watch Japanese baseball. Clearly a top league, or MLB wouldn't be stealing their best players. I watch minor league baseball too. I don't watch MLB due to the asinine commentary I have to suffer (at least Nippon League I can't understand a word), as well as the fact it's utterly corrupt and run by an asshole. To be quite honest, if I could watch MLB without commentary or commercials, I'd watch it religiously.

The guys plugging away in the minors and non-MLB affiliated leagues are where the REAL baseball is, where it's driven by love for the sport rather than money. I also love old baseball. Gimme a recording of a 1935 radio broadcast over ES-fucking-PN's baseball coverage any day of the week.

Working on your logic, nobody watching the Negro leagues truly loved baseball, because they weren't watching the "top level". Watching sport as it's highest level has NOTHING whatsoever to do with whether you love a sport. I love motor racing. I wouldn't dare to say that because someone like Formula 3 but not Formula 1 they weren't a fan of the sport. Your's is one of the most retarded and ill-informed comments I've ever read on here. (And that's saying something.) In fact, if you ONLY watch MLB, then clearly you're an MLB fan, NOT a actual fan of BASEBALL itself. Yes, it's the top league, but given I live 3000 miles away from the nearest team in the majors, I have no allegiance to any team. I'd much rather watch my local team in the Pacific Coast League play than a bunch of guys who are paid more than I'll earn in the next decade, and who are interrupted every few minutes to sell me shit I don't need.

In fact I am in awe at your arrogance. Yeah, I love the stats. If loving stats wasn't socially acceptable, Bill James wouldn't be a very rich man. Not to mention that I'm a grown adult and have long since stopped giving a shit about whether something is socially acceptable. For christs sake we're talking stats, not pedophilia. You think I honestly give a shit if anyone thinks I'm a geek? I wear it as a badge of fucking honour my friend.

Your comment is idiotic, short sighted and ridiculous.

Re:If its so likely, they why hasn't it happened?

[Gizzmonic]

In fact I am in awe at your arrogance. Yeah, I love the stats. If loving stats wasn't socially acceptable, Bill James wouldn't be a very rich man. Not to mention that I'm a grown adult and have long since stopped giving a shit about whether something is socially acceptable. For christs sake we're talking stats, not pedophilia. You think I honestly give a shit if anyone thinks I'm a geek? I wear it as a badge of fucking honour my friend.

You cared enough to write an illogical (Japanese/minor league baseball less commercialized than MLB? Obviously you've never been to a game of either one), irate essay about why you don't give a shit.

Re:If its so likely, they why hasn't it happened?

[Goldberg's Pants]

Troll? Crack smoking dog fucking piece of shit mod.

THAT is a troll.

Nerves

[digidave]

This doesn't take into account that once a player achieves an impressive hit streak he gets more media attention, people start asking him about Dimaggio's record, and every time he steps up to the plate he's a bit more nervous about it than the last time, making it slightly less likely that he'll get a hit.

Re:Nerves

[p0tat03]

That kind of error can be accounted for by tracking their batting averages over time. If we have a model for batting average deterioration due to stress, then the simulation will still work as a good approximation.

Re:Nerves

[EdIII]

That's like saying that a famous player would be more likely to be popular with the women and get the "clap" thereby depriving him of a few games.

In any case, what you are talking about would affect all players equally, therefore it would cancel itself out in their research.

Re:Nerves

[Kjella]

In any case, what you are talking about would affect all players equally, therefore it would cancel itself out in their research. Not when they use it the way they use it, and say streaks of 39 to 109 is to be expected. If the difficulty increases by the length of the streak, 56 could be a far more exceptional streak than their research indicates.

Re:Nerves

[The One and Only]

You know, for a lot of successful athletes stress either doesn't affect them much or it actually works the opposite way: it makes them more successful. These people are who we call "clutch performers". Of course, a lot of talented but non-clutch performers would be eliminated were this compensated for, but a lot of clutch performers would do better.

Re:Nerves

[sykodoc]

So all the players after Demaggio are displaying the Heisenberg principal of sports players!! They are being observed by the public, with feedback of that observation being provided by the media, and so the outcome becomes unknowable.

awesome.

Re:Nerves

[cleatsupkeep]

I remember reading an article saying that "clutch performers" don't really exist - and that the reason we believe they do is because of the same biases that make us cling to our beliefs - taking note of something when it fits your belief and tossing it away when it disagrees.

Wikipedia to the rescue (http://en.wikipedia.org/wiki/Clutch_(sports))

Some sports analysts have presented evidence that while individual plays and moments may resonate as "clutch" because of their importance, there is no such thing as "clutch ability" or an inherently clutch player. One example of such an argument is presented in the 2006 book Baseball Between the Numbers published by Baseball Prospectus, which compiles evidence that no baseball players are demonstrably consistently clutch over the course of a career, and that the numbers of allegedly clutch players in clutch situations are in fact no different from players reputed to be "chokers."[1]

The rest of the page does well with some very good examples from baseball - including Derek Jeter and Reggie Jackson (both Yankees - maybe the author was a Red Sox fan).

Re:Nerves

[coren2000]

Some people get better under pressure. eg. goalies in hockey.

Re:Nerves

[altoz]

It also doesn't take into account the fact that the media, fans, even to some extent opposing players and umpires want the streak to continue. At least with DiMaggio's streak, there were 4-5 games where he should not have had a hit but was creditted with one (2 errors that were scored as hits, 2 times where he should have been walked in the 9th but was pitched to, etc). For whatever reason, people want to see streaks continue and that's far MORE likely than nerves causing someone to have less of a chance to get a hit. The pitchers, the fielders, the umpires are probably MORE nervous than the hitter when there's a streak of that length.

BTW, hitting streaks are a media creation. Nobody cared about them until DiMaggio.

Re:Nerves

[Anonymous Coward]

I think 56 is a formidable streak, and history supports that conclusion. If a study predicts that 39-109 should be achievable, why doesn't anyone get close to 56? A power hitter will never have a 56-game hitting streak, thanks to walks (both intentional and semi-intentional). A guy like Ichiro has the foot speed and lack of power that provides a lot of infield hits, and no incentive to walk him. But most of the guys who start their careers as speedsters are then pressured into bulking up to hit for power. This is the name of the game in the AL, where you need to hit 15HR to play 2nd base. Even the NL is beginning to catch on to the quest for power. I wonder if steroids will ever be phased out, and if so, will the game change in a significant way?

Deistic proof?

[Anonymous Coward]

I think this is basically a form of statistical proof that there is, indeed, a God.

Re:Deistic proof?

[Loadmaster]

What would God need with an impressive hitting streak? Besides, Jesus can't hit a curve ball so he'd never make it anyway.

Re:Deistic proof?

[TheRealMindChild]

Apparently, Joeboo can't either :(

Re:Deistic proof?

[spisska]

Besides, Jesus can't hit a curve ball so he'd never make it anyway.

The curve he's alright with, and can manage a bit of opposite field power. It's the backdoor slider that gives him trouble. And the knuckleball.

Those pitches are a bit too deceitful. I heard that after 40 days of batting practice he was offered some kind of deal that would let him see the pitches before they were thrown but he turned it down. Oh well. I guess that's why he's still playing in Iowa.

Re:Deistic proof?

[Anonymous Coward]

No need to introduce FSM.

If we combine this research with Anthropic principle and "Many worlds" interpretation of QM, according to Occam's Razor, we get only one logical explanation for this anomaly:

- In the past, this amazing record in the past intrigued or impressed and thereby actually diverted someone who was capable of destroying our world from actually doing so. Was that someone a diabolic Evil Genius mastermind, or a suicidal person in command chain of nuclear ballistic missiles during Cold War, inquiring minds would like to know!

Re:Deistic proof?

[FunkSoulBrother]

There are lots of guys named Jesus in the major leagues, perhaps one of them will do it.

How to Make Baseball Even MORE Boring?

[morari]

Talk about the statistics of anyone at bat..

Re:How to Make Baseball Even MORE Boring?

[pchan-]

You don't understand. Baseball is so boring, the fans find the statistics exciting!

Re:How to Make Baseball Even MORE Boring?

[garett_spencley]

I was once at a friend's BBQ and a lot of the other guests were really into sports and talking a lot about their various sporting events etc. I made a comment about how baseball was one of those sports that is fun to play but boring as hell to watch. One of the guys responded with, simply, "I disagree". To which I replied "You're right. It's pretty boring to play too." He wasn't very amused.

Talk about a great way to make an awkward social event even more awkward :(

Re:How to Make Baseball Even MORE Boring?

[davolfman]

It's decent to watch provided you are in the stands, reasonably close to the field. Otherwise you're just looking through a straw.

Re:How to Make Baseball Even MORE Boring?

[rob1980]

It's definitely one of those sports that is boring to watch on TV, at the very least. Going to a college or minor league game is a good way to spend an afternoon with the kids or some friends, as long as you eat beforehand so you don't talk yourself into paying $4 for a hot dog, $6.50 for a bag of peanuts, etc etc etc. I've heard hockey's the same way, but where I live it's pretty tough to see a live hockey game let alone one on TV.

Re:How to Make Baseball Even MORE Boring?

[jd]

Do you have the statistics to prove that?

Re:How to Make Baseball Even MORE Boring?

[zach_d]

... maybe it's my nationality talking, but I fail to see how hockey's boring on TV.

now, if I was defending curling, I know it's my nationality, but hockey's got some pace to it.

Re:How to Make Baseball Even MORE Boring?

[Pollardito]

... maybe it's my nationality talking, but I fail to see how hockey's boring on TV. i finally went to a hockey game recently after a lifetime of finding the game infinitely boring to watch on TV, and it's definitely a lot better in person. so much of the checking and player-to-player action is going on off the screen in TV, and it's way easier to follow the puck in person than it is on the screen

Re:How to Make Baseball Even MORE Boring?

[Anonymous Coward]

"Baseball is five minutes of action packed into three hours." -- Yogi Berra

Re:How to Make Baseball Even MORE Boring?

[jocknerd]

While I can't agree or disagree with you on whether you find baseball boring, it is still my favorite sport. While I'd rather watch a college football game than a baseball game on television, I can read about baseball. I can't read about football. I can study baseball.

I find soccer extremely boring, but its probably because I don't really know the strategy in the sport. I understand the strategy in baseball. While the game on the field goes a bit slow, the strategy that is taking place behind the scenes is fascinating. Thats one of the reasons I can't stand the DH. I prefer the strategy that comes from having a pitcher in the batting lineup over watching the extra offense a DH provides.

If you haven't ever watched Ken Burn's Baseball, I recommend you renting it from Netflix.

Re:How to Make Baseball Even MORE Boring?

[Boronx]

It's the ultimate sport: everyone's going about 5 times faster than they ought while wielding deadly weapons.

Re:How to Make Baseball Even MORE Boring?

[krelian]

I love baseball. I view it as a large scale RPG only that I am not the DM nor a player (when I want to actually play I fire up ootp which allows me to play several adventures in a short amount of time or even include some of the heroes from old lore into my adventure.

I don't find baseball on T.V very interesting but it's a fun sport to follow up on

Re:How to Make Baseball Even MORE Boring?

[NateTech]

Usually if baseball appears boring to someone, it's because they know very little of the nuances of the game. Defense is quite a mental challenge in many situations, but people just see a bunch of guys standing there.

That's not what the players are thinking, however.

In order to learn the nuances and strategies, usually one must have PLAYED on a team, to really get it. Most people don't or have only played in non-competitive softball leagues, and completely miss probably 50%-75% of the real "action" going on.

As far as statistics go... the reason baseball fans find them interesting, isn't so much because they're just fans... you need to know those statistics for strategy on the field, defensively.

Thus, fans that follow stats, aren't doing it because they like stats -- they're doing it because it helps them think about what the players and coaches are thinking when a particular guy takes the batting box.

Baseball can be very subtle when played to its best, and Americans aren't much for subtlety these days. Just like they also don't get real satire.

Re:How to Make Baseball Even MORE Boring?

[garett_spencley]

Actually I have played baseball and I understand the game and it's rules and nuances quite well.

I still find it boring to watch and for the exact same reason that I enjoy playing Chess yet find Chess games to be boring to watch.

They take too long between plays and the entire game lasts far too long. Call it a short attention span if you must but it's actually a problem that I have with most sports, baseball just happens to be one of the worst.

I prefer seeing CONSTANT action. Tennis is a rare example of a sport that I enjoy watching because there are little or no timeouts. No interruptions. No long setups. It's just play and play and play until the game is over. I can get into it. With American Football, Hockey, Baseball etc. it's almost like constant commercial interruptions every couple of seconds.

Not to mention that baseball lasts 9 innings so the whole ordeal takes ages. Granted it's nowhere near as bad as Cricket (like someone else pointed out) ... but then, as you may have guessed, I don't like Cricket either.

Re:How to Make Baseball Even MORE Boring?

[NateTech]

Ever considered adding to the experience to give your head something to do? I like to keep a scorecard during most games I attend. It fills in the "down-time" nicely.

Learning how the shorthand used on a baseball scorecard, and developing your own style so you can "read the game back" to yourself later, is an interesting thing to do, and keeps your head in the game... you can start to see how a player is doing that day vs. the last time you saw him play, etc... did he hit to right field more than left? Etc. (Also can be pretty invaluable if you ever find yourself coaching a recreational league team.)

I understand your need for constant action. It goes away as you get older.

A nice baseball game, my scorecard, a brautwurst/hot dog, a beer, and my AM headphones (I like listening to the radio announcers instead of the inane announcers are most stadiums, and you get some other background information on the players)...

And I'll meet you at the ballpark ANYTIME! Take me out to the balllllll-game... heh.

I'll save the "excitement" for the airport.

(I fly small aircraft.)

Now, commercial airlines... THAT's boring! Baseball is FAR more interesting than riding in high speed aluminum tubing stuffed to the gills like the lower levels of the Titanic!

Just think, we live in a country where every single one of us, not just chosen "military" or other priveledged people can fly one of THESE:

http://www.cirrusdesign.com/

And instead, we drive around in minivans on weekends to soccer games. Sad. Teach the kid to fly instead.

Of course it's not that simple

[Anonymous Coward]

A batter with, say, a .300 batting average does not have a 30% chance of getting a hit each time he's up to bat. There are inevitably going to be days when you face someone like Randy Johnson or Roger Clemens or (cross fingers for 2008) Felix Hernandez. Couple that with some fantastic defenses and it's not surprising that a good batter goes a day without a hit.

So players have some days with a smaller chance of getting a hit than others (say, when the 2007 Mariners were running Horacio Ramirez out there with our god-awful defense). There's a reason advanced baseball statistics are more complex than what these guys did.

Re:Of course it's not that simple

[glitch23]

A batter with, say, a .300 batting average does not have a 30% chance of getting a hit each time he's up to bat.

There is a legitimate reason for that. A batting average is not a prediction for the future but a record of the past. Of course, the same applies to all other statistics in any subject for that matter.

typo

[Anonymous Coward]

i found a typo streak!!!

"very alternate universe produced a steak f 39 games or better"

2 typos in a row!!!

Re:typo

[Chris Burkhardt]

Pfft. That's nothing. I onc mitsyped, lik, fivve wrds in a row.

Re:typo

[smitty97]

Looks like the editors are on top of it, its been fixed to: "a steak of 39"

i came here to make an insightful comment

[circletimessquare]

unfortunately, not many of my comments are insightful, so with my batting average, you will have to refer to a parallel universe

there you will find that this comment contains something worthwhile reading. sorry

Re:i came here to make an insightful comment

[laejoh]

I guess my batting average must be pretty low as I ma(i)nly write bash shell scripts, not Microsoft MS-DOS ones!

Changing game of baseball

[__aanonl8035]

One of the key points mentioned in this article is when does the hitting game streak occur? They mention that it was much more likely to occur during the early 1900's which is known as the deadball era. The baseball wasn't as springy and they tended to use the same ball during the entire game. During that time it was more efficient to try and knock the ball between the holes in the fielders and get a double or single then to try and hit it out of the park.

I think it would be more impressive to take a subset of the data, and compare from 1930 up until the present. Of course, there have been other major changes to; glove sizes, introduction of the slider for a pitch, steroid use.

too simplistic

[ndenissen]

From reading the article (which is light on the details) it seems like they used nothing but batting average, at bats, and games played.

The problem is this doesn't control for variances in the quality of pitching. The chances of going that many games without running into a hot pitcher isn't accounted for.

Imagine you average a 75% chance of getting a hit in any individual game. If you face three average pitchers, your chances are (.75)^3 but if you face a good pitcher an average pitcher and a bad pitcher it might be (.5)(.75)(1.0) which gives a different probability, despite the same average number of hits.

In order to be realistic the calculation would need to account for the deviation from average in the ability of the pitchers (which would likely be higher 100 years ago because of fewer player and segregation, and now because of expansion, as compared to the 1950s)

What they don't report is how often there are long (but not record) streaks in their model, so there is no way of knowing how accurately it reproduces reality.

Re:too simplistic

[DannyO152]

On the other hand, one doesn't get the benefit of running into the belly-itchers. My feeling is that, on average, the superstars, the ones with above 340 career averages, generally feasted on the mediocre to minor pitchers.

What this study doesn't take into account is how long it takes to live through a streak. DiMaggio needed two months. Besides the strain of day to day playing (and if it's a pennant race, you know the hot hitter is going to be in the lineup) there's also the way the weather and the light changes during the season. There used to be more day games and double-headers back in the 30s-40s-50s when batting averages were highest. Travel was by train and by bus and took longer. There seems to be a week every season when a cold or flu is making the rounds of the club. Then there's situational issues. 7th inning and behind, man on second base, the hitter is 0-3 and 30 games into the streak. I say the pitcher semi-intentionally walks the batter and amid a chorus of boos the streak goes poof. Here's another consideration, the opposing players and pitchers know the hitter has a streak when it gets past 20 games and the pitching gets a bit more careful and the batter has to extend the streak via pitchers' mistakes, and that makes it less likely.

if what I say is true, it should follow that the incidence of any consecutive games with a hit streak beyond 15 in a MLB season should be lower than the probability suggested by the league batting averages (which are depressed in the NL by pitchers and the other bottom 4 from the lineup.)

Too many assumptions?

[kevinatilusa]

From the descriptions I've seen of their research, it seems that they're treating all games identically for the purpose of determining a typical season's behavior. While this may me necessary to make the computation tractable, it's not realistic, and introduces a sizable bias towards long hitting streaks.

In reality, a league is typically very imbalanced from team to team and from pitcher to pitcher (probably even more so in the game of the early 20th century than now). It's easier to get hits off of two successive average pitchers than it is to get hits both off of a very good and a very bad pitcher. For example (to oversimplify a good deal):

Say the league is split 50/50 between "good" pitchers (pitchers you'll get a hit off of 50% of games) and "bad" pitchers (pitchers you'll get a hit off of 80% of games). In a typical 20 game stretch, you'll encounter 10 good pitchers and 10 bad ones, and your odds of getting a hit in all 20 games would be (0.50)^10(0.80)^10, about 1/9537.

Under their analyis as I understand it, they'd replace all the pitchers by mediocre pitchers who you'd get a hit off of 65% of the time, and your odds would be (0.65)^20, about 1/5517.

This one assumption almost doubled your chances of getting a hit in all 20 games.

There are other biases as well going the other way (ignoring the effect of hitting slumps, for example), but this one jumped out at me.

Re:Too many assumptions?

[mortonda]

Wait. You're saying that you can make statistic lie? Say it ain't so!
</sarcasm>

Bogus

[DoofusOfDeath]

Shouldn't we say that the probability of it happening was 1.0, because it did happen?

It seems to me that if their experiments report anything else, then either their models are erroneously inaccurate, or they got something else wrong.

Re:Bogus

[Miseph]

No, because the probability for ANYTHING, given enough chances, is 1.

What they are actually saying is that reality appears to follow a probability bell curve.

You could also say that, in 1,230,000 years of baseball games, we could be almost certain of a hitting streak longer than 56 games.

Re:Bogus

[Anonymous Coward]

Your lack of understanding of statistics is horrifying. I just flipped a coin ten times. The most likely outcome is 5 heads, and the fact that I got 7 does not show that the usual model is in any way inaccurate.

Re:Bogus

[Anonymous Coward]

"No, because the probability for ANYTHING, given enough chances, is 1."

ehhh... no.

I'm pretty sure that is not the case for rolling a 13 with a pair of standard dice or drawing 5-of-a-kind from a standard deck.

Re:Bogus

[DoofusOfDeath]

I just flipped a coin ten times. The most likely outcome is 5 heads, and the fact that I got 7 does not show that the usual model is in any way inaccurate.

I disagree on two counts. First, when you say the likely outcome is 5 heads, your asserting that the mean of the PDF for the coin is 0.5. I.e., your model includes a mean of 0.5 as a detail. Because you're taking that as an established fact, no matter how many heads you got in a row could change your mind about the PDF's true mean - you've already taken a mean of 0.5 as a given.

But I was making a very different point in my OP. The particular baseball stat is a "state of nature". It happened. If we assume that the universe is deterministic, then it's nonsensical to assign a probability other than 1.0 to that event. Any model that produces an answer other than 1.0 for that event is probably replacing a variable that in the real world has a fixed value, with a random variable. Thus, the researchers had to introduce an inaccuracy in their model in order to make anything about their outcome be uncertain. It's more of a modeling issue than a probability issue.

Re:Bogus

[Reziac]

Hell, in 1,230,000 years of baseball games, *I* could get a 56 game hitting streak. ;)

True story: I am the world's worst volleyball server. I'm lucky to whack the ball in the correct direction, let alone get it over the net. One day in the 9th grade, I scored 14 spikes in a row. Everyone (including the gym teacher) is staring at me like "WTF?! What have you done with the real Rez??" Needless to say it was a one-time freak event. :?~

Re:Bogus

[Anonymous Coward]

Indeed, now where is they guy throwing 190 in darts? I mean the probability of ANYTHING given enough chances is 1, and the FSM knows they darts players get enough chances

Re:Bogus

[grammar fascist]

But I was making a very different point in my OP. The particular baseball stat is a "state of nature". It happened. If we assume that the universe is deterministic, then it's nonsensical to assign a probability other than 1.0 to that event. Any model that produces an answer other than 1.0 for that event is probably replacing a variable that in the real world has a fixed value, with a random variable.

Funny, we just talked about that a few days ago. The crux is that you're not necessarily right: it depends on whether you're approaching the problem with a frequentist ("frequency of occurrence") or Bayesian ("degree of belief") interpretation of probability. The Bayesian view is more widely applicable (for instance, this study) and easier to model in (because you can easily compose distributions that don't integrate nicely).

Stodgily advocating a single definition and purpose of "probability" is cute and all, in a "get off my lawn" sort of way. But it's a little irksome to those of us who use Bayesian models to solve AI / machine learning / computer vision problems - among other folks.

Still, feel free to tell everyone you know that a coin flip that has already happened but that you haven't observed the outcome of has heads probabilities of 0.0 and 1.0, period. That won't help anybody who has to compute an expected value over outcomes in similar situations, though. To do that, you need a framework in which "How likely was that?" is a valid question.

Also, the assumption that the universe is deterministic is fairly iffy, especially given our current understanding of quantum mechanics, wherein the only true state of a small-scale object is actually a probability distribution over states.

I probably sound a lot pissier than I am right now.

Re:Bogus

[PopeGumby]

No, because the probability for ANYTHING, given enough chances, is 1.

Or rather, APPROACHES 1.

If I toss a coin a million times, the odds on getting a single head toss approach 1, but not quite 1, because I could toss a million tails.

Clearly...

[Corpuscavernosa]

... they didn't take into account my 162 game hitting streak in "The Bigs" on PS3. With settings on easy.

Isn't this a monkey typing thing?

[PC and Sony Fanboy]

Isn't this the same thing as saying that an large number of monkeys typing for a large period of time is more likely to properly re-create the complete works of tolkien ... by accident?

Re:Isn't this a monkey typing thing?

[webmaster404]

No, It would be like taking monkeys, giving them RPG style intelligence points, and running them via an RNG and see which one can write the book to stay on the best seller list for the longest time.

Re:Isn't this a monkey typing thing?

[Anonymous Coward]

how many times could I reroll my stats? And what race of monkey?

How to make STATISTICS even more boring?

[sayfawa]

Involve baseball. There, fixed.

Also, go Jays.

You can't do statistics with a random # generator

[rufusdufus]

Our simulations did something very much like this, except instead of a coin, we used random numbers generated by a computer.
It is not mathematically sound to do statistics with a random number generator. Computers do not actually generate random numbers, but instead, they can only make pseudo-random numbers that have a certain distribution.
Any 'simulation' done in this way will always have a bias.
In order to get correct statistics, you must actually compute the statistics.

news?

[superwiz]

maybe. for nerds? i doubt it. "the math guys"... ok, definitely not news for nerds -- too dismissive of the experts

Re:news?

[sgbett]

more importantly (re: your sig) who would ever need a slashdot post number over 2,147,483,647!

So basically...

[davidbrit2]

They took a bunch of measured statistics, ran a simulation with outcomes biased using said statistics, and then acted surprised when the simulation results ended up pretty close to what actually happened?

Re:So basically...

[kevinatilusa]

They took a bunch of measured statistics, ran a simulation with outcomes biased using said statistics, and then acted surprised when the simulation results ended up pretty close to what actually happened? I think their point was that they took a set of numbers that were generally considered unremarkable (the overall statistical distribution of batting totals from the last 100+ years) and tried to show that a number that most people considered very unusual (the 56 game streak) was in fact also typical given this other, "unremarkable" set of data.

Anohter unreported weird fact

[mrfantasy]

In every simulation, a ground ball went between Bill Buckner's legs in the 1986 World Series.

Re:Anohter unreported weird fact

[siwelwerd]

Wow, 5 people here knew enough about baseball to mod that funny.

Re:You can't do statistics with a random # generat

[kevinatilusa]

It is not mathematically sound to do statistics with a random number generator. Computers do not actually generate random numbers, but instead, they can only make pseudo-random numbers that have a certain distribution. Any 'simulation' done in this way will always have a bias. In order to get correct statistics, you must actually compute the statistics. Sure, the proper way to put it mathematically would have been "we did a Monte-Carlo based simulation of the probability distribution of the longest hitting streak under our model due to the intractability of direct computation", but this is an editorial in the New York Times, not a mathematical journal! As a side note, just because a computation is performed on a set of pseudorandom numbers does not mean it is biased...usually the whole point of pseudorandomness is that the discrepancy between computations involving them and identical computations involving true random numbers will typically be quite small.

Re:You can't do statistics with a random # generat

[Anonymous Coward]

I think it's safe to say that most all statistics uses a random number generator. Computers *do* have the capability to produce true random numbers, as shown with /dev/random, which relies on an entropy pool and is suitable for cryptographic key generation.

What about slumps?

[Squirmy McPhee]

By assuming the hitter's probability of getting a hit is equal to his season average the researchers don't take into account that most, if not all, batters have a higher batting average at some points in the season than they do in others. As one with experience in Monte Carlo simulations I know that taking that into account would complicate the analysis considerably, but I suspect their results would be a bit different if they even did something as simple as using a 10-game moving average of the batter's average.

Re:What about slumps?

[jd]

Yes, you'd need the variance and not just the mean, and you'd need a suitable distribution, which will probably not be symmetrical and certainly not a single spike.

Re:What about slumps?

[Squirmy McPhee]

Yes, you'd need the variance and not just the mean, and you'd need a suitable distribution

That would be another way to do it, sure. You could also use a Markov chain with states corresponding to slumping/not slumping (or something like that). Bootstrapping might not be a bad way to go, either. It all depends on how much computational effort you want to expend and how well suited these various methods are to the actual statistics (that's a project unto itself). Even then, the decisions a batter and a team's manager make in the real world depend very much on the in-game situation and whether or not a player is slumping. If Joe DiMaggio isn't going into game 45 of his hitting streak, for example, then maybe on his last at-bat of the game the manager has him sacrifice bunt instead of trying to get a hit.

Because of that, I can understand why the study's authors took the approach they did -- from a statistical standpoint, you can argue with the method, but more rigorous methods won't necessarily be reflective of the real world (or a "real" parallel universe). That's why I suggested using something like a 10-game moving average: I think it strikes a balance between rigor and real-world insight, particularly since this isn't exactly an earth-shatteringly important topic. That's just armchair statistics, though.

Re:What about slumps?

[djp928]

Except that slumps are already accounted for--they're just normal random distribution. Slumps and "hot streaks" can only be identified after the fact, and they turn out to be just what you'd expect in a random deviation from the mean. Toss an unbiased coin 1000 times and you'll see "hot streaks" and "slumps" of heads.

Re:What about slumps?

[Squirmy McPhee]

Except that slumps are already accounted for--they're just normal random distribution. Slumps and "hot streaks" can only be identified after the fact, and they turn out to be just what you'd expect in a random deviation from the mean. Toss an unbiased coin 1000 times and you'll see "hot streaks" and "slumps" of heads.

You're assuming the probability distribution doesn't change over the course of a season. That's a reasonable assumption when you're tossing a coin -- the weight distribution of the coin doesn't change from toss to toss -- but I'm not sure that's a reasonable assumption for a batter going through the emotional and physical ups and downs of a baseball season and facing pitchers of varying skill. Coins don't have good and bad days, but people do.

Does Joe DiMaggio's Streak Deserve an Asterisk?

[harryjohnston]

This seems relevant:

http://abcnews.go.com/Technology/WhosCounting/story?id=3694104&page=1

Disclaimer: I'm not an American, so I know next to nothing about baseball - and care less!

Re:Does Joe DiMaggio's Streak Deserve an Asterisk?

[Vellmont]

The thing I find strange is the idea of
The record book.

As if there's some big "official" book that's published as
"The Record Book Of Baseball", and MLB officials all sit around arguing about asterisks. I'm no baseball fan, but I never thought there was "The" record book. Isn't there just a series of "A" record books?

I've never been a big sports fan, but what drives me crazy, especially about a game like this where you win or lose, is how people get all hung up about this record or that record. What does it matter? The players play the game, the people watch the game, they get caught up the in the competition, the game is over. Then people go home and start whipping out their pencil and calculator. That just seems kind of odd to me. The records just seem so created and artificial.

Re:Does Joe DiMaggio's Streak Deserve an Asterisk?

[Otter]

I'm no baseball fan, but I never thought there was "The" record book. Isn't there just a series of "A" record books?

My understanding is that MLB (and the other major North American leagues) does maintain an official record book, but that the Roger Maris asterisk is a myth.

Re:Does Joe DiMaggio's Streak Deserve an Asterisk?

[Paradise Pete]

but that the Roger Maris asterisk is a myth.

No it's not. The asterisk was on there for many years. Babe Ruth was revered, and Maris failing to break it in 154 games, which was the season length when Ruth set it, was all the excuse needed.

Re:Does Joe DiMaggio's Streak Deserve an Asterisk?

[Otter]

Babe Ruth was revered, and Maris failing to break it in 154 games, which was the season length when Ruth set it...

Yes, I know what the asterisk story is, but my understanding is that it's not literally true (although the record book did note both records, unlike the practice for most 154 game records). Both Wikipedia and Bud Selig disagree with you, although admittedly neither is a completely convincing source.

Re:Does Joe DiMaggio's Streak Deserve an Asterisk?

[Paradise Pete]

Hmm. Admittedly, I was going on my memory. I was too young to remember the 1961 season, but soon after I became aware of baseball and even then "everybody knew" about the asterisk. I see this NY Times obituary, but I wouldn't say it's a convincer. There's Billy Crystal's movie 61*, and he'd remember better than I, and presumably he did some research, but I'm not going to watch it again just so I can see what it says.

It's too bad the Wikipedia article doesn't cite a reference for the "urban legend" statement. But considering how, uh, fanatical baseball fans can be, the fact that it is still in the article, unchallenged, must mean something.

Re:Does Joe DiMaggio's Streak Deserve an Asterisk?

[Paradise Pete]

Reading further, this article seems pretty clear about the whole thing. So I think you're right, there was no official asterisk, though essentially Frick put one there in spirit.

Thanks for the education. (Though I could probably put those brain cells to better use than remembering another baseball item :-)

Re:You can't do statistics with a random # generat

[Anonymous Coward]

No, it won't always have a "bias". Bias is a technical term here, and infact there is not likely to be bias. Bias is where the long-run average value of simulated variables is not equal to the actual average value of the thing you're simulating. For example, rolling a chipped die to draw numbers uniformly from 1 to 6 will probably cause this.

The problem with pseudorandom number generation tends to be dependence between samples (barring a more serious bug, which has happened... but this is always a problem, and there can also be bugs in the rest of the code anyway). Now this correlation is a problem for cryptography maybe, since there is intense interest in every bit of entropy in a very short signal, and a lot of clever guys hacking at it.

However in statistics, you basically just use the random numbers as "fuel" for a sequence of very stupid computations (more or less, glorified averages and averages of squares, &c.). The functions used in statistics are just too stupid to find out that the numbers have inter-dependence, so that they tend to give the same results for pseudo-random numbers as for real numbers. This is thanks to a lot of hard work from many fields, to improve pseudorandom number generators.

In fact, and as a tangent, theoretical computer scientists tend to believe that any randomness in an algorithm can be replaced by deterministic functions! (although they don't believe this as widely as they believe P!=NP). Since we can consider any statistical procedure an algorithm, the effect (at least philosophically) that this would have on many applied fields is mind-boggling. I would love to a proof and some general techniques for this "derandomization" - if there were one, we could finally absolve ourselves of our state of sin. (It would also imho inform the "free will" debate a bit.)

On the point of baseball

[earlymon]

A lot of people think baseball is boring - today it is, but take it from a geezer, not always so.

I blame television. I can no longer watch a ball game on TV. Might as well be Entertainment tonight. They used to have a camera behind the backstop so you could see the pitch, the swing (from behind) and the infield. Another camera to go to the outfield, and maybe one for the infield. They game has strategy. It has finesse. It even has - to use a term no longer apparently known in the software world - elegance.

Now, it's unshaven bums (I'm an unshaven bum today, so OK) close-up, on the mound, with bad hair cuts curling from under the caps, with their follicles in high def while they spit and scratch. High res, high def and no sense of a team at work - if there still are any!

Baseball is a noble game of strategy, ruined by Madison Avenue's need to sell multi-hundred dollar sneakers to our kids.

Want to check out a good ball game? Here you go - http://www.startrek.com/startrek/view/series/DS9/episode/103565.html

So, if baseball's good enough for Klingons and Vulcans and Ferengi, it's good enough for /. - and this covers the alternate universe part as well.

Now - get off my lawn!

Re:On the point of baseball

[Reziac]

Bah, Vulcans could never play baseball; they'd be too caught up in the stats! It'd be exactly like Gene Mauch and the Angels -- everything done right per the stats book, but never, ever would they try anything that was out of spec. It's not the stats that win games; it's the quarter of an inch you reach beyond what your stats say you can. (I love little ball, but Mauch made me crazy.)

I haven't been where I could get sports TV reception (or even radio reception) in 11 years. Before that, I worked my business hours around Angels games. :) But the trend was already there -- too much dwelling on details that don't really tell us anything. I want to see the pitcher's presence on the mound, not the zits behind his ear. I want to see the batter's body language, not his chaw-drip. I want to see the runner's balance between the bases, not his crappy haircut. Baseball may be a game of inches and microcosms, but it's first and foremost a game of *balance*, and that means the whole team, seen as a team.

I think what happened was that because we COULD do extreme zoom and the like, we DID, whether or not we SHOULD. Football is going that way too -- sometimes after a play that was zoomed in too tight, I find myself thinking "Wha'happened?" because the whole view was too lacking.

True story: for a while I had such bad TV reception that all I could see was shadows. I knew every pitcher by his motion, every batter by his stance -- and none of their faces, cuz I'd never seen 'em. My sister comes in and wants to know why I'm intently watching static. "See here," I tell her with great enthusaism, "this is X on the mound, and Y at bat, and watch how that runner is leaning toward second, etc, etc." She peers closely at the TV, and finally says "I don't see *anything*!!"

Re:On the point of baseball

[peektwice]

I disagree on the point of baseball being ruined. There is a decidedly downhill slide occurring and it has much to do with marketing (among many other things), as you correctly point out. However, it's not beyond repair. Complicity between owners and players in the steroid scandal, marketing "multi-hundred dollar sneakers to our kids", player strikes, new stadium building binges funded by taxpayers, and all the maladies that affect baseball will never be able to overcome the nobility of baseball, unless we the fans allow it.

However, take a kid to little league practice, or go watch a double-A game sometime. You'll see that all is not lost. I went to the Springfield (MO) Cardinals vs. St. Louis Cardinals game just this Saturday, and saw Albert Pujols catching the first pitch thrown by a kid with Down Syndrome. That kid was smiling from ear-to-ear. He threw out the first pitch, and Albert caught it and gave him the ball.

Later, I saw some small minor league kid hit into an easy out. I also saw his first base coach, at a level in baseball where players sometimes begin to become "uncoachable" *cough*J.D. Drew*cough*, actually coach him. He gave him pointers on what to look for from the pitcher, where to try and hit the ball, and then gave him an encouraging slap on the behind.

My kid's little league coach is a woman. She's one of the few coaches I've seen that can actually coach. She fits more instruction into an hour of practice than most of the men I've seen coach. She doesn't have to yell to do it, and the kid's pay close attention.

Take heart, my /. friend. Baseball will prevail. Just look at the face of any kid getting an autograph from their favorite, larger-than-life player and you will see.

I'll be taking my leave of your lawn now.

In an alternative baseball universe...

[__aaclcg7560]

Tickets, hot dogs and beer would be a lot more affordable.

Cricket

[flyingfsck]

Actually, baseball is very exciting compared to cricket.

Re:Cricket

[Alpha Whisky]

Actually, the English invented baseball, but we gave it up to play the vastly superior game "Rounders". Rounders is now only played by schoolchildren and even then only when they don't have enough space in their timetable for the infinitely superior game of Cricket. We consider it quaint that you colonials still play that obsolete game. It's nice that someone preserves the old ways.

Re:Cricket

[Opie812]

troll? come on that's funny shit!

Re:Cricket

[RealGrouchy]

Actually, baseball is very exciting compared to cricket. Except baseball doesn't have deadly robots.

- RG>

Re:Cricket

[Alpha Whisky]

You make an interesting and mostly valid point Phil, I don't know why they modded you troll, you must have touched a nerve somewhere. But of course American Football bears more of a resemblance to Rugby Football than Association Football, and you don't need any special equipment to play Rugby either (except access to a hospital). Rugby games don't stop for a rest every ten seconds either. It's hardly surprising that Football is associated with violence, it did originate from armies, winding down after a battle, kicking the severed heads of their enemies up and down the town.

Something like the modding on /.

[Anonymous Coward]

The parent and the post immediately after it make the same point. The parent is modded 1 and the one after is modded 5.

The point is that the statisticians didn't accurately model the chance of a hitless game.

it's basically a sure thing.

[iminplaya]

Yeah, and so is the Cubs winning the World Series more than once in a hundred years

Re:it's basically a sure thing.

[Anonymous Coward]

Surprisingly, that was one thing the experimenters didn't find, but they fully expect the Cubbies to claim a modern-era WS when they run their second or third batch of 10,000 unis.

Re:it's basically a sure thing.

[Al Dimond]

I just ran a quick calculation assuming that every team in the NL has an equal chance of winning the NL pennant, and then a 50% chance of winning the World Series after that. This was so I'd only have to keep track of the number of teams in the NL instead of in both leagues. It doesn't account for a lot of stuff, and I don't care.
 
For the first 53 years of the WS drought, 1909 through 1961 in an 8-team National League
 

octave:1> (1 - 1/8 * 1/2)^53
ans = 0.032695

There's only a 3.3% chance of the Cubs not winning the WS. The league started expanding in 1962, and for the remaining 46 years, the odds of the Cubs not winning are
 

(1 - 1/10 * 1/2)^7 * (1 - 1/12 * 1/2)^24 * (1 - 1/14 * 1/2)^5 * (1 - 1/16 * 1/2)^10
ans = 0.15262

15.2%. The total probability that any given team wouldn't win a World Series over 1909-2007 comes out to just under a half-percent.
 
The Cubs, though, for many of those years, didn't have the same chance to win the NL pennant as any other team. Losing 7 straight World Series between 1910 and 1945 is awful luck (1/128 probability of losing those Series), but not appearing in another one since then is systematic ineptitude. Not making the postseason from 1946 through 1983, and having no fewer than 3 postseason-less years between each appearance in the various expanded playoff formats that have arisen since then, is not bad luck, it's bad management.

tl;dr: through 1945, blame the goat. Since 1945, blame the suits.

  - A lifelong Cubs fan.

Re:Lies, damned lies, and statistics

[Anonymous Coward]

Remember, any event ultimately has a 50-50 chance. It either happens or it doesn't. The particular outcome is rarely significant, it is a binary proposition.

Tell that to the casinos.

Re:Lies, damned lies, and statistics

[demeteloaf]

See, I think the quote you quoted really doesn't sum up the article at all, or at least, it's very misleading. The article doesn't say that streaks of 56 games are a common occurrence. What it says is that the fact that baseball history has a record of a 56 game hitting streak isn't that rare, given players historical batting averages.

Think of it this way. I flip a coin 10 million times and find the longest streak of heads. Say it's 50 (I have no idea how likely that is and i'm too lazy to actually simulate it, so i'm just going to make up numbers). I flip another 10 million coins, and find that the longest streak is 53. I flip another 10 million coins and find the longest streak is 40. I do this another 10,000 times and record the record in each case, and get a distribution for the longest streak in any given 10 million flips. Now, i'm not going to claim that 50 heads in a row isn't an extremely rare streak. But i can say what the odds are of getting a streak of at least 50 heads when i flip 10 million coins are.

That's what these researchers did. Simulated 10,000 seasons from 1871-2005 (using players historical batting averages as their likelihood to get a hit) and found what the longest hitting streak in each 'alternate history' is. Their claim is that 56 games is not an unusually high number (in fact It looks to be right about the median.) So sure, DiMaggio's streak was great, and incredibly rare. But it's interesting to see what the odds are of having a hitting streak that long.

I've switched domains...

[xactuary]

I've switched domains to operating systems and can now say that it took 42 googleplex simulations before I found a parallel universe where Vista doesn't suck. As you would expect, that's also the only parallel universe that had Steve Jobs throwing chairs.

Re:I've switched domains...

[Rod Beauvex]

As you would expect, that's also the only parallel universe that had Steve Jobs throwing chairs.

I had to read this four time for my brain to process it correctly.

Re:You can't do statistics with a random # generat

[UnknownSoldier]

There is no such thing as true randomness. You can't measure something without effecting it.

Consciousness can effect randomness, as this Princeton page proves.

Feedback...

[argent]

There's no feedback here.

Don't forget that the makeup of teams, the behavior of other players, and even the rules of baseball all depend on what happens in the game. If someone was setting a 109-game hitting streak in the 1890s, then they would be facing more determined pitchers and probably better pitchers by the time they were more than 20 or 30 games into the run. It seems pretty good odds that would have changed their batting average for that year. :)

How are real hitting streaks distributed in time? Do they bunch up in the 1800s and early 1900s the way their simulations did?

Were there changes in the rules between the 1890s and the 1940s that might have reduced the effect of this kind of feedback mechanism?

Someone please mod parent Troll.

[Enderandrew]

Someone please mod parent Troll.

Ted Williams might be surprised...

[emaja]

(if he was still alive) to read that no one has hit .400+ since 1930, since he hit .406 is '41.

Baseball is one of those things that if your father didn't like, you probably won't either. It seems to be a passion that is passed down from generation to generation.

Today, it seems a bit archaic or even boring to some because of the slower pace, but some of us appreciate it for those very same qualities.

Re:Ted Williams might be surprised...

[Anonymous Coward]

Hitting .400 is not technically a record, but the .400 threshold is going to be difficult for anyone else to cross. Williams had a 154 game schedule to deal with. We are now up to 162. More importantly, middle relief and closers are now part of basic strategy. From the 7th inning to the end of the game, important batters (like the guys who can hit .400) will face a lefty-lefty or righty-righty matchup. The best these guys can hope for is a favorable matchup vs. a fresh reliever who throws smoke.

Back in 1941, nobody would bring in a lefty-specialist just to pitch to Williams. Today, that would be standard operating procedure.

Re:You can't do statistics with a random # generat

[Vellmont]

Computers do not actually generate random numbers

That'll be a surprise to the multiple true random number generators build into most operating systems. There's many sources of random data in a computer. Timing between keystrokes, timing of mouse movements, network latency between packets, and of course hardware random number generators that use thermal noise as its source.

So to put it mildly, computers can, and DO generate truly random numbers that are completely unpredictable and free from bias.

(Oh, BTW, to do a Monte-Carlo simulation (which the referenced article is) you actually don't need true random numbers, you only need a pseudo-random source that's free from bias. Those pseudo-random sources do exist, and aren't that even that difficult to code.)

just slow down

[plopez]

You just have to slow down. Put away the x-box and the crackberry. It has it's own pace and ebb and flow. drink a few cold ones, fire up the grill. A nice saturdy or sunday relaxing watching a game is never wasted. Or go out to a park and see a real live game. Get out of your parent's basement and get some sun. Even better, call in to work for a mental health day and go to a park. If you don't have access to a mjor league park a minor league game can be fun too.

In an ADD culture, a nice relaxing baseball game can be great.

Re:just slow down

[sjames]

It's also a question of HOW you watch the game. It's a lot more interesting when you understand the deeper strategy and try to guess what the manager and players are thinking, look at how the play should have happened vs. how it did and why that might be. The slower way the game unfolds allows time to talk about it with friends.

Of course, some of the perceived slowness is just due to not understanding what is quietly happening. The runner slowly stretching his lead, the pitcher deciding if a throw to first is in order, the fine line between catching the runner flat-footed and a balk. Then there's the fielders repositioning based on what they think their pitcher can get the batter to do.

I also find it interesting that baseball is one of the few games where when something that 'shouldn't happen' takes place (like a ball lost in ivy), the rules explicitly call for the umpires to decide what should have happened and set things as if it did. They get that right practically every time.

Re:just slow down

[Dhalka226]

I also find it interesting that baseball is one of the few games where when something that 'shouldn't happen' takes place (like a ball lost in ivy), the rules explicitly call for the umpires to decide what should have happened and set things as if it did.

I'm not sure if the "ball lost in ivy" thing was a reference to Wrigley Field, but as a life-long Cubs fan I can tell you the umpire has essentially no discretion in that case. If the fielder throws up his hands, indicating that the ball has become lost in the ivy, it's a ground-rule double. The only discretion involved is in the following two scenarios:

1. If there's a baserunner on ahead of the person who got the hit, the umpires get to decide whether or not he would have been able to score on the play based on where he had gotten to at the time of the call. This very seldom means anything, however, because balls that get stuck in the ivy tend to have been quite hard-hit and usually with a single hop--meaning the baserunner is almost never half way around third base if he was standing anywhere other than third to begin with, and if he WAS on third there's nothing to decide.

2. If the fielder attempted to dig the ball out before throwing his hands up, the ball is live. I wouldn't call this discretion so much as a caveat to the rule and a decision on their part.

There ARE areas of umpire discretion, though. For example interfering with the baserunner makes them not only safe at the next base, but possibly more bases depending on what the umpire thinks would have happened had the interference not occurred. I like these things as they add more of a human element to it that I think is engaging and important.

I can also see where it's completely necessary. Baseball is one of few sports where one team isn't really trying to stop the other. Oh, sure, they're trying to "stop" you from getting a hit or run, but the focus is nearly always on the ball. Did the ball beat the runner to the bag? Did the ball tag the runner? Contrast with basketball where you're dribbling with a defender in your face or football where the entire game is built around who stops the opposing team better and you can see where baseball is unique and requires somebody to make these sorts of decisions. It's part of the allure of the game to me.

Re:just slow down

[sjames]

I was referring to Wrigley. It is true that there is a ground rule for the Ivy, but mostly because it is the obviously correct ruling (it's hard to hit the ball to the wall uncaught and not get a double and triples are just plain uncommon there) Last year, there was an incident elsewhere where the ball hit the wall below the line and managed to come to rest on top of a scoreboard. The umps correctly (IMHO) ruled it a double. Many of the ground rules are a lot like legal precidents.

That reminds me of another thing I like about baseball. Because of the age of the game and where it was oftem played, there's a fair amount of room when it comes to a 'regulation field'. It not only adds an extra dimension to the play, but also means that a regulation field is practical from the sandlot level up. While recreational players can always agree on an irregular field for any game, it's nice that the official professional rules for Baseball actually accomodate the process. It also leads to traditional things like the Ivy (which I'm convinced grows mostly due to composted baseballs :-) that no other professional sport would tolerate.

Comparison of Sports

[buildguy]

Interesting comparison made on this page, but I'm not sure if it is accurate. http://en.wikipedia.org/wiki/Don_Bradman#World_sport_context

why the assumptions

[Anonymous Coward]

In reality, a league is typically very imbalanced from team to team and from pitcher to pitcher (probably even more so in the game of the early 20th century than now). The reason the imbalance decreases with time is the rise of competition. In his book "Full House", Stephen J. Gould detailed this effect for both baseball (in particular, but he noted sports in general) and in the rise of and success of complex life forms. The effect of a random walk from zero athletic achievement (or size or complexity or etc.) is limited still by the wall of maximum human ability (or etc.). You're bound to find records being advanced by large amounts and often at first, trailing off to smaller advances less frequently as time goes on. The book cites the example of the present slow growth of men's sporting achievements compared to the relatively faster growth of women's sporting achievements in the same athletic endeavors where the difference is simply the length of time of women's participation and records having been kept.

Among all batters and at-bats in the major leagues, the batting average hovers right around .260 overall and for every sub-interval, with (IIRC) a handful of moderate single-season excursions. Note that that spans first 60 years, last 60, deadball, steroid era, everything.

It describes "the death of the .400 hitter". The best hitters aren't getting any better because there's only so well a human can perform given the laws of physical reality. However, the worst hitters are getting better because of the "efficient market" effect: there's no room in the league for lousy hitters because more high-quality hitters are available from larger pools (from human population in an absolute sense and from the popularity of a sport in particular). Same with pitchers. That's why the best hitters' batting average falls with time and why the best pitchers' opposing batting average goes up with time. It's also why the worst hitters' and pitchers' numbers improve with time; all the numbers converge to steady-state optimality as quality improves in an efficient market.

A top-quality hitter in an early era would have many more mediocre pitchers to exploit than in a later era. Similarly, a top-quality pitcher in an early era would have many more mediocre hitters to exploit than in a later one.

It's easier to get hits off of two successive average pitchers than it is to get hits both off of a very good and a very bad pitcher. For example...
[pertinent example] ... True enough (and your numbers sound reasonable; I'm not certain what the likelihood of an n-game hitting streak is...), but that's not a big enough difference to surmount the sheer number of chances at hitting streaks throughout the history of baseball. Take even your most pessimistic example (or one much more pessimistic, even), and sum it over the number of possible 56-game-batter-hitting-streaks over the last 100 years of games played and the result will be bigger than unity. In fact (I think this is the case...), if each individual player in the history of baseball, no matter how great or terrible, had exactly one shot at a 56-game hitting streak, given the aforementioned .260 overall average, you would expect at least one of them to get it and probably more, and you would expect the largest streak to be somewhat larger still.

Your complaint about the inhomogeneity's affecting the outcome of the calculation would be more valid if the distribution of player abilities, match-ups, and the like were not uniform; but in fact they are. They do indeed make that assumption as you note, but it is a justified assumption.

What baffles me is why they are running "simulations" on this sort of thing rather than dealing with closed-form probability. It's cool to get the computers to do simulate baseball, and there's a market for that entertainment value, but you don't need to plink the pseudo-random number generator millions of times to find your way around the problem.

Re:You can't do statistics with a random # generat

[Anonymous Coward]

Bitches don't know 'bout my Blum Blum Shub.

Or is it proof that...

[clonan]

Giving grad students unlimited computational power does NOT provide quality science.

streaks and slumps

[Anonymous Coward]

Your assertion of slump effects sounds similar to "the myth of the hot hand" in analysis of basketball statistics. When you examine streaks, you find that the probability of making a shot following one successful shot is not any higher (or lower!) than the probability of making one after an unsuccessful shot. Same for "cold" hands. The distribution of shooting streaks really is, upon careful analysis, what you would expect based on examining only the mean. You don't gain anything by accounting for streaks, whether short or long or of highly-skilled or mediocre shooters.

I know the above analysis has been done for basketball, and I suspect there exists but have not seen a similar analysis of baseball hitting streaks. I suspect that using, say, a 10-game moving average as you suggest (or similar streak/slump accounting tactic) would be as useless in baseball as it is in basketball. (I'm much more familiar with baseball than basketball, both stats and as a sport).

So intuitively, you'd expect slumps and streaks to influence subsequent attempts. There's even a good body of argument from psychology, climate, environment, physiology (fatigue) etc. that there might. Actually examining the statistics, however, reveals that it just ain't so. and I think that is an earth-shatteringly important point; it's not about something trivial like sports, but about the power of statistical insight we can gain from physical reality. (Stephen J. Gould himself said he wouldn't begrudge anyone for thinking that baseball was as "earth-shatteringly important" a topic as anything else...)

Now that I have waxed poetic, I see that my captcha is "predict".

Re:You can't do statistics with a random # generat

[ucblockhead]

None of those things are truly random. Unless you are dealing with quantum effects, you are not dealing with something truly random.

In particular, timing between keystrokes is not at all random. In fact, one can use the timings between keypresses to figure out who the typist is!

Re:You can't do statistics with a random # generat

[ampathee]

There is no such thing as true randomness. You can't measure something without effecting it. Non-sequiteur. And there are plenty of sources of true random data.

Consciousness can effect randomness, as this Princeton page proves. Rubbish. Anyone can see that the global consciousness project is a load of crap. Spikes occur in random data. This is normal. All they are doing is looking for spikes that occur near what they consider "important" world events, and pointing at them saying "significant!".

In related news ...

[PPH]

... simulation runs have not yet identified an alternate universe in which a Slashdotter gets a date with Jessica Alba.

Every alternate universe produced a steak . . .

[corifornia2]

Every alternate universe produced a steak
That sounds like good eatin'.

Re:You can't do statistics with a random # generat

[Vellmont]

Unless you are dealing with quantum effects, you are not dealing with something truly random.


From wikipedia on "electronic (thermal) noise":

In any electronic circuit, there exist random variations in current or voltage caused by the random movement of the electrons carrying the current as they are jolted around by thermal energy.

Is that quantum mechanical enough for you?

As for network latency between packets, while it may not be random on a quantum-mechanical level, it's still unpredictable unless you can get on the same lan segment as the target computer. The keyboard timings are taken on a small enough time scale that they're quite unpredictable, and not related to the typist.

Re:You can't do statistics with a random # generat

[Anonymous Coward]

"Timing between keystrokes"
"timing of mouse movements"
"network latency between packets"
"hardware random number generators that use thermal noise as its source"
The term "truly random" is not something to be tossed around. Only truly random thing is the measurement of an unentangled quantum state. Something like timing between keystrokes clearly doesn't fit the bill(depends on age, sex etc.), unless it is inside the box and Schrodingers cat is walking on it, of course.
Network latency between packets might depend on your income (do you have money for a good qual. cable - with low stable latency?).
You might argue that the last one fits the bill, but I'm pretty sure you can't prove the outcome is unbiased - way too many factors to consider to know/prove how random it really is (although that might sound counter intuitive).

I guess you can get random number generators based on some radioactive source, which actually would be truly random.

To sum up: Don't say something is truly random just because it "feels" random.

Other tidbits about DiMaggio's streak

[jocknerd]

After the streak ended, he started a new 16 game hitting streak. That means he hit safely in 72 of 73 games.

During the streak Joe DiMaggio had a batting average of .408, a slugging average of .717, he faced four (4) future hall of fame pitchers, and he played in the 1941 All-Star Game (he went one-for-four, scored a run, and drove in a run). Source is http://www.baseball-almanac.com/feats/feats3.shtml

During Joe DiMaggio's streak, Ted Williams actually had a higher batting average. William's batted .412 and finished with a .406 average for the year.

Joe DiMaggio had a 61 game hitting streak while playing for the San Francisco Seals in the Pacific Coast League in 1933.

Re:You can't do statistics with a random # generat

[lostchicken]

This isn't how modern statistics is done. The pseudo-random number generators used in statistical research are entirely predictable when their initial seed is known, but are otherwise statistically random. They must obey certain requirements of "statistical randomness" that make the output look like pure entropy for essentially any form of real statistical examination, other than an attempt at determining the sequence directly. Monte carlo computation is always done with PRNGs so that the experiments are actually repeatable.

The modern PRNG, something like the Mersenne twister algorithm, is random enough that if you repeated the experiment done in the article with a true entropy source a bunch of times, and compared them to the results from running the PRNG based simulation, you should find identical distribution of results. If you don't, you'd have a statistical find much, much more interesting than baseball. The point of a good algorithm is that it won't have a visible "bias" in results.

Now, it is possible that a really bad PRNG could impact research results, and they have in the past. The RANDU algorithm is a particularly good example, but it was really, really, really astronomically bad. I doubt the authors used something like this.

What about consecutive games on base?

[Anonymous Coward]

As impressive as DiMaggio's streak is, I'm much more curious about Ted Williams' consecutive games on base streak. Joe D holds second place in this stat: 74. Williams got on based 152 out of 154 games in 1949, with a streak of 84 in there.
It is considered a MAJOR achievement if someone goes 60 consecutive games on base (rarer than even a 40 game hitting streak).

Since they ran the stats, they could have tested both...

Re: yet anohter unreported weird fact

[Anonymous Coward]

I've also performed about 10,000 *ahem* "simulations" in which I hit a home run.

In real life though I rarely make it to first. Oh wait. We're talking about baseball?

in lieu of wit

[Normal_Deviate]

I applaud the author for using (not "utilizing") a word that is slightly shorter than the correct word. "Alternative" would be a reasonable description of a universe that did not happen. "Alternate" means something like "oscillating". Presumably the author did not mean to imply that we are hopping back and forth between two universes. A pedantic quibble you say? Why yes, yes it is.

Re:in lieu of wit

[ne0n]

I particularly savored the author's alphabetic parsimony in the last sentence of TFS.

Every alternate universe produced a steak of 39 games or better; one reached 109 games.

In other news..

[Anonymous Coward]

The man who bought 1 lottery ticket was much less likely to have won than the guy that bought 10, but yet most lottery winners only purchased a single ticket.

When you are trying to figure out why an unlikely event happened to a particular person, you must also analyze the parameters of that particular person. In other words, since 99.9% of lottery players only buy a single ticket, it turns out that only slightly less than 99.9% of winners are single-ticket purchases.

So, it really isn't too surprising that someone with a decent batting average ended up with this record because there are far more "decent" batters than there are "exceptional" batters and the difference between exceptional and decent plays a small role in probability to this extreme.

Re:You can't do statistics with a random # generat

[Frequency Domain]

MODS - Parent post is not informative, it is flat-out wrong. A pseudo-random number generator is is considered unacceptable if it can't pass a Turing-like test - if I gave you two sequences where one was pseudo-random and the other was "truly" random, you would be unable to tell which was which using any statistical test you can dream up. If one of the sequences yielded biased results for some known distributional property, that would itself be grounds for rejecting it.

Re:You can't do statistics with a random # generat

[multipartmixed]

You're saying that computers _typically_ use thermal noise as a source of entropy for random-number generation?

The only one I'm aware of would be the Yamaha DX7 (and their ilk); that's how the random-noise low-frequency oscillator is fed.

Re:How likely is it that the researchers have sex?

[Anonymous Coward]

Re:How likely is it that the researchers have sex? (Score:3, Funny)
by Anonymous Coward on Sunday March 30, @05:40PM (#22915280)

I think you're fucking sad Christopher. Check out how often you post on Slashdot. Get a fucking life. I rarely come here yet I see posts from you all the fucking time. Face facts - you're a loser and none of these fucking posts you make mean anything. You are wasting your life with trivial bullshit. This is the only time I will offer you help.

Either find something meaningful to do with your life or go fucking kill yourself. Asshole. [ Reply to This | Parent ]

Whoever modded this funny should have their user ID yanked and their IP address banned from use on slashdot.

Re:You can't do statistics with a random # generat

[4D6963]

It is not mathematically sound to do statistics with a random number generator.

Why do people like you get modded up, as you sound like you never even heard the word "Chi-square" before. People don't get modded up for being correct but for *sounding* knowledgeable.

Re:You can't do statistics with a random # generat

[CTachyon]

Modern Intel motherboards (i810 forward) and AMD motherboards (768 forward) have a hardware RNG (Random Number Generator) that IIRC is based on diode noise. That's straight up quantum randomness, and most modern Linux distros automatically detect and use it if available.

Apropos of Nothing...

[gyrogeerloose]

Joe D was my first cousin once removed. That is, he was my father's mother's sister's son. Yeah, I'm a geezer for sure. Unfortunately, as much as I love baseball, none of Joe's genes filtered down my way. I couldn't hit a big-league fastball on the best day I ever had.

Re:How likely is it that the researchers have sex?

[Anonymous Coward]

Go back to bed, Chris.

Is this as screwed up as it sounds?

[tji]

"The result: Joltin' Joe's record is not merely likely, it's basically a sure thing. Every alternate universe produced a steak of 39 games or better; one reached 109 games. Joe DiMaggio was not the likeliest player in the history of the game to accomplish the record, not by a long shot."

Is this just poorly written, or is their conclusion really this silly? The article seemed to say that they just took the player's batting average, and calculated how likely it is that he would get at least one hit in a game. How is this worthy of an academic paper? Basically, their outcome is: The higher a player's batting average, the more likely he is to have a streak longer than Dimaggio.. genius.

It doesn't account for any of the subtleties which make this type of streak rare and special. Off the top of my head, these include:

- Batting (and athletic performance in general) is streaky. Players always have a hot streak, where they bat .400 for a month, .300 for a few months, and a month at .156 thrown somewhere in there.

- Everyone has an "off night", which is all it takes to break the streak.

- If you're playing that well, they pitch around you. You might get intentionally walked, or thrown garbage and walked. So, you get less real at-bats to work with in some games.

- Some pitchers you just don't match up well against. That left hander with nasty junk is all but un-hittable.

- As the streak gets longer, the pressure gets higher, which impacts your performance. Just ask Paul Molitor, who had a 39 game streak in 1999, or Roger Maris who had health problems related to the pressure of beating Babe Ruth's single season home run record in 1961.

- As the streak gets longer, pitchers are more aware of it and pitch to you differently because of it.

Any of the above can put an end to a streak, all it takes is one game.

A conclusion of "it's basically a sure thing" is obviously horseshit, given the fact that the only guy who has even come close enough to the record to talk about is Molitor, and he wasn't even close. If your computer simulation says it's a sure thing, common sense says you have a flawed computer simulation.

The guys who dominated their simulations, in real life never approached a streal of DiMaggio's duration:

Ty Cobb: 40
Willie Keeler: 44

And, the one that should have been a real big clue to re-asses their analysis:

Hugh Duffy: 27

Re:Is this as screwed up as it sounds?

[Anonymous Coward]

the only guy who has even come close enough to the record to talk about is Molitor, and he wasn't even close. Rose had a 44 game hitting streak (the national league record). See http://www.associatedcontent.com/article/75289/longest_hitting_streaks_in_baseball.html?page=4

If Dimaggio had gotten a hit in the 57th game (and everything else stayed the same), he would have had a 73 game hitting streak. It's the kind of record where one doesn't get close.

Re:You can't do statistics with a random # generat

[ZorbaTHut]

[i]Something like timing between keystrokes clearly doesn't fit the bill(depends on age, sex etc.)[/i]

You're not processing it correctly.

Yes, you can definitely find a correlation between typing speed and age. Can't really question that. But if you're using keystrokes as a random number source, you don't use the high bits. You use the low bits. If there's an even number of microseconds between keystrokes, it's a 0. If there's an odd number of microseconds between keystrokes, it's a 1.

I would find it extremely unlikely that [i]that[/i] particular number is biased based on age, sex, etc.

Obviously the same things can be used for network latency and mouse movement timing.

The Monroe Factor

[Slur]

Okay, okay, but what are the odds that Joe DiMaggio would have such a streak, and land Marilyn Monroe? Somebody needs to get on that simulation asap. Here are my statistics, by the way...

Has to be said...

[Chris Mattern]

I just picked up Deep Space Nine Season One, so I gotta say... ...this is not linear.

Excuse my ignorance...

[Buchenskjoll]

but what did he do? Did he hit the ball 56 times in a row or what. Baseball isn't very popular around here.

Shameless plug...

[MajorSnafu]

Just in case there are any other baseball nerds here, check out this game - Out of the Park Baseball. Allows you to simulate baseball history as in the article, plus play as GM of a team. (I'm not involved with the game in any way, just a fan.)

What's a "hitting streak"?

[1u3hr]

I'm not American, but I did play baseball few times at school, so I know the basics. But I have no idea what is meant by a "hitting streak". He scored a home run in every game? Every innings? Or just got off home base? Or what?

Re:What's a "hitting streak"?

[Quill_28]

Hitting streak is where you get at least one hit every game.

So it is possible that the team could walk you every at bat and you lose you hitting streak.

Re:What's a "hitting streak"?

[EricWright]

Not true. For a hitting streak to end, you have to have one official "at-bat" in the game and fail to get a hit. Walks and sacrifice flies/bunts count as a plate appearance, but not an at-bat.

Testing the Model

[Fractal Dice]

So ... the model says streaks are probable. Reality says they're rare. So which is the more reasonable conclusion: that we're in a rare reality or that the model is not an accurate reflection of reality? Seems to me that the later is the better choice. Like seers of old, I don't care how interesting the theory and show of a model are, the only thing that matters is: "are the predictions accurate"

Hitting Streak 101

[dtmos]

Baseball Overview
(This is a quick overview. I am omitting exceptions and details not needed for pedagogy.)

In baseball, there are two, nine-person teams, and four bases (home, first base, second base, and third base). The offensive team members take turns at batting, one at a time, with the other members off the field and not in play, while all nine members of the defensive team are in the field. The goal in baseball is to hit the ball thrown by the pitcher of the opposing team, then run around the bases, both starting and ending at home ("circling the bases"). This results in a "run," or one point for the batter/runner's team. Once the batter has hit the ball, however, the defensive team in the field can get the batter/runner "out," requiring him to leave the field without scoring, in one of three main ways:

1. By touching him with the ball when the batter/runner is not touching a base;
2. By catching a ball the batter/runner has hit in the air, before it has hit the ground; or
3. By touching a base to which the batter/runner must run (e.g., first base if he's just hit the ball) while holding the ball, before the batter/runner can touch the base.

Definition:: A hit occurs when a batter/runner hits the ball and reaches a base safely (i.e., without being made out). Yes, this can be confusing; it's not sufficient for the batter just to hit (i.e., make contact with) the ball; he has to also reach a base safely as a result.

There are lots of other ways the batter can be made out, but the most significant is the "strike out," the rules for which I won't bore you with now, but involve an inability to hit the ball thrown by the pitcher to begin with. Much of the strategy, and resulting fascination with the game, involves the game-within-a-game between the pitcher and the hitter.

When the defensive team has made three batter/runners out it is called a half-inning, and the teams exchange places (i.e., the offensive team takes the field and becomes the defensive team). Not surprisingly, when both teams have made three outs it is called an inning and, by rule, there are nine innings in a baseball game. As a result, every member of each team will be able to bat a minimum of three times per game; the average is something over four, I think. Thus, a batter has an opportunity to get a hit between four and five times per game.

A Hitting Streak
Definition: A hitting streak is a streak of consecutive games in which a particular batter gets at least one hit. As it happens, at the professional level only the very best players get a hit three out of every ten times they go to bat (a success rate, or "batting average," of 0.300); such "three hundred" hitters are greatly sought, even though even they fail at their job 70% of the time. (It's a hard game to play well; this is one reason baseball is called "the game of failure.") A typical player might have a batting average of 0.275. Another of the fascinations with the game is how this trivial batting average difference between the great and average player, 0.025, or a difference of one hit every forty times at bat, significantly affects play. But I digress.

A three-hundred hitter has a likelihood of 1 - (0.700)^4.5 = 1 - 0.201 = 0.799 of getting at least one hit in an average game (if we assume that he is at-bat an average of 4.5 times per game). His chance of doing so in 56 consecutive games, however, is (0.799)^56 = 3.52E-6. As others have mentioned, however, there are additional subtleties in the game. For example, if a pitcher is good, the opposing team will get few hits and quickly get its three outs per half-inning. This means, however, that there is a feedback effect: The average number of at-bats a player will have against a good pitcher is less than against a poor one, because his teammates are making outs instead of their own hits. This makes it harder for him to get the hit in the game to keep his streak alive -- instead of his 4.5 at-bats, he may get onl

Re:You can't do statistics with a random # generat

[Anonymous Coward]

Just because it's quantum mechanical, it doesnt mean it is truly random. Great care must be taken to claim that. The measured state have to be completely unentangled, which, cosidering how difficult this is even when u try, I seriously doubt electronic noise is. Depends on temperature, fx. What values would you look at in an electronic circuit to extract to true randomness? Can you point something out that is not somehow entangled to the environment? No, you cant.
Network latency is a really bad idea. Is he/she running bittorrent? Well, thats more likely if it's a teenager...

Keyboard timings is a laughable bad idea. Sure it is "hard" to predict keyboard timings, but with an arbitrarily large amount of computing power and data samples you prob. could do better than random guessing. Hence it is not truly random.

Now, I don't think you need truly random numbers to run monte carlo. I mean - you can't prove anything with that anyway, so you don't really gain anything with truly random numbers. The model doesnt seem to be very good and monte carlo is at its heart not useful for proofs.

Pressure?

[Quill_28]

Stats don't take into account the pressure that would build on a person as the streak got higher.

If you have ever had to hit free throws at the end of a basketball game you know what I mean.

To have that pressure over the course of a few weeks would be unbelievable and draining but emotionally and physically.

Even if one had the skill I don't know how many people could take the pressure.

It is also doesn't seem to account for pitching match-ups, weather, etc.

Pressure

[openldev]

They don't take into consideration the increasing pressure that will be on you to continue that hitting streak, which may adversely affect your hitting ability.

Not Another One-Shot Calculation

[Effugas]

OK, so, since DiMaggio had his 56 game streak 60 years ago, what are the odds (by their measurement) of a 55 game streak occuring some time between now and then?

Almost guaranteed, by their method.

What is the likelihood that we'd see no such streaks?

Almost none by their method.

Did we see any such streaks? No, no we didn't.

Pretty fatal flaw.

Something says their probability calculations were probably a bit off. There are two things they're not taking into account: First, as has been mentioned elsewhere, baseball is not a one-shot game. Things happen as a streak picks up, and you necessarily end up with less and less data about what those things are as a streak progresses (since you have fewer and fewer samples of behavior on a 20-game streak, a 30 game streak, and so on). The game changes, the players change, the pitches change.

Second, players, pitchers, and the game itself have changed over the last 60 years. One reason we may not be seeing DiMaggio's records hit again, is that the skill levels are just so rarified.

Re:You can't do statistics with a random # generat

[holmedog]

I swear, one of these days I'm going to make a mechanical device that looks very similiar to a cup with dice in it (just really small). Then we are going to dump those dice, take a picture of it with a built in camera and feed it back to the pc. I'll make it as a usb device.

I'll make millions for reinventing (true) random...

Ignoring emotional pressure of streak

[Maxo-Texas]

Joltin Joe was not affected by the pressure late in the streak. For many people, the knowledge that they were in a history making streak would affect their average hitting rate. Some excel- some crumble.

Segregation also changed competition

[guanxi]

The competition was also much weaker in the earlier years of "Major League" baseball. Until 1947, all the best Latino and African-American players were excluded. Could you imagine what the competition would look like today if the major leagues were re-segregated? Pujols, Santana, Bonds, A-Rod, Jeter, Ortiz, Ramirez, etc. etc. ... all gone. The "Major Leagues" would be a joke, and it would debatable which league was better.

And even in 1947, there were only two non-whites in the majors, Jackie Robinson and Larry Doby. It was symbolic, but most of the best non-whites were still excluded. IIRC, the Tigers and Red Sox didn't hire their first non-white players until 1959. I've read that, if you had the talent of Willie Mays or Hank Aaron, sure you could get a job. But if you were just an every day player, or a backup, there was no room for you on the roster.

I once posted the question to rec.sport.baseball on Usenet: At what date could we say that, substantially, the number of non-white players in the majors was based on ability and not skin color? The consensus, FWIW, was the 1970s.

Re:You can't do statistics with a random # generat

[kidgenius]

Just use a bubble out of the game "trouble" and hook up a camera.

Up Next: Money Managers

[Anonymous Coward]

It would be interesting to see if they could say the same thing about some star money managers. How much is skill, and how much is luck?

Why 1942

[ChrisMaple]

Others have already mentioned technical changes in the game affecting the probabilities. History also changes the probabilities. In 1942 we were in World War II, and there was a draft. Some good players were already gone, making it easier for the outstanding remaining players to set individual records.

The Walrus ran an article about this...

[penguinstorm]

The Walrus, a Canadian magazine, ran an article about this a while ago:
http://www.walrusmagazine.com/articles/2007.10-joe-dimaggio-56-games/

I don't have time to re-read it right now, but as I recall the basic thesis was that it's *highly* likely that the streak didn't really happen, and that it was...ahem...aided along by willing assistance on the part of officials.

Good grief. What would Charlie Brown say? Our nation turns it lonely eyes to you.

So...is there a difference between this type of assistance and the...blurg...assistance the modern home run kings had in their pursuit of Hank Aaron's record?

Meanwhile, in an alternate Universe:

[Lewrker]

Damn it, Scotty, I'm a doctor, not a baseball player!

Re:You can't do statistics with a random # generat

[Frequency Domain]

Too bad there's not a "disinformative" mod for posts which propagate misinformation based on ignorance rather than trolling.

Dimaggio may have the streak, but he's not 1337

[Firebones]

What do Bobo Newsom and Mario Mendoza have over Joltin' Joe and his streak? Membership in the All-Time 1337 Hall of Fame.

Re:You can't do statistics with a random # generat

[Anonymous Coward]

I would like to see the mathematical prove, LOL. As a researcher "unlikely" just doesn't quite cut it for me. It is easy to say that it is unlikely that p=np, but I doubt I will run away with 1. mil for that.

Anyways... I actually don't think it is that unlikely that there could be some person out there where the prob. of hitting a key on an odd microsecond is 50.000000 (continue with 10^(10^(number of atoms in the universe)) 0s)) 0000001%.
However, it is important to note, that it is not my job to prove there is, it is your job to prove there isn't and never will be (if you want to say that it is truly random).

Obviously the same things can be said for network latency and mouse movement timing.

Re:You can't do statistics with a random # generat

[ZorbaTHut]

And, for that matter, quantum random number generators.