Slashdot Mirror


Confidentiality Expires For 1940 Census Records

Hugh Pickens writes writes "In spring of 1940, the Census Bureau sent out more than 120,000 fact-gatherers, known as 'enumerators,' to survey the nation's 33 million homes and 7 million farms. Now as the 72 years of confidentiality expires, the National Archives website buckled under the load as the 1940 census records were released and 1.9 million users hit the archives servers in the first four hours the data went public and at one point, the Archives said, its computers were receiving 100,000 requests per second. Data miners will have the opportunity to pick and chip through more than 3.8 million digital images of census schedules, maps and other sociological minutiae. What will we learn from this mother lode? The pivotal year 1940 'marked the beginnings of a shift from a depressed peacetime to a prosperous wartime,' says David E. Kyvig, author of Daily Life in the United States, 1920-1939. The vast data dump, Kyvig says, will allow historians 'to look closely at particular communities and how people within them were doing in terms of employment, income and material comforts.' The 1940 census was the first Census that looked deeper into the details of much of American life. 'As we see how the country evolved over the subsequent 20 years, where we have aggregate census data ... we ought to be able to see more clearly how government spending bettered everyday life, confirmed Keynesian economic theory and revealed that, before the war, the New Deal did too little, rather than too much, to stimulate the U.S. economy."" Get all 18TB of it while it's hot.

27 of 311 comments (clear)

  1. correlation != causation by starworks5 · · Score: 5, Insightful

    Just because the government was able to implement a Keynesian solution to that economic problem, does not mean that it holds the solution to every economic problem, for instance one that involves post - peak natural resource production.

    1. Re:correlation != causation by starworks5 · · Score: 3, Interesting

      Keynesian economics did increase the total workforce participation, but your right about the destroyed industrialized countries being a part of it, so did the neo-colonialism of the late 19th and early 20th century. Now that we can no longer access such cheap resources, and don't have a significant advantage in manufacturing, we (and also Europe) are left with an inflated economic bubble that's collapsing. Simply put is that our total factor productivity is too low and resources too high, to be able to demand the amount of money that we expect from our labor. And though Keynesian economics might help by being able to tax unproductive consumption, it also hinges on the ability to target more productive investments, which is essentially trading consumer spending for capital expenditures. But even if you completely automate production of a thing, say for instance the mining of copper which now requires demolishing entire mountains, its not going to lower the price enough to make it as affordable as it used to be.

    2. Re:correlation != causation by supadjg · · Score: 4, Insightful

      It's from Darren Aronofsky film "Pi", which has some great lines:

      "Hold on. You have to slow down. You're losing it. You have to take a breath. Listen to yourself. You're connecting a computer bug I had with a computer bug you might have had and some religious hogwash. You want to find the number 216 in the world, you will be able to find it everywhere. 216 steps from a mere street corner to your front door. 216 seconds you spend riding on the elevator. When your mind becomes obsessed with anything, you will filter everything else out and find that thing everywhere."

      "As soon as you discard scientific rigor, you're no longer a mathematician, you're a numerologist."

    3. Re:correlation != causation by repapetilto · · Score: 4, Insightful

      It's impossible to say what would have happened without the war.

    4. Re:correlation != causation by repapetilto · · Score: 3, Insightful

      Keynes is not only good for the economy, it's good for the soul.

      Really?? Before Keynes no country ever survived a depression?

    5. Re:correlation != causation by jellomizer · · Score: 3, Insightful

      Not to mention after the war. The United States was the largest nation that had the least collateral damage. So the decades to follow while Europe and Japan was rebuilding the United States had a near monopoly on trade. The USSR was a threat however the way they implemented communism it created a situation where people didn't have that much motivation to build very competitive products, with a few exceptions mostly in weapons.
      Combined with people living on rations for years, a government forced saving program. When the war was over, people had money, they wanted to spend it and with the GI Bill a lot of these people went to college and got better educated. As well as their time in the war gave a lot of these people discipline that they wouldn't have gotten else ware. While some of the Stimulus spending helped, but not so much in terms of spending but in the fact that it rebuild and improved key areas of the infrastructure.

      Monopoly + (Disciplined + Educated) Work Force + With a lot of money saved up + Improved Infrastructure = Dominate Economy.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
    6. Re:correlation != causation by jcombel · · Score: 5, Interesting

      not sure if trolling, or just revisionist

      fannie+freddie were not forced by law to to give subprime loans. they were compelled by the market forces, as propelled by de/unregulated banks (2004 lowered Debt Capital Rule, unregulated derivatives and CDO market, Gramm-Leach-Bliley Act, DIDMCA, adjustable-rate mortages), which allowed the major institutions to over-leverage themselves while dealing out predatory ARMs.

      if fannie+freddie had not existed the 2008 FC would have still happened in the private sector alone. northern rock, countrywide, bear stearns, lehman brothers, merril lynch would have still all collapsed/required government takeover. the (de)regulatory framework simply allowed them astronomic profits at substantial risk, with the knowledge that any failure would cause systemic collapse, thus requiring government action, thus mitigating any risk to the personal wealth of the execs and traders.

      yes, fannie+freddie were headed by some fuckups that made decisions very similar to the large banks. but they were the decisions of private executives; these organizations were not compelled by law to seek inappropriate mortgages and then leverage them on the CDO market. they were compelled by high profits and low effective risk, just like the other speculative lenders.

    7. Re:correlation != causation by Opportunist · · Score: 3, Insightful

      If I didn't fear China's economic power yet, I would start right now.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    8. Re:correlation != causation by aslagle · · Score: 4, Informative

      Wow....just....I don't even know how to respond to the sheer number of fallacies in that paragraph.

      Instead, I'll focus on the biggest whopper:

      Russia became paranoid and autocratic as a defensive measure,

      WTF?!? Are you seriously saying that Russia *wasn't* paranoid and autocratic until *after* WWII? Stalin was General Secretary of the Communist Party from 1922 on, and used that position to consolidate power. His centralized planning of the economy resulted in the famine that caused mass uprisings, which led Stalin to command the "Great Purge" in 1937-38.

    9. Re:correlation != causation by starworks5 · · Score: 3, Insightful

      The autocracy was a result of constant existential threats, much like what occurred in the USA.

      I have a saying "terrorists need tyrants, like tyrants need terrorists", but it generally applies to the relationship fear and control.

      His centralized planning also led to the industrialization of the country, but there was a chicken and the egg problem where you have to move people from farming, and you dont have anything to sell but agriculture to build factories. Many non communist countries have had this same exact problem, even countries that recieve IMF development (often capitalist dictators), but we like to use it as fodder against the communists.

      its not as if our dictators are less dispicable than theirs (chile and iran come to mind), or that our development from agriculture to industry was less worse (irish famine) ( US slavery) (germ warfare).

    10. Re:correlation != causation by tnk1 · · Score: 4, Informative

      That would all make sense, if Stalin hadn't been planning all along to attack Germany, he just needed to wait longer for his forces to redeploy and his officer corps to rebuild after killing 90% of them. And yes, maybe he needed to ramp up production, but his military woes were really caused by the lapdog morons he put in command and his own micromanagement. I mean, he needed to dig up Zhukov after executing Tukhachevsky, the guy who pretty much invented the deep operations concepts that won the war for the Soviets. If the Red Army had had a reasonable tactical doctrine, as well as professional military leaders running the show, you can be certain that the Germans would not have gotten anywhere near as far as they did into the USSR. They may have even been repulsed.

      And let's not forget that the Red Army had absolutely no compunction about attacking the Finns during the Winter War. They even shelled some of their own troops to provide the reason for the invasion. The only reason Finland wasn't a Soviet Socialist Republic was the sheer incompetence of the Red Army staff, which is understandable because it was filled with lapdogs, and generals and colonels recently promoted from the lofty grade of lieutenant due to "staff rotation via gunshot to the back of the head".

      As for the USSR's economy, they had no concept of "sustainability" and "equality" in their economic focus, unless you consider that everyone in their sphere was equal in that they needed to do what Moscow said. If you were lucky, you'd be like Cuba, where you got subsidies so that you could continue to stay in power and piss off the Americans. That's nothing more than aid to prop up your friends, not economic equality. That's like saying we give aid to Pakistan because we think that they deserve equality and sustainability.

      There is also nothing new about Russia being paranoid and autocratic at any point. Russia has been paranoid and autocratic since before the reign of Ivan the Terrible, let alone during Stalin. They're still paranoid and autocratic, albeit to a lesser extent. The #1 reason that the USSR carved out its sphere of influence is that Stalin pretty much assumed that the Western powers would do what he would do, which is to say invade when they smelled weakness. And while Operation Unthinkable existed, it was a theoretical plan that was drawn up due to was assumed to be Stalin's next step (ie. invasion of Western Europe), not because they wanted to invade the USSR and take over. I should also point out there was a reason it was called "Unthinkable". For those who don't know, it was because they didn't want to do it, and they were pretty sure that they would either fail miserably or the victory would be at so terrible a cost that it would have been Pyrrhic at best.

      Of course, there were certainly many misunderstandings between the superpowers during the Cold War. And certainly the US got itself dirty playing in the mud with the Soviets, but I can't think of a single case where the US or the West legitimately considered an attack, or even a pre-emptive strike on the USSR that was not specifically for the purposes of defending against the gigantic Red Army presence in either Eastern Europe or their efforts to spread communist revolution around the world. As it stands, I think the fact there was no war is a testament to how both sides realized that it wasn't worth ending the world to spread their power via global war.

  2. Re:Why was it confidential? by ClioCJS · · Score: 5, Funny

    Because it's somewhat sensitive information.

    --
    -Clio
    Karma: Bad (mostly from not giving a fuck)
    Blog: http://clintjcl.wordpress.com
  3. 18 Terabytes?! by ndogg · · Score: 4, Interesting

    Someone gonna torrent that?

    --
    // file: mice.h
    #include "frickin_lasers.h"
    1. Re:18 Terabytes?! by arth1 · · Score: 5, Insightful

      No, the reason for the 100,000 requests per second is largely - drumroll - mormons.
      No, I'm not kidding. Honestly.

      Part of their religion is that they hold baptisms for dead people, so they allegedly get a chance to become mormons after their death, and thus gain greater rewards[*]. In order to do that, they engage extensively in genealogy. Really extensively.

      Their accuracy isn't much to brag about, though. I discovered through a search engine that someone in the LDS church had done a baptism "for" my departed father. And got most of his details wrong, including his birth year and family relations. But now it's "official" as far as they're concerned.
      They refuse to strike this from their records - I am "welcome to" submit correct information, but I don't want them to have that either. How about they stick to their own, and leave the rest of us alone?

      But yeah, mormons cause a huge part of the traffic.

      [*]: Like being reunited with one's loved ones, or becoming a Mr. and Mrs. God of a new planet. No, I'm not making this up. Religion is stranger than fiction.

    2. Re:18 Terabytes?! by dasunt · · Score: 4, Interesting

      Their accuracy isn't much to brag about, though. I discovered through a search engine that someone in the LDS church had done a baptism "for" my departed father. And got most of his details wrong, including his birth year and family relations. But now it's "official" as far as they're concerned.

      I've done a fair bit of genealogy. It's a decent geek hobby, it doesn't cost too much, takes a large amount of time, and requires good problem solving skills and the ability to judge and verify information. Plus it tends to tie in with a lot of history and geography, which I'm interested in. I'd recommend it.

      I'm not a professional genealogist, but I have found situations which may explain the conflicting information for your father. It could be due to two people with similar names, or misinformation on public records, etc.

      Someone I'm probably descended from lived about 50 miles away from someone else with the same name in colonial times. They were about the same age. They both married, and some of their children share the same name as well. Its common to see details from both individuals in other family trees.

      A similar situation exists with myself - when I went to school, there was someone in the same school with my name. Different birthdate, but roughly the same age. Anyone who attempts a family tree with me in it will probably run into the same problem.

      Public records are not immune to this either. Some of them show interesting errors. The state and the federal government disagree on the date my grandmother died - the state thinks she died a day earlier than the feds. Anyone who hasn't seen the records would consider any genealogical research with the wrong date to be "sloppy". Nowadays, with the Internet, its pretty easy to get both records. But even 15 years ago, having retrieved just one record wouldn't be unusual. Another case would be a great-grandmother of mine, who had the amazing ability to age only 8 or 9 years between each census - she kept lying about her age on every census in order to be younger than she actually was. (In addition, her children could never agree on her father's name either - marriage records and the death certificate give conflicting information.)

      Of course, a large problem with genealogy today, especially Internet genealogy, is the severe amount of copying that goes on among amateur genealogists, especially with the lack of verification and citation for the source of information. Citations are very important when it comes to research - there are going to be mistakes in records, and you always want to know the sources when it comes to conflicting information in order to verify which one is correct. Someone may be listed as a son or daughter on the census, but instead turn out to be a stepson or stepdaughter or other relative. Or perhaps a person's name was recorded incorrectly. Blindly following this information results in flawed family trees. But some people are not patient enough to do this, and instead tend to add people without verification or hunting down the source. These are the same people who tend to copy from other individuals family trees, which compounds the problem.

      This is one reason why I won't publish my family tree, in its current form, online - I have links and information in my family tree that are, quite frankly, a "best guess". As long as the notes and citations are included, it's clear that the information requires further verification, but if put on-line, the information would most likely be copied into countless other family trees and stripped of citations and notes. I'd rather not do that. ;)

      As for the LDS's obsession with genealogy, I tend to really appreciate it. The amount of preservation of old records the LDS has done is amazing, regardless of the reasoning behind that. And really, post-death baptism shouldn't be too upsetting. If you're religion or lack of religion is so weak that a religious ceremony once you're long dead will put you in jeopardy, I think you're belief is misplaced.

  4. Re:As Krugman says by starworks5 · · Score: 5, Interesting

    This is called "The Broken Window Fallacy", but is essentially a cornerstone of alot of economic policy, because rebuilding things (that were destroyed) creates jobs. In fact I hear alot about 'creating jobs', for example recent talks in my state for a casino, even though its a negative sum game. Even part of our throw away culture is defined by the measurement of GDP for economic success, since the sale of a single part contributes less than the sale of a whole new device. Now its no surprise that warfare, exploitation of and shipping resources around the world, may have not been the most efficient use of our time. But I have a belief that had the greedy capitalist pigs, not gone to war to protect their 'private property' from the 'communist looters', we would have ended up with a more 'free market' than we have now. Furthermore the more homogeneous development and lower diminished returns on both natural and human capital, would have increased aggregate human development and economic productivity, and probably have reduced the population and factored resource inefficiencies.

    America has a god given right to demand a bigger piece of the pie, even if that means destroying some of it in the process, because were exceptional and gods chosen people. Which is essentially our 19th and early 20th century intellectual rhetoric, we found what was essentially virgin land that we exploited, in order to create our version of order in the world.

  5. Re:Keynsian Theories by starworks5 · · Score: 5, Informative

    Technically speaking all money is now a form of debt, and what governments tend to do anymore is issue debt in their currency, and then inflate the currency to keep the debt/gdp level low enough to prevent default.

    Its mathematically impossible for everyone to pay off all debt in the system that we currently have now, the inflation is what makes the active pursuit of money (and therefore production) obligatory.

  6. Re:Keynsian Theories by repapetilto · · Score: 3, Insightful

    I have gathered that point, however there is more than one way to get rid of debt.

    I think that is kind of tangential to what I was asking though. Keynes advocated "counter-cyclical" spending, correct? Inflate during recession and deflate during boom times (yes, people and businesses will default because of this policy). What we have instead is constant inflation.

  7. Re:Why was it confidential? by uberdilligaff · · Score: 4, Informative

    Because that was the law under which the sensitive information was collected. The average US life expectancy in 1935 was 61.7 years, so 72 years would mean that the privacy issues would be moot for most of the folks enumerated in the Census -- they weren't expected to be around to complain 72 years later. The laws that established the Census go way back -- I don't know when the 72 years criterion was established. Life expectancy was even shorter the farther back you go.

    --
    Against stupidity, the Gods themselves contend in vain. --Friederich Schiller
  8. Re:Why was it confidential? by vlm · · Score: 5, Interesting

    LOL human death back then, as now, was a bathtub distribution, like electronics parts. So most people died as little kids or elderly. Back then pretty much all preemies died as a general rule, for example, unlike now. All the "average" means is the ratio of how many died as a baby vs died as an old man. I'd guess that means about two kids died young for every 8 or so that died elderly, which seems to fit in with actual genealogical data I have on my ancestors...

    The "real story" (in quotes because even pages of and pages of this stuff is still merely a summary of the real sources) can be read at

    http://www.census.gov/history/www/reference/genealogy/the_72_year_rule.html

    The exact number 72 was selected because in 1952 they wanted to give away the 1880 census information. Essentially declassify it by transfer from the BC to the NA. I think you can see the math there, 1952 - 72 = 1880 The exact 72 year range has stuck since then.

    The legal BS behind the general range of "more than 70 years" was selected, as you'll read at the link above, because the census officers had to / have to take an oath to never release the data. Assuming someone lied on their application and got hired anyway at 10 (unlikely), and assuming that even in extenuating circumstances there are no govt employees of any sort over the age of 82 (unlikely), that means waiting 72 years means the oath takers successfully did their duty and while it was in their power, blah blah blah, they never released the data. Essentially its your usual govt corruption. Technically according to the rule of the law the folks who gathered your 2010 census data will Never permit the release of the 2010 census data .... Never ... of course they'll be dead or retired eventually at which point it'll be released anyway in 2082, assuming the country doesn't self destruct first, at which time the oath takers will all be dead or retired.

    Its legal bullshit because if you're convicted of a crime by a judge, just because a judge dies or retires doesn't mean you're a free man. Another example would be the priest who married me and my wife about a dozen years ago by the process of signing the marriage license recently died... that does not automagically make us single. Also from my military experience the death of a guy who classified a document doesn't automagically free that document.

    If someone invents an immortality treatment, we'll have to come up with some new legal technicality bullshit. But for now 72 years works and is the tradition.

    --
    "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
  9. Re:1940 regulation? by SirGarlon · · Score: 4, Insightful

    I'm not claiming that all regulation is good. The examples you cite (regulating price of electricity, the TSA) are clear examples of stupid, counterproductive regulation.

    My position is that without Federal law to suppress a host of discriminatory practices, the bigots would still be in charge. Without Federal law to prevent dioxin in the ground water, companies would still be dumping toxic waste. The quality of life we take for granted did not magically emerge as we all became enlightened. It required the big stick of government regulation to stop the elite from abusing the rest of the public.

    --
    [Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
  10. Re:As Krugman says by Shining+Celebi · · Score: 5, Interesting

    Krugman's exactly the kind of ivory-tower asshat who believes that there's such a thing as a good war.

    No, he doesn't. He's said the right kind of war would help the economy, not that any war is a good thing. Krugman is consistently anti-war. He's also said an arms built-up to fight an alien invasion would be good for the economy and have great secondary effects (from research and whatnot.) He's also not seriously advocated that we arm up to fight an imaginary alien invasion.

    But you can sit their in your smug, self-righteous libertarianism and keep pretending Krugman is a hack who hasn't consistently made very accurate predictions and the Nobel prize in economics is a fraud

  11. Re:As Krugman says by repapetilto · · Score: 4, Informative

    The economics nobel prize is pretty strange. It is actually the "Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel".

  12. Re:Why was it confidential? by Skater · · Score: 4, Funny

    Essentially its your usual govt corruption.

    I like how you just equated people following the law with government corruption. No bias here! Good show.

  13. Re:As Krugman says by repapetilto · · Score: 3, Insightful

    The main claim is that government often creates more problems than it solves, then gets stuck in a loop of never ending quick fixes each generating more unintended consequences.

  14. Copyright vs Census by Immerial · · Score: 3, Interesting

    Anybody else find it interesting/sad that the time limit on copyrights is longer than the privacy time limit on the Census records? Just a clear indication that corporations are valued above people.

  15. Re:As Krugman says by Gilmoure · · Score: 4, Insightful

    If the central bank prints up 20 trillion dollars and it doesn't help anything, Krugman and his ilk simply shrug and says "Eh, must not have been enough."

    No, Krugman says it wasn't spent correctly and yes, the $700B of the stimulus was half what he prescribed. Instead of being injected at the bottom of the economy (working folk and main street businesses), it was piled on to the top, where it was used to keep paying out bonuses to the very people that caused this mess in the first place.

    When the financial industry makes up 40% of the economy, purely by shuffling paper and gambling, that's actually acting as a drag on the economy and not helping things. The TBTF banks should have been broken up *and* liquidity should have been aimed at jobs creation. Instead, you have a gov't that's heavily influenced (run?) by bankers and financiers who can't see any further than the edges of Manhattan. No idea how we're going to get out of this one, unless they go and dig a moat around New Yawk and stop listening to all the yammerheads over there.

    --
    I drank what? -- Socrates