Slashdot Mirror


The Black Hole Image Data Was Spread Across 5 Petabytes Stored On About Half a Ton of Hard Drives (vice.com)

An anonymous reader quotes a report from Motherboard: On Wednesday, an international team of scientists published the first image of a black hole ever. It looked like a SpaghettiO, and yet the image was an incredible scientific achievement that gave humanity a glimpse of one of the universe's most destructive forces and confirmed long-held theories -- namely, that black holes exist. Storing the raw data for the image was a feat itself -- tiny portions of data spread across five petabytes stored on multiple hard drives, the equivalent of 5,000 years worth of MP3s. Katie Bouman, a computer scientist and assistant professor at the California Institute of Technology, led the development of the algorithm that imaged the black hole. An image of her posing with some of the data drives went viral as observers praised her success.

The massive amounts of data were essential to creating the image of the black hole. Bouman and other scientists coordinated radio telescopes all over the Earth, each pointed at the black hole and gathering data at different times. The data scientists then pieced this information together and used an algorithm to fill in the blanks and generate a likely image of the black hole. The five petabytes of data took up such a massive amount of digital and physical space it couldn't be sent over the internet. Instead, the hard drives were flown to processing centers in Germany and Boston where the data was assembled. On Reddit's /r/datahoarder subreddit, a community dedicated to spreading the passion of hoarding vast amounts of data, the drives were bigger news than the scientific achievement itself.

11 of 293 comments (clear)

  1. Re:I have some questions by Sique · · Score: 4, Informative
    2. and 3. are related. Yes, the data is sparse and error prone, and yes, that's why collecting all the date and weeding out data, that has errors, took so long. The main problem was that the software to analyze the data had to be developed first, and there were several teams independently of each other developing software. The image you see now is basicly the image all teams agree upon. The images the teams created each had more detail though.

    And we are talking about radiotelescopes here. What you get is a signal from an antenna, and you have to recalculate the sources of the different waves the antenna recorded. The datapoints are just long lists of energy measurements from the different antennas.

    We knew beforehand that M87 (a large eliptical galaxy about 55 million light years away) had a supermassive Black Hole at its center. There were estimates of its size from redshift measurements of the movements of the galaxy's center. Thus this is not a discovery we stumpled upon, this was a carefully selected target, and there were expectations beforehand how the picture should look like. A physicist who wrote his doctorial thesis on how a picture of a Black Hole should look like, gave a speech three month ago (albeit in German): Andreas Mueller: Foto eines Schwarzen Loches.

    --
    .sig: Sique *sigh*
  2. Re:Never underestimate the bandwidth.... by geekmux · · Score: 4, Informative

    of a C130 loaded with Flashdrives flying at 700 mph. The latency is a bitch though.

    If a prop-driven C-130 is traveling at 700MPH, your latency problem will be expiring very soon.

    Unfortunately, it will be replaced by a much larger data loss problem...

  3. Re:"a likely image of the black hole" - LOL by ledow · · Score: 4, Informative

    Erm...

    I think you miss that this is a "real picture" of a black hole. It's black. That's the hole.

    It bends space, time and light - correct. Anything past it's event horizon is lost forever, correct. But anything on the periphery isn't and arcs rounds and is fired back into space at random, almost... like a mirror. Light acts like a planet in orbit around the object, which means you can see all kinds of artifacts not caused by anything else, and can see light focused, diverted and spread from behind, in-front and the side of the object in question, producing bright halos of light - maybe from our own side of the galaxy - that orbited around the hole and came back our way.

    And it's doing that in all dimensions. And depending on the tilt of the accretion disc, you'll see parts of that disc caught up in it / blocking light, which is why the halo isn't even - the accretion disc is tilted from our viewpoint.

    Black holes are "invisible". But their presence makes everything near them go really weird and not like a standard piece of space at all.

    You can even measure the Schwarzchild radius from the size of the haloes because parts of it will be directly related to certain multiples of the radius.

    The black hole is only a point to us because it warps space, too. Whether it's actually a "point" in its local reference frame is another matter entirely.

  4. Like he cares about cancer or fusion... by Anonymous Coward · · Score: 0, Informative

    Your comment makes no sense. "Trumptards" would not give a fuck about curing cancer and would prefer oil over fusion.
    You're missing his (angry) point anyway.

    He may not have said it in the nicest way, but he definitely got a point there, from his limited POV.
    So why do people here think it's appropriate to act like assholes *themselves* too, mod him "-1, Troll", and talk like you do?

    Shouldn't you be agreeing that there are definitely more pressing concerns, but that because of $reasons, this is definitely not useless?
    He's missing out, and standing there with his fly open, and you're calling him a moron and neither teach him nor tell him about his dick showing.

    That is not nice. That is, in fact, exactly imitating what you just hated on and downmodded.
    So go to your corner, be ashamed, and mod yourselves "-1, Troll" too.

  5. Re:I wonder how much Bouman actually contributed. by Anonymous Coward · · Score: 2, Informative

    Yeah, there it is. Time for the whiny toxic manbabies to show up and cry about the evil woman. Fucking incels.

  6. Re:I have some questions by noodler · · Score: 3, Informative

    "what if there is a star there which is 4k times our sun"

    What if.. What if...
    We actually LOOKED and there is no such star. There is a cloud-like disk with a black center. See picture in TFA.

    "(the end and beginning of everything, where time exist and not exist, where your dreams become true, etc.)"

    Where do you get this bullshit from???? Hollywood?

    "your conclusions and assuming that something 4000 times more massive is likely to be much more brighter and to have a much stronger gravitational force, you move to "black hole""

    We haven't seen a star that is 4000 times more massive than our sun anywhere in the universe.
    This is one of the puzzle pieces. We see gigantic sources of gravity in very small volumes, but no star at the center. In fact, no light at all from the center.
    This is what we observe.

    Another piece of the puzzle is that our current theories also say the same thing. You can't make a star bigger than a certain size. If you try to do it you will concentrate so much mass in a small space that it will collapse into a small point and will trap light. So theory predicts there should be very massive and very small objects that look black to us.
    So these theories explain and predict our observations.

    "And why assuming a black or light-less or matter-less center rather than the much more logical tremendously bright and massive one?"

    Again, we have never seen a star that is this big and bright. All stars that we have seen so far in our galaxy and in other galaxies are limited in size. Apparently the universe does not have big stars like you propose.
    If you can find one they you will become a science superstar.

    "Because a century-old theory (about which I will better not talk) told you so?"

    Everything we have observed so far is exactly like that theory predicted. If you don't want to talk about it you will never understand why we expected black holes in the first place.
    The theory says that mass bends space. We have observed this phenomenon without black holes. It is a true property of this universe.

    Some smart person once noted that, according to this theory there can be a region of space where gravity is so strong that light can't escape it. A kind of black hole. That was about 100 years ago and at that time we had no instruments to observe such phenomena. So it was 'only' a theory.
    Over the years, as our instruments became better and better, we started finding such regions of space. But we never could get a good picture due to limitations of our instruments. But it was enough to develop our theories more.
    Now we have pointed our telescopes towards a place where the theories predict a black hole could be. And what do we find? A picture of a black hole surrounded by a fuzzy disk! It even includes other effects that the theory predicts, like that one side of the dust disk looks brighter.
    So this picture is exactly like the 'old theory' predicted!
    In over 100 years no one managed to create a different theory that predicted this picture.

    Now, you could argue that the picture could be the result some other process. But then you would need to provide the mechanics of this other process. You have to make your own theory of why this picture looks like it does.
    But then your theory will ALSO need to explain why GPS needs time correction, why stars behind the sun can be observed to the side of the sun and why light always travels at a single speed and hundreds of other effects that are explained by "the old theory".
    Thousands of professional physicists haven't been able to create a different theory that explains all these effects.
    So good luck trying to invent your own. :)

    If you do not provide an alternative explanation that covers everything we have observed then you have no right complaining. You should first study the 'old theory' and try to understand why it has been so successful and no one has managed to invalidate it so far.

    But think of it this way. If this 'old theory' wa

  7. Re:I wonder how much Bouman actually contributed. by ath1901 · · Score: 4, Informative

    She designed the algorithm.

    The NY Times says it wasn't the algorithm used to make the final picture. https://www.nytimes.com/2019/0...

    While she led the development of an algorithm to take a picture of a black hole, an effort that was the subject of a TED Talk she gave in 2016, her colleagues said that technique was not ultimately used to create this particular image.

    But that doesn't diminish her contribution to the project or her skills. She is clearly a skilled scientist but you have to read her actual articles to see that. By misrepresenting her role in the project you miss an opportunity to give her credit for the cool things she actually did. Not to mention the other 39 women and 160 men who also worked on the project.

  8. Re:I wonder how much Bouman actually contributed. by fahrbot-bot · · Score: 4, Informative

    She clearly didn't write the majority of the code.
    However it's entirely possible she's responsible for the math and/or the actual algorithm the code implements.

    From The internet’s idiots are already trying to discredit Katie Bouman’s historic accomplishments:

    The criticism claiming Bouman is just one name of a few on the research paper shows a misunderstanding of how academic papers work. Bouman is the first author of her paper “Computational Imaging for VLBI Image Reconstruction.” The first author on a research paper is typically the person who made the most important contributions. Alongside Bouman, Michael D. Johnson, Daniel Zoran, Vincent L. Fish, Sheperd S. Doeleman, and William T. Freeman worked to produce their findings.

    “Of course Bouman will not have written all of the code, just like Englert and Higgs are not solely responsible for the discovery of the Higgs boson. ..." Wade said.

    In the discussion on Hacker News, and even in our own Facebook comment section, multiple users claim Bouman’s colleague Andrew Chael wrote over 850,000 lines of the 900,000 lines of code used to discover the black hole. Chael tweeted to her defense, saying that without Bouman and her contribution to the software, the project would never have been a success.

    So, with respect to a successful outcome, does it really matter how *many* lines of code she (or someone) wrote, especially if her/their code and/or other contributions made everything work?

    I imagine we've all heard the joke about getting an itemized bill like: $0.50: Pushing a button; $99.50: Knowing what button to push.

    --
    It must have been something you assimilated. . . .
  9. Re:Katie Bouman did jack shit by smooth+wombat · · Score: 4, Informative
    and most of the code that produced the image was Chael's and he is the guy who created the Git Hub project

    Completely false. From Chael's own words:

    "I did not write "850,000 lines of code" -- many of those "lines" tracked by github are in model files," Chael clarified. "There are about 68,000 lines in the current software, and I don't care how many of those I personally authored." . . . "While I appreciate the congratulations on a result that I worked hard on for years, if you are congratulating me because you have a sexist vendetta against Katie, please go away and reconsider your priorities in life," Chael wrote.

    BTW, Chael is gay. So congratulations on supporting him.

    As to Bouman herself, she isn't the one taking credit. She has said repeatedly it was a collaborative effort:

    "No one of us could've done it alone," Bouman told CNN. "It came together because of lots of different people from many backgrounds."

    So yeah it seems a lot of political science was involved here.

    Sure was, it came from only one group who was so incensed a woman could do anything remarkable it had to jump up and down, wave its hands, and put out fake information to make itself feel better.

    --
    We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
  10. Re:I wonder how much Bouman actually contributed. by smooth+wombat · · Score: 3, Informative
    Another scared white guy who can't handle the truth. Chael's own words:

    "I did not write "850,000 lines of code" -- many of those "lines" tracked by github are in model files," Chael clarified. "There are about 68,000 lines in the current software, and I don't care how many of those I personally authored."

    Also:

    (3/7) the work of many others who wrote code, debugged, and figured out how to use the code on challenging EHT data. With a few others, Katie also developed the imaging framework that rigorously tested all three codes and shaped the entire paper (https://t.co/hgJrv3gOE5);

    But congratulations on supporting a gay guy in his efforts. I'm sure he's very pleased with your support. Now go scurry back to your white supremacist site with your tail between your legs because you played yourself.

    --
    We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
  11. Re:I wonder how much Bouman actually contributed. by fahrbot-bot · · Score: 4, Informative

    Following up whit this information from To undermine Katherine Bouman's role in the Black Hole photo, trolls held up a white man as the real hero -- until he fought back

    The misleading posts said Chael alone had authored "850,000 of the 900,000 lines of code that were written in the historic black-hole image algorithm!" ... However, the effort quickly backfired.

    Though it may have been nice to receive more recognition, Chael immediately took to Twitter to explain that the online trolls had exaggerated his contributions, and he defended Bouman's work. In addition, Chael said that as an openly gay man, he is also an underrepresented demographic in STEM.

    Chael disputed the incorrect posts

    I did not write "850,000 lines of code" -- many of those "lines" tracked by github are in model files. There are about 68,000 lines in the current software, and I don't care how many of those I personally authored.

    [... several tweets referenced ...]

    While I appreciate the congratulations on a result that I worked hard on for years, if you are congratulating me because you have a sexist vendetta against Katie, please go away and reconsider your priorities in life.

    Chael wrote the code for one of three scripted code pipelines that scientists used to transform telescope data into a coherent image.

    Bouman has emphasized collaboration

    Though Bouman has received a lot of attention, she has maintained that the black hole image was the product of teamwork.

    "No one of us could've done it alone," Bouman told CNN. "It came together because of lots of different people from many backgrounds." The Event Horizon Telescope project was composed of an international team of more than 200 researchers.

    --
    It must have been something you assimilated. . . .