Slashdot Mirror
The Black Hole Image Data Was Spread Across 5 Petabytes Stored On About Half a Ton of Hard Drives (vice.com)
An anonymous reader quotes a report from Motherboard: On Wednesday, an international team of scientists published the first image of a black hole ever. It looked like a SpaghettiO, and yet the image was an incredible scientific achievement that gave humanity a glimpse of one of the universe's most destructive forces and confirmed long-held theories -- namely, that black holes exist. Storing the raw data for the image was a feat itself -- tiny portions of data spread across five petabytes stored on multiple hard drives, the equivalent of 5,000 years worth of MP3s. Katie Bouman, a computer scientist and assistant professor at the California Institute of Technology, led the development of the algorithm that imaged the black hole. An image of her posing with some of the data drives went viral as observers praised her success.
The massive amounts of data were essential to creating the image of the black hole. Bouman and other scientists coordinated radio telescopes all over the Earth, each pointed at the black hole and gathering data at different times. The data scientists then pieced this information together and used an algorithm to fill in the blanks and generate a likely image of the black hole. The five petabytes of data took up such a massive amount of digital and physical space it couldn't be sent over the internet. Instead, the hard drives were flown to processing centers in Germany and Boston where the data was assembled. On Reddit's /r/datahoarder subreddit, a community dedicated to spreading the passion of hoarding vast amounts of data, the drives were bigger news than the scientific achievement itself.
The massive amounts of data were essential to creating the image of the black hole. Bouman and other scientists coordinated radio telescopes all over the Earth, each pointed at the black hole and gathering data at different times. The data scientists then pieced this information together and used an algorithm to fill in the blanks and generate a likely image of the black hole. The five petabytes of data took up such a massive amount of digital and physical space it couldn't be sent over the internet. Instead, the hard drives were flown to processing centers in Germany and Boston where the data was assembled. On Reddit's /r/datahoarder subreddit, a community dedicated to spreading the passion of hoarding vast amounts of data, the drives were bigger news than the scientific achievement itself.
And we are talking about radiotelescopes here. What you get is a signal from an antenna, and you have to recalculate the sources of the different waves the antenna recorded. The datapoints are just long lists of energy measurements from the different antennas.
We knew beforehand that M87 (a large eliptical galaxy about 55 million light years away) had a supermassive Black Hole at its center. There were estimates of its size from redshift measurements of the movements of the galaxy's center. Thus this is not a discovery we stumpled upon, this was a carefully selected target, and there were expectations beforehand how the picture should look like. A physicist who wrote his doctorial thesis on how a picture of a Black Hole should look like, gave a speech three month ago (albeit in German): Andreas Mueller: Foto eines Schwarzen Loches.
of a C130 loaded with Flashdrives flying at 700 mph. The latency is a bitch though.
If a prop-driven C-130 is traveling at 700MPH, your latency problem will be expiring very soon.
Unfortunately, it will be replaced by a much larger data loss problem...
Erm...
I think you miss that this is a "real picture" of a black hole. It's black. That's the hole.
It bends space, time and light - correct. Anything past it's event horizon is lost forever, correct. But anything on the periphery isn't and arcs rounds and is fired back into space at random, almost... like a mirror. Light acts like a planet in orbit around the object, which means you can see all kinds of artifacts not caused by anything else, and can see light focused, diverted and spread from behind, in-front and the side of the object in question, producing bright halos of light - maybe from our own side of the galaxy - that orbited around the hole and came back our way.
And it's doing that in all dimensions. And depending on the tilt of the accretion disc, you'll see parts of that disc caught up in it / blocking light, which is why the halo isn't even - the accretion disc is tilted from our viewpoint.
Black holes are "invisible". But their presence makes everything near them go really weird and not like a standard piece of space at all.
You can even measure the Schwarzchild radius from the size of the haloes because parts of it will be directly related to certain multiples of the radius.
The black hole is only a point to us because it warps space, too. Whether it's actually a "point" in its local reference frame is another matter entirely.
She designed the algorithm.
The NY Times says it wasn't the algorithm used to make the final picture. https://www.nytimes.com/2019/0...
While she led the development of an algorithm to take a picture of a black hole, an effort that was the subject of a TED Talk she gave in 2016, her colleagues said that technique was not ultimately used to create this particular image.
But that doesn't diminish her contribution to the project or her skills. She is clearly a skilled scientist but you have to read her actual articles to see that. By misrepresenting her role in the project you miss an opportunity to give her credit for the cool things she actually did. Not to mention the other 39 women and 160 men who also worked on the project.
She clearly didn't write the majority of the code.
However it's entirely possible she's responsible for the math and/or the actual algorithm the code implements.
From The internet’s idiots are already trying to discredit Katie Bouman’s historic accomplishments:
The criticism claiming Bouman is just one name of a few on the research paper shows a misunderstanding of how academic papers work. Bouman is the first author of her paper “Computational Imaging for VLBI Image Reconstruction.” The first author on a research paper is typically the person who made the most important contributions. Alongside Bouman, Michael D. Johnson, Daniel Zoran, Vincent L. Fish, Sheperd S. Doeleman, and William T. Freeman worked to produce their findings.
“Of course Bouman will not have written all of the code, just like Englert and Higgs are not solely responsible for the discovery of the Higgs boson. ..." Wade said.
In the discussion on Hacker News, and even in our own Facebook comment section, multiple users claim Bouman’s colleague Andrew Chael wrote over 850,000 lines of the 900,000 lines of code used to discover the black hole. Chael tweeted to her defense, saying that without Bouman and her contribution to the software, the project would never have been a success.
So, with respect to a successful outcome, does it really matter how *many* lines of code she (or someone) wrote, especially if her/their code and/or other contributions made everything work?
I imagine we've all heard the joke about getting an itemized bill like: $0.50: Pushing a button; $99.50: Knowing what button to push.
It must have been something you assimilated. . . .
Completely false. From Chael's own words:
BTW, Chael is gay. So congratulations on supporting him.
As to Bouman herself, she isn't the one taking credit. She has said repeatedly it was a collaborative effort:
So yeah it seems a lot of political science was involved here.
Sure was, it came from only one group who was so incensed a woman could do anything remarkable it had to jump up and down, wave its hands, and put out fake information to make itself feel better.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
Following up whit this information from To undermine Katherine Bouman's role in the Black Hole photo, trolls held up a white man as the real hero -- until he fought back
The misleading posts said Chael alone had authored "850,000 of the 900,000 lines of code that were written in the historic black-hole image algorithm!" ...
However, the effort quickly backfired.
Though it may have been nice to receive more recognition, Chael immediately took to Twitter to explain that the online trolls had exaggerated his contributions, and he defended Bouman's work. In addition, Chael said that as an openly gay man, he is also an underrepresented demographic in STEM.
Chael disputed the incorrect posts
I did not write "850,000 lines of code" -- many of those "lines" tracked by github are in model files. There are about 68,000 lines in the current software, and I don't care how many of those I personally authored.
[... several tweets referenced ...]
While I appreciate the congratulations on a result that I worked hard on for years, if you are congratulating me because you have a sexist vendetta against Katie, please go away and reconsider your priorities in life.
Chael wrote the code for one of three scripted code pipelines that scientists used to transform telescope data into a coherent image.
Bouman has emphasized collaboration
Though Bouman has received a lot of attention, she has maintained that the black hole image was the product of teamwork.
"No one of us could've done it alone," Bouman told CNN. "It came together because of lots of different people from many backgrounds." The Event Horizon Telescope project was composed of an international team of more than 200 researchers.
It must have been something you assimilated. . . .