Slashdot Mirror

← Back to Stories (view on slashdot.org)

SpaceX Landing Video Cleanup Making Progress

Posted by timothy on from the from-worse-to-bad dept.

Maddog Batty (112434) writes 'The fine people at the NASA Space Flight Forum are making good progress on restoring the corrupted landing video reported earlier. It worth looking at the original video to see how bad it was and then at the latest restored video. It is now possible to see the legs being deployed, the sea coming closer and a big flame ball as the rocket plume hits the water. An impressive improvement so far and it is still being actively worked on so further refinements are likely.' Like Maddog Batty, I'd suggest watching the restored version first (note: the video is lower on the page), to see just what a big improvement's been made so far.

54 comments

  1. Special edition by abies · · Score: 2, Funny

    Will they add small robots and funny looking animals in background later? I don't think that even cleaned version represents artistic vision of landing they had in mind.

  2. Just do it again by Starvingboy · · Score: 2

    To heck with fixing up the video, just launch another rocket already and try again.

    1. Re:Just do it again by arse+maker · · Score: 5, Interesting

      They will shortly, there was a planned launch last month but it has been pushed back for various reasons. http://www.space.com/25822-spa...

      The fact they have this thing vertical at well below terminal velocity and apparently not spinning means the rest is just details. Controlling it down from supersonic is the hard part. They have made many successful landings with grasshopper from a vertical, low speed non spinning state.

    2. Re:Just do it again by Anonymous Coward · · Score: 4, Insightful

      This landing in particular holds some interesting extra data though, as this landing was during a storm strong enough to keep ships out of the area. So seeing how well the rocket performs in such extreme circumstances, where you can have considerable lateral wind loading, in as much detail as possible could be quite useful for later engineering work.

    3. Re:Just do it again by StripedCow · · Score: 5, Funny

      Or they could just pay the license fee to unlock the DRM.

      --
      If Pandora's box is destined to be opened, *I* want to be the one to open it.
    4. Re:Just do it again by VideoPrincess · · Score: 2

      This is true - in the repaired video, the rocket can be seen adjusting its course significantly as it descends towards the sea. The control software seems to be able to cope with it well, which is good!

    5. Re:Just do it again by Anonymous Coward · · Score: 1

      They should use the same software that's used on those CSI shows. Those police can enhance a cheap security camera image into a full color HD quality image that can zoom in and read a newspaper article from 3 blocks away.

    6. Re:Just do it again by Anonymous Coward · · Score: 0

      Actually, I think the "most important" parts are the leg extension segments. This was the first time that they had launched with legs, and the grasshopper/F9R-dev1 test flights had always had stationary legs. Getting video of the legs extending was super cool.

  3. cheap webcam by Cheeze · · Score: 4, Funny

    looks like they tried to use video conference software over a dialup modem with a webcam from 2001.

    --
    Why read the article when I can just make up a snap judgement?
    1. Re:cheap webcam by geogob · · Score: 2, Insightful

      The technical challenges of running a telemetry link with something falling through the sky from a moving aircraft has little to do with that of plugging two televisions together over a wired network. I'd expect the bandwidth for the video to be, in fact, comparable to that of a dial modem, especially considering that the bandwidth is shared with a lot of other housekeeping data which are probably much more important and useful than the video feed.

    2. Re:cheap webcam by kaiser423 · · Score: 5, Interesting

      Video feeds are typically their own streams. They're typically in the couple of Mbit range, but really can be anything. We've had 10+Mbps video links, but they're typically high frame rate versus high resolution. The thing to remember here is that you can't do any real fancy compression or modulations schemes typically, so every a couple of Mb/s really isn't that high of resolution. This is because you know that you're dropping bits, you're signal is fading unpredictably, the signal propagation path is changing wildly, etc so things like QAM don't work, and compression actually hurts because you're often getting errors in the blocks, etc really throws a wrench in the whole thing. So you almost have to ship the video raw over some fairly inefficient modulation scheme like FM or SOQPSK (more efficient, but more likely to burst-lose lots of data).

      I took a quick look at the embedded video stream, and it looks like there would have been a better way to pack it (looks like some asynchronous frames inside, with multiple sync words inside needing to be correct to get a good frame, made it harder than it had to be. But still, this isn't easy stuff. I expect them to come out shooting next time though. They really didn't have much in the area to grab the video with good fidelity because they had other things to focus on, but this time I expect a bit more.

      I do telemetry chase form aircraft, boats, etc for exactly this type of thing for a living. Fun job :)

    3. Re:cheap webcam by Anonymous Coward · · Score: 3, Interesting

      Well considering HOW they actually got the video, I guess it's not too shabby. Here's a quote from Musk from the recent Dragon v2 unveiling:

      "As far as the soft landing of the boost phase, it was interesting, when we got the corrupted video back, because we really actually had a real difficult time getting the telemetry. In fact, I'll tell you a funny thing. We actually had to - because normally we get the bulk of the telemetry from a boat. We also have a backup, an AP3 that was going to go up, and the P3 got iced up, the boats couldn't go out, so I sent my plane up with my pilots, and... we had to design and fabricate an antenna that exactly fit in the window of the plane. We started off with a pizza dish and we were able to do a double loop antenna with a pizza dish and point it out the window to get the link."

    4. Re:cheap webcam by mvpel · · Score: 3, Interesting

      The story we heard from SpaceX is that due to the weather, they couldn't get a boat out in the 15-foot seas, and the NASA chase plane they usually use wasn't available, so they went up in Elon's private jet equipped with an antenna jerry-rigged from a pizza pan stuck in the window. Maybe next time they'll use a Pringles can instead. :D

      One of the mpeg experts was lamenting the encoding - there was no error correction enabled in the MPEG encoding, and the images were interlaced from an NTSC 29.97 down to a 15-fps feed into the encoder, and interlacing crushes the compression ratio, I gather. They've been informed of this, so hopefully the next launch will see some rock-solid quality video.

    5. Re:cheap webcam by kaiser423 · · Score: 1

      I typically supply the NASA chase plane for downrange TM :) We could have supported, just weren't asked to (likely a cost thing and potential safety thing).

  4. Digital vs analog by Hamsterdan · · Score: 2

    Old debate, but analog video would at least be watchable (like analog vs DTV reception). Digital is nice and all, but it's all or nothing. Or they could have added error correction (what was the codec BTW?)

    --
    I've got better things to do tonight than die.
    1. Re:Digital vs analog by geogob · · Score: 1

      But you wouldn't get it through the telemetry data feed. You'd have to have an separate analog transmitter just for the video feed. I'd say that it would be worth it if the video feed was critical. I doubt it is... it's most likely only a nice to have feature.

    2. Re:Digital vs analog by werepants · · Score: 1

      I believe the codec was FFMPEG - they've got one of the codec's authors helping out on the recovery effort.

    3. Re:Digital vs analog by arse+maker · · Score: 1

      Sure but they weren't making a TV show. They have the telemetry (from what I can tell), its just the video that is so poor. Getting high bandwidth data from below the horizon of a fast moving object is hardly easy.

      They will have nice enough videos when they bring the first stage back to land :)

    4. Re:Digital vs analog by Anonymous Coward · · Score: 0

      Highly doubtful as ffmpeg isn't a codec... it's a software stack that uses other libraries (including libavcodec) to do the codec processing.

    5. Re:Digital vs analog by citizenr · · Score: 1

      FEC is a standard feature of any digital video broadcast

      problem is they only have TS file from the receiver with badly interpreted FEC, not a raw stream of I/Q samples that can be worked on more extensively, so only way to recover anything now is to try and guess what went wrong in the receiver and what got shifted/xored.

      It would be prudent in the future for SpaceX to have SDR as a backup receiver.

      --
      Who logs in to gdm? Not I, said the duck.
    6. Re:Digital vs analog by Anonymous Coward · · Score: 0

      Correction: The codec was MPEG-2, which is why there was all of the talk about I-Frames and tweaking headers in the original stream. Much of the "error correction" is simply fixing up MPEG headers where it appears that the header may have been garbled up.

      Yes, the author of FFMPEG is involved with this cleanup effort, but that is because he is familiar with the MPEG specification. That software seems to have been used by SpaceX in the encoding process of the video image in the first place though.

    7. Re:Digital vs analog by VideoPrincess · · Score: 3, Informative

      Codec was MPEG4 in an MPEG-TS transport stream. The author of ffmpeg confirms it here.

    8. Re:Digital vs analog by VideoPrincess · · Score: 3, Informative

      No, codec was MPEG4 as explained by the author of ffmpeg!

    9. Re:Digital vs analog by werepants · · Score: 1

      Good to know - I defer to you, as I have next to no idea what I'm talking about with video stuff.

    10. Re:Digital vs analog by Anonymous Coward · · Score: 0

      Old debate, but analog video would at least be watchable (like analog vs DTV reception). Digital is nice and all, but it's all or nothing. Or they could have added error correction (what was the codec BTW?)

      Obviously digital isn't all or nothing as they did get something.

    11. Re:Digital vs analog by tibit · · Score: 1

      This! I can't agree more.

      --
      A successful API design takes a mixture of software design and pedagogy.
    12. Re:Digital vs analog by Hamsterdan · · Score: 1

      http://en.wikipedia.org/wiki/D...

      --
      I've got better things to do tonight than die.
  5. Summary of techniques used? by Whip · · Score: 3, Interesting

    I would *love* to see a summary of the types of problems the video stream has, and the techniques used to recover them. Anyone feel like sorting through the ~70 pages of thread and cataloging them? :)

    1. Re:Summary of techniques used? by Anonymous Coward · · Score: 1

      Sorry- the people who could/would do this for you are.. you know.. actually helping with the restoration.

    2. Re:Summary of techniques used? by werepants · · Score: 4, Informative

      I've been following the thread, but I don't know shit about video codecs or recovery so my understanding is limited at best. That said, it seems like FFMPEG (the codec used, I think) gains a lot of its compression by containing occasional keyframes that contain the full image, and then calculating deltas for providing subsequent frames. So, if you miss a few keyframes, you get huge swaths of video that are totally unintelligible. And, I think errors can cascade down into many subsequent frames because of the way the original image is modified and modified again.

      As well, I get the impression that blocks within images can have the same sorts of issues, where an early bit or two in error can corrupt the entire thing. So, the effort has seemed to focus on trying to go through and fix keyframes first, and sometimes human pattern recognition can pick out the errors quickly, sometimes it looks like it has been a frame-by-frame trial and error where someone flips a bit and sees what comes out.

      Given ~20 seconds of video, ~30 FPS, and probably several hundred blocks per frame, that's on the order of 100,000 pieces that are being repaired by human trial-and-error. It's a pretty herculean effort led by some extremely capable people.

    3. Re:Summary of techniques used? by dinfinity · · Score: 4, Informative

      There's a wiki here: http://spacexlanding.wikispace...

    4. Re:Summary of techniques used? by VideoPrincess · · Score: 5, Informative

      Hi, I'm the user "Princess" on the NSF site and I've mainly been involved with cleaning up the file at the TS level. I can answer any questions you like. The best summary for the Slashdot audience would be this one by Lourens, it explains things simply without dumbing things down. The types of problems we have are basically that bits have been either flipped or (rarely) omitted. The flips tend to clump together, i.e. you'll get an area that's good and then an area that's awful. The work is approximately divided into two parts: fixing up the file, and fixing up the video that results. I work on fixing the file, and from that I can find extra frames and pieces of MPEG4 data for the video people. Fixing the video is done by using a modified version of ffmpeg that can change macroblock pointers, ordering, luma and chroma. This work is not done on the file directly and can't easily be mapped back to the file, so it's not just a question of flipping bits once you get to the video level. Other technical info: The video itself is a broadcast (fixed bandwidth) MPEG-TS stream containing one video stream, a 704x480 MPEG4 stream at approx. 15 fps (technically half the NTSC framerate which is 15000 / 1001 fps).

    5. Re:Summary of techniques used? by VideoPrincess · · Score: 2

      Some of us can spare the time! Ask any questions you want, I've been fixing the MPEG-TS data in the file.

    6. Re:Summary of techniques used? by Anonymous Coward · · Score: 1

      It's actually ~15fps - the camera itself is apparently running NTSC 29.97 fps and interlacing two frames to produce one frame to hand off to a non-interlaced MPEG encoding for transmission over the radio. "FFMPEG" is the open-source video software tool - see http://www.ffmpeg.org/

      The cascading corruption occurs because each "macroblock" - an 8x8 square block of pixels - which makes up the video image can depend on the block above it or the block to its left, and for pframes, the corresponding block in the previous frame, and so you wind up with an error avalanche, as it were, as an invalid value spreads through each frame and through subsequent frames until the next iframe comes up 19 frames later to reset everything.

      The Wiki page mentioned below has some good links to key explanatory forum posts.

    7. Re:Summary of techniques used? by adisakp · · Score: 1

      FFMPEG is a program that supports hundreds of Codecs. It's likely they used a H264 or a compatible variant as the actual codec because a) its currently the most commonly used advanced video compression algorithm, b) there is plentiful hardware and software support for encoding / decoding, and c) it has a very good tradeoff for quality, bitrate, and processing horsepower required.

      There is no guarantee you will get keyframes using H264 if you are compressing video without detectable screen cuts. Some H264 compliant codecs, like the very commonly used x264 library (used by FFMPEG), do not even need to have dedicated keyframes at all but rather use a technique called Periodic Intra Refresh to encode videos without keyframes. Periodic Intra Refresh provides much better streaming /live capture performance since it lowers both encode and decode latency for transmission and it lessens the incidence and severity of data rate spikes when using variable bit rate compression schemes.

    8. Re:Summary of techniques used? by werepants · · Score: 1

      Ya, I realized that shortly after posting this. A more knowledgeable poster said it was MPEG4 in an MPEG-TS transport stream.

    9. Re:Summary of techniques used? by Anonymous Coward · · Score: 0

      Ah, that's an absolutely wonderful link, thank you! That had a lot of the details I'm looking for.

      One outstanding question I had, though -- do we have any idea why the corruption existed to start with? Was it just low signal strength so some bits got lost in the noise?

  6. I WANT TO BELIEVE by Thud457 · · Score: 2

    It still looks like it was filmed with a Logitech potato.

    --

    the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff

    1. Re:I WANT TO BELIEVE by Anonymous Coward · · Score: 1

      Unfortunately the hight quality cameras are specified for 0-70 degrees Celsius and indoors use only.

    2. Re:I WANT TO BELIEVE by mvpel · · Score: 1

      Not to mention melting due to aerodynamic heating:
      http://www.youtube.com/watch?v=7cvYIHIgH-s

  7. Conflicting advice. Great editting, timofee ! by Anonymous Coward · · Score: 0

    Poster: "It worth looking at the original video to see how bad it was *and then* at the latest restored video."

    Timofee: "I'd suggest watching the restored version first"

    Do Slashdot "editors" even read the stories they comment on ?

  8. PROOFREAD... by Anonymous Coward · · Score: 0

    "It worth looking at"

    Well done.

  9. Uncrop and enhance... by Anonymous Coward · · Score: 0

    .. then zoom out and enhance even more.

    And we make laugh of arguably bad tv shows that extract DNA from random piece of hair found in the crack between two tiles on the floor near a crime scene...

  10. VLC to the rescue! by who_stole_my_kidneys · · Score: 0

    they just did a conversion in VLC and bamn! clear video!

  11. Re:MPEG compression by Anonymous Coward · · Score: 0

    Your analysis is essentially correct.
    I frames are essentially JPEG images.
    P frames are deltas against the previous I or P frame.
    B frames are deltas against the previous and following I and P frames.
    https://en.wikipedia.org/wiki/...

  12. For those of you carping by Anonymous Coward · · Score: 1

    Keep in mind that this was transmitting in a storm. As such, they were losing data. All in all, what they have cleaned up, has been impressive.

  13. you wouldn't throw away a car after one use... by Thud457 · · Score: 2

    Somebody needs to throw together one of those X-Files "I WANT TO BELIEVE" parody posters with a fuzzy Falcon 9 1st stage coming down for a fiery landing exactly as described by Ezekiel.

    --

    the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff

  14. Next time by Anonymous Coward · · Score: 0

    use data partitioning and other error resilient features on the mpeg encoder. Defaults do not suit everyone, especially on video encoding field.

  15. Just do it again by Anonymous Coward · · Score: 0

    This is the video documentation of the very first soft "landing" of a rocket's first/booster stage. EVER. As such, it is documenting a huge milestone in rocketry/aeronautics (SpaceX was also able to get uncorrupted telemetry data). Maybe it's not quite on the same scale of the Wright Brother's first flight at Kittyhawk, but just getting the video from a subsequent launch isn't the same thing. This was the first major step in SpaceX's project to make a fully reusable launch vehicle. If they are eventually successful, they will have started a revolution in spaceflight. Or at least in access to space.

    Besides, the next scheduled launch for SpaceX is scheduled for NET (not earlier than) June 12th, ~9pm EDT, so a night launch when there is not much hope of very good video of the booster's landing.

  16. CSI to the rescue by Anonymous Coward · · Score: 0

    * video shows random noise because camera broke
    * "We need to see some legs, can you do that?"
    * Forensic shaman presses magic button "ENHANCE!"
    * posts it on /.

  17. Mod this funny! by SpammersAreScum · · Score: 1

    Somebody with a major Humor Deficiency apparently moderated the parent.