Slashdot Mirror


Is Time Moving Forward Or Backward? Computers Learn To Spot the Difference

sciencehabit (1205606) writes For the first time, scientists have taught computers to figure out the direction of time in videos, a result that could help researchers better understand our own perception of time. Regardless of any possible applications, "we just thought it was a great problem," says one of the study's authors. Teaching computers to see the arrow of time combines computer science, physics, and human perception to get at the heart of the question, "How do we understand the visual world?" The researchers "broke down 180 YouTube videos into square patches of a few hundred pixels, which they further divided into four-by-four grids. Combining standard techniques for discovering objects in still photographs with motion detection algorithms, the researchers identified 4000 typical patterns of motion, or 'flow words,' across a grid’s 16 cells. ... When they tested their program on the remaining 60 videos, the trained computers could correctly determine whether a video ran forward or backward 80% of the time."

78 comments

  1. Easy by ZombieBraintrust · · Score: 3, Funny
    Beginning

    hey guys..

    End

    Click Like To Subscribe. Follow Hashtag blah on twitter, blah blah reddit. Hand waves to bottom of screen.

    1. Re:Easy by ZahrGnosis · · Score: 2

      Actually, if you guessed that a randomly selected set of youtube videos were being played, you know... FORWARD, you'd probably be correct more than 80% of the time without having to actually think at all. I assume their 80% result was based on something more difficult, but it's still kind of a silly sounding number without context.

    2. Re:Easy by ZombieBraintrust · · Score: 1

      I assumed they took youtube vids and then played them backwards half the time. Then they asked the computer to guess forward or backward. I think it would be easier to guess based on the audio than the vidio. You could just put the audio through speach regonition. If the sound makes sense as words in some language then it is playing forwards. But good for them to look at the harder problem.

    3. Re:Easy by Anonymous Coward · · Score: 0

      But the point is not to tell if sound is moving forward or backwards.

      The point is to detect if a video (possibly even without sound) is moving forward or backwards.

      But you could easily mess up this program by showing it a video of Michael Jackson doing the Moonwalk.

    4. Re:Easy by Dcnjoe60 · · Score: 3, Insightful

      But the point is not to tell if sound is moving forward or backwards.

      The point is to detect if a video (possibly even without sound) is moving forward or backwards.

      But you could easily mess up this program by showing it a video of Michael Jackson doing the Moonwalk.

      However, whether detecting if sound or video is moving backwards or not has nothing to do with detecting if time is moving backwards or not. Playing a file backwards still occurs with time moving forwards.

    5. Re:Easy by Gallefray · · Score: 1

      http://en.wikipedia.org/wiki/B... [wikipedia.org]

      Except the headline *doesn't* end in a question mark. It clearly finishes it with the letter `e`, ergo Betteridge's Law of Headlines doesn't apply.

    6. Re:Easy by Anonymous Coward · · Score: 0

      I know of at least one show where the most sound effects were made by playing sounds backwards.

    7. Re:Easy by michelcolman · · Score: 1

      If the sound makes sense as words in some language then it is playing forwards.

      Except for certain Prince songs.

    8. Re:Easy by michelcolman · · Score: 1

      Yes, that's a different law: if a headline ends in a question mark, the author of the article got everything backwards.

    9. Re:Easy by Jane+Q.+Public · · Score: 1

      I assume their 80% result was based on something more difficult, but it's still kind of a silly sounding number without context.

      On our next episode: Scientists Use Computers To Determine Which Direction The Earth Is Turning.

    10. Re:Easy by david_thornley · · Score: 1

      Consider Weird Al Yankovic's "Amish Paradise" video. There's one scene in which everything but Al seems to be moving in reverse. (In reality, it was shot with Al walking and singing backwards, then reversed.)

      --
      "When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
  2. How'd it do on Top Secret? by Overzeetop · · Score: 4, Interesting

    https://www.youtube.com/watch?...

    (and..."Europe on 5 Quaaludes a Day," forward version: https://www.youtube.com/watch?...

    --
    Is it just my observation, or are there way too many stupid people in the world?
  3. E L O by Anonymous Coward · · Score: 0

    !!

  4. Sooo.... by clam666 · · Score: 0

    This saves us from the terrorists how?

    This sounds like the beginning of evil technology so that I can't use my DVR to skip commercials.

    --
    I'm a satanic clam.
    1. Re:Sooo.... by Meshach · · Score: 0

      This saves us from the terrorists how?

      Is that the only measure of success?

      --
      "Maybe this world is another planet's hell"
      Aldous Huxley
    2. Re:Sooo.... by clam666 · · Score: 2

      Is that the only measure of success?

      ...and this is why the terrorists are winning. If you're in 'murica, get some Jesus. If you aren't, stop takin' meh jobs!

      --
      I'm a satanic clam.
    3. Re:Sooo.... by Dishevel · · Score: 3, Insightful

      This saves us from the terrorists how?

      Is that the only measure of success?

      Not. It is not the only measure. We also measure success by thinking of the children. Any other questions?

      --
      Why is it so hard to only have politicians for a few years, then have them go away?
  5. I know by nitehawk214 · · Score: 2

    I will call you yesterday and let you know.

    --
    I'm a good cook. I'm a fantastic eater. - Steven Brust
    1. Re:I know by geekoid · · Score: 2

      I was in yesterday, and you never called.

      --
      The Kruger Dunning explains most post on /. http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
    2. Re:I know by Anonymous Coward · · Score: 0

      So meant next yesterday, not this past one....

    3. Re:I know by nitehawk214 · · Score: 1

      How about the second Thursday of last week?

      --
      I'm a good cook. I'm a fantastic eater. - Steven Brust
  6. time or not by buswolley · · Score: 4, Insightful

    This means nothing. It is not detecting time per se. It is detecting things violations like objects don't fall up, or other such experienced pattern that is the result of time.

    --

    A Good Troll is better than a Bad Human.

    1. Re:time or not by gnick · · Score: 0

      It doesn't even have to do that. I just watched 180 YouTube videos and guessed that they were all moving forward. I blew that 80% number out of the water.

      --
      He's getting rather old, but he's a good mouse.
    2. Re:time or not by rogoshen1 · · Score: 2

      Hi gnick, this is your boss. Step into my office please.

    3. Re:time or not by Anonymous Coward · · Score: 0

      This means nothing. It is not detecting time per se. It is detecting things violations like objects don't fall up, or other such experienced pattern that is the result of time.

      Oh, come on!!! Think of the practical applications for a computer program that can determine if you are watching a youtube video forwards or backwards. Here's what we can use it for:
      1) ummmmmm.... hmmmmmmm....
      2) well..... i guess maybe..... Nope....
      3) ahhhhhhhh.. Shucks......

      Okay, it really has no valuable use. But an article about it did make it to Slashdot!!! :)

    4. Re:time or not by Your.Master · · Score: 1

      What is the difference?

      How is detecting a violation like objects falling up different from detecting time?

      I can only imagine you think the real test is some different scenario?

    5. Re:time or not by Anonymous Coward · · Score: 0

      Aw, gee whiz, Mom, don't make me come upstairs from the basement!

  7. 80 only? by Anonymous Coward · · Score: 0

    Mission accomplished?

  8. ok by hurfy · · Score: 2

    If I close my eyes and say "forward" what percentage do I get right?

    1. Re:ok by TheCreeep · · Score: 1

      Depends on the data set used. In this case 50%

    2. Re:ok by Carnildo · · Score: 2

      More significantly, if you see left-to-right motion and say "forward", what percentage do you get right? I suspect there's a bias in videos towards left-to-right motion of subjects (or conversely, right-to-left motion of backgrounds), and I don't see anything in the paper about controlling for it.

      --
      "They redundantly repeated themselves over and over again incessantly without end ad infinitum" -- ibid.
  9. Time isn't moving at all by kruach+aum · · Score: 1

    Events unfold in time, but time itself doesn't move. Substitute space for time to make the absurdity clearer: "Is space moving forwards or backwards?" Space isn't moving, we move through space.

    1. Re:Time isn't moving at all by Anonymous Coward · · Score: 0

      Events unfold in time, but time itself doesn't move. Substitute space for time to make the absurdity clearer: "Is space moving forwards or backwards?" Space isn't moving, we move through space.

      That's simply wrong. You have no idea what you are talking about.

    2. Re:Time isn't moving at all by TheCreeep · · Score: 1

      You're imposing an arbitrary abstraction on a complicated natural phenomenon.
      Space *is* not doing anything. We observe things in the real world and we construct useful models that contain notions such as "time" and "space".

      In this particular case, time is analogous to the frame index of the video. If that index is increasing, time is moving forward. If it is decreasing, time is moving backwards in this video. Everybody understands what it means so stop being pedantic.

    3. Re:Time isn't moving at all by sjames · · Score: 2

      If you prefer, now moves through time. At the quantum level there is no directional preference yet at the macro level, there is a distinct preference. Why that is so is an open question.

    4. Re:Time isn't moving at all by kruach+aum · · Score: 1

      I'm not doing anything of the sort. I'm clarifying that the title is incoherent. If time could move, growing older would make no sense; you would age because time would move past you, rather than you move through time. And because everyone has a different age, it would mean that time moves differently past everyone. Everyone (and every thing) would have their own personal time bubble, rather than time being just a dimension of a shared world.

    5. Re:Time isn't moving at all by kruach+aum · · Score: 0

      You're simply wrong. You have no idea what I know.

    6. Re:Time isn't moving at all by Charliemopps · · Score: 1

      Events unfold in time, but time itself doesn't move. Substitute space for time to make the absurdity clearer: "Is space moving forwards or backwards?" Space isn't moving, we move through space.

      Since you're being pedantic, so can I. Space can travel. It moves all the time. It warps and contorts, and acts very much like a "thing" Time also warps and contorts. So if you want to get all picky, the article is still correct :-p

    7. Re:Time isn't moving at all by kruach+aum · · Score: 2

      For space to be able to travel there needs to be some frame of reference against which it can be judged to have travelled. As the frame of reference for travel is space itself, if space could travel we wouldn't be able to tell.

    8. Re:Time isn't moving at all by lazy+genes · · Score: 0

      I briefly understood time on the quantum level. All I can tell you is it is very complex and It had to do with the physical geometry of the fabric of space time/higgs field.

    9. Re:Time isn't moving at all by Ultra64 · · Score: 1
    10. Re:Time isn't moving at all by Anonymous Coward · · Score: 0

      People do age at different rates....there is no absolute/common/shared time....
      The twin left and returned on his rocketship to find his twin dead "for centuries"....

    11. Re:Time isn't moving at all by Anonymous Coward · · Score: 0

      Time is what a clock tells you...and physically identical clocks moving/accelerating relative to the other, or sitting in different gravitational fields relative to the other will report different amounts of elapsed time when brought back together....

  10. Lazy AI by Anonymous Coward · · Score: 0

    Can we stop the lazy AI experiments which required zero effort?

    1. Re:Lazy AI by TheCreeep · · Score: 1

      Can you stop the lazy commenting experiments that require zero effort? This was by no means lazy, it was a complicated computer vision task that required a lot of preparation and experimentation.

  11. Sounds like bad methology by imsabbel · · Score: 2

    You tube videos? And analysation on block level?

    Won't the fact that the video codec has a direct timeline (with predictive frames, etc) override the rsults?

    --
    HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
    1. Re:Sounds like bad methology by Minwee · · Score: 3, Funny

      Be quiet! You'll jeopardize our funding.

    2. Re:Sounds like bad methology by TheCreeep · · Score: 1

      The only reason that the codec would influence the outcome is if it would generate artifacts that are somehow informative of the direction of time. In practice, codecs aim to keep as much information as possible while reducing the required space by finding regularities in the video. If the quality is sufficiently high, there would be next to no artifacts. Only in cases where the quality is extremely low there are tell-tale signs of the direction of time. Everyone has seen damaged Xvid movies I assume.

    3. Re:Sounds like bad methology by mrbobjoe · · Score: 2
      Good question about the codec, they took that into account somewhat by including a dataset that used a codec with only intra-frames (I imagine this was MJPEG).

      From section 3.3:

      We also filmed a small number of video clips using a camera which could record in a video codec which used only intra-frame, rather than inter-frame coding, meaning that there was no possibility of compression artefacts holding any time-direction information. This dataset comprises 13 HD videos of tennis balls being rolled along a floor and colliding with other rolling or static balls.

      The algorithms tested did well on this dataset as well, 12/13 classified correctly.

    4. Re:Sounds like bad methology by geekoid · · Score: 1

      No.

      Maybe your opinion should be with held until you get more data then a /. summary.

      --
      The Kruger Dunning explains most post on /. http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
    5. Re:Sounds like bad methology by mikael · · Score: 1

      I'd imagine they decompress the video into it's constituent frames. That's easy to do with various Linux command line tools. Now you have to determine whether each adjacent pair of images are moving forwards or backwards in time. You can split this task up into small tiles to make use of parallel processing. Now you've got various sorts of movement; no change (eg. blue sky), upwards movement (smoke, clouds, rockets), sideways movement (cars, people), downwards movement (stuff falling, parachutists). Each of those will have it's own pattern of pixel movement and colors.

      If you can understand what an object is, you can impose some sort of expectations on how it will move. You just need to look at some of those early comedy movies where the directors discovered how to play a film reel backwards. A tractor/trailer going backwards wasn't unusual, but someone lying on the ground, rolling backwards then jumping back into a standing position on the trailer was. Another one would be paratroopers receive an order to retreat, standing in a field, inflating their parachutes and jumping upwards into the back of an aircraft. So some rules are: human figures don't jump higher than 2 or 3 feet without help of a trampoline or unless they are a super hero. Smoke doesn't concentrate itself back into a small tube. Liquids don't fall upwards into the ceiling. If the system can understand those rules, it can tell when a video is being played backwards or forwards.

      --
      Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
    6. Re:Sounds like bad methology by saccade.com · · Score: 1

      +1. It seems like the results are perhaps keying off the compression artifacts introduced rather than any fundamental image data. Moreover, the compression artifacts are consistent from video to video, forming a consistent training set.

  12. Forward, obviously. by Anonymous Coward · · Score: 0

    Otherwise I would have gotten first post.

  13. I know how we can REALLY put it to the test... by Anonymous Coward · · Score: 1

    Hard Mode: Test it on Back to the Future.

    1. Re:I know how we can REALLY put it to the test... by danbert8 · · Score: 1

      pfft, test it on any Ace Ventura movie where Jim Carrey does his own reverse taking sequence!

      --
      Yes it's an anecdote! Were you expecting original research in a Slashdot comment?
  14. Maybe they'll be able to spot mirrored images too by Anonymous Coward · · Score: 0

    Content ID has been foiled by silly tricks in the past like mirroring videos, adding new frames and letterboxing before.

  15. Try the videos slashdotters link to... by damn_registrars · · Score: 1

    The political videos that people like to link to from here often praise the past, and demonize the present. While they leave me wishing I could have my time back I would love to know if the comptuers could recognize time as moving forward.

    (That said, as they are mostly political speeches with no significant moving object the identification could be done by speech pattern recognition)

    --
    Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
    1. Re:Try the videos slashdotters link to... by david_thornley · · Score: 1

      Praising the past and demonizing the present goes back as far as we have records. Not that they do it right nowadays.

      --
      "When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
  16. history racing up to correct itself/us by Anonymous Coward · · Score: 0

    who's time is it? do we need have our time divided into minutes or is that our worth as a commodity can be enforced? the spirit of creation may be tiring of our tardy way of ignoring our undeniable obligations to each other, starving people etc...? time's up for being heartless? see you there.... thanks again moms

  17. Oh really? by ArcadeMan · · Score: 2

    Have they tried their software on the music video Amish Paradise by "Weird Al" Yankovic?

  18. Hitchhiker's Guide prototype by Gestahl · · Score: 1

    Behold, the first piece of the the trans-dimensional Hitchhiker's Guide. One of the first things it has to do is figure out is which way time runs in whatever reality it finds itself in.

  19. Shameless Ad by lorinc · · Score: 1

    Seriously, the poster was presented yesterday at CVPR and ends up on /. today. There is nothing sensational about it. I'm getting sick of science turning into PR stunts all the time.

  20. Is entropy decreasing? by jeffasselin · · Score: 1

    We can't even make machines that can figure out if entropy is increasing or decreasing in a video recording (something most humans can do unsciously). But we'll have human-equivalent AI is in our grasp within 10 years. And the singularity is coming within 20.

    Sure.

    --
    If he explores all forms and substances Straight homeward to their symbol-essences; He shall not die.
  21. Memento by Baby+Duck · · Score: 1

    But would it recognize the time flow of a movie like Memento? Every sequence moves forward, but each subsequent sequence predates the previous one, with overlap. There are also flashback sequences.

    --

    "Love heals scars love left." -- Henry Rollins

  22. Obviously time is moving "forward" by Applehu+Akbar · · Score: 1

    That being the label we apply to the direction we observe time to be moving in. If there is some other direction time could be moving in, please demonstrate it so we could label it appropriately.

  23. Probably cheating by penguinoid · · Score: 1

    Odds are it's probably cheating somehow, eg discovered which direction based on text fading/scrolling, or backwards voice, or something. Also, 80% success rate is rather poor, though I suppose some sort of things would be hard to tell (if they have little change in entropy).

    --
    Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
  24. Nothing new here. by Dcnjoe60 · · Score: 1

    It would seem that before you could make a computer detect time moving backwards, you would first have to devise a way to actually make time move backwards. Running a video in reverse still occurs with time moving forward. That's nothing new. VCRs did this with a little LED. DVDs do it with an on-screen display.

    Now, if they found a way to actually make time move backwards, that would be something.

  25. ...may one day save the universe. by Anonymous Coward · · Score: 0

    In what way does this aid anyone in better understanding human perception of time? I mean, fuck, bravo on the pure computer science of it all, but these sorts of lofty interdisciplinary tie-ins reek terribly of hyperbole.

  26. Chinese retirees walking backwards by magarity · · Score: 1

    To really confuse it, point it out the window of an apartment complex anywhere in China. At any given time at least one retiree is walking around the grounds backwards as a form of exercise and or coordination boost (I haven't figured out which yet).

    1. Re:Chinese retirees walking backwards by Anonymous Coward · · Score: 0

      At any given time at least one retiree is walking around the grounds backwards as a form of exercise and or coordination boost (I haven't figured out which yet).

      Actually, they're spies sending coded messages to Chinese satellites. In the next version they plan to do it with interpretative dance.

  27. Abrupt transitions in optical flow? by RandCraw · · Score: 3, Informative

    Dr Freeman spoke about this work at CVPR this week. In the videos I saw he identified small markers of temporal transition as indicative of moving forward or backward. Those they labeled as backward appeared to recognize asymmetric movement -- as in gradual acceleration followed by sudden deceleration as uniquely forward flow (as when a hand swings down and strickes a table top) -- an asymmetry that cannot occur in reverse (as in sudden acceleration followed by gradual deceleration).

    Dr Freeman did not propose this as the causal phenomenon in question, but that made the most sense to me in light of the motions he identified as evidence for backward motion.

    1. Re:Abrupt transitions in optical flow? by Anonymous Coward · · Score: 0

      What about the sudden accleration and gradual deceleration of a golf ball, BB fired from a gun? They are less common cased but they would cause a serious problem for ruled based on "typical" types of motion. The real key would be to write rules that check what direction entropy is going but even then there are many cases that might cause issues.

    2. Re:Abrupt transitions in optical flow? by RandCraw · · Score: 1

      I suspect most events like you describe probably would occur too quickly for conventional cameras to capture, but I see your point. It seems to me that kind of mition would have to take the form of a percussive force that arises without visible warning -- like the launch of an explosive powered bullet, and unlike the launch of a golfball being struck by a moving golf club that rapidly approached the stationary ball before making contact. And as you suggest, I doubt the firing of a bullet is the kind of motion seen most often in videos.

      I suspect the vast majority of conventional motion sequences follow a path of 1) slow accel followed by slow decel (providing little clue as to directon of time), or 2) slow accel followed by fast decel (something that occurs often in forward moving time sequences, as a moving object is stopped suddenly by an impact). Thus path #2 is probably frequent enough and visible enough to be the anomaly that lets Freeman's group recognize the backward passage of time.

  28. Forward and Backward are just definitions by YoungManKlaus · · Score: 1

    it only depends on the way you look at things

  29. Things.. by Anonymous Coward · · Score: 0

    .. tend to move from left to right, since we so used to read that way, movie makers are used to it to. .. tend to move against us, people shoot movies of people moving against us, much more boring to see something moving away.

    Do I get 80% correct, on Youtube movies?

  30. Whoah! by Anonymous Coward · · Score: 0

    FTS: "[...] broke down 180 YouTube videos [... then ...] tested their program on the remaining 60 videos"

    Whoah. They used all 240 YouTube videos?! That's, like, insane!