Slashdot Mirror

← Back to Stories (view on slashdot.org)

Choose a Better Train With Web Scraping (hackaday.com)

Posted by Soulskill on from the it's-never-on-time-when-you-need-it-to-be dept.

szczys writes: Tired of his trains being constantly late, Eric Evenchick headed to the Via Rail (Canada's communter train service) website to find which trains had a better on-time rate. Unfortunately they only offer three days worth of data through the dropdown selections — but a bit of investigating showed the GET requests were open for about the last six months. Evenchick built a web-scraper with Python, along with a web interface that queries the resulting SQL db. The harvested data shows system-wide delays that average more than twelve minutes (mostly due to commercial rail having the right-of-way). The good that comes of this? You can now choose your train based on smallest likelihood of delay..

5 of 50 comments (clear)

  1. Canada's communter train service by xxxJonBoyxxx · · Score: 3, Informative

    >> Canada's communter train service

    But do they have anything for commuters?

  2. See Via Rail limiting the GET requests in... by Ecuador · · Score: 3, Funny

    See Via Rail limiting the GET requests in 3... 2... 1...
    Well, OK, there's the weekend ahead, perhaps Monday? :)
    In any case it does look like commuter rail is a 2nd class citizen in Canada.

    --
    Violence is the last refuge of the incompetent. Polar Scope Align for iOS
  3. Violating ToS? by Anonymous Coward · · Score: 3, Insightful

    Check the site's terms of service, scraping site contents may be in violation of the ToS.

    I wrote a similar app about 15 years ago to scrape the Edmonton Transit System's route schedules (conveniently posted in generally well structured HTML at the time) so I could build a relational system and try and sort out predictive routes / times. Then I found out what I was doing was in violation of their ToS, I stopped my scraping service immediately (before getting called on it).

  4. Neither commuter nor "communter" by Roadmaster · · Score: 5, Informative

    VIA Rail is NOT a commuter train service. It offers "intercity passenger rail services", not commuter service, which Wikipedia defines better than I can: "Commuter rail, also called suburban rail, is a passenger rail transport service that primarily operates between a city centre, and the middle to outer suburbs...". Again, not what VIA Rail primarily does.

    Examples of agencies which offer commuter rail service in Canada include Greater Toronto's GO Transit trains and Montreal's AMT. These do, indeed, offer service between communities forming part of a greater metropolitan area and said area's city centre. At least in Montreal, the AMT has some exclusive tracks and agreements on shared tracks which prioritize commuter trains over other scheduled trains at rush hour.

  5. Some train companies have an API for that... by Anonymous Coward · · Score: 4, Informative

    See the National Rail Enquiries APIs. Loads of information on train timetables, delays, maintenance schedules, and almost all for free.
    http://www.programmableweb.com/api/national-rail-enquiries