Slashdot Mirror


GitHub's Website Remains Broken After a Data Storage System Failed Earlier Today (theregister.co.uk)

Github engineers are trying to repair the data storage system underpinning the code hosting website, which has been presenting users with a "What!?" error for much of the Sunday. From a report: Depending on where you are, you may have been working on some Sunday evening programming, or getting up to speed with work on a Monday morning, using resources on GitHub.com -- and possibly failing miserably as a result of the outage. From about 4pm US West Coast time on Sunday, the website has been stuttering and spluttering. Specifically, the site is still up and serving pages -- it's just intermittently serving out-of-date files, and ignoring submitted Gists, bug reports, and posts. Sometimes, it appears to be serving a read-only cache or older backup of itself, although some fresh code pushes are coming through onto the site. From the status page, it appears a data storage system died, forcing the platform's engineers to move the dot-com's files over to another box. In the meantime, some older versions of files and repos are being served to visitors and users. "We're continuing to work on migrating a data storage system in order to restore access to GitHub.com," the team said just after 5pm PT, adding in the past few minutes: "We are continuing to repair a data storage system for GitHub.com. You may see inconsistent results during this process."

66 comments

  1. INb4 by darkain · · Score: 4, Funny

    INb4 "Microsoft Broke It!" - they've yet to actually take over. They're still waiting on government approval of the acquisition.

    1. Re:INb4 by mentil · · Score: 1

      GitHub's resident Guru must Meditate before it can approve the acquisition.

      --
      Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    2. Re:INb4 by Tablizer · · Score: 1

      [re] "Microsoft Broke It!" - they've yet to actually take over. They're still waiting on government approval of the acquisition.

      Maybe MS doesn't want it after this.

    3. Re:INb4 by Anonymous Coward · · Score: 0

      What? The track record shows it's a good match.

      Oh and apparently the EU gave the green light.

    4. Re: INb4 by Anonymous Coward · · Score: 2, Interesting

      Actually DNS records show it already transfered to Microsoft.

    5. Re: INb4 by Anonymous Coward · · Score: 1

      Obviously Microsoft converted GitHub over to Windows 10 when they bought it. I guess it just decided to auto-update and eat all their (our) files....

    6. Re:INb4 by Highdude702 · · Score: 1

      Makes you wonder if they left a dead man switch to fuck over Microsoft..

  2. Microsoft by hcs_$reboot · · Score: 2

    just signed the GitHub purchase.

    --
    Slashdot, fix the reply notifications... You won't get away with it...
    1. Re:Microsoft by Anonymous Coward · · Score: 0

      I know who you are. How do I know?

      LOOK BEHIND YOU!

    2. Re:Microsoft by Anonymous Coward · · Score: 0

      It's creimer naked!! *eyes melt* *head asplodes in horror*

  3. First day under Microsoft by SuperKendall · · Score: 2

    "All ops staff required to use pure HoloLens interface with floating console and keyboard!"

    *Github immediately convulses*

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
    1. Re:First day under Microsoft by Anonymous Coward · · Score: 0

      And SuperKendall bends over and awaits Trump's micropenis to penetrate his quivering asshole like a good, little alt-right, incel cuck.

    2. Re: First day under Microsoft by Anonymous Coward · · Score: 0

      Anti-Trump stalker trolls sure do have vivid homosexual fantasies involving the President.

  4. Use SourceForge Instead by mentil · · Score: 0

    If you'd been using SourceForge instead, you wouldn't have to deal with the outage.
    This Slashvertisement brought to you by SlashdotMedia(TM).

    --
    Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    1. Re: Use SourceForge Instead by Anonymous Coward · · Score: 0

      ...not to mention the ads, shitty user interface, and 90s-era design. There was probably another acquisition or two by stupid corporate overlords.

    2. Re: Use SourceForge Instead by Tough+Love · · Score: 1

      And somehow managed to muddle through to this day without serious drama. While Sourceforge isn't to my taste, they still host a lot of projects and the price is right.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    3. Re: Use SourceForge Instead by jd · · Score: 1

      Nobody uses sourceforge. Outdated codebase. I forget which fork is still open source for gforge, but it's better.

      For commercial projects, Atlassian is now industry standard. It's good for funded open source, too, if a little overkill.

      Of course, a central server on a distributed system like git seems pointless.

      --
      It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
    4. Re: Use SourceForge Instead by Anonymous Coward · · Score: 0

      And somehow managed to muddle through to this day without serious drama. .

      seriously? they have had massive amounts of drama over the years from the outages earlier this year to attacks and various other downtimes. Sure I don't think they are any better/worse than GitHub on that front but lets not kid ourselves in pretending they have had no drama.

    5. Re: Use SourceForge Instead by Tough+Love · · Score: 3, Interesting

      For what it's worth, sourceforge is still here and googlecode isn't. Probably says more about Google's baby-eating culture than anything.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    6. Re: Use SourceForge Instead by Anonymous Coward · · Score: 0

      Distributed version control is pointless.

    7. Re: Use SourceForge Instead by Anonymous Coward · · Score: 1

      Right. Because you'd never have more than one developer in one location working on a project. Sheesh.

    8. Re: Use SourceForge Instead by Anonymous Coward · · Score: 0

      It does protect you from catastrophic server crashes.

    9. Re: Use SourceForge Instead by Anonymous Coward · · Score: 0

      Ah! I see what you did there.

  5. What is there storage back end? and what VM system by Joe_Dragon · · Score: 1

    What is there storage back end? and what VM system? are they useing?

  6. Kind of just like how they trashed Sidekick.... by TheNarrator · · Score: 1, Funny

    Does anyone remember back when they trashed most of the cloud data of Sidekick smartphone users shortly after they acquired the company? This was even covered by Slashdot!

    https://hardware.slashdot.org/...

    Looks like a repeat of the bumbling.

    1. Re:Kind of just like how they trashed Sidekick.... by Anonymous Coward · · Score: 0

      Except for that part where the Microsoft deal hasn't even closed yet. But, hey, don't let pesky things like facts get in your way.

    2. Re:Kind of just like how they trashed Sidekick.... by Tough+Love · · Score: 2

      The storage system heard that it would soon be acquired by Microsoft and had a heart attack :)

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    3. Re:Kind of just like how they trashed Sidekick.... by Anonymous Coward · · Score: 0

      *crickets*

    4. Re:Kind of just like how they trashed Sidekick.... by Tough+Love · · Score: 1

      Microsoft probably sent their advance team in years ago to soften them up like they did with Nokia.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    5. Re:Kind of just like how they trashed Sidekick.... by Anonymous Coward · · Score: 0

      are you a creimer sock pocket?

  7. not a bad failure mode by Anonymous Coward · · Score: 0

    you may have been working on some Sunday evening programming, or getting up to speed with work on a Monday morning, using resources on GitHub.com -- and possibly failing miserably as a result of the outage

    if not being able to make posts and gists breaks your workflow to the point of "failing miserably"...then you are just a miserable failure :-)

    1. Re:not a bad failure mode by Tough+Love · · Score: 1

      if not being able to make posts and gists breaks your workflow to the point of "failing miserably"...then you are just a miserable failure :-)

      I know right? And any shining star such as yourself out to be able to still succeed brilliantly even if their source repo serves out random files.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
  8. Which data storage system? by Tough+Love · · Score: 4, Insightful

    Which data storage sytem would that be? Brand and model. I want to know so that I can think badly of it, or possibly of the persons who configured it, but most probably the vendor.

    --
    When all you have is a hammer, every problem starts to look like a thumb.
    1. Re:Which data storage system? by avandesande · · Score: 1

      You can break the best systems by configuring them wrong.

      --
      love is just extroverted narcissism
    2. Re:Which data storage system? by Tough+Love · · Score: 2

      You can break the worst systems by configuring them correctly.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
  9. Re:What is there storage back end? and what VM sys by Tough+Love · · Score: 2
    --
    When all you have is a hammer, every problem starts to look like a thumb.
  10. Fiasco by Anonymous Coward · · Score: 0

    It's

    the

    whole

    Hotmail

    Debacle

    all
    over
    again. :-D

    1. Re:Fiasco by Anonymous Coward · · Score: 0

      Memes are strong today. Who is going to be the first to post M$? Oops it was me...

    2. Re:Fiasco by Anonymous Coward · · Score: 0

      Well, what ever was old, is new again, with our new and enlightened (ahum) generation of computer wanabees

      Time to release a new version of OS/2 or eCommStation as it's called now... though the OS/2 name is catchier, especially the WARP one

      OS/2 WARP
      when neither windows, linux nor Mac OSX tickle your funny bone anymore.

      OS/2 WARP to go where no-one has gone before !

      captcha : ailments

  11. Re:What is there storage back end? and what VM sys by Tough+Love · · Score: 1

    Another possible clue:

    I’m a developer on Openstack’s object storage system, and I believe that Openstack is uniquely positioned to achieve this vision.

    In contrast, I believe that Openstack is a steaming pile of impenetrable Python poo, especially its storage architecture. If this is indeed their platform then I am not at all surprised to hear that Github admins are now busy failing their files over manually.

    --
    When all you have is a hammer, every problem starts to look like a thumb.
  12. They should have used NodeJS by Anonymous Coward · · Score: 0

    NodeJS is an event driven not blocking io server that can be used to build high performance applications. Everyone knows that Python uses blocking IO. Python was developed before async was discovered.

    sarcasm

    1. Re:They should have used NodeJS by Tough+Love · · Score: 1

      Nice troll, that was sweet.

      --
      When all you have is a hammer, every problem starts to look like a thumb.
  13. What's the problem? by skdffff · · Score: 2

    Microsoft would just run CHKDSK to fix it.

    1. Re:What's the problem? by Tough+Love · · Score: 1

      Indeed they would, and when they do complete the acquisition they will copy all the storage to drive c: so it is easier to run CHKDSK. If anything goes wrong then Abort, retry, ignore?

      --
      When all you have is a hammer, every problem starts to look like a thumb.
    2. Re:What's the problem? by Anonymous Coward · · Score: 0

      Dear OP, please meet ReFS. ReFS, please meet OP.

      "the ReFS developers designed ReFS as a filesystem which does not fail, so ReFS does not have any built-in means of repairing"

      https://en.wikipedia.org/wiki/ReFS#Stability_and_known_problems

      Microsoft loves hubris. ReeeeeeeeFS is unrecoverable because it Microsoft thinks it cannot fail. The one and true snowflake OS, it's even in the name. I have had one HP MicroServer running as a lab test bed with ReFS for two months in a NAS role to test its stability. ECC RAM , latest UEFI version, medium SATA drive, OS on a small SSD drive C, data on that large HDD. Seemed to work okay until one day the HDD volume was gone. Its ReFS volume was unrecoverable without any indication of what happened. I just found it had dismounted on its own. Rebooted the server to see if anything would come back, but it didn't. The HDD was alright and could be read flawlessly in RAW mode using Linux. ReFS regarded the thing as bricked and then I found there are zero recovery utilities from the OS available, only paid-for versions by third parties with a hit or miss success rate for a substantial price for a test bed. Lost a terabyte of movies used as test data. Wouldn't touch ReeeeFS with a ten foot pole again.

  14. Oh dear! Another Windows 10 upgrade gone haywire by Anonymous Coward · · Score: 0

    I'm tellin' you people... these contraptions aren't really prime time

  15. A hub for gits... by Chas · · Score: 1

    Thanks Microsoft.

    Breaking shit and blowing stuff up...

    --


    Chas - The one, the only.
    THANK GOD!!!
  16. They need an IT miracle worker! by Anonymous Coward · · Score: 0

    They just need to figure out which one of his twenty five personalities Chris is pretending to be right now! But the nice thing is the storage closets at Github will be clean after. And all the fridges in the break room too!

    (Just don't go near Chris when you hear those awful slurping and gnashing sounds!)

  17. Fecal Midas Touch! by Anonymous Coward · · Score: 0

    Yeah, Microsoft has this. Everything it touches turns into shit.

  18. Terminate and Stay Resident by Anonymous Coward · · Score: 1

    Isn't that what Microsoft did to most acquisitions?

  19. Probably upgraded to Win10.... by gweihir · · Score: 1

    That thing does seem to delete files whenever it feels like.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  20. Tinfoil hat perhaps but .... by Anonymous Coward · · Score: 0

    This is Github, it doesn't exactly run on a single server nor have a single point of failure. Rather it's called a Fire Drill in IT, and done by Sociopathic PHB managers. The gist of it is you create an outage forcing your target (say... a company you just aquired or a team) - to act. You drill the hell out of them to turn over documentation, training, etc. in the process.

    It's NOT the same as cross training. They will use the outage(s) as weapons to implement some other changes which can be anything from changing who manages it to structural.

      More outages, more weapons.

  21. Re: THERE WILL BE CONSEQUENCES FOR YOUR LIES KEN D by Anonymous Coward · · Score: 0

    I'm curious. Are you just some bored IT guy who likes a laugh so post as AC, emails himself a reminder of stories trolled so you can come back later and read some lols?

  22. Linux has fallen, has fallen by Anonymous Coward · · Score: 0

    Because they opened their recta to LGBT tapeworms.
    Besides, it turns out that their "CoC" demands its own abolition. To begin with, the "CoC" itself doesn't escape its scope, for according to the same "CoC":

    "**This Code of Conduct applies** both within project spaces and **in public spaces when an individual is representing the project or its community**. Examples of representing a project or community include **using an official project e-mail**, ..."

    The "CoC", being spread in public spaces as, ostensibly, a "representation" of, among others, the Linux developer community, is subjected to itself. Indeed, said "representation" uses "an official project e-mail":

    "Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by **contacting the Technical Advisory Board (TAB) at **."

    As per the above statute, harassment is deemed not aligned to the "CoC", neither is any threat:

    "**Maintainers have the right and responsibility to remove, edit, or reject** comments, commits, code, wiki edits, issues, and **other contributions that are not aligned to this Code of Conduct**, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, **threatening, offensive**, or harmful."

    But it so happens that "Maintainers who do not follow or enforce the Code of Conduct in good faith **may face temporary or permanent repercussions** as determined by other members of the project’s leadership" is a blatant threat of harassment. Of course the "CoC" itself is a threat if it is a law at all.

    Hence this "CoC" transgresses this "CoC". If it is not de jure destroyed by itself (the "CoC" compels the "maintainers" to permaban themselves, taking said "CoC" with them*) then it is weak, laxity obtains and nobody need pay attention to it, de facto destroying it, which is the better case, as nobody has hitherto pointed out this obvious loophole. On the other hand, this silence could betray a real blindness or "rigorous" discharge on the repressive apparatus' part (offensive and therefore *not rigorous* in avoiding offence against the CoC), either of which may reflect pretty badly on the Linux kernel's code quality, architecture, *merit*...

    * They say "**Maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions** as determined by other members of the project’s leadership" (CoC) but then, since they don't *cut* their unCoC out (for reasons expounded in a previous post), they neither follow (on its way out) nor enforce it (by kicking said CoC in its unballs), they must, according to their CoC, "face temporary or permanent repercussions".

  23. They installed Windows 10, and the files are gone by Anonymous Coward · · Score: 0

    most likely situation, in my opinion. Another reason to leave GitHub and switch to GitLab.

  24. Linus was blackmailed by Anonymous Coward · · Score: 0, Troll

    Free Software world hero Linus Torvalds was forced to resign from the Linux kernel project by blackmail. He fell for a honeytrap and was threatened with a #MeToo purge if he didn't resign. It's a corporate power grab, using "Social Just-Us" as a tool.

    1. Re:Linus was blackmailed by Anonymous Coward · · Score: 0

      Are we talking about the same Linus Torvalds that is now back in charge of Linux?

  25. We don't need backups... by Anonymous Coward · · Score: 0

    Err!?

  26. Re: THERE WILL BE CONSEQUENCES FOR YOUR LIES KEN D by Highdude702 · · Score: 1

    That's a god damn good idea! Will help all those boring days at work! Thank you kind sir for adding another troll to slashdot for the lulz.

  27. Re:What is there storage back end? and what VM sys by bluefoxlucid · · Score: 1

    The storage back-end is called The Cloud, and it stores The Bits. Fortunately, everything is backed up a million times in Git, so somebody has a copy.

  28. Git working for me, but not Travis on the PRs by physburn · · Score: 1

    Git seem to be working for me. But travis isn't starting on the Pull requests.

  29. Re:What is there storage back end? and what VM sys by saider · · Score: 1

    Except for the github source code, which is in Subversion.

    --


    Remember, You are unique...just like everyone else.
  30. Just bring back Google Code by Gabest · · Score: 1

    It was only running on Subversion, but I it was perfect for a small team. And the revision discussion feature was awesome.