Slashdot Mirror

← Back to Stories (view on slashdot.org)

GitLab Says It Found Lost Data On a Staging Server (theregister.co.uk)

Posted by msmash on from the reviving-again dept.

GitLab.com, the wannabe GitHub alternative that went down hard earlier this week and reported data loss, has said that some data is gone but that its services are now operational again. From a report The Register: The incident did not result in Git repos disappearing. Which may be why the company's PR reps characterised the lost data as "peripheral metadata that was written during a 6-hour window". But in a prose account of the incident, GitLab says "issues, merge requests, users, comments, snippets, etc" were lost. The Register imagines many developers may not be entirely happy with those data types being considered peripheral to their efforts. GitLab's PR flaks added that the incident impacted "less than 1% of our user base." But the firm's incident log says 707 users have lost data. The startup, which has raised over $25 million, added that it lost six hours of data and asserted that the lost doesn't include users' code.

1 of 101 comments (clear)

  1. Bad incident; great response by Wuhao · · Score: 5, Interesting

    Obviously, data loss is embarrassing. I think we all appreciate the importance of not only having multiple backups, but testing to ensure that your backups work, and are sufficient to fully restore operations. GitLab is just the latest in a long tradition of sites and services that have found themselves facing the consequences of not regularly testing their recovery plans.

    But I do respect their response. They quickly recognized what had happened, and they diagnosed what went wrong with their backups. They did not try to use PR-speak to conceal their mistake -- they publicly copped to it, in plain industry-standard language that their users would understand, and even offered a livestream of their team resolving the issue. I think this has been a masterclass in how to recover from a blunder. I bet you that this is not a mistake GitLab will be repeating anytime soon.

    Also, I think it's very fortunate that they're in the git repo business, and presumably users who had data that was affected by the loss still have a copy in their own local repos. Thank god for distributed SCM.