Slashdot Mirror

← Back to Stories (view on slashdot.org)

The 32-Bit Dog Ate 16 Million Kids' CS Homework (code.org)

Posted by EditorDavid on from the blaming-the-Cloud dept.

"Any student progress from 9:19 to 10:33 a.m. on Friday was not saved..." explained the embarrassed CTO of the educational non-profit Code.org, "and unfortunately cannot be recovered." Slashdot reader theodp writes: Code.org CTO Jeremy Stone gave the kids an impromptu lesson on the powers of two with his explanation of why The Cloud ate their homework. "The way we store student coding activity is in a table that until today had a 32-bit index... The database table could only store 4 billion rows of coding activity information [and] we didn't realize we were running up to the limit, and the table got full. We have now made a new student activity table that is storing progress by students. With the new table, we are switching to a 64-bit index which will hold up to 18 quintillion rows of information.
The issue also took the site offline, temporarily making the work of 16 million K-12 students who have used the nonprofit's Code Studio disappear. "On the plus side, this new table will be able to store student coding information for millions of years," explains the site's CTO. But besides Friday's missing saves, "On the down side, until we've moved everything over to the new table, some students' code from before today may temporarily not appear, so please be patient with us as we fix it."

94 of 161 comments (clear)

  1. Well then. by Anonymous Coward · · Score: 5, Funny

    That doesn't inspire a whole lot of trust in the system. Who did they get to code this thing, elementary school kids?!?

    1. Re:Well then. by omnichad · · Score: 1

      Wasn't that long ago that Youtube ran into the same problem for view counts (on Gangnam Style). They were even using signed integers - gotta be prepared for negative views, I guess.

    2. Re:Well then. by 93+Escort+Wagon · · Score: 1

      Who did they get to code this thing, elementary school kids?!?

      Larry Ellison personally set it up.

      ... But he did have to ask his next door neighbor's kid for advice on a few occasions.

      --
      #DeleteChrome
    3. Re: Well then. by Anonymous Coward · · Score: 4, Insightful

      Are you kidding or just have no memory of the past? Sites were incredibly fragile from before. Outages were the norm and you could take down most weak terribly written PHP sites by sneezing at them the wrong way. Maybe you haven't been here long, but we used to have this thing called "Slashdotting" which would take websites down just by being linked to by this webpage. Nothing had any ability to scale and of it did 99â... of the time that hardware was wildly over provisioned.

      This has nothing to do with your "hur Hur millennials" bullshit. 32 bit and 64 bit numbers, and problems with picking the right one had been around since the beginning of time. The person that picked 32 bit instead of 64 bit was more likely to be some grizzly old-timer used to drive and memory space being the main constraint. The evil millennial characatur you hate so much would have made it 128 bit and wasted all that space because in this day and age, why the fuck not?

    4. Re:Well then. by Hognoxious · · Score: 1

      Didn't the post/thread count overflow here about six owners back?

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    5. Re:Well then. by arglebargle_xiv · · Score: 1

      Note that the WTF isn't the use of a 32-bit index, it's HowTF you can code a system that requires 4 billion rows in a database. Announcing they've fixed it by moving to a 64-bit row index is like announcing that you're fixed a problem with plugging a 100A device into a 10A circuit by replacing the fuse that keeps blowing with a piece of 00AWG wire current-welded across the terminals.

    6. Re:Well then. by doccus · · Score: 1

      32 bits outa be enough for anyone... ;-)

  2. Using the cloud is so safe and secure... by QuietLagoon · · Score: 3, Insightful

    At least there was a back-up... Or not... Not even a 24-hour transaction log... Or not... Way to go code.org... set that example...

    1. Re:Using the cloud is so safe and secure... by halivar · · Score: 4, Informative

      How do you back up data that was never stored? Or logs for transactions that never completed? And how, even if you had those transactions, would you meaningfully restore them when the restoration process itself would simply repeat the result of overflowing the available indexes?

      This isn't a typical disaster recovery scenario. The architecture itself is at fault, and the data is lost.

    2. Re:Using the cloud is so safe and secure... by phantomfive · · Score: 1

      Uh, what kind of data are you talking about that was never stored?
      The obvious thing is to restore at the most recent backup. Some data will be lost of course, but that's better than losing all data, which apparently these people did.

      --
      "First they came for the slanderers and i said nothing."
    3. Re:Using the cloud is so safe and secure... by Anonymous Coward · · Score: 2, Informative

      They didn't lose all data. The lost every every insert into a table the occurred after its index reached it's maximum value. As the database insert was the method of storing the data, there's nothing to recover.

    4. Re: Using the cloud is so safe and secure... by Anonymous Coward · · Score: 2

      Shouldn't whatever interface these students were using have told them the save failed?

    5. Re:Using the cloud is so safe and secure... by wonkey_monkey · · Score: 1

      Some data will be lost of course, but that's better than losing all data, which apparently these people did.

      No they didn't.

      Any student progress from 9:19 to 10:33 a.m

      So a grand total of about 74 minutes' work was lost.

      --
      systemd is Roko's Basilisk.
    6. Re:Using the cloud is so safe and secure... by quonset · · Score: 1

      So a grand total of about 74 minutes' work was lost.

      Which, from the way some programmers on here talk, could have been a whole three lines of code.

    7. Re:Using the cloud is so safe and secure... by fisted · · Score: 1

      So a grand total of about 74 minutes' work was lost.

      Times the number of kids working at that time.

      --
      CLI paste? paste.pr0.tips!
    8. Re: Using the cloud is so safe and secure... by Anonymous Coward · · Score: 1

      The prudent student stores the files locally, copies and pastes them into the browser then hits "send".

      I would love to be able to say I had not personally learned that lesson the hard way.

    9. Re:Using the cloud is so safe and secure... by phantomfive · · Score: 1

      ok, so apparently not all data was lost, I misread the info. Bad on me, I should have read more carefully, and my other comment should be modded down for being wrong.

      --
      "First they came for the slanderers and i said nothing."
    10. Re:Using the cloud is so safe and secure... by wonkey_monkey · · Score: 1

      Doesn't really matter. The loss could be more or less undone in 74 minutes. Doesn't matter whether it's ten kids or a billion kids.

      --
      systemd is Roko's Basilisk.
    11. Re: Using the cloud is so safe and secure... by tepples · · Score: 1

      The prudent student stores the files locally

      Provided said prudent student owns a device with a text editor capable of storing files locally. Does an old hand-me-down iPad with a Bluetooth keyboard count?

    12. Re:Using the cloud is so safe and secure... by ShanghaiBill · · Score: 1

      Or logs for transactions that never completed?

      You log the transaction just before you execute it. The point of the log is that you can re-roll it to reconstruct the database in the event of failure. That doesn't work if you only log things that didn't fail.

    13. Re:Using the cloud is so safe and secure... by rtb61 · · Score: 1

      Technically speaking a whole lot less than 74 minutes if they were coding properly and taking notes and keeping track of their work. So maybe 20 minutes and for some particular skilled coders maybe 10, depending how long it took them to figure out the original code structure.

      --
      Chaos - everything, everywhere, everywhen
    14. Re:Using the cloud is so safe and secure... by QuietLagoon · · Score: 1

      How do you back up data that was never stored? Or logs for transactions that never completed?...

      There's an input-logging capability that can be replayed in times like this. Since it is separate from the part that caused this problem, the data would be retrievable. Simple.

    15. Re:Using the cloud is so safe and secure... by gumbi+west · · Score: 1

      right, so there is a second bug that the interface didn't admit to the inability to save work.

    16. Re: Using the cloud is so safe and secure... by Dog-Cow · · Score: 1

      How many kids use a hand-me-down iPad and BT keyboard to submit work to code.org? And yes, the iPad has an app pre-installed which can save text files locally.

      You probably deserve to have the iPad shoved up your anus until it rips through your colon.

    17. Re:Using the cloud is so safe and secure... by strikethree · · Score: 1

      You are more correct than you realize about the architecture issue.

      The architecture should have been taking everything, dumping it to a "log" to be processed, and then processed. Once the processing is done, the "logs" could be deleted. If they had done this, recovery would have been easy.

      I mean really, "you" expect data to always be processed perfectly? Not a chance. There is always room for hiccups. Store the raw data until you can process it. Simple. Obvious. And clearly not done in this instance.

      Please note that the "you" in this sentence is not halivar but rather anyone writing an application to store data.

      --
      "Someone needs to talk to the tree of liberty about its ghoulish drinking problem." by ohnocitizen
    18. Re:Using the cloud is so safe and secure... by Holi · · Score: 1

      Why did the device report a successful save? I mean, if they need to tell you your files weren't saved then what kind of caching operation do they have that allows for over an hour of saves failing and no notification sent to the user?

      --
      Sorry, teleporters just kill you and then make a copy. A perfect, soul-less copy.
    19. Re:Using the cloud is so safe and secure... by halivar · · Score: 1

      It's probably something as simple and stupid as catchless try (which I have seen on DB ops more times than I care to think about) in misguided attempt at "graceful degradation."

  3. And a valuable lesson learned: by aix+tom · · Score: 5, Insightful

    Don't trust the cloud as the only place you store your work.

    1. Re:And a valuable lesson learned: by Tablizer · · Score: 2

      Don't trust the cloud as the only place you store your work.

      A generalized version is don't trust any one system. Put copies on different servers/devices.

      Of course there's a break-even point where the labor to manage backups exceeds that lost on average to failures.

      --
      Table-ized A.I.
    2. Re:And a valuable lesson learned: by Anonymous Coward · · Score: 1

      Don't trust the cloud

      Full stop.

    3. Re: And a valuable lesson learned: by im_thatoneguy · · Score: 1

      Don't trust

      Fixed that for you.

    4. Re: And a valuable lesson learned: by Dog-Cow · · Score: 1

      Don't

  4. Don't look at it that way... by mmell · · Score: 5, Insightful

    Consider this a real-world lesson for our youth in the ways that design choices can have unanticipated effects on implementation, manageability and viability of software in the long haul. For extra credit, the kids that are affected should be encouraged to explore what they could have done to mitigate the risk caused by some grown-up's oversight.

    1. Re:Don't look at it that way... by Anonymous Coward · · Score: 2

      The people that originally made the decision to go with a 32 bit table are probably long gone. The real lesson here is don't waste time worrying about things you won't be around to have to deal with.

    2. Re:Don't look at it that way... by Anonymous Coward · · Score: 1

      Gladiators? Lions and tigers?

    3. Re:Don't look at it that way... by Dog-Cow · · Score: 3, Insightful

      Y2K wasn't a "bug". It was a reasonable design decision made when storage (both RAM and long-term) was expensive and scarce. Computer systems were new, and no one had any idea how long programs would be running.

      On the flip-side, 64 bit ints have been cheap for ages now. Code.org programmers were just lazy fucks.

    4. Re:Don't look at it that way... by TheRaven64 · · Score: 2

      For a lot of things, a 64-bit id is overkill, but if you do use a smaller int, then always set something up to notify you when any of the top few bits is set so that you have a nice long time to migrate the data.

      --
      I am TheRaven on Soylent News
    5. Re: Don't look at it that way... by reanjr · · Score: 3, Interesting

      For a lot of things, 32 bit is overkill, but you don't see people storing 24 bit numbers. This is a fundamental problem with premature optimization. You should always use the largest precise integer available unless you have a compelling, evidence-based reason not to. The onus should be on the 32bit users to demonstrate their choice is better.

    6. Re: Don't look at it that way... by doccus · · Score: 1

      Yup, but I recall people saying a few years ago the same thing as in... "realistically, most people will never use more than 4 gigs of RAM".. Never thought I would have to switch to a 64 bit OS for years yet.

    7. Re: Don't look at it that way... by TheRaven64 · · Score: 1

      If you're creating a database table, 64 bits doubles the size of the index column. If you're creating a database table in normal form, then you'll have a number of tables that are simply pairs of two indexes, so you will double the size of entire tables. That's going to have big cache and memory overheads and is definitely not a simple 'let's always do that' choice.

      --
      I am TheRaven on Soylent News
    8. Re: Don't look at it that way... by Bengie · · Score: 1

      Unless your table only has an index, doubling the size of the index is quite moot. IF you knew anything about data alignment, SQL data page layouts, or the slew of meta-data stored along with the data, you would realize it does not matter in nearly every case. It will matter if you're programming in a compile language that allows you to control structure layouts and you need to do a lot of CPU intensive work or need to work with SIMD, but outside of that, 64bit.

    9. Re: Don't look at it that way... by TheRaven64 · · Score: 1

      Any relational database in any normal form will include a lot of tables that just contain a pair of indexes. If your databases are not in a normal form, then your maintenance headaches are going to be far bigger than the size of index fields.

      --
      I am TheRaven on Soylent News
    10. Re:Don't look at it that way... by RockDoctor · · Score: 1

      Those of us who worked the overnight shift for the Millennium Bug remember it well. Shouldn't we be coming up to another rollover point some time soon? Unix Epoch hits 2^32 seconds, or something?

      --
      Birds are not dinosaur descendants;birds are dinosaurs, for all useful meanings of "birds", "are" and "dinosaurs"
    11. Re: Don't look at it that way... by Bengie · · Score: 1

      That only applies to many-to-many relations and overly-normalized is nearly as bad as under. Not including that if you have 2 tables with a 32bit int vs one table with a 64bit it, it's a wash. Even in these situations, if you're doing seeks of large many-to-many tables, the random access out-weighs the cost of the extra 4 bytes per field.

      I recently was doing performance testing of a tight loop and was comparing 32bit vs 64bit on a 32bit CPU, and the performance was within 10% of each-other. Even when having to do two register loads and combining the registers to emulate 64bit arithmetic, the CPU was able to OoO+pipeline most of the cost. In the end, I have had more situations where using 32bit ints was slower than 64bit.

  5. We will never learn by Xarin · · Score: 5, Funny

    4 billion rows of coding activity is all we will ever need

    1. Re:We will never learn by newcastlejon · · Score: 4, Insightful

      What sort of DBMS are they using that doesn't notify the admin when a table is nearly full? What sort of client are they using that doesn't tell the user when an attempt to write to a DB fails?

      --
      If God forks the Universe every time you roll a die, he'd better have a damned good memory.
    2. Re:We will never learn by dwywit · · Score: 1

      Maybe that's how they found out. Tech support tickets start flowing in a bit after 9:30 - "I can't insert my changes." They finally suspend activity and investigate after a few dozen tickets all show the same symptoms.

      There's no excuse for not notifying an Admin that a table is about to reach limit.

      --
      They sentenced me to twenty years of boredom
    3. Re:We will never learn by Mashiki · · Score: 2

      If you can't make it fit onto a 8-bit eeprom chip you're doing it wrong?

      --
      Om, nomnomnom...
    4. Re:We will never learn by __aaclcg7560 · · Score: 1

      Funny you should mentioned that. The college instructor for the Introduction to CIS class told us in the early 1990's that 4GB (32-bit) was all the memory anyone would ever need for a PC. For the most part, he was right. When I upgraded my PCs last year, I finally broke through the 4GB barrier. Not because 4GB wasn't enough. I had to get new memory modules and 8GB (two 4GB sticks) was on sale.

    5. Re:We will never learn by freeze128 · · Score: 1

      What sort of client are they using that doesn't tell the user when an attempt to write to a DB fails?

      The kind that is written by code.org developers.

    6. Re:We will never learn by AmiMoJo · · Score: 1

      Didn't Slashdot have a problem with 32 bit post IDs one time? I can't find the story now, but I could swear I remember a temporary fall back to static pages when the 32 bit counter overflowed many years ago.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  6. Platform skill matches student skill by gweihir · · Score: 1, Insightful

    It is no surprise to me that the ones creating and operating this platform are just as incompetent as the "graduates" they produce. Mediocrity breeds mediocrity...

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    1. Re:Platform skill matches student skill by gweihir · · Score: 1

      Indeed.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  7. the people dont care. by nimbius · · Score: 4, Insightful

    "The way we store student coding activity is in a table that until today had a 32-bit index... The database table could only store 4 billion rows of coding activity information

    if it can only store four billion rows, it isnt "the cloud." its just a KVM instance running on a shared hosting facility then, isnt it.

    we didn't realize we were running up to the limit, and the table got full.

    so not only were you incapable of scaling your infrastructure or your program to handle four billion rows --something every sysadmin on the planet is capable of-- you weren't even competent enough to set up monitoring for it.

    We have now made a new student activity table that is storing progress by students.

    the ones that lost all their data dont care. the students will leave to try something else, the educators will fall back on lesson plans that werent written by a corporate think tank, and your 'hour of code' will remain just another hour of minecraft in a kids life.

    With the new table, we are switching to a 64-bit index which will hold up to 18 quintillion rows of information.

    you dont get it. no one fucking cares about your SQL table limits but you, and youre oblivious to the fact that a table with eighteen quintillion rows would never load. code.org will be no different than the spanish or french class in a kids life. a fractional percentage of them will actually go on to use it as a career.

    --
    Good people go to bed earlier.
    1. Re: the people dont care. by Dracos · · Score: 4, Funny

      09:19 to 10:33 is 74 minutes, not 20. Did you learn arithmetic on code.org?

    2. Re:the people dont care. by wonkey_monkey · · Score: 1

      and youre oblivious to the fact that a table with eighteen quintillion rows would never load

      I think you're oblivious to the fact that no one person is going to fetch every single row. You do know how a database works, don't you?

      so not only were you incapable of scaling your infrastructure or your program to handle four billion rows --something every sysadmin on the planet is capable of-- you weren't even competent enough to set up monitoring for it.

      They were capable of the first part, and are now doing it. They just didn't realise it needed to be done.

      the ones that lost all their data dont care.

      No-one lost all their data. At most they lost 74 minutes' work.

      Read the whole thing next time, eh?

      --
      systemd is Roko's Basilisk.
    3. Re:the people dont care. by thegarbz · · Score: 1

      you dont get it. no one fucking cares about your SQL table limits

      Okay okay, you're over stimulated. Have a beer and go back to bed.

    4. Re:the people dont care. by Dog-Cow · · Score: 1

      if it can only store four billion rows, it isnt "the cloud." its just a KVM instance running on a shared hosting facility then, isnt it.

      There is no relationship between the index's datatype size and the kind of system the RDMS is hosted on. Also, a KVM instance running in a shared hosting facility (on? is it on the roof of the building?) is running in the cloud. That's what the cloud is... shared (virtual) servers, optionally maintained by someone else.

      so not only were you incapable of scaling your infrastructure or your program to handle four billion rows --something every sysadmin on the planet is capable of-- you weren't even competent enough to set up monitoring for it.

      Sysadmins are not responsible for database schema design or implementation. The issue was not a matter of scaling.

      You have demonstrated that you are more of a fucked-up shit than the code.org developer who decided to use a 32-bit index. How does it feel to be worse than the pile of shit the story is about?

  8. More important lesson by saboosh · · Score: 5, Insightful

    Thank you for teaching the kids the importance of taking responsibility and being honest and open about your mistakes. It's okay to make mistakes as long as we learn from them. Too many people today are afraid of making mistakes and cover them up.

    1. Re:More important lesson by saboosh · · Score: 5, Insightful

      I find people like anecdotes here so please allow me to add: I was raised by very "tough" parents with a very "tough" form of discipline. Mistakes meant punishment. Today I have a 9 year old daughter who, like any other human being, makes mistakes. A few years ago I noticed a very strange phenomena with regards to "dealing" with her mistakes". When I would get upset with her and punish her for spilling on the couch or forgetting to clean her room I would see her make it again and as time went on she would get, either, more defensive about it or try to lie about it. At some point my fiancee asked that I try a different approach: Try being kind and loving with my response and take time with her to show empathy, to share that Im not perfect either and to figure out another way of handling whatever the mistake was.... the taking-time part is probably the toughest for me because it means work, im sure many can relate.... but, strangely, I noticed that she was making the mistakes I handled the new way a lot less... and she seemed to be ok with handling them a new way. She started to clean her room on her own and even though her coordination did not allow her to stop from spilling she was more careful about where she took her drinks and cleaned them up more quickly.... its really ass backwards to me... and to top it off, she seems less anxious around me and my responses and seems less defensive... Im not a psychologist and wont pretend to understand the how or why of it, I just know she seems less distracted and anxious and I seem to get more hugs from here and I will take that over trying to "force" her to learn anyday.

    2. Re:More important lesson by Morpeth · · Score: 1

      Sorry don't have any mods points to give you, but I would.

      Code.org does a lot of good, they had an issues, they talked about it openly and addressed it -- not sure why some people are acting like code.org started murdering puppies or something.

      --

      'The unexamined life is not worth living' - Socrates
    3. Re:More important lesson by Dutch+Gun · · Score: 4, Insightful

      Caveat: It's okay to make mistakes as long as no one was hurt or killed by easily preventable errors. Obviously, that doesn't apply here, so I definitely agree. Sharing your experience and turning it into a teachable moment ensures others learn from it as well.

      It would have been less embarrassing for them to just make up some excuse about a temporary outage, or blame a DDOS attack, or Russian hackers. It's good to remember that when lambasting them about what idiots they are for not noticing this before their DB puked on them. It's tempting to do, but really does nothing but stroke your own ego while at the same time encouraging people to try to hide their mistakes to avoid this sort of public shaming.

      So, yeah, kudos for them for owning up to their own mistake.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    4. Re:More important lesson by Anonymous Coward · · Score: 1

      It's not ass backwards, (to me at least) it's obvious. Humans are very social animals, they live and work in groups with sophisticated and complex social interactions. Actively wanting to be useful and helpful is a natural part of that as much as selfishness is. Punisment als the only way to respond to mistakes assumes that avoiding punishment is the only thing that motivates, and overlooks the fact that cooperation in itself is a strong motivator for social beings, which is what you can see with your daughter now. Punishment is a negative experience. The reason it stimulates lying is that it's more likely to get associated not with the mistake itself but with the response of the parent to the mistake. Think about it, the link to the mistake is indirect, it has to be made via the parent's response, the link to the parent's response is direct and therefore stronger. Much stronger. Learning to lie is a perfectly natural way to avoid that response, and not what you as a parent intend to teach your kid.

    5. Re:More important lesson by AmiMoJo · · Score: 1

      I noticed something similar with the way Japanese parents tend to talk to their kids. Not all of them, but it seems to be the more normal way of doing things over there.

      By treating them more like an adult, not getting angry and shouty but instead helping them to understand why soiling the sofa is a problem, involving them in correcting the error (cleaning up) and seeing mistakes as something to learn from and aid in personal improvement their kids seem to be a lot more responsible and calmer.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    6. Re:More important lesson by yithar7153 · · Score: 1

      I had parents like this and I would do the same thing. I would just get better at hiding my mistakes rather than taking the time to improve, because no matter what I did, it wouldn't be enough.

  9. Wasn't any Code.org dev around for Slashdot's fail by El+Cubano · · Score: 4, Interesting

    Seriously, was not a single developer or architect from Code.org around when Slashdot overflowed its 24-bit index? I know it has been a few years now, but I'm sure there are folks here who remember threading breaking and all other sorts of problems when it happened. Remember: https://slashdot.org/story/06/11/09/1534204/slashdot-posting-bug-infuriates-haggard-admins

    Granted, that was Slashdot, and while annoying, it was hardly the end of the world This problem with Code.org clearly reinforces "cloud bad" to people who are already fearful of putting their data in the cloud.

    I am guessing that Code.org didn't bother tracking things like how to close to various limits they were getting, but I bet that they are now. In any event, when this happened to Slashdot 10+ years ago, I suppose you could argue that we weren't as advanced. In 2016-2017 there is no excuse for such a critical architectural flaw. To me, it completely undermines my confidence in their entire platform. What other time bombs are ticking under the surface there?

  10. Well duh by cyber-vandal · · Score: 5, Funny

    It's code.org not databasedesign.org

  11. table partitioning? by oneiros27 · · Score: 1

    I admit, I've mostly done it for speed purposes, but my understanding is that the record limit is per partition, so you could also use it to deal with record limits.

    They could either partition based on user IDs (might be faster to select by for the bulk of the queries), or by date (making it easier to manage autonumber fields).

    --
    Build it, and they will come^Hplain.
  12. 64bit by ledow · · Score: 4, Informative

    Honestly don't get why everything these days isn't just 64-bit by default.

    You can hit 32-bit limits just buying a memory chip, or bog-standard storage. 4 billion is not a big number in those terms.

    32-bit times are dead.
    32-bit filesizes are dead.
    32-bit memory sizes are dead.
    32-bit file counters are dead.
    Hell, it's not inconceivable that in some things 32-bit user counters could die - with account recreation and spam accounts, surely the big people are having to deal with that.

    Just stop faffing about and use 64-bit for everything, by default, from the start. 8 bytes isn't a huge amount of overhead nowadays.

    But starting with the assumption "4 billion is enough" when some people have more than 4bn in their bank account, some services have more than 4bn users, and people can buy 4bn-whatevers in their local electronics store is stupid.

    But 4 billions lots of 4 billion is not a limit that you will hit for a very, very, very long time. Even 128-bit isn't unseen - IPv6, ZFS, GPUs - and that's 4 billion lots of 4 billion 64-bit numbers each of which is capable of holding 4 billion lots of 4 billion.

    Supercomputer architectures did this a long time ago, translating and assuming everything is 128-bit so that you never have to worry about a limit.

    Why does it take so long for basics like web servers and databases to get there? 64-bit by default, MINIMUM. Anything that incurs a performance hit on that is old, and up to the user to resolve.

    1. Re:64bit by thegarbz · · Score: 2, Insightful

      But starting with the assumption "4 billion is enough" when some people have more than 4bn in their bank account

      Yep, I should bog down my computer processes because someone else is rich. Incidentally how many bits does it take to represent the number 4bn? While we're at it do you realise that the number of planets that humans have colonised is 1? Let's build a database with a 25 year life expectancy, how many bits would you assign to the index? 64bits? Your approach is the reason computers are frigging slow. It's the reason why I wait for ages to open up Chrome on a Quad 1.4Ghz Snapdragon.

      How about instead of just blindly wasting resources you actually learn about statements of requirements and project scopes.

      Supercomputer architectures did this a long time ago, translating and assuming everything is 128-bit so that you never have to worry about a limit.

      Didn't you just say we should use 64bit for everything by default?

      Why does it take so long for basics like web servers and databases to get there? 64-bit by default, MINIMUM.

      No thanks. I'll target 8bit minimum and scale up as needed.

    2. Re:64bit by Anonymous Coward · · Score: 1

      I work for a massive entity. We bought thousands of computers with 8GB of RAM. We then installed 32-bit Windows 7 on them, effectively rendering more than half of that RAM useless. Why? Because 64-bit Windows 7 broke ONE *WEB* in-house designed, written, and maintained application we use, and we allegedly couldn't afford updating it. People do dumb shit.

    3. Re:64bit by tepples · · Score: 1

      Why does it take so long for basics like web servers and databases to get there?

      Because the PHP language on 32-bit architectures doesn't support 64-bit integers. All you get are 32-bit actual integers and the 52-bit type you get by (ab)using a double-precision floating point value as an integer.

  13. Oh the irony by luis_a_espinal · · Score: 3, Insightful

    Code.org CTO Jeremy Stone gave the kids an impromptu lesson on the powers of two with his explanation of why The Cloud ate their homework. "The way we store student coding activity is in a table that until today had a 32-bit index... The database table could only store 4 billion rows of coding activity information [and] we didn't realize we were running up to the limit, and the table got full. We have now made a new student activity table that is storing progress by students. With the new table, we are switching to a 64-bit index which will hold up to 18 quintillion rows of information.

    The of seeing a programming education site using 32-bit indexes without any form of index space monitoring is both hilarious and surreal.

    Who the hell runs a cloud-based, massively accessible operation with 32-bit indexes? And who the hell runs a production system without database monitoring?

  14. Deja vu by jdavidb · · Score: 3, Funny

    I remember when Slashdot had this exact same problem with comment ids!

    --
    Secession is the right of all sentient beings.
  15. Thats what they get by Anonymous Coward · · Score: 1

    For trusting the "cloud".

    1. Re:Thats what they get by 93+Escort+Wagon · · Score: 1

      For trusting the "cloud".

      Yeah, a local database table using a 32-bit index would've handled the situation just fine...

      --
      #DeleteChrome
  16. Even programmers should know better though by SuperKendall · · Score: 1

    I would say having a 32 bit number as some kind of ID for activity is not even a database design issue, it's almost a pure programing issue. Any programmer should know better than to keep a unique ID in some kind of 32 bit value... heck the "fix" to move to a 64-bit value is better but not as good as using a for-real UUID which is really more of a standard (and even larger than the 64-bit value), and also something any programmer should know about.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
    1. Re:Even programmers should know better though by Daetrin · · Score: 1

      Using UUIDs as indexes in a DB comes with a whole different set of problems and such a decision definitely shouldn't be made on the basis of "well at least this will prevent us from ever overflowing the ID." I'm not a real database expert myself (though i'm what passes for one in the context of my (rather small) company) but even i shudder at the thought of having to create or maintain a DB centered around the concept of "let's use UUIDs for indexes!"

      Here's a post outlining the general reasons why it can be a bad idea:
      https://rclayton.silvrback.com...

      --
      This Space Intentionally Left Blank
    2. Re:Even programmers should know better though by Bengie · · Score: 1

      Indexing UUIDs on a large table is a management nightmare. Take about page fragmentation. Joining two large GUID based tables can cause horrible performance corner cases even when clustered. UUIDs should be limited to public data/APIs, but not actually used internally. I do love them for keeping my data sane, but the performance optimizations and management overhead means you need to really think about the issue before you index them. NEVER cluster index them.

    3. Re:Even programmers should know better though by Bengie · · Score: 1

      In before anyone points out about a "work around" for UUID clustered index fragmentation. And don't use sequential UUIDs, I will hate you. They are how you get collisions across systems or even in the same system if two tables have the same seed.

  17. No Dog by Princeofcups · · Score: 1

    According to TFS, nothing was lost. They just can't access their stuff until it's moved over to the new database. No disaster. No lesson. No dog. Just off line for a few days.

    BFD

    --
    The only thing worse than a Democrat is a Republican.
    1. Re:No Dog by JohnFen · · Score: 1

      According to TFS, nothing was lost.

      Well, except for any data generated from 9:19 to 10:33 a.m

  18. Example #4,294,967,296 by JohnFen · · Score: 1

    Why to avoid trusting cloud services with any data that you can't afford to lose.

  19. Periodic testing and data reviews anyone? by gb7djk · · Score: 2

    Perhaps this will kick someone into looking at the database, as a whole, on a periodic basis to check other limits. Maybe do the odd test transaction or spot trends in other tables which are unexpected? Maybe run some regression tests? Then use this information to tweak the data model in controlled fashion before it breaks.

    You know, like grown ups do...

  20. Re:fucking idiots by tepples · · Score: 2

    Let me guess: Your C89 compiler defined an int as 16-bit and a long as 32-bit, and it provided no 64-bit type. The C type "long long" wasn't part of the standard until 1999.

  21. Re:Wasn't any Code.org dev around for Slashdot's f by Gravis+Zero · · Score: 1

    Code.org correctly reinforces "cloud bad" to people who should be fearful of putting their data in the cloud.

    FTFY.

    --
    Anons need not reply. Questions end with a question mark.
  22. Just the other day - posts about not needing math by dbIII · · Score: 2

    Not long ago there were some posts here about programmers not needing to know any mathematics.
    It didn't take very long for an article to appear that showed the consequences of not cracking open some books.

    Who would have thought - Knuth seems to have a bit more of a point than the guy who taught himself PHP.

  23. This bug by MichaelSmith · · Score: 1

    Glad to see it would never happen to slashdot.

    --
    http://michaelsmith.id.au
  24. Love those numbers! by hyades1 · · Score: 1

    With the new table, we are switching to a 64-bit index which will hold up to 18 quintillion rows of information.

    Is that bigger than a bajillion?

    --
    I've calculated my velocity with such exquisite precision that I have no idea where I am.
  25. Won't work forever by allo · · Score: 1

    in a few million years, the table will be full again. And then nobody expected it, again.

  26. Re:Wasn't any Code.org dev around for Slashdot's f by rhazz · · Score: 1

    To me, it completely undermines my confidence in their entire platform

    So do you avoid all companies who have ever had a free product down one time for at least 74 minutes and was completely open and honest about it?

  27. Why a singular index? by phorm · · Score: 2

    A singular index seems like a weird thing to have in this case anyways. Wouldn't it be better to have a multi-column index on something like userid+item rather than an index of all items?

  28. Don't use integers for indexes by cwsumner · · Score: 1

    Contrary to popular belief, don't use integers for primary indexes. Multi column "natural" indexes can handle way more rows.

    Just because the old databases used "record numbers", doesn't mean you have to... ;-)

  29. Thinking "outside the box" by woboyle · · Score: 1

    This the sort of thing that happens when engineers (especially software engineers) don't think outside of the box and considering the consequences of the code they write.

    --
    Sometimes, real fast is almost as good as real-time.