At this point all my whiteboards are full of boxes of each database cluster, the machines in that cluster, which have passed their checksum tests. (innodb checksums each 16k page), which replayed their replay/undo logs, where in binlogs each was writing/reading/executing etc...
So lots of waiting now on the checksum validators. I don't want to put a machine back in and find out in a week there was a database page that was corrupt because the battery-backed write-back cache on the RAID card didn't work as advertised. (which happens on about 95% of RAID cards, in my experience, because they're mostly crap, even the most expensive ones...)
Also whenever there's any doubt about something's integrity, we backup or snapshot the potentially corrupt version before operating on it. That operation can take time too.
But we intentionally don't have databases come back up on boot because if there was a blip, we want to do an integrity check first. (we run InnoDB, so it's ACID, but we're paranoid...)
We have clusters of 2 identical databases in separate cabinets, separate switches, separate Internap power feeds... so normally losing one database in each cluster doesn't matter: the other one gets used. But when we lose every single database, in all clusters, all at once... that's the time to be paranoid and double check stuff.
The reporter who talked to me obviously wanted a fun slant for her article: "Look at all this insecure crap out there!"
Things we talked about that she decided to ignore in her article:
-- we've been working on challenge/response logins in JavaScript so passwords don't go in the clear. it's like Digest auth but in JS instead. We had this working when we talked to her, and since then it's gone into final user testing on our public test site. it'll probably go live this weekend. (I remember when I talked to her I compared it to HTTP Digest Auth and I had to explain what Digest auth was to her..... this is a _security_ reporter?)
-- we never said SSL wasn't important or security wasn't a priority. we told her it HAS BEEN a priority, but performance stuff keeps getting in the way. in fact, we have SSL stuff working and it's going live at the same time as the challenge/response logins. we just told her that it's hard to do right when you have a shitload of servers.
-- we let users bind their login session to their IP, so damage from cookie theft over non-SSL is mitigated
-- we don't let users do any major action (like, oh, change the account's password) without the original password.
-- we have no many anti-hijacking measures in place to let owners of accounts restore their stolen accounts. and you know what? it's not because of SSL... it's because of people just people plain dumb/trusting/gullible. SSL isn't a magic security wand.
Anyway, please recognize an article on a security site wants a "security's terrible!" slant. Who wants to read an article saying, "Yup, security's pretty good and improving." The security situation isn't as grim as it's made out to be.
Just think of a dumbass name that no real company would try and use.... Ximian, LoserJabber, Pygmy, Knoqueror, hell -- anything with a K, G, Gnu, or GTK at the beginning... like gtktalog. Or anything with an annoying mix of capital and lower-case letters: SQmaiL.
I mean, seriously... if you're writing open-source software and don't have a dumb/geeky/clever name for it, how good can it be, ya know?:-)
Hah --- you joke, but it's almost true.
In my operating systems class at University of Washington (in Seattle), we looked at parts of NT code, and our professor (an ex- microsoft kernel programmer) was trying to arrange for the next quarter to have access to the full Windows 2000 source code. I'm not sure if that ever happened, though.
Maintain one light version of the site for Avantago, Lynx, WebTV, whatever, and maintain your full-screen full version. Ideally you're using a templating system that lets both be auto-generated from the same content source.
Then, let users decide which they access. If they want to torture themselves using the full-screen version on their little device, it's their choice.
The challenge for making an embedded web browser isn't in porting the code, it's in laying out the page such that you don't have to scroll so far in both directions.
WebTV and Pocket Explorer for Windows CE do a pretty nice job at this... scaling images and everything so that normal webpages look decent, without having to do a special site design specifically for them.
From the screenshots, the embedded Koqueror didn't appear to do any of this. It looks like you have to scroll quite a bit.
I've already started to see some comments to the effect of, "That's great that they run a site, but they should monitor the content somehow..."
We _do_ monitor the content.
There are many mechanisms running behind the scenes that screen the content based on sets of "bag" regexes and then flag booths. At any time, booths can be in the "Probably okay", "Probably bad", "Verified Good", or "Verified Bad" state, along with a date that the booth was set to that state last. There are then jobs working all the time scanning booths more and adjusting the states of booths that have changed since their last update.
FreeVote volunteers and employees then manually verify booths that are in the "probably bad" state and place them in either "Verified Bad" or "Verified Good".
In addition, visitors to the booths can rank the quality/content of the booth, and that raises more flags we look at.
There is a ton of moderation being done on the site, both automatic and by hand. More code has been written for our admin area than any other part of the site.
The real problem is the combination of:
a) people's immaturity
b) people's intolerance
I'm not sure either one is solvable.
My issue with Canada is that they're extremely intolerant. A bad booth will go up and immediately they start threatening lawsuits and calling my advertisers complaining, even if we shut it down within a day or so of its creation.
I don't start websites to make money --- I do them all for fun. I really hate having to deal with this crap because IANAL, I don't want to be a learn, don't want to pretend to be a lawyer, and just hate dealing with this stuff.
Slashdot Mirror
At this point all my whiteboards are full of boxes of each database cluster, the machines in that cluster, which have passed their checksum tests. (innodb checksums each 16k page), which replayed their replay/undo logs, where in binlogs each was writing/reading/executing etc...
So lots of waiting now on the checksum validators. I don't want to put a machine back in and find out in a week there was a database page that was corrupt because the battery-backed write-back cache on the RAID card didn't work as advertised. (which happens on about 95% of RAID cards, in my experience, because they're mostly crap, even the most expensive ones...)
Also whenever there's any doubt about something's integrity, we backup or snapshot the potentially corrupt version before operating on it. That operation can take time too.
It's going to be a fun night.
They all came back up when the power came back.
...)
But we intentionally don't have databases come back up on boot because if there was a blip, we want to do an integrity check first. (we run InnoDB, so it's ACID, but we're paranoid
We have clusters of 2 identical databases in separate cabinets, separate switches, separate Internap power feeds... so normally losing one database in each cluster doesn't matter: the other one gets used. But when we lose every single database, in all clusters, all at once... that's the time to be paranoid and double check stuff.
IIRC, 256MB of Flash. Then a smaller amount of another type of NVRAM.
I'm Brad Fitzpatrick, from LiveJournal.
The reporter who talked to me obviously wanted a fun slant for her article: "Look at all this insecure crap out there!"
Things we talked about that she decided to ignore in her article:
-- we've been working on challenge/response logins in JavaScript so passwords don't go in the clear. it's like Digest auth but in JS instead. We had this working when we talked to her, and since then it's gone into final user testing on our public test site. it'll probably go live this weekend. (I remember when I talked to her I compared it to HTTP Digest Auth and I had to explain what Digest auth was to her..... this is a _security_ reporter?)
-- we never said SSL wasn't important or security wasn't a priority. we told her it HAS BEEN a priority, but performance stuff keeps getting in the way. in fact, we have SSL stuff working and it's going live at the same time as the challenge/response logins. we just told her that it's hard to do right when you have a shitload of servers.
-- we let users bind their login session to their IP, so damage from cookie theft over non-SSL is mitigated
-- we don't let users do any major action (like, oh, change the account's password) without the original password.
-- we have no many anti-hijacking measures in place to let owners of accounts restore their stolen accounts. and you know what? it's not because of SSL... it's because of people just people plain dumb/trusting/gullible. SSL isn't a magic security wand.
Anyway, please recognize an article on a security site wants a "security's terrible!" slant. Who wants to read an article saying, "Yup, security's pretty good and improving." The security situation isn't as grim as it's made out to be.
Guess I get into the habit of typing RAID. That's like how I can't type the word "serve" without typing "server" first and then deleting the "r".
The indy machine looks neat... it's all black and locked up with a sticker on the front that says, "Resist Corporate Greed".
Sorry, this is hardly relevant.
I'd expect you to be educated enough to know how to spell "their", but you don't.
I was just about the post the same thing myself. If I had moderation points right now, you'd get them.
I can't stand how bad the grammar is on Slashdot.
http://www.privacyfoundation.org/education/webbug. html
I mean, seriously... if you're writing open-source software and don't have a dumb/geeky/clever name for it, how good can it be, ya know? :-)
hahah .... just sometimes? :-)
Hell, I can hardly think off a large db application where I haven't used LEFT JOIN.
The row size limit is what really turned me away from Postgres originally.
They both suck in a lot of areas.... don't bash one without bashing the other.
I just had to point out how ridiculous it is that the Linux Operating Systems course website was created in Microsoft Frontpage.
I hate our CS department. So Microsoft-ish. Bleh.
*sigh*
Our class worked on modifiying NACHOS.
The current class is working on modifying Linux.
Maintain one light version of the site for Avantago, Lynx, WebTV, whatever, and maintain your full-screen full version. Ideally you're using a templating system that lets both be auto-generated from the same content source.
Then, let users decide which they access. If they want to torture themselves using the full-screen version on their little device, it's their choice.
WebTV and Pocket Explorer for Windows CE do a pretty nice job at this ... scaling images and everything so that normal webpages look decent, without having to do a special site design specifically for them.
From the screenshots, the embedded Koqueror didn't appear to do any of this. It looks like you have to scroll quite a bit.
*shrug*
Perhaps your first course should be in the use of wget . :-)
Somehow I think it takes more than that to be a Unix sysadmin. :-)
This is another reasons that people need to have the source to their programs and build it themselves, instead of blindly binary packages.
I can't even remember how much time I spent typing in games for my Apple II ......
Dimensional Media
22 W. 19th St., 2nd Fl.
New York, NY 10011 USA
http://www.3dmedia.com/ phone: 212-620-4100
e-mail: info@3dmedia.com
At least "aero" is intuitive ... the rest are pretty damn stupid.
.pro ??
I've already started to see some comments to the effect of, "That's great that they run a site, but they should monitor the content somehow..."
We _do_ monitor the content.
There are many mechanisms running behind the scenes that screen the content based on sets of "bag" regexes and then flag booths. At any time, booths can be in the "Probably okay", "Probably bad", "Verified Good", or "Verified Bad" state, along with a date that the booth was set to that state last. There are then jobs working all the time scanning booths more and adjusting the states of booths that have changed since their last update.
FreeVote volunteers and employees then manually verify booths that are in the "probably bad" state and place them in either "Verified Bad" or "Verified Good".
In addition, visitors to the booths can rank the quality/content of the booth, and that raises more flags we look at.
There is a ton of moderation being done on the site, both automatic and by hand. More code has been written for our admin area than any other part of the site.
The real problem is the combination of:
a) people's immaturity
b) people's intolerance
I'm not sure either one is solvable.
My issue with Canada is that they're extremely intolerant. A bad booth will go up and immediately they start threatening lawsuits and calling my advertisers complaining, even if we shut it down within a day or so of its creation.
I don't start websites to make money --- I do them all for fun. I really hate having to deal with this crap because IANAL, I don't want to be a learn, don't want to pretend to be a lawyer, and just hate dealing with this stuff.
So depressing.