Agreed. At the time, I didn't know how to use the tools (Veritas Foundation Suite) well enough to go outside the wizard. Now that I'm familiar enough, the wizard supports RAID 0+1 and RAID 1+0. We're planning an array rebuild, but these things take time.:-)
Luckily, I'm only dealing with 28 drives, so we only have about 1 failure per year. Nothing the hot spares can't deal with.
I finally broke down and replaced my 486/66. It was my NAT box, and it did ok serving ssh, ntp, dhcp, qmail and apache. But when I added MySQL, the old IDE (not EIDE, think 512 Meg drives) interface started showing it's age.
So I dumpster dove at work and surfaced with a K6-2/350. The EIDE-33 interface works like a charm, and being able to boot up with more than 64 Meg of RAM helps. And it feels so darn fast!
Good point about Software RAID working well on modern systems.
There are even times that you have to have software RAID. My work systems are all software RAID because we have 2 controllers. Controller 1 has a RAID 0 stripe set and Controller 2 has another RAID 0 stripe set. We then mirror those stripes. If a single controller, cable, drive, or drive cabinet power supply goes out, we can still get to the data. Can't do that with a hardware RAID card.
Well, there are ways to do it, but they all involve intelligent external cabinates that present a RAID drive as a normal disk. That costs way more $ than work is willing to pay, and we still buy Sun hardware.
That passive nmap won't work very well on well formed packets. nmap OS identifaction works by exercising the TCP/IP stack. It requires an open and close port to even attempt identification. It uses the undefined portions of the spec to determine what OS you're using.
Setting up a connection is about the most well defined portion of the spec.
Still, it was a hard lock, no screen blanking. (Or are you sure it hasn't locked up while the console screen blanker was active?)
Most of the lockups are while the blanker is active (the machine sits in a closet.) The one time I saw it lockup, it hung leaving the boot up fsck progess indicator on the screen. No errors were displayed.
The machine has 2 different 3Com cards. But as I think back, I did see the machine lock up with both cards out. That leaves the Video card, motherboard, RAM or CPU. Guess I swap out the RAM, since that's cheap-n-easy.
I'm having a similiar problem. What kind of symptoms did you see? I get a blank screen, and the machine requires a power cycle.
What kind of NIC was it?
How did you diagnose? Just start some massive dd's and ftp's?
I was getting ready to buy a new motherboard, because I thought the IDE chipset was going bad (had a power supply overheat, so it's possible but unlikely.) The NIC is much more likely.
BTW, I've always wondered about how the transition day is calculated. I've never been able to find a simple description of it.
*snip* Is it just a 'last Sunday in October/April' or something like that hard coded in the library?
Depends on the Country and the Hemisphere. The US (and most of North America) is first sunday in April and last sunday in Octoboer. I didn't realize that the Southern Hemisphere observers DST the other six months (ie, their summer). Seems obvious in hindsight.
Web Exhibits has history and start/stop days by country.
I shamelessly stole a bit of DST calculation code from their web site.
To date, I've had 4 Sun machines that've randomly rebooted. I sent the core file to Sun, and an engineer showed up the next day with a shiny new processor.
6 weeks later, I'm still trying to figure out why my new Dual P4 Xeon linux boxes are hardlocking daily. Looks like it something to do with a combination of SMT, Gigabit Ethernet, and BigBrother.
And as far as memory problems: The Sun boxes tell me which memory module generated the ECC error. Linux just tells me that an ECC error occured. So now I have to take the box offline and run memtest86 all weekend long, hoping that chip chooses to flake out while I'm testing.
Regarding performance, it depends on the Application. MySQL on Sun kicks the pants off of MySQL on Intel. Sun's massive Processor/RAM I/O kicks ass. Sure, Intel motherboards have more bang for the buck, but Suns have more bang. This may be changing with the likes of RDRAM and DDR SDRAM, but don't count out the UltraSparc III's.
Don't get me wrong, I love Linux. But Sun has honed their support to a razor sharp edge. Linux will get there, but it'll take a while.
I'll start by putting most of my attention into driving and a little bit into the conversation. That works pretty well. But then I start getting involved in the conversation. After a while, the passengers start telling me to shut up and drive.
Buy 4 more Class C subnets, and transition the people on the leased subnets now.
Then when you move, its just a matter of updating the routing.
Of course, changing the subnet routes could be another problem. A lot of the big routers don't propigate routes for anything smaller than a/19 subnet. I could be wrong about the size, but a/24 is right out. If your 4 existing subnets are all together, you've got a/22, and that might just work. Getting the 8 subnets together for a/21 would be better.
If you want more info, I googled a good site. The explainations/advantages/disadvantages are mediocre, but the diagrams of disk blocks are worth 1000 words.
It took me a while to figure out, but the numbers ("0123456710530+1") in the upper right hand corner are links to different RAID level explainations.
It even explained RAID 2, which I haven't seen before.
Re:Not designed to work with LCD screens
on
USB KVMs Compared
·
· Score: 1
I traced this to the 'official' Belkin VGA cables - they sucked too. Replacing them with other (not expensive) makes of VGA extension leads improved the video quality on the CRT enormously. This was trivially proved by just using the leads as extension leads, taking the switch out of the equation. When using the Belkin leads, video quality was crap.
Man, I'm a sucker. The talked me into upgrading to the Pro cables ($50 / set).
Looking at the 2, the standard cables use unsheilded video without the iron core RF shield. The Pro cables uses a sheilded video cable with the iron core, with Gold Plated contacts!
My CRT display is great with the Pro cables, but sucked with the standard cables. Good thing work paid for the cables. Now I just feel like a sucker, instead of a poor sucker.
The $40k is per processor. If you're upgrading to Suns for performance, you're probably getting at least a Quad processor.
With all the features we wanted on a Quad processor, Oracle was $250K per machine. The features we wanted (partitioning, RAC under 9i) are sweet. Esp. RAC - imagine Oracle breeding with a Beowulf Cluster... yum!
Eventually we decided that even with the Cluster, it would be faster to buy 4 more Suns than 1 Oracle liscense.
My old 486 wouldn't handle a drive > 512Meg (including CD-ROMS). So I bought an EIDE card. A quick check on Pricewatch for "ISA EIDE Card" shows that Promise still makes them. It'll be pretty slow (the ISA bus can only transfer like 7 or 8 Meg/sec.) If the machine has PCI, you can go that route. But if this is an old machine, you probably don't need a performance powerhouse anyway.
My ISA card worked fine up to 8 Gig drives, but I would've needed a BIOS Overlay to use drives larger than 8 Gigs. YMMV.
Although... PriceWatch shows the cards as costing about $34 w/ S&H, so I guess the cheaper option would be to buy the BIOS upgrade.
So does MySQL not flush disk writes to disk before a transaction commits? Your reply seems to suggest that MySQL waits for the kernel to flush disk buffers to disk -- if that's the case, it definately sounds like there is a potential for data loss.
Transactions? You must be thinking of another database. MySQL only does transactions if you use the not-supported-out-of-the-box InnoDB table type. And that defeats the whole point of MySQL, because InnoDB is about the same speed as PostgreSQL.
The "Data" I've lost was usually still in the middle of being written. Some of the writes takes more than a few seconds to complete, so I guess the few seconds overlaps with "not done yet". That's what I get for posting drunk;-)
MySQL does flush the data out of disk, but its less agressive flushing indexes out to disk. In general though, most of my repairs after a crash are just to be safe. MySQL has a "clean" bit in the tables, so we check tables that are labelled unclean. Most of the time the unclean tables are fine, and the bit is cleared. It still takes a while to verify though. Sorta like checking your ext3 filesystem after a crash. MySQL isn't journaled, but our filesystem is. That might help a bit.
That was more likely because at the time, MySQL was the only proper open source database around and that had good interfaces in web languages. Postgresql was still unstable, slow and full of dumb limitation (32 KB row sizes anyone ?). Interbase was closed source.
That's an arguement that I'll listen too. The only arguement proposed was "It's faster."
Most of MySQL limitations are not limitation on what you can do but on how easy you can do it. Stored proc, nested selects and views are all nice to use but you can write clean code without using them (it's just more lines of codes). There's no "hack" to do here.
That would explain the 600 Meg Perl Hash I have to load up, because MySQL can't do sub-selects.:-)
Clean code is what we ended up with. But it would be much eaiser to maintain 1 SQL statement that UPDATEs a table bases of the results of a SELECT statement (or for that matter, lets me update more than 1 table in a single UPDATE statement). Instead we have to mainted approx 20 lines of Perl code and 2 SQL statements per. Plus all that data is now clogging my network, instead of residing in RAM on the server.
You have to repair the tables before restarting. MySQL claims to be atomic in its data operations, and it appears that repairs are mostly just re-indexing to make sure the indexes agree with the data. I've never lost data due to a crash (well, nothing that hadn't been written within a few seconds of the crash.)
The time to repair is causing us a few issues. The 60GB of data we have takes approx 1.5 hours to repair after a crash (single process repair.) We're currently experimenting with how many processes the hardware can support for optimum repair time. 32 processes is looking good, but that's a Quad UltaSparc2 w/ Fiber Channel.
I've had to deal with quite a few crashes recently. It looks like our hardware is a lot bigger than MySQL can test on in house. The Sparc boxes (mentioned above) have had about 10 crashes in the past year. The Quad Xeon box has had 0 crashes in the 2.5 years that I've maintained it. YMMV.
I was the lead developer on a e-commerce site. The guy that started the company in his basement picked MySQL because the benchmarks beat the shit out all the other DBs. Now we're stuck with it, due to legacy.
MySQL works great, as long as you're prepared to make a bunch of bastard hacks to get around the limitations. We started over from scratch, and designed around MySQL. The Objects we have work great, as long as we use MySQL. If we ever move to something that is ACID compliant, we'll probably have to redesign from scratch to get it working. It would definately be easier.
But our site is freaking fast! We're stuck with MySQL because the whole App had to be designed around MySQL to make it run reasonably.
Depends if those specs are for the individual Drive or the Bus.
My 5400 RPM IDE drive pushes about 12 Megs/sec on UDMA66. Sure, the Bus supports 66 Megs/sec, but the drive sure doesn't.
The volumes at work have 3 10k RPM SCSI drives on Fiber Channel. That bus is 1Gbps (125 Megs/sec), but the drives will only push ~25Meg/sec. With the RAID0 I can substain around 75 Meg/sec Read/Write. Since we mirror acrossed controllers, I can do 150 Meg/sec Read, 75 Meg/sec write. 'Course, these are all magic benchmark numbers.
Now, if the inidivudual Drives will are 1Gbps, that would be sweet. It would take me 5 10k RPM SCSI drives to do that. If I grab 5 of those new 180G drives, I can get a RAID0 900 Gbps at 1Gbps for somewhere in the neighborhood of $20k. But since 5 drives are less reliable that 1 drive, I really need to buy 10 drives and RAID 0+1. More hardware... yada yada, probably talking about $50k for 900 Gig @ 1Gbps.
So I'll keep my eye on these, and try to find some specs I can read. Might be useful, might be useless.
We're an ASP. We use Perl, Linux, MySQL, and a bunch of CPAN modules. We had a meeting with MySQL recently, and I asked, point blank, if what we were doing was Ok. MySQL said that as long as we stay an ASP, we're fine (but we should really buy a Support Liscense). Difficulties would occur if we decided to install our software on customers computers.
We are working on improving a few CPAN modules, and I plan to contribute that back. But as long as we stay an ASP, we're not legally required to.
Slashdot Mirror
But we almost had it!
This explains why I always remember those pizza adds in my pr0n collection.
Agreed. At the time, I didn't know how to use the tools (Veritas Foundation Suite) well enough to go outside the wizard. Now that I'm familiar enough, the wizard supports RAID 0+1 and RAID 1+0. We're planning an array rebuild, but these things take time. :-)
Luckily, I'm only dealing with 28 drives, so we only have about 1 failure per year. Nothing the hot spares can't deal with.
I finally broke down and replaced my 486/66. It was my NAT box, and it did ok serving ssh, ntp, dhcp, qmail and apache. But when I added MySQL, the old IDE (not EIDE, think 512 Meg drives) interface started showing it's age.
So I dumpster dove at work and surfaced with a K6-2/350. The EIDE-33 interface works like a charm, and being able to boot up with more than 64 Meg of RAM helps. And it feels so darn fast!
Good point about Software RAID working well on modern systems.
There are even times that you have to have software RAID. My work systems are all software RAID because we have 2 controllers. Controller 1 has a RAID 0 stripe set and Controller 2 has another RAID 0 stripe set. We then mirror those stripes. If a single controller, cable, drive, or drive cabinet power supply goes out, we can still get to the data. Can't do that with a hardware RAID card.
Well, there are ways to do it, but they all involve intelligent external cabinates that present a RAID drive as a normal disk. That costs way more $ than work is willing to pay, and we still buy Sun hardware.
Setting up a connection is about the most well defined portion of the spec.
Still, it was a hard lock, no screen blanking. (Or are you sure it hasn't locked up while the console screen blanker was active?)
Most of the lockups are while the blanker is active (the machine sits in a closet.) The one time I saw it lockup, it hung leaving the boot up fsck progess indicator on the screen. No errors were displayed.
The machine has 2 different 3Com cards. But as I think back, I did see the machine lock up with both cards out. That leaves the Video card, motherboard, RAM or CPU. Guess I swap out the RAM, since that's cheap-n-easy.
I'm having a similiar problem. What kind of symptoms did you see? I get a blank screen, and the machine requires a power cycle.
What kind of NIC was it?
How did you diagnose? Just start some massive dd's and ftp's?
I was getting ready to buy a new motherboard, because I thought the IDE chipset was going bad (had a power supply overheat, so it's possible but unlikely.) The NIC is much more likely.
Any info is appreciated!
*snip*
Is it just a 'last Sunday in October/April' or something like that hard coded in the library?
Depends on the Country and the Hemisphere. The US (and most of North America) is first sunday in April and last sunday in Octoboer. I didn't realize that the Southern Hemisphere observers DST the other six months (ie, their summer). Seems obvious in hindsight.
Web Exhibits has history and start/stop days by country.
I shamelessly stole a bit of DST calculation code from their web site.
At least Sun can tell you what the error is.
To date, I've had 4 Sun machines that've randomly rebooted. I sent the core file to Sun, and an engineer showed up the next day with a shiny new processor.
6 weeks later, I'm still trying to figure out why my new Dual P4 Xeon linux boxes are hardlocking daily. Looks like it something to do with a combination of SMT, Gigabit Ethernet, and BigBrother.
And as far as memory problems: The Sun boxes tell me which memory module generated the ECC error. Linux just tells me that an ECC error occured. So now I have to take the box offline and run memtest86 all weekend long, hoping that chip chooses to flake out while I'm testing.
Regarding performance, it depends on the Application. MySQL on Sun kicks the pants off of MySQL on Intel. Sun's massive Processor/RAM I/O kicks ass. Sure, Intel motherboards have more bang for the buck, but Suns have more bang. This may be changing with the likes of RDRAM and DDR SDRAM, but don't count out the UltraSparc III's.
Don't get me wrong, I love Linux. But Sun has honed their support to a razor sharp edge. Linux will get there, but it'll take a while.
I can't.
I'll start by putting most of my attention into driving and a little bit into the conversation. That works pretty well. But then I start getting involved in the conversation. After a while, the passengers start telling me to shut up and drive.
Buy 4 more Class C subnets, and transition the people on the leased subnets now.
/19 subnet. I could be wrong about the size, but a /24 is right out. If your 4 existing subnets are all together, you've got a /22, and that might just work. Getting the 8 subnets together for a /21 would be better.
Then when you move, its just a matter of updating the routing.
Of course, changing the subnet routes could be another problem. A lot of the big routers don't propigate routes for anything smaller than a
RAID Info
It took me a while to figure out, but the numbers ("0123456710530+1") in the upper right hand corner are links to different RAID level explainations.
It even explained RAID 2, which I haven't seen before.
Man, I'm a sucker. The talked me into upgrading to the Pro cables ($50 / set).
Looking at the 2, the standard cables use unsheilded video without the iron core RF shield. The Pro cables uses a sheilded video cable with the iron core, with Gold Plated contacts!
My CRT display is great with the Pro cables, but sucked with the standard cables. Good thing work paid for the cables. Now I just feel like a sucker, instead of a poor sucker.
With all the features we wanted on a Quad processor, Oracle was $250K per machine. The features we wanted (partitioning, RAC under 9i) are sweet. Esp. RAC - imagine Oracle breeding with a Beowulf Cluster... yum!
Eventually we decided that even with the Cluster, it would be faster to buy 4 more Suns than 1 Oracle liscense.
My old 486 wouldn't handle a drive > 512Meg (including CD-ROMS). So I bought an EIDE card. A quick check on Pricewatch for "ISA EIDE Card" shows that Promise still makes them. It'll be pretty slow (the ISA bus can only transfer like 7 or 8 Meg/sec.) If the machine has PCI, you can go that route. But if this is an old machine, you probably don't need a performance powerhouse anyway.
My ISA card worked fine up to 8 Gig drives, but I would've needed a BIOS Overlay to use drives larger than 8 Gigs. YMMV.
Although... PriceWatch shows the cards as costing about $34 w/ S&H, so I guess the cheaper option would be to buy the BIOS upgrade.
Transactions? You must be thinking of another database. MySQL only does transactions if you use the not-supported-out-of-the-box InnoDB table type. And that defeats the whole point of MySQL, because InnoDB is about the same speed as PostgreSQL.
The "Data" I've lost was usually still in the middle of being written. Some of the writes takes more than a few seconds to complete, so I guess the few seconds overlaps with "not done yet". That's what I get for posting drunk ;-)
MySQL does flush the data out of disk, but its less agressive flushing indexes out to disk. In general though, most of my repairs after a crash are just to be safe. MySQL has a "clean" bit in the tables, so we check tables that are labelled unclean. Most of the time the unclean tables are fine, and the bit is cleared. It still takes a while to verify though. Sorta like checking your ext3 filesystem after a crash. MySQL isn't journaled, but our filesystem is. That might help a bit.
Have you considered using PostgreSQL?
We're too far gone. See my other comments
That was more likely because at the time, MySQL was the only proper open source database around and that had good interfaces in web languages. Postgresql was still unstable, slow and full of dumb limitation (32 KB row sizes anyone ?). Interbase was closed source.
That's an arguement that I'll listen too. The only arguement proposed was "It's faster."
Most of MySQL limitations are not limitation on what you can do but on how easy you can do it. Stored proc, nested selects and views are all nice to use but you can write clean code without using them (it's just more lines of codes). There's no "hack" to do here.
:-)
That would explain the 600 Meg Perl Hash I have to load up, because MySQL can't do sub-selects.
Clean code is what we ended up with. But it would be much eaiser to maintain 1 SQL statement that UPDATEs a table bases of the results of a SELECT statement (or for that matter, lets me update more than 1 table in a single UPDATE statement). Instead we have to mainted approx 20 lines of Perl code and 2 SQL statements per. Plus all that data is now clogging my network, instead of residing in RAM on the server.
You have to repair the tables before restarting. MySQL claims to be atomic in its data operations, and it appears that repairs are mostly just re-indexing to make sure the indexes agree with the data. I've never lost data due to a crash (well, nothing that hadn't been written within a few seconds of the crash.)
The time to repair is causing us a few issues. The 60GB of data we have takes approx 1.5 hours to repair after a crash (single process repair.) We're currently experimenting with how many processes the hardware can support for optimum repair time. 32 processes is looking good, but that's a Quad UltaSparc2 w/ Fiber Channel.
I've had to deal with quite a few crashes recently. It looks like our hardware is a lot bigger than MySQL can test on in house. The Sparc boxes (mentioned above) have had about 10 crashes in the past year. The Quad Xeon box has had 0 crashes in the 2.5 years that I've maintained it. YMMV.
I have to agree.
I was the lead developer on a e-commerce site. The guy that started the company in his basement picked MySQL because the benchmarks beat the shit out all the other DBs. Now we're stuck with it, due to legacy.
MySQL works great, as long as you're prepared to make a bunch of bastard hacks to get around the limitations. We started over from scratch, and designed around MySQL. The Objects we have work great, as long as we use MySQL. If we ever move to something that is ACID compliant, we'll probably have to redesign from scratch to get it working. It would definately be easier.
But our site is freaking fast! We're stuck with MySQL because the whole App had to be designed around MySQL to make it run reasonably.
Depends if those specs are for the individual Drive or the Bus.
My 5400 RPM IDE drive pushes about 12 Megs/sec on UDMA66. Sure, the Bus supports 66 Megs/sec, but the drive sure doesn't.
The volumes at work have 3 10k RPM SCSI drives on Fiber Channel. That bus is 1Gbps (125 Megs/sec), but the drives will only push ~25Meg/sec. With the RAID0 I can substain around 75 Meg/sec Read/Write. Since we mirror acrossed controllers, I can do 150 Meg/sec Read, 75 Meg/sec write. 'Course, these are all magic benchmark numbers.
Now, if the inidivudual Drives will are 1Gbps, that would be sweet. It would take me 5 10k RPM SCSI drives to do that. If I grab 5 of those new 180G drives, I can get a RAID0 900 Gbps at 1Gbps for somewhere in the neighborhood of $20k. But since 5 drives are less reliable that 1 drive, I really need to buy 10 drives and RAID 0+1. More hardware... yada yada, probably talking about $50k for 900 Gig @ 1Gbps.
So I'll keep my eye on these, and try to find some specs I can read. Might be useful, might be useless.
We are working on improving a few CPAN modules, and I plan to contribute that back. But as long as we stay an ASP, we're not legally required to.
I never thought that I would see the day when a Linux Desktop is being ported for usability.