Don't ever assume that the market trades on fundamentals.
People think the market trades on fundamentals? I thought that stopped when Day Trading got big. We're in the glorious new era of marketing for stocks as much as products.
MP3/DVD ripping is a poor "big write operation". Even at 8x DVD speed (which I only get for a small percentage of the disk), it's writing at 10 MBps. And that's only if you rip the ISO to HDD, then re-compress. If you re-compress on the fly, it's less. These slower-than-your-disk sequential writes will be stuttering the drive head, at a rate that depends on how much your OS will buffer before writing a chunk at full speed. The SSD should handle this stuttering better than a HDD.
It's different if the rip reads from DVD, writes uncompressed to the HDD, then in a 2nd thread/process reads from HDD, compresses, and writes to HDD. Now you're talking in the neighborhood of 30 MBps... except you can't get that on a HDD because of the seeking between the write, read, and write location.
For large compiles... you're not maxing your disk I/O, you're maxing your disk IO Ops. SSD had many more IOps than HDD, so this will be faster.
Now, recording multiple channels of streaming video, maybe. If you're recording the compressed streams, and then it's not that high of MBps. If you're streaming raw HD, then no single disk is going to handle it anyway.
Here's the cool thing about I/O in general. 70 MBps is a *huge* amount of data. You can copy that 9GB DVD image in about 2 minutes. When's the last time you've ever seen any real life I/O operation go that fast? The best I've managed to pull off is about 32 MBps sustained, because I was scp'ing something from one computer to another, and I saturated the GigE link (no jumbo packets). Well, I've gotten better, but most people don't have $250k fiber channel disk arrays to play with. And even then, it wasn't a whole lot better.
Disclaimer: I'm not talking about those crap-ass first-gen SSD drives; those drives that get beat by a 100 MB IDE drive on an ISA card. I'm talking about real SSD drives, and it looks like Intel just released the first one.
XFS's goal was to be great at streaming and editing media, so it's more optimized for large sequential I/O.
VxFS's goal is to make Oracle faster, so it's more optimized for small random I/O.
VxFS is more comparable to JFS than XFS, from a design goal point of view.
</warning>
Yes, I'm painting with a very broad brush, but those two optimization goals tend to be exclusive.
Can't comment on sparse files. The only time I use sparse files is testing if something handles sparse files. They tend not to be used much with databases, which was my "enterprise focus".
Fast file delete is misleading on VxFS. It appears to be *very* fast, but it's actually pretty slow. The filename is unlinked quickly, and the free space is reclaimed very slowly.
They all appear to be fully 64bit. According to wikipedia, XFS has a max volume size of 8EiB (Exabytes), and VxFS has a max volume size of 16EiB. So both appear to be using full 64bit integers, but XFS is using signed 64bit integers.
The main reason I don't mention XFS is that I've never used it.:-) I got VxFS and VxVM with Veritas Cluster, so I used them.
VxFS includes a kernel module. You can't boot off it (no grub support), and it's installed after installation, so it can't be your root FS. It can be any other mount point. I generally use it for my MySQL and PostgreSQL data partitions. I would use it for/home if I had to deal with users.
VxFS by itself doesn't support all of those features (moving from stripe to concat, changing stripe width etc). Some of those come from VxVM (Veritas Volume Manager), which is well enough integrated with VxFS that I can resize a logical volume and filesystem with a single command.
VxFS is the only FS that I've used that can be resized while mounted. Actually, it must be resized while mounted. I've expanded and shrunk filesystems many times while MySQL was under load. It increases the disk I/O a bit, so MySQL runs a bit slower, but otherwise there was no impact.
Not only that, I've had a machine reboot (my fault) in the middle of a complex operation (restrip the RAID0 portions of the RAID 0+1 array in preparation to convert to a RAID 1+0). VxVM and VxFS mounted the volume fine, MySQL started serving, then VxVM picked up where it left off and completed successfully. No data lost.
In addition, a dirty 100G+ volume takes about 15 seconds to fsck. Suck that ext3.
On any server that can wake me up in the middle of the night, I'll gladly pay for the Veritas Foundation Suite.
I don't find a benefit or obvious advantage in a device that requires wear-leveling to keep from wearing itself out. The fact that it degrades its storage capacity gracefully instead of all at once doesn't offset that swap files can really work over mass storage devices and the first bad sectors have been known to start showing up after only weeks of use in some cases.
Magnetic media does this too, just not as intelligently. Magnetic media waits until a sector is nearing failure, then reads the data (hopefully) and moves it to a new sector.
You can query your magnetic drive to get a list of bad sectors. The list grows over time.
I would agree that SSDs can take a large chunk out of the high-performance HD market, but only for read-heavy environments....But in write-heavy environments the IOPS capabilities of flash becomes irrelevant because any write-heavy environment will also quickly wear the flash device out.
This has been sufficiently debunked. See comment 24888609. Even in the enterprise storage arena. Those guys will probably have fewer drives to replace, since magnetic drives don't have built in wear leveling.
And I'm looking forward to newer and cooler things that ZFS will do with pools of both types of media. It already uses flash drives as a read cache for magnetic media, what else will it do?
There are some steam trains out there that are running and are over 100 years old... do we really think that a CPU or a RAM or a motherboard can live that long?
I assume that these steam engines have been taken apart and rebuilt so many times that none of the original parts are still present.
A train is more analogous to a computer. Swap out the motherboard when it fails, swap out a drive in the RAID1 array when it fails, etc. And you *can* keep a computer running that long... if you can find the parts.
The selection isn't always great, but the books generally are cheaper than amazon. I find the monthly mailer catalog helpful. I can go through the catalog in about 10 minutes and get a list of "maybes". Then I'll read the whole review online to make up my mind.
There is a good selection of harder to find "classic" Sci-Fi material. Any Sci-Fi reader should be able to cover their commitment just back-filling their library with classics.
It is a book of the month club, so you have a minimum commitment and you'll get 2 books in the mail every month, unless you send back the mailer or reply online. I called them up and had them stop sending me the books automatically about a decade ago (after I meet my commitment).
They've been running a lot of specials lately, usually along the lines of "Buy two, get the cheaper one free", so I've been stocking up. I've taken this opportunity to fill in as much of Terry Prachett's Discworld series as SFBC has. They have a quarter to a third of the series. Or if I have an odd number, I'll throw a "maybe" into the cart, and it'll cost me about $1 in S&H.
One of the things I love about the SFBC is all the books are hardcover and are the same height and depth. The thickness and the font size vary, but every SFBC book stacks neatly on my bookshelf. No more bookshelf Tetris! You do miss out on the fancy color maps that some books have. When I care, I get those book from Amazon.
All in all, I buy about 50% of my books from SFBC. I get the must-have new releases from Amazon (if I remember to pre-order) or a brick and mortar. Anything I can wait a few months for comes from SFBC.
Maybe it's just me, but it appears that I'm reading the summary of that research topic differently.
Groups with diverse functional expertise, education, or personality can increase performance by enhancing creativity or group problem-solving. In contrast, more visible diversity, such as race, gender, or age, can have negative effects unless its managed properly, says Margaret Neale.
This implies that the skillset to properly manage a diverse good is non-trivial, lending support to the GP's post.
How wide is the bulletin board? What kind of hinge system did you build to handle that much weight?
I guess part of my confusion is the size of your closet. I was visualizing my office closet, which is 2 feet deep and 8 feet wide with sliding doors. Now I'm thinking your closet is a coat closet, about 3 feet wide and 3 feet deep.
No reason I can't build 3 of these units, and install them side by side in my closet.
In Oracle, you'd: SELECT t1.id, t1.name, t2.id, t2.name, t3.id, t3.name
FROM t1 LEFT JOIN t2 ON t1.id = t2.fk_t1_id
LEFT JOIN t3 ON t2.id = t3.fk_t2_id
WHERE t1.name = $my_user;
In MySQL that's slow. It's faster to: SELECT t1.id, t1.name FROM t1 WHERE t1.name = $my_user; SELECT t2.id, t2.name FROM t2 WHERE t2.fk_t1_id = $t1_id; SELECT t3.id, t3.name FROM t3 WHERE t3.fk_t2_id = $t2_id;
In Oracle that's slow.
Obviously, this is a simple example. But that's exactly how we load trees of objects. The associated INSERT/UPDATE statements to save the objects back to the database are even slower. At the time, the benchmarks said the speed optimization was worth while. I didn't know the maintence headache it would cause.
Refactoring the fast MySQL version into a fast Oracle version takes a good understanding of what the original code is trying to accomplish. Understanding is harder to acheive spread out like that.
These days, there are good Object-Relational Mapping tools. They were a bit harder to come by when we started the project (ie, too much money). It would be much simplier to refactor all the code to use a good ORM tool than to port to Oracle. That is a project I'm working on, since it can be done one class at a time instead of requiring an entire port.
I'll leave the rant about MySQL's stance on transaction (prior to InnoDB) for a different thread. Suffice it to say, we drank that koolaid too. Transactions are nearly impossible to refactor into code that was never designed for them.
While OSS alternatives do exist, migration is non-trivial. Particularly when you've been drinking the MySQL koolaid. My day job is a MySQL shop. Our code was written to run fast using MySQL, not an ACID database. Transactions were not available, and were not designed for. Rewritting all the code that does multi-table joins in code instead of in SQL (because that the only way to make MySQL fast) will take a lot of time.
The time it will take to migrate won't kill us, but it will cost us customers. Since we just started making a profit, it's a very big concern.
In my defense, nobody in the basement knew any better. MySQL is still a lot better than the flat binary files we were using. If I could rewrite, MySQL wouldn't be a consideration. But that's not a viable business option as long as MySQL is available.
> More specifically, watching a variable change value, while stepping > through the lines of a Perl script using the debugger, requires that > the programmer manually or programmatically echo that variable's value, > by issuing a print command ("p") followed by the variable name, one > way or another.
$ perl -d -e 1
DB h...
h [db_cmd] Get help on command w expr Add a watch expression
h h Complete help page W expr|* Delete a/all watch exprs...
Admittedly, the watch expressions don't detect changes in deep data structures. But it works just find on dependant expressions. I'll regularrly set watches on a loop index and a data structure being indexed by said loop var. The watch expressions will display both results.
In addition, the 'x' command works much better than 'p'. 'x' dumps the data structure, so it will display arbitrarily complex and deep data.
There are several threads about `screens`, a text based "window manager". Other interesting concepts were Ion and Rat Poison, Pane Managers vs. Window Managers.
Slashdot Mirror
Don't ever assume that the market trades on fundamentals.
People think the market trades on fundamentals? I thought that stopped when Day Trading got big. We're in the glorious new era of marketing for stocks as much as products.
MP3/DVD ripping is a poor "big write operation". Even at 8x DVD speed (which I only get for a small percentage of the disk), it's writing at 10 MBps. And that's only if you rip the ISO to HDD, then re-compress. If you re-compress on the fly, it's less. These slower-than-your-disk sequential writes will be stuttering the drive head, at a rate that depends on how much your OS will buffer before writing a chunk at full speed. The SSD should handle this stuttering better than a HDD.
It's different if the rip reads from DVD, writes uncompressed to the HDD, then in a 2nd thread/process reads from HDD, compresses, and writes to HDD. Now you're talking in the neighborhood of 30 MBps... except you can't get that on a HDD because of the seeking between the write, read, and write location.
For large compiles... you're not maxing your disk I/O, you're maxing your disk IO Ops. SSD had many more IOps than HDD, so this will be faster.
Now, recording multiple channels of streaming video, maybe. If you're recording the compressed streams, and then it's not that high of MBps. If you're streaming raw HD, then no single disk is going to handle it anyway.
Here's the cool thing about I/O in general. 70 MBps is a *huge* amount of data. You can copy that 9GB DVD image in about 2 minutes. When's the last time you've ever seen any real life I/O operation go that fast? The best I've managed to pull off is about 32 MBps sustained, because I was scp'ing something from one computer to another, and I saturated the GigE link (no jumbo packets). Well, I've gotten better, but most people don't have $250k fiber channel disk arrays to play with. And even then, it wasn't a whole lot better.
Disclaimer: I'm not talking about those crap-ass first-gen SSD drives; those drives that get beat by a 100 MB IDE drive on an ISA card. I'm talking about real SSD drives, and it looks like Intel just released the first one.
XFS's goal was to be great at streaming and editing media, so it's more optimized for large sequential I/O.
... which makes XFS more appropriate for a MythTV box. What was I talking about again?
XFS has a different design goal than VxFS.
<warning title="Gross Generalizations">
</warning>
Yes, I'm painting with a very broad brush, but those two optimization goals tend to be exclusive.
Can't comment on sparse files. The only time I use sparse files is testing if something handles sparse files. They tend not to be used much with databases, which was my "enterprise focus".
Fast file delete is misleading on VxFS. It appears to be *very* fast, but it's actually pretty slow. The filename is unlinked quickly, and the free space is reclaimed very slowly.
They all appear to be fully 64bit. According to wikipedia, XFS has a max volume size of 8EiB (Exabytes), and VxFS has a max volume size of 16EiB. So both appear to be using full 64bit integers, but XFS is using signed 64bit integers.
The main reason I don't mention XFS is that I've never used it. :-) I got VxFS and VxVM with Veritas Cluster, so I used them.
That's new to me. Thanks!
I'll upgrade my personal machines. It's always been such a PITA to umount /home to resize it.
VxFS includes a kernel module. You can't boot off it (no grub support), and it's installed after installation, so it can't be your root FS. It can be any other mount point. I generally use it for my MySQL and PostgreSQL data partitions. I would use it for /home if I had to deal with users.
VxFS by itself doesn't support all of those features (moving from stripe to concat, changing stripe width etc). Some of those come from VxVM (Veritas Volume Manager), which is well enough integrated with VxFS that I can resize a logical volume and filesystem with a single command.
VxFS is the only FS that I've used that can be resized while mounted. Actually, it must be resized while mounted. I've expanded and shrunk filesystems many times while MySQL was under load. It increases the disk I/O a bit, so MySQL runs a bit slower, but otherwise there was no impact.
Not only that, I've had a machine reboot (my fault) in the middle of a complex operation (restrip the RAID0 portions of the RAID 0+1 array in preparation to convert to a RAID 1+0). VxVM and VxFS mounted the volume fine, MySQL started serving, then VxVM picked up where it left off and completed successfully. No data lost.
In addition, a dirty 100G+ volume takes about 15 seconds to fsck. Suck that ext3.
On any server that can wake me up in the middle of the night, I'll gladly pay for the Veritas Foundation Suite.
I don't find a benefit or obvious advantage in a device that requires wear-leveling to keep from wearing itself out. The fact that it degrades its storage capacity gracefully instead of all at once doesn't offset that swap files can really work over mass storage devices and the first bad sectors have been known to start showing up after only weeks of use in some cases.
Magnetic media does this too, just not as intelligently. Magnetic media waits until a sector is nearing failure, then reads the data (hopefully) and moves it to a new sector.
You can query your magnetic drive to get a list of bad sectors. The list grows over time.
I would agree that SSDs can take a large chunk out of the high-performance HD market, but only for read-heavy environments. ...But in write-heavy environments the IOPS capabilities of flash becomes irrelevant because any write-heavy environment will also quickly wear the flash device out.
This has been sufficiently debunked. See comment 24888609. Even in the enterprise storage arena. Those guys will probably have fewer drives to replace, since magnetic drives don't have built in wear leveling.
And I'm looking forward to newer and cooler things that ZFS will do with pools of both types of media. It already uses flash drives as a read cache for magnetic media, what else will it do?
There are some steam trains out there that are running and are over 100 years old... do we really think that a CPU or a RAM or a motherboard can live that long?
I assume that these steam engines have been taken apart and rebuilt so many times that none of the original parts are still present.
A train is more analogous to a computer. Swap out the motherboard when it fails, swap out a drive in the RAID1 array when it fails, etc. And you *can* keep a computer running that long... if you can find the parts.
I remember when it was only half-full of nerds.
Oh, and get off my lawn!
> > I give a talk on the consequences of Moore's Law to a freshman class...
> I ahve [sic] no doubt it will happen, but we are generations away. My son(now 10)...
You've given the exact same timetable; the recipients of this lecture are much closer to your son's generation than your generation.
Don't you people do code outside of work? What kind of "for the love of coding" programmer are you?
Admittedly, my personal project code is a wee bit dated (it's older than my children), but it's more current than my resume...
I've been a member of SFBC for about 15 years.
The selection isn't always great, but the books generally are cheaper than amazon. I find the monthly mailer catalog helpful. I can go through the catalog in about 10 minutes and get a list of "maybes". Then I'll read the whole review online to make up my mind.
There is a good selection of harder to find "classic" Sci-Fi material. Any Sci-Fi reader should be able to cover their commitment just back-filling their library with classics.
It is a book of the month club, so you have a minimum commitment and you'll get 2 books in the mail every month, unless you send back the mailer or reply online. I called them up and had them stop sending me the books automatically about a decade ago (after I meet my commitment).
They've been running a lot of specials lately, usually along the lines of "Buy two, get the cheaper one free", so I've been stocking up. I've taken this opportunity to fill in as much of Terry Prachett's Discworld series as SFBC has. They have a quarter to a third of the series. Or if I have an odd number, I'll throw a "maybe" into the cart, and it'll cost me about $1 in S&H.
One of the things I love about the SFBC is all the books are hardcover and are the same height and depth. The thickness and the font size vary, but every SFBC book stacks neatly on my bookshelf. No more bookshelf Tetris! You do miss out on the fancy color maps that some books have. When I care, I get those book from Amazon.
All in all, I buy about 50% of my books from SFBC. I get the must-have new releases from Amazon (if I remember to pre-order) or a brick and mortar. Anything I can wait a few months for comes from SFBC.
> Seriously, get off this planet.
We're trying. STFU.
Last I checked, all my VRTS* packages hadn't changed to SYMC*. Although I'm not running the latest version, so YMMV. :-D
This implies that the skillset to properly manage a diverse good is non-trivial, lending support to the GP's post.
How wide is the bulletin board? What kind of hinge system did you build to handle that much weight?
I guess part of my confusion is the size of your closet. I was visualizing my office closet, which is 2 feet deep and 8 feet wide with sliding doors. Now I'm thinking your closet is a coat closet, about 3 feet wide and 3 feet deep.
No reason I can't build 3 of these units, and install them side by side in my closet.
+digg
I think I understand what you built, but it would help if I could see it.
Do you have any photos avaialble? Maybe a project page?
> Unfortunately I don't think it comes close to something
> like lucene. But that's another kettle of fish.
You might want to check out Solr. It's a search server front end for Lucene.
Disclaimer: I work for CNET, who open sourced the project.
The website is at http://incubator.apache.org/solr
The wiki is at http://wiki.apache.org/solr
In Oracle, you'd:
SELECT t1.id, t1.name, t2.id, t2.name, t3.id, t3.name
FROM t1 LEFT JOIN t2 ON t1.id = t2.fk_t1_id
LEFT JOIN t3 ON t2.id = t3.fk_t2_id
WHERE t1.name = $my_user;
In MySQL that's slow. It's faster to:
SELECT t1.id, t1.name FROM t1 WHERE t1.name = $my_user;
SELECT t2.id, t2.name FROM t2 WHERE t2.fk_t1_id = $t1_id;
SELECT t3.id, t3.name FROM t3 WHERE t3.fk_t2_id = $t2_id;
In Oracle that's slow.
Obviously, this is a simple example. But that's exactly how we load trees of objects. The associated INSERT/UPDATE statements to save the objects back to the database are even slower. At the time, the benchmarks said the speed optimization was worth while. I didn't know the maintence headache it would cause.
Refactoring the fast MySQL version into a fast Oracle version takes a good understanding of what the original code is trying to accomplish. Understanding is harder to acheive spread out like that.
These days, there are good Object-Relational Mapping tools. They were a bit harder to come by when we started the project (ie, too much money). It would be much simplier to refactor all the code to use a good ORM tool than to port to Oracle. That is a project I'm working on, since it can be done one class at a time instead of requiring an entire port.
I'll leave the rant about MySQL's stance on transaction (prior to InnoDB) for a different thread. Suffice it to say, we drank that koolaid too. Transactions are nearly impossible to refactor into code that was never designed for them.
While OSS alternatives do exist, migration is non-trivial. Particularly when you've been drinking the MySQL koolaid. My day job is a MySQL shop. Our code was written to run fast using MySQL, not an ACID database. Transactions were not available, and were not designed for. Rewritting all the code that does multi-table joins in code instead of in SQL (because that the only way to make MySQL fast) will take a lot of time.
The time it will take to migrate won't kill us, but it will cost us customers. Since we just started making a profit, it's a very big concern.
In my defense, nobody in the basement knew any better. MySQL is still a lot better than the flat binary files we were using. If I could rewrite, MySQL wouldn't be a consideration. But that's not a viable business option as long as MySQL is available.
> More specifically, watching a variable change value, while stepping
... ...
> through the lines of a Perl script using the debugger, requires that
> the programmer manually or programmatically echo that variable's value,
> by issuing a print command ("p") followed by the variable name, one
> way or another.
$ perl -d -e 1
DB h
h [db_cmd] Get help on command w expr Add a watch expression
h h Complete help page W expr|* Delete a/all watch exprs
Admittedly, the watch expressions don't detect changes in deep data structures. But it works just find on dependant expressions. I'll regularrly set watches on a loop index and a data structure being indexed by said loop var. The watch expressions will display both results.
In addition, the 'x' command works much better than 'p'. 'x' dumps the data structure, so it will display arbitrarily complex and deep data.
Read through this Ask /.: http://ask.slashdot.org/article.pl?sid=05/11/27/02 52214&tid=189&tid=185&tid=4
There are several threads about `screens`, a text based "window manager". Other interesting concepts were Ion and Rat Poison, Pane Managers vs. Window Managers.
% uname -a ... 2.6.12-1.1387_FC4smp #1 SMP Fri Jul 1 12:55:24 EDT 2005 i686 i686 i386 GNU/Linux
Linux
% uptime
17:27:23 up 27 days, 1:27, 75 users, load average: 3.08, 2.80, 1.95
I'm using some sort of on-board Intel video chipset.