I'd be skeptical of someone, especially in IT, who has no online presence. I believe it is good to build up a brand around your name on the internet. You should be in control of what potential future employers see when they Google you. It is better for you to be in the top spot for your name then someone else talking about you, or something with the same name.
When I first joined the internet, I asked my parents if I could have a website on Geocities. They said no. I didn't listen and went behind their back to create it. I didn't have any info besides my first name, so there was no real harm (and I was smart enough not to meet up alone with random strangers, not that I was ever propositioned).
In 2001 I bought a domain named after me and blogged on it (although, it took years for me to admit it was a blog, since those had a bad stigma attached:) ). More recently I've posted the occasional technical entry. Because of this I've been cold called (well, e-mailed) from major companies asking if I want to interview.
It is 2009, nearly everyone has some sort of online presence now. It is unlikely you'll be targeted just for having your name out there. It is much more likely bad things will happen when a company you deal with is hacked and your information is stolen that way. Plus, you can use your presence as a defense. If I wasn't the top result on Google for my name, people might think I was an anti-semite author.
I pay income tax, but barely ever use freeways (mainly when I'm on the bus). Why should I subsidize your use of the roads? Also, congestion tolling makes the roads more efficient and keeps the average speed higher.
When you draw the internet while diagramming on a whiteboard, what do you draw? Most people draw a cloud as an abstraction for the stuff "out there". I've never seen anyone draw a spiderweb when drawing a diagram that includes the internet.
I've tried to explain the cloud to slashdotters before. If you don't like the word cloud, you don't have to call it that. A less buzzword-y and perhaps more accurate term would be "utility computing". Turn on the faucet and out comes your data.
Yeah, I noticed that this morning when I read about the investment. They closed a bunch of older facilities in Asia, laying off the workers, and are building the new fancy fabs in the US (and creating high paying jobs in the process).
Of course, the next thing that came to my mind is whether Slashdot would cover that aspect of the story. Sure enough, Slashdot's summary completely disregards that Intel is creating jobs in America. I suspect there are two reasons for this: 1. It hurts Slashdot's agenda if they report about companies insourcing, readers should only know about outsourcing by "the evil corporations". 2. Because Intel is the big bad wolf and we can't report anything good they do.
It's not my fault that company didn't do it's due diligence. If you lied on your resume and were actually incompetent, I wouldn't expect you to get past the phone screen, at least where I work.
Umm, hate to burst your bubble, but I was being pro-American. This country has a long history of being built by immigrants and I want to keep it that way.
I know other countries have protectionist policies, do you notice how they are not as successful as the US? We are not the biggest economic power in the world in spite of our open borders, but because of our open borders.
If the company is on the brink of disaster and is receiving bailout money in order to save the company (which I oppose), then they should do everything in their power to stay afloat. If the H1-B workers are providing a better deal than Americans, or have skills that can not be replaced, then getting rid of them to support American jobs endangers the company and goes against the whole bailout effort.
The reality, particularly in the tech industry, is that non-Americans are leaders in the various fields. Pick up any industry-related journal, and 90% of the articles are by people of non-American decent.
Very true. This is to be expected because America makes up only, what, 4% of the global population? This alone means we'll have only a small percent of the top-talent natively.
We probably have a higher percent in actuality because our wealth allows more people to go to higher education, whereas large swaths of the world are prevented from reaching their potential, either through poverty, health, or non-free governments. This is a huge shame; I can only imagine the scientific progress and quality of life improvements we'd make if everyone were allowed to live up to their full potential.
You can train people all you want, it won't necessarily make them smarter.
My team at work has five engineers and a manager. I'm the only one that was born in the US. Some of them have become citizens and others are here on visas. They are extremely smart and know their shit. There is a shortage of top-notch talent, and the only way for a company to remain competitive is to hire people from outside the US. In my opinion it is better to bring them here to work than to set up an office in their native country (offshore) because the employees make more and they spend most of it within the US. That's a net win.
I'm worried by the increasing number of stories on/. up in arms about companies bringing in *gasp* foreigners. America was founded by non-natives and our economic strength comes from the thousands of immigrants who come here for a better life by getting good jobs or starting businesses.
The contempt for the foreigners coming here on H1-B visas, and the companies that hire them, disgusts me. What makes you any better or more deserving than these people? The fact that you were born in the US? Please. These people have the should have the same right as all of us to come here and be successful. By preventing people from immigrating, especially talented, smart people, we are damaging the future of this country. The ability to attract the best and the brightest to come here is one of our greatest strengths. Erecting barriers to trade and enacting protectionism, especially during this economy, will lead to our downfall as a nation.
The economy isn't a zero-sum game. Allowing foreigners to come here to work enhances their life and the life of those in this country. If you believe you are inherently more entitled to a job than someone from another country, just because you were born here, then you are a xenophobic prick.
I came across Mr. Show on YouTube. I think I was linked there by some website. I ended up watching a bunch of the clips. A few months later I went and bought the whole series on DVD through Amazon. I would have never known about the show if it wasn't for YouTube.
Well, the storage cost per gigabyte-month is $0.15 and the storage cost for transferring a gigabyte outbound is $0.17, so if you download the stored data once per month then transfer is the dominant cost.(1)
Someone is always paying for the cost of storage and transfer. Before the person who owns the bucket would pay for both, but now they can make the accessor pay for the transfer. Amazon isn't offering either for free, so as long as they are making a marginal profit on storage and transfer, they aren't eating any loss.
In that case I agree with toddestan. The memory market is too competitive for that. I believe a bunch of memory manufacturers are hurting this year (and their stock is being pummeled) because their margins on flash chips are extremely small. The price differentiation will occur based on the performance of the firmware and controller. The memory is dirt cheap, but the manufacturers with good firmware (such as Intel) will be able to provide a price premium until other SSD manufacturers catch up.
Really? Due to the physical moving components of a hard disk, they tend to bottom out at $30. With smaller disks, you get diminishing returns because the price of the moving parts dominates. With SSD, however, the price scales fairly linearly all the way to the bottom. Applications that don't need much storage (such as netbooks or low end laptop/desktops) will move to SSD when the price SSDs for the necessary amount of storage is less than $30. If you only need 30 GB in a netbook, SSD is a more logical choice.
Also, it depends on how you quantify priced below regular drives. I assume you are talking about $/GB. However, if you look at $/IOPS (IOs per second), then Intel's SSD is way cheaper than any hard disk. SSDs are going to quickly take over database servers in 2009.
Outside of one instance where it launched 250 XL nodes, it seems to be performing pretty well. Their software takes into account a large number of data points (30-50) when deciding to scale up or down. It also takes into account the average launch time of instances, so it can be ahead of the curve, while at the same time not launching more than it needs.
Contemporary libertarians remind me of children who never learned to share.
I consider myself a libertarian and I donate to philanthropic causes. However, I believe that the free market is a better way to organize the distribution of the my wealth and that I have a better idea of where to donate my money than the government. You want to take my money and give it to people who you believe are worthy. I want everyone to keep their money and (optionally) give it to organizations they individually believe are worthy. Big difference there, see?
I should have been more specific/clear. If you read do a full read of a terabyte disk a dozen times, you are likely to see an unrecoverable read error:
"Typically, [Unrecoverable Error Rate (UER) for read operations] will be 1 per 10^14 bits read for consumer class drives and 1 per 10^15 for enterprise class drives. This can be alarming, because you could also say that consumer class drives should see 1 UER per 12.5 TBytes of data read."
That quote is from a Sun blog that has lots of information about Mean Time To Data Loss. His other posts are interesting as well.
The individual functions map and reduce are quite standard. The innovation here is the systems work they've done to make it work on such a large scale. All the programmer needs to worry about is implementing the two functions, they don't have to worry about distributing the work, ensuring fault tolerance, or anything else for that matter. That is the innovation.
They mention in the article that if you try and sort a petabyte you WILL get hard disk and computer failures. Hell, you can only read a terabyte hard disk a few times before you encounter unrecoverable errors. The system for executing those maps and reduces is what is important here. The important parts are in the design details, such as dealing with stragglers. If you have 4000 identical machines, you won't necessarily get equal performance. If a few of those machines have a bit flipped and started without disk cache, they might see a huge decrease in read/write performance. The system needs to recognize this and schedule the work differently. That can make a huge difference in execution time. If you graph the percentile complete of a MR job, you'll often see that it quickly reaches 95% and then plateaus. The last 5% may take 20% of the time, and good scheduling is required to bring this time down.
But like I said, the innovation isn't in the idea of using a Map and Reduce function, it is the system that executes the work.
Almost, but not quite. MapReduce has a slightly different format than just map() and reduce(). Here is the signature of map and reduce from a theoretical functional language:
Slashdot Mirror
I think you should read that as:
(Mac with an Intel processor) or (a Power PC G5, G4, or G3 processor and built-in FireWireî)
and not
(Mac with an Intel processor or a Power PC G5, G4, or G3 processor) and (built-in FireWireî)
I'd be skeptical of someone, especially in IT, who has no online presence. I believe it is good to build up a brand around your name on the internet. You should be in control of what potential future employers see when they Google you. It is better for you to be in the top spot for your name then someone else talking about you, or something with the same name.
When I first joined the internet, I asked my parents if I could have a website on Geocities. They said no. I didn't listen and went behind their back to create it. I didn't have any info besides my first name, so there was no real harm (and I was smart enough not to meet up alone with random strangers, not that I was ever propositioned).
In 2001 I bought a domain named after me and blogged on it (although, it took years for me to admit it was a blog, since those had a bad stigma attached :) ). More recently I've posted the occasional technical entry. Because of this I've been cold called (well, e-mailed) from major companies asking if I want to interview.
It is 2009, nearly everyone has some sort of online presence now. It is unlikely you'll be targeted just for having your name out there. It is much more likely bad things will happen when a company you deal with is hacked and your information is stolen that way. Plus, you can use your presence as a defense. If I wasn't the top result on Google for my name, people might think I was an anti-semite author.
Andrew Hitchcock
This isn't Wikipedia, we call them "KB" here. Thanks.
I pay income tax, but barely ever use freeways (mainly when I'm on the bus). Why should I subsidize your use of the roads? Also, congestion tolling makes the roads more efficient and keeps the average speed higher.
When you draw the internet while diagramming on a whiteboard, what do you draw? Most people draw a cloud as an abstraction for the stuff "out there". I've never seen anyone draw a spiderweb when drawing a diagram that includes the internet.
I've tried to explain the cloud to slashdotters before. If you don't like the word cloud, you don't have to call it that. A less buzzword-y and perhaps more accurate term would be "utility computing". Turn on the faucet and out comes your data.
Yeah, I noticed that this morning when I read about the investment. They closed a bunch of older facilities in Asia, laying off the workers, and are building the new fancy fabs in the US (and creating high paying jobs in the process).
Of course, the next thing that came to my mind is whether Slashdot would cover that aspect of the story. Sure enough, Slashdot's summary completely disregards that Intel is creating jobs in America. I suspect there are two reasons for this: 1. It hurts Slashdot's agenda if they report about companies insourcing, readers should only know about outsourcing by "the evil corporations". 2. Because Intel is the big bad wolf and we can't report anything good they do.
It's not my fault that company didn't do it's due diligence. If you lied on your resume and were actually incompetent, I wouldn't expect you to get past the phone screen, at least where I work.
Umm, hate to burst your bubble, but I was being pro-American. This country has a long history of being built by immigrants and I want to keep it that way.
I know other countries have protectionist policies, do you notice how they are not as successful as the US? We are not the biggest economic power in the world in spite of our open borders, but because of our open borders.
If the company is on the brink of disaster and is receiving bailout money in order to save the company (which I oppose), then they should do everything in their power to stay afloat. If the H1-B workers are providing a better deal than Americans, or have skills that can not be replaced, then getting rid of them to support American jobs endangers the company and goes against the whole bailout effort.
Or maybe Americans have an inflated view of their own ability. I'm humbled by the foreigners and H1-B visa holders I work with.
The reality, particularly in the tech industry, is that non-Americans are leaders in the various fields. Pick up any industry-related journal, and 90% of the articles are by people of non-American decent.
Very true. This is to be expected because America makes up only, what, 4% of the global population? This alone means we'll have only a small percent of the top-talent natively.
We probably have a higher percent in actuality because our wealth allows more people to go to higher education, whereas large swaths of the world are prevented from reaching their potential, either through poverty, health, or non-free governments. This is a huge shame; I can only imagine the scientific progress and quality of life improvements we'd make if everyone were allowed to live up to their full potential.
You can train people all you want, it won't necessarily make them smarter.
My team at work has five engineers and a manager. I'm the only one that was born in the US. Some of them have become citizens and others are here on visas. They are extremely smart and know their shit. There is a shortage of top-notch talent, and the only way for a company to remain competitive is to hire people from outside the US. In my opinion it is better to bring them here to work than to set up an office in their native country (offshore) because the employees make more and they spend most of it within the US. That's a net win.
I'm worried by the increasing number of stories on /. up in arms about companies bringing in *gasp* foreigners. America was founded by non-natives and our economic strength comes from the thousands of immigrants who come here for a better life by getting good jobs or starting businesses.
The contempt for the foreigners coming here on H1-B visas, and the companies that hire them, disgusts me. What makes you any better or more deserving than these people? The fact that you were born in the US? Please. These people have the should have the same right as all of us to come here and be successful. By preventing people from immigrating, especially talented, smart people, we are damaging the future of this country. The ability to attract the best and the brightest to come here is one of our greatest strengths. Erecting barriers to trade and enacting protectionism, especially during this economy, will lead to our downfall as a nation.
The economy isn't a zero-sum game. Allowing foreigners to come here to work enhances their life and the life of those in this country. If you believe you are inherently more entitled to a job than someone from another country, just because you were born here, then you are a xenophobic prick.
a Chrome checkbox in the toolbar that automatically removes all the wikipedia entries from a google search
And replaces them with links to Knol!
I came across Mr. Show on YouTube. I think I was linked there by some website. I ended up watching a bunch of the clips. A few months later I went and bought the whole series on DVD through Amazon. I would have never known about the show if it wasn't for YouTube.
Amazon MP3 is available in the UK now.
And if you are annoyed by how US-centric many big companies are, then maybe you should move here and reap the benefits :)
Fine, you can accept $1.29 for the latest hits from Apple, but I'll stick with Amazon, where the top 100 tracks are 79 cents.
Well, the storage cost per gigabyte-month is $0.15 and the storage cost for transferring a gigabyte outbound is $0.17, so if you download the stored data once per month then transfer is the dominant cost.(1)
Someone is always paying for the cost of storage and transfer. Before the person who owns the bucket would pay for both, but now they can make the accessor pay for the transfer. Amazon isn't offering either for free, so as long as they are making a marginal profit on storage and transfer, they aren't eating any loss.
(1) Both numbers are for the base tier.
In that case I agree with toddestan. The memory market is too competitive for that. I believe a bunch of memory manufacturers are hurting this year (and their stock is being pummeled) because their margins on flash chips are extremely small. The price differentiation will occur based on the performance of the firmware and controller. The memory is dirt cheap, but the manufacturers with good firmware (such as Intel) will be able to provide a price premium until other SSD manufacturers catch up.
Really? Due to the physical moving components of a hard disk, they tend to bottom out at $30. With smaller disks, you get diminishing returns because the price of the moving parts dominates. With SSD, however, the price scales fairly linearly all the way to the bottom. Applications that don't need much storage (such as netbooks or low end laptop/desktops) will move to SSD when the price SSDs for the necessary amount of storage is less than $30. If you only need 30 GB in a netbook, SSD is a more logical choice.
Also, it depends on how you quantify priced below regular drives. I assume you are talking about $/GB. However, if you look at $/IOPS (IOs per second), then Intel's SSD is way cheaper than any hard disk. SSDs are going to quickly take over database servers in 2009.
I posted this as a comment on the blog post, but I'm copying it here as well:
http://blogs.smugmug.com/don/2008/06/03/skynet-lives-aka-ec2-smugmug/
Outside of one instance where it launched 250 XL nodes, it seems to be performing pretty well. Their software takes into account a large number of data points (30-50) when deciding to scale up or down. It also takes into account the average launch time of instances, so it can be ahead of the curve, while at the same time not launching more than it needs.
Contemporary libertarians remind me of children who never learned to share.
I consider myself a libertarian and I donate to philanthropic causes. However, I believe that the free market is a better way to organize the distribution of the my wealth and that I have a better idea of where to donate my money than the government. You want to take my money and give it to people who you believe are worthy. I want everyone to keep their money and (optionally) give it to organizations they individually believe are worthy. Big difference there, see?
I should have been more specific/clear. If you read do a full read of a terabyte disk a dozen times, you are likely to see an unrecoverable read error:
"Typically, [Unrecoverable Error Rate (UER) for read operations] will be 1 per 10^14 bits read for consumer class drives and 1 per 10^15 for enterprise class drives. This can be alarming, because you could also say that consumer class drives should see 1 UER per 12.5 TBytes of data read."
That quote is from a Sun blog that has lots of information about Mean Time To Data Loss. His other posts are interesting as well.
The individual functions map and reduce are quite standard. The innovation here is the systems work they've done to make it work on such a large scale. All the programmer needs to worry about is implementing the two functions, they don't have to worry about distributing the work, ensuring fault tolerance, or anything else for that matter. That is the innovation.
They mention in the article that if you try and sort a petabyte you WILL get hard disk and computer failures. Hell, you can only read a terabyte hard disk a few times before you encounter unrecoverable errors. The system for executing those maps and reduces is what is important here. The important parts are in the design details, such as dealing with stragglers. If you have 4000 identical machines, you won't necessarily get equal performance. If a few of those machines have a bit flipped and started without disk cache, they might see a huge decrease in read/write performance. The system needs to recognize this and schedule the work differently. That can make a huge difference in execution time. If you graph the percentile complete of a MR job, you'll often see that it quickly reaches 95% and then plateaus. The last 5% may take 20% of the time, and good scheduling is required to bring this time down.
But like I said, the innovation isn't in the idea of using a Map and Reduce function, it is the system that executes the work.
Almost, but not quite. MapReduce has a slightly different format than just map() and reduce(). Here is the signature of map and reduce from a theoretical functional language:
map(): A* -> B*
reduce(): B* -> C
Whereas in MapReduce:
map: (K, V)* -> (K1, V1)*
reduce: (K1, (V1)*)* -> (K2, V2)*
I think that is mostly accurate. Read more accurate/detailed report in MapReduce revisited[PDF].