I feel like competition is the problem and maybe you point out the right argument. Where I live (Charlotte, NC) there is basically no competition for fast internet (over 5Mbps), there is one cable operator (time warner), one phone operator (ATT); that's it. When I lived in France, pretty much all operators were present in all cities. But in france, the local loop is not tied to an operator. But in the US, the local loop is tied to an operator, so the operator is king and there is less competition.
I dont really buy the population density argument for justifying the cost because most of the cost is in the local loop and not in the main infrastructure.
I am the first one to hate social networks for their privacy concerns. But what we actually need is social scoring for app reviews and all. You could build a recommendation based on the friendship relationships. Since I am not going to be friend with "spamer1234", the impact of their score whould be very low. Of course, you need something more than just take the average of your friends. You need something closer to "personalized page rank" or a graph based inference system. But anything in that matter would essentially solve the spamer problems.
I picked up your paper on slashdot actually (and probably from one of your posts). But I had never realized that you (as in the author of the paper) was posting on slashdot. That's pretty cool.
What is interesting about diverse double compiling is that it is feasible by a large technically savvy entity (but difficult in practice).
I must say that the trusting trust problem has become huge in the recent years. We pretty much have to rely on somebody for compiling our software and not just compilers. Compiling your own firefox or chromium or vlc is close to being impossible. There are too many dependences or the build chain is too complicated. (A friend of mine run out of disk space compiling chromium...) But assuming that you trust debian (or anyother large entity with technical expertise), you can rely on debian to make sure trusting trust problems do not appear. (And provided writing a C compiler is a standard student project, it should be easy to get multiple compilers to make sure none is backdoored.)
There is something else. Bruce Schneier is a public figure in the cryptography area. Scientist need to fight for money and a large part of it comes from reputation and fame. If Bruce said something that appear wrong to security researchers, they would speak up, just to be "the one that knows better".
"Even the compiler can be compromised. Ken Thompson showed that."
Well, double compiling techniques can be used to certify a compiler. (Though it actually assume that you have access to an other safe compiler, which is a little bit complicated, but doable)
If we could do it with 99% efficiency then the cost of being lazy would be acceptable. But 15% seems like a lot. (And frankly, I am kind of doubting that you can actually get that much.) Essentially it means that the power is 15% more expensive (at least) just for the luxury of not plugin a cable. If I could not get out of my car but still get gas at a 15% surcharge, well, I'd get out of my car and pump my own gas...
TFA does not mention any GPU core. Did I miss something?
People often say that the core does not matter, the memory bandwidth does. That is only true for applications doing "bookkeeping"
The application defines how much "computation per byte" it does and you want an architecture that can play well with a wide range of "computation per byte". When the kernels are a little bit complicated, getting the right data to the core can take many instructions (so many cycles even if the data fits in L1 cache), so you want the "computation" part of the kernel to use as little instructions as possible. That way the "inner loops" of your kernel can saturate the memory controller.
If your "inner loop" takes too many instruction, you won't be able to saturate the memory controller. Then all the bandwidth in the world will not help you.
That means little. can it do vectout = cosine(vectin);? In how many cycle? can it do multiply-add in a single instruction and in a single cycle? Can it do that in EACH pipe?
That is a bit strange to count Integer operations. Most of the computations one do nowadays are mostly floating point. Also,there is no mention of memory bandiwdth and cache size. Though I'll stay tuned.
Seeing the notification email in slashdot, I recognized who I had replied to. Team Twizzer seems inactive and there have been little updates to your blog. What kind of cool stuff are you up to? (Beside walking around the world and taking raidoactivity measurements.)
That could work. But you need to give people incentive in release an exclusive version. Red Hat or Canonical could have the fund necessary to generate such an exclusive games. Or maybe one such effort could be crowdfunded. But nobody is going to develop a $10 million game and release it only on Linux without a significant incentive.
Overall, he is right. I bought gaming systems for a single game. For instance, I bought the Wii just to play FireEmblem. I was already interested but it is only on FE's release that I bought it. Once I had it, I played other things as well. But a single exclusive game I was interested in convinced me to buy.
I think that the same thing could happen for Linux. But I am no sure it will ever happen. Will there ever be a Linux exclusive game? If you were a game developper, would you commit to realse your fancy need AAA game ONLY on Linux and not on Windows? That seems like a stupid move unless the company receives a ridiculous amount of money cash for the exclusivity.
I don't think that compatibility with Linux will be sufficient to see an "explosion", it is an exclusivity one need. And being linux exclusive look a lot like betting on a three legged horse.
I am all with you on this. I was mostly designing approxiamtion algorithm for weird-ass scheduling problem in the past. Recently I have been hired by a university to conduct research on "Big Data" and the problems faced have little to do with Combinatorial Optimization anymore.
The problem is more on the lines of: Here are the medical record from 20 hospital on the east coast. You got everything: patient files, radios, MRI, blood test, cardios, doctor notes, nurses notes (as text files or images). Your job is to figure "stuff" out of that mess.
Or: Here comes all the logs of my comercial website, the price of all items, how the website was laid out, where the user clicked and scrolled, the commercial displayed on the site and the log of what they bought. How can I improve my bottom line?
But I agree with you that the term "Big Data" have been so used recently that we are drowning in it. And everybody tries to claim that they are doing it to benefit from the hype. Just look at how many conferences and workshop are doing big data (in their title) now while there was none two years ago. (Hint: I stopped counting at 5 conferences and 12 workshops.)
"Big Data" is actually a pretty clear problem. It is not when your problems grows faster than your ressources. It is when you are faced with processing massive amount of unstructured data that flow in your system. The data might be untrustable or forged or incomplete. You might want to read on the "Vs of Big Data", it describe pretty well the type of problem it encompasses. Obvisouly not everybody faces such difficult to process data.
Essentially it is a big word you put to describe most modern data analytics. But most NP-Hard problem do not qualify as they are usually fairly static and the input of the problem is usually quite accurate (if the input is inaccurate, you'll end up doing heuristics anyway because you can't do much better anyway).
But even some classical polynomial problem are relevant to Big Data. For instance, pick a regular problem in data analytics such as the "most common subset in transactions" and drop 10% more data to your current problem. How do you "update" your solution to the problem without recomputing everything? (I never studied that one, so maybe it is easy.)
I agree with you on the fact that these problems are not new. Though their reach as grown tremendously in the last 5/10 years with the democratization electronic records.
No that is not the point. The point of OpenAccess papers is to allow a larger communicatino of the papers by removing the barrier of ridiculously high access fees. Accessing a single paper can cost $50 for a researcher that do not have the proper subscription. OpenAccess journals are mainly designed to take the editors and publishers which ask for a ridiculously high publication fee. or cost of access.
Open Access does not mean that anything get published in there. Though as a reviewer for many computer science journal, I can guarantee you that everybody can publish in there... assuming the level of contribution and style are up to standard of scientific method and writing. That is a very difficult thing to achieve for a non academic because of the time comitment in "learning" how to write these papers.
cores or processors, are typically not that fast at doing graphical rendering. GPUs are typically much more efficinet at that task. (Hey that's what they have been built for.)
I have no problems with hosting my own software, or setting up something in EC2 or a VPS. Others might want to pay others to handle things for them, perhaps with advertising. However, none of this is possible when FOSS effort goes almost exclusively into applications that only work over X11. We're still fighting against the Microsoft of the 90s and the world has moved on...
I think that this comment is really unfair to the opensource community. There have been vast projects to get web technologies up and running. The apache webserver, php, mysql, most nosql stuff are in the infrastructure level. On the application level, there have been webmails like squirrelmail, facebook alternatives with diaspora, Instant Messenger alternative based on jabber protocols, skype replacement with sip client like ekiga, flicker/youtube replcament with media goblin, dropbox replacement with owncloud. There have been efforst to package that easily with freedombox.
You (and GP) make it sound like the free software movement has been sitting on a bench idle. But I find that there have been lots of project designed to "free" the user from the domination of large actors. Maybe they are not good enough, maybe they need more work. I am a user of owncloud, and clearly that needs more work. But it is usable and it allows me not to use google calendar, google news reader and dropbox.
These projects were all presented on Slashdot, they are not small obscure stuff. Did you check them out? Did you use them? They were looking for money, did you help them?
I tend to agree that we want small companies and small government to avoid the problem of having one entity that overpowers the rest.
The thing is, many endeavours require global supervision, global management. And I am not sure how to achieve that. Nuclear powerplants are ridiculously expensive. Phone infrastructure is ridiculously expensive. The LHC is ridiculously expensive. Very few people will argue we should not have build them. But with low power governments and companies with limited budget, I am not sure how to reach that. Also many things to be made cheap require a significant scale. I am not sure how that fits in.
Slashdot Mirror
I feel like competition is the problem and maybe you point out the right argument. Where I live (Charlotte, NC) there is basically no competition for fast internet (over 5Mbps), there is one cable operator (time warner), one phone operator (ATT); that's it. When I lived in France, pretty much all operators were present in all cities. But in france, the local loop is not tied to an operator. But in the US, the local loop is tied to an operator, so the operator is king and there is less competition.
I dont really buy the population density argument for justifying the cost because most of the cost is in the local loop and not in the main infrastructure.
Well, you can't trust any software you do not have the source code of. So of course you can not trust the MS compiler.
I am the first one to hate social networks for their privacy concerns. But what we actually need is social scoring for app reviews and all. You could build a recommendation based on the friendship relationships. Since I am not going to be friend with "spamer1234", the impact of their score whould be very low. Of course, you need something more than just take the average of your friends. You need something closer to "personalized page rank" or a graph based inference system. But anything in that matter would essentially solve the spamer problems.
I picked up your paper on slashdot actually (and probably from one of your posts). But I had never realized that you (as in the author of the paper) was posting on slashdot. That's pretty cool.
What is interesting about diverse double compiling is that it is feasible by a large technically savvy entity (but difficult in practice).
I must say that the trusting trust problem has become huge in the recent years. We pretty much have to rely on somebody for compiling our software and not just compilers. Compiling your own firefox or chromium or vlc is close to being impossible. There are too many dependences or the build chain is too complicated. (A friend of mine run out of disk space compiling chromium...) But assuming that you trust debian (or anyother large entity with technical expertise), you can rely on debian to make sure trusting trust problems do not appear. (And provided writing a C compiler is a standard student project, it should be easy to get multiple compilers to make sure none is backdoored.)
There is something else. Bruce Schneier is a public figure in the cryptography area. Scientist need to fight for money and a large part of it comes from reputation and fame. If Bruce said something that appear wrong to security researchers, they would speak up, just to be "the one that knows better".
"Even the compiler can be compromised. Ken Thompson showed that."
Well, double compiling techniques can be used to certify a compiler. (Though it actually assume that you have access to an other safe compiler, which is a little bit complicated, but doable)
http://arxiv.org/abs/1004.5534
If we could do it with 99% efficiency then the cost of being lazy would be acceptable. But 15% seems like a lot. (And frankly, I am kind of doubting that you can actually get that much.) Essentially it means that the power is 15% more expensive (at least) just for the luxury of not plugin a cable. If I could not get out of my car but still get gas at a 15% surcharge, well, I'd get out of my car and pump my own gas...
TFA does not mention any GPU core. Did I miss something?
People often say that the core does not matter, the memory bandwidth does. That is only true for applications doing "bookkeeping"
The application defines how much "computation per byte" it does and you want an architecture that can play well with a wide range of "computation per byte". When the kernels are a little bit complicated, getting the right data to the core can take many instructions (so many cycles even if the data fits in L1 cache), so you want the "computation" part of the kernel to use as little instructions as possible. That way the "inner loops" of your kernel can saturate the memory controller.
If your "inner loop" takes too many instruction, you won't be able to saturate the memory controller. Then all the bandwidth in the world will not help you.
That means little. can it do vectout = cosine(vectin);? In how many cycle? can it do multiply-add in a single instruction and in a single cycle? Can it do that in EACH pipe?
That is a bit strange to count Integer operations. Most of the computations one do nowadays are mostly floating point. Also,there is no mention of memory bandiwdth and cache size. Though I'll stay tuned.
Interesting error indeed.
Seeing the notification email in slashdot, I recognized who I had replied to. Team Twizzer seems inactive and there have been little updates to your blog. What kind of cool stuff are you up to? (Beside walking around the world and taking raidoactivity measurements.)
randall munroe actually put up a fairly insightful chart of radiation levels: http://xkcd.com/radiation/
That could work. But you need to give people incentive in release an exclusive version. Red Hat or Canonical could have the fund necessary to generate such an exclusive games. Or maybe one such effort could be crowdfunded. But nobody is going to develop a $10 million game and release it only on Linux without a significant incentive.
Yes, a delayed exclusive would probably be enough. That is what OUYA tried with its proram to get 6 month exclusive games.
Overall, he is right. I bought gaming systems for a single game. For instance, I bought the Wii just to play FireEmblem. I was already interested but it is only on FE's release that I bought it. Once I had it, I played other things as well. But a single exclusive game I was interested in convinced me to buy.
I think that the same thing could happen for Linux. But I am no sure it will ever happen. Will there ever be a Linux exclusive game? If you were a game developper, would you commit to realse your fancy need AAA game ONLY on Linux and not on Windows? That seems like a stupid move unless the company receives a ridiculous amount of money cash for the exclusivity.
I don't think that compatibility with Linux will be sufficient to see an "explosion", it is an exclusivity one need. And being linux exclusive look a lot like betting on a three legged horse.
I am all with you on this. I was mostly designing approxiamtion algorithm for weird-ass scheduling problem in the past. Recently I have been hired by a university to conduct research on "Big Data" and the problems faced have little to do with Combinatorial Optimization anymore.
The problem is more on the lines of: Here are the medical record from 20 hospital on the east coast. You got everything: patient files, radios, MRI, blood test, cardios, doctor notes, nurses notes (as text files or images). Your job is to figure "stuff" out of that mess.
Or: Here comes all the logs of my comercial website, the price of all items, how the website was laid out, where the user clicked and scrolled, the commercial displayed on the site and the log of what they bought. How can I improve my bottom line?
But I agree with you that the term "Big Data" have been so used recently that we are drowning in it. And everybody tries to claim that they are doing it to benefit from the hype. Just look at how many conferences and workshop are doing big data (in their title) now while there was none two years ago. (Hint: I stopped counting at 5 conferences and 12 workshops.)
"Big Data" is actually a pretty clear problem. It is not when your problems grows faster than your ressources. It is when you are faced with processing massive amount of unstructured data that flow in your system. The data might be untrustable or forged or incomplete. You might want to read on the "Vs of Big Data", it describe pretty well the type of problem it encompasses. Obvisouly not everybody faces such difficult to process data.
Essentially it is a big word you put to describe most modern data analytics. But most NP-Hard problem do not qualify as they are usually fairly static and the input of the problem is usually quite accurate (if the input is inaccurate, you'll end up doing heuristics anyway because you can't do much better anyway).
But even some classical polynomial problem are relevant to Big Data. For instance, pick a regular problem in data analytics such as the "most common subset in transactions" and drop 10% more data to your current problem. How do you "update" your solution to the problem without recomputing everything? (I never studied that one, so maybe it is easy.)
I agree with you on the fact that these problems are not new. Though their reach as grown tremendously in the last 5/10 years with the democratization electronic records.
No that is not the point. The point of OpenAccess papers is to allow a larger communicatino of the papers by removing the barrier of ridiculously high access fees. Accessing a single paper can cost $50 for a researcher that do not have the proper subscription. OpenAccess journals are mainly designed to take the editors and publishers which ask for a ridiculously high publication fee. or cost of access.
Open Access does not mean that anything get published in there. Though as a reviewer for many computer science journal, I can guarantee you that everybody can publish in there... assuming the level of contribution and style are up to standard of scientific method and writing. That is a very difficult thing to achieve for a non academic because of the time comitment in "learning" how to write these papers.
not to mention the headaches if you work with any other company that isn't using Open Office files themselves
It is gendarmerie we are talking about. Why are you making the assumption that they are working? Let alone with external people.
They did multiple pilot test and all went all. I don't think they will migrate back anytime soon.
cores or processors, are typically not that fast at doing graphical rendering. GPUs are typically much more efficinet at that task. (Hey that's what they have been built for.)
You would not want them to chip!
I have no problems with hosting my own software, or setting up something in EC2 or a VPS. Others might want to pay others to handle things for them, perhaps with advertising. However, none of this is possible when FOSS effort goes almost exclusively into applications that only work over X11. We're still fighting against the Microsoft of the 90s and the world has moved on...
I think that this comment is really unfair to the opensource community. There have been vast projects to get web technologies up and running. The apache webserver, php, mysql, most nosql stuff are in the infrastructure level. On the application level, there have been webmails like squirrelmail, facebook alternatives with diaspora, Instant Messenger alternative based on jabber protocols, skype replacement with sip client like ekiga, flicker/youtube replcament with media goblin, dropbox replacement with owncloud. There have been efforst to package that easily with freedombox.
You (and GP) make it sound like the free software movement has been sitting on a bench idle. But I find that there have been lots of project designed to "free" the user from the domination of large actors. Maybe they are not good enough, maybe they need more work. I am a user of owncloud, and clearly that needs more work. But it is usable and it allows me not to use google calendar, google news reader and dropbox.
These projects were all presented on Slashdot, they are not small obscure stuff. Did you check them out? Did you use them? They were looking for money, did you help them?
Actually, how much power does it need to operate? There must be some energy cost per time unit. But I could not figure it out.
I tend to agree that we want small companies and small government to avoid the problem of having one entity that overpowers the rest.
The thing is, many endeavours require global supervision, global management. And I am not sure how to achieve that. Nuclear powerplants are ridiculously expensive. Phone infrastructure is ridiculously expensive. The LHC is ridiculously expensive. Very few people will argue we should not have build them. But with low power governments and companies with limited budget, I am not sure how to reach that. Also many things to be made cheap require a significant scale. I am not sure how that fits in.