The blurb hit on a fundamental reason data mining is still at (or beyond) the horizon...defining relations between the various elements is hard. Available datasets are not themselves in anything like normal relational form, and so have potential internal inconsistencies. And that gets in the way before you even have the chance to try to form intelligent inferences based on relations between data sets, which of course are terribly inconsistent.
Consider the following boring but difficult task I was given: two large organizations were to merge, each with a portfolio of about 100,000 items. Each item had a short history, some descriptive information, and some data such as internal quality ratings or sector assignments. This data was available (for various reasons) as big CSV file dumps. Questions to answer were: (1) how much overlap did the portfolios have? (2) were the sector distributions similar?
These are very simple, concrete questions. But you can imagine that since the categorizations differed, and descriptors differed within the CSV files, let alone between the two, the questions were difficult to answer. It required a lot of approximate matching, governed intelligently (or so I flatter myself).
Contrast this situation with what people typically think of as data-mining: answering interesting questions, and you can appreciate that without a whole lot of intelligence, artificial or otherwise, those questions will be unanswerable.
OK -- I just tried Eclipse and PyDev for the day. I found the process of creating Eclipse projects corresponding to my existing working directories slightly byzantine, but not too bad.
The PyDev addins for Eclipse have some nice features to them, such as code completion, pyLint, outlines and the like. I did not manage to get code completion or Tasks to work -- I hope/imagine another half-day or so of futzing around might solve that problem. I thought SubClipse (the subverson addin) was also very nice.
However, before you try switching to Eclipse have a look at Bug 14654 and Bug 5138. Are you really going to find an editor usable when it fails to get double-clicking right? You might think the problems will be fixed soon, but before you assume that consider that folks have known about the issue for four years!
I'm afraid I will still be counted among those people who use a text editor.
To expound on the problem with mid-size companies, imagine yourself in the position of the telecommuter. You, Mr. Special, have already fought (and barely won) an uphill battle with your managers, convincing them that you will be a valuable employee despite telecommuting. Now, you also need them to create a new corporate entity, just for Mr. Special.
A small company will know you and need you enough to go through the minimal hassle. A large one can create the new entity easily, if they haven't done so already for unrelated reasons, and in addition can probably amortize the cost and bother over many telecommuters from your state. The midsize one? I fear the bureaucracy and politics there!
I think it's quite debatable whether the telecommuter is exploiting NY or NY is exploiting the telecommuter. I lean toward the latter.
That aside, although I believe you are quite right that some states might want to try to write their tax codes to fight this scheme, it would be operationally quite difficult for them. Their current operational (rather than legal) strength lies in the fact that W-2's go to them from the state of employment.
Once those W-2's are going instead to (in this case) Tennessee, they no longer have any record at all of the employee's existence. The person is at this point a resident of Tennessee and employed entirely in Tennessee. New York would be no more likely to receive his/her records than they would be to receive mine, working and living as I do in Chicago.
For companies that are either small enough to nimbly do so, or large enough to handle the red tape for a large number of employees, it seems the solution to this problem would be to create new corporate entities in each state containing some of their telecommuters. The telecommuters would then be made employees of their local corporate entities.
That really screws the 10-500 employee businesses that make up the backbone of the US economy, of course. They have too much infrastructure to just go ahead and do this for the fairly nominal setup cost a small company would encounter, but too little to already be incorporated in multiple locations.
I think you underestimate just how little influence ICANN can legally wield outside the U.S.
I made no legal arguments at all.
If China mandates their ISPs use local root servers do you really think a significant percentage of people will not use that DNS server
Yes. Yes, I do.
90% of people using a DNS server do not even know what it is
Quite true -- I would guess 99%. But that's because right now, it works. The Chinese (among others) will not be able to resist the temptation to meddle, particularly by blackholing sites they dislike. The public will then work around the problem on their own -- as they already do, using proxies, for blocked IPs.
why would people switch their root server to a different one if their current one is already working?
Indeed. Which is why everyone who understands the issue thinks this EU push is so silly.
You know that a number of ICANN regulated servers are physically located in foreign countries and subject to the laws there right?
Yes.
That means a number of them will instantly be governed by any EU/UN directives that come down the pipe
No, it doesn't. The servers themselves are private property, which developed economies do not, as a matter of course, seize.
I suggest learning about the topic somewhere other than Slashdot.
There will be issues with both ICANN servers not properly resolving hosts/domains outside the U.S. which most users will interpret as ICANN being broken
That's debatable. Assuming all the alt-roots are unified, you may be right. But then those alt-roots are just providing something extra on top of ICANN. Like I said, I have no idea how much demand there really is for that.
when there is a conflict and an ISP has to decide which is "proper," well most ISPs are not in the U.S. and if the law where they are says they have to provide users with the UN/EU results or lose their common carrier status/face a fine, then they will provide the non-ICANN results or build a hack to provide both
In cases of conflict, users will end up preferring to use ICANN for the current generic TLDs, and then maybe the alt-roots for their relevant ccTLDs (or perhaps even all ccTLDs), depending on the site and the quality of the alt-roots. Certainly people in a place like China will want to be able to resolve sites their government prefers to blackhole.
Remember, this is ultimately about what the end users prefer. No country can easily dictate what nameservers people point their workstations at. If the alt-roots provide better (essentially: extended) resolution than ICANN, and the extra domains are ones people actually care about, the alt-roots will become popular. If the alt-roots screw up, everyone will ignore them as much as possible.
if the US government decides to flex its muscles and remove brazil from the top level domain list of US controller root servers
That's a pretty paranoid if. Countries (like Iraq recently) where this sort of thing might actually be an issue are likely to be in enough chaos that any alt-root is going to be in a quandary about what to do.
So some places outside the US, as is their right, are going to set up their own root servers. This kind of thing has been done many times before. Those other alt-roots have never been very heavily subscribed. Naturally that reference level could change, if other countries mandate that their ISPs use the new alt-roots.
But you know what? To the extent that the data coming out of the latest alt-roots conflict with the ICANN, they will be generally perceived as broken, particularly but not exclusively from the point of view of users in the US. For example, domain names will fail to resolve, or will resolve to the "wrong" place. If the new alt-roots do much of anything differently, users will start pointing their DNS clients at nameservers that resolve up to the ICANN. So for example if China sets up something that won't resolve (say) freechina.net, the individual users will soon learn to point their DNS clients at US nameservers.
The only way I can see these new alt-roots being heavily subscribed is if they make sure they agree with the ICANN everywhere ICANN has a route to a name, and if their use is legally mandated so that ISPs are forced to go through the hassle of changing. If they do that, the only value that they could possibly add would be of including extra domains that resolve for the alt-roots, and that ICANN does not yet have. Is there really a lot of demand for such a thing? I'm not sure.
I appreciate the difficulty of dealing with users installing lots of software, but I have experienced the "lockdown solution" in three different organizations (two of them very large), and feel it worked poorly for me in all of them.
Here's why: (1) Response times. When I made a request for installation of, or permission to install, software needed for my work responsibilities, response times ranged from 45 minutes to a couple days. 45 minutes is little enough time to find something else to do in. A period of days is not. I have yet to encounter a tech desk that can reliably respond to even such a simple request in a timely manner, never losing it.
(2) Interconnections. Those times when I installed a piece of software were often followed shortly thereafter by the need to install some other, related (or substitute) program. That meant another delay of 45 minutes to a couple hours (or more). Chain a few of those along, and you easily waste a day or two.
(3) Questioning and denial. Large organizations have a list of "approved" software and biases toward denying the use of anything not on the list. For example, at one point I had a strong need to do some time series analysis. Appropriate tools for this include SAS, SPSS, Matlab....and GNU R. The first three, since they cost thousands of dollars, would have required cost review, tech review, et cetera. Installation for those types of packages took months. (I think that Matlab took about 3 1/2 months when we bought it). We needed results within a week or so, so R was the obvious choice. But of course, few sysadmins have (and none of ours had) heard of GNU R. Before we could get it installed we went through a long and frustrating round of "what is this?" and "why do you need it?" and "why can't you use X instead". Had the sysadmins just trusted that we had done our research, it would have been far less painful.
It doesn't take more than one or two such experiences for the users to develop a deep distaste for dealing with a lockdown.
I, on the other hand, am long a house and therefore short REITs. I think your position is smarter, but I do enjoy the intangible freedoms of owning my own residence.
With respect to your desired swaps, you might not be able to trade them as a retail investor, but if the more limited tenors are acceptable to you, you can almost certainly buy or sell exchange-traded Eurodollar futures and options.
codergeek42 writes Then how will you print yellow stuff, stupid?
and justforaday writes (with less pointless hostility) And then you can pay an assload for repairs and get a nice stern talking to from the repair guy about how you should never ever ever put the wrong color toner in the wrong bin
I think you're both missing the point. It doesn't matter if the printer is utterly destroyed and never prints again, since this needs to be done just once per model. For example, by an organization like EFF or Consumer Reports.
I understand the marking is done with yellow ink. It seems one would be able to expose a lot of these printers by replacing (or contaminating) the yellow ink with black.
As a member of the Globalized Corporate Oligarchy, let me just say that we have the U.S./Costa Rican/Chinese/Azerbaijanian Shadow Government under our complete control.
I can state unequivocally that 9-11 was not the work of our pawns.
However, our control of the Australian/South African/Polish Shadow Government has always been tenuous at best, and of course they have always had the motivation and resources for these sorts of things.
The rest of the world's Shadow Governments have generally been spending the last few decades implementing our labor reform acts and performing corporate audits, so of course they are beyond suspicion.
We in the Globalized Corporate Oligarchy seek to implement our vision statement across all of our puppets (shadow governments and media machines alike) in order to bring new exciting products that add value for all our stakeholders and customers. We apologize for the inconvenience of this attack.
I'm about to hook up a Python app to a PostgreSQL database. Last year when I checked, I found that psycopg was the best (fastest and most reliable) connector package. I decided that because it was the only one I tried that didn't choke on huge 500MB loan model runs.
Although the conclusion of the Pythagorean theorem was known (long before Pythagoras) by many early civilizations, there's no evidence any of them made it into a theorem by proving it.
a friend of mine got a -3 on a question on a test. The girl sitting next to him got a -1 on the same question with a near identical response. He complained and the situation was resolved by giving the girl a -3 instead of a -1.
Heh. Back when I used to teach calculus, I did this (or at least threatened to do it a lot). I was a real BTFH.
Basically I would be careful never to err on the parsimonious side with partial credit. Frequently, two friends with fairly similar answers would walk up to me with their graded papers, having fairly similar answers but different scores. I would always out that the student with the higher score was the beneficiary of a generous mistake on my part.
"But, if you want me to, I could reduce your score to even things out." No one ever took me up on that offer, no matter how close their friends were. Usually, they left feeling relieved I had not in fact reduced the higher score, and perhaps a little peeved at their friend for putting them at risk.
Slashdot Mirror
The blurb hit on a fundamental reason data mining is still at (or beyond) the horizon...defining relations between the various elements is hard. Available datasets are not themselves in anything like normal relational form, and so have potential internal inconsistencies. And that gets in the way before you even have the chance to try to form intelligent inferences based on relations between data sets, which of course are terribly inconsistent.
Consider the following boring but difficult task I was given: two large organizations were to merge, each with a portfolio of about 100,000 items. Each item had a short history, some descriptive information, and some data such as internal quality ratings or sector assignments. This data was available (for various reasons) as big CSV file dumps. Questions to answer were: (1) how much overlap did the portfolios have? (2) were the sector distributions similar?
These are very simple, concrete questions. But you can imagine that since the categorizations differed, and descriptors differed within the CSV files, let alone between the two, the questions were difficult to answer. It required a lot of approximate matching, governed intelligently (or so I flatter myself).
Contrast this situation with what people typically think of as data-mining: answering interesting questions, and you can appreciate that without a whole lot of intelligence, artificial or otherwise, those questions will be unanswerable.
I'm rather annoyed that "gwerdna" or whatever his name was didn't tell us
Hmm. Maybe we should ask Andrew G?
(Hint: backwards)
>> [...] Linus is an optomist [...]
>Is that a cross between an optimist and an optometrist?
No, it's the ippisote of a pessomist.
OK -- I just tried Eclipse and PyDev for the day. I found the process of creating Eclipse projects corresponding to my existing working directories slightly byzantine, but not too bad.
The PyDev addins for Eclipse have some nice features to them, such as code completion, pyLint, outlines and the like. I did not manage to get code completion or Tasks to work -- I hope/imagine another half-day or so of futzing around might solve that problem. I thought SubClipse (the subverson addin) was also very nice.
However, before you try switching to Eclipse have a look at Bug 14654 and Bug 5138. Are you really going to find an editor usable when it fails to get double-clicking right? You might think the problems will be fixed soon, but before you assume that consider that folks have known about the issue for four years!
I'm afraid I will still be counted among those people who use a text editor.
the types of music I listen to (Classical, "Western Art Music", Jazz, Opera) aren't served well by iTMS anyway.
And poorly served by CDDB, etc., I might add.
To expound on the problem with mid-size companies, imagine yourself in the position of the telecommuter. You, Mr. Special, have already fought (and barely won) an uphill battle with your managers, convincing them that you will be a valuable employee despite telecommuting. Now, you also need them to create a new corporate entity, just for Mr. Special.
A small company will know you and need you enough to go through the minimal hassle. A large one can create the new entity easily, if they haven't done so already for unrelated reasons, and in addition can probably amortize the cost and bother over many telecommuters from your state. The midsize one? I fear the bureaucracy and politics there!
I think it's quite debatable whether the telecommuter is exploiting NY or NY is exploiting the telecommuter. I lean toward the latter.
That aside, although I believe you are quite right that some states might want to try to write their tax codes to fight this scheme, it would be operationally quite difficult for them. Their current operational (rather than legal) strength lies in the fact that W-2's go to them from the state of employment.
Once those W-2's are going instead to (in this case) Tennessee, they no longer have any record at all of the employee's existence. The person is at this point a resident of Tennessee and employed entirely in Tennessee. New York would be no more likely to receive his/her records than they would be to receive mine, working and living as I do in Chicago.
For companies that are either small enough to nimbly do so, or large enough to handle the red tape for a large number of employees, it seems the solution to this problem would be to create new corporate entities in each state containing some of their telecommuters. The telecommuters would then be made employees of their local corporate entities.
That really screws the 10-500 employee businesses that make up the backbone of the US economy, of course. They have too much infrastructure to just go ahead and do this for the fairly nominal setup cost a small company would encounter, but too little to already be incorporated in multiple locations.
I think you underestimate just how little influence ICANN can legally wield outside the U.S.
I made no legal arguments at all.
If China mandates their ISPs use local root servers do you really think a significant percentage of people will not use that DNS server
Yes. Yes, I do.
90% of people using a DNS server do not even know what it is
Quite true -- I would guess 99%. But that's because right now, it works. The Chinese (among others) will not be able to resist the temptation to meddle, particularly by blackholing sites they dislike. The public will then work around the problem on their own -- as they already do, using proxies, for blocked IPs.
why would people switch their root server to a different one if their current one is already working?
Indeed. Which is why everyone who understands the issue thinks this EU push is so silly.
You know that a number of ICANN regulated servers are physically located in foreign countries and subject to the laws there right?
Yes.
That means a number of them will instantly be governed by any EU/UN directives that come down the pipe
No, it doesn't. The servers themselves are private property, which developed economies do not, as a matter of course, seize.
I suggest learning about the topic somewhere other than Slashdot.
There will be issues with both ICANN servers not properly resolving hosts/domains outside the U.S. which most users will interpret as ICANN being broken
That's debatable. Assuming all the alt-roots are unified, you may be right. But then those alt-roots are just providing something extra on top of ICANN. Like I said, I have no idea how much demand there really is for that.
when there is a conflict and an ISP has to decide which is "proper," well most ISPs are not in the U.S. and if the law where they are says they have to provide users with the UN/EU results or lose their common carrier status/face a fine, then they will provide the non-ICANN results or build a hack to provide both
In cases of conflict, users will end up preferring to use ICANN for the current generic TLDs, and then maybe the alt-roots for their relevant ccTLDs (or perhaps even all ccTLDs), depending on the site and the quality of the alt-roots. Certainly people in a place like China will want to be able to resolve sites their government prefers to blackhole.
Remember, this is ultimately about what the end users prefer. No country can easily dictate what nameservers people point their workstations at. If the alt-roots provide better (essentially: extended) resolution than ICANN, and the extra domains are ones people actually care about, the alt-roots will become popular. If the alt-roots screw up, everyone will ignore them as much as possible.
if the US government decides to flex its muscles and remove brazil from the top level domain list of US controller root servers
That's a pretty paranoid if. Countries (like Iraq recently) where this sort of thing might actually be an issue are likely to be in enough chaos that any alt-root is going to be in a quandary about what to do.
So some places outside the US, as is their right, are going to set up their own root servers. This kind of thing has been done many times before. Those other alt-roots have never been very heavily subscribed. Naturally that reference level could change, if other countries mandate that their ISPs use the new alt-roots.
But you know what? To the extent that the data coming out of the latest alt-roots conflict with the ICANN, they will be generally perceived as broken, particularly but not exclusively from the point of view of users in the US. For example, domain names will fail to resolve, or will resolve to the "wrong" place. If the new alt-roots do much of anything differently, users will start pointing their DNS clients at nameservers that resolve up to the ICANN. So for example if China sets up something that won't resolve (say) freechina.net, the individual users will soon learn to point their DNS clients at US nameservers.
The only way I can see these new alt-roots being heavily subscribed is if they make sure they agree with the ICANN everywhere ICANN has a route to a name, and if their use is legally mandated so that ISPs are forced to go through the hassle of changing. If they do that, the only value that they could possibly add would be of including extra domains that resolve for the alt-roots, and that ICANN does not yet have. Is there really a lot of demand for such a thing? I'm not sure.
I appreciate the difficulty of dealing with users installing lots of software, but I have experienced the "lockdown solution" in three different organizations (two of them very large), and feel it worked poorly for me in all of them.
Here's why:
(1) Response times. When I made a request for installation of, or permission to install, software needed for my work responsibilities, response times ranged from 45 minutes to a couple days. 45 minutes is little enough time to find something else to do in. A period of days is not. I have yet to encounter a tech desk that can reliably respond to even such a simple request in a timely manner, never losing it.
(2) Interconnections. Those times when I installed a piece of software were often followed shortly thereafter by the need to install some other, related (or substitute) program. That meant another delay of 45 minutes to a couple hours (or more). Chain a few of those along, and you easily waste a day or two.
(3) Questioning and denial. Large organizations have a list of "approved" software and biases toward denying the use of anything not on the list. For example, at one point I had a strong need to do some time series analysis. Appropriate tools for this include SAS, SPSS, Matlab....and GNU R. The first three, since they cost thousands of dollars, would have required cost review, tech review, et cetera. Installation for those types of packages took months. (I think that Matlab took about 3 1/2 months when we bought it). We needed results within a week or so, so R was the obvious choice. But of course, few sysadmins have (and none of ours had) heard of GNU R. Before we could get it installed we went through a long and frustrating round of "what is this?" and "why do you need it?" and "why can't you use X instead". Had the sysadmins just trusted that we had done our research, it would have been far less painful.
It doesn't take more than one or two such experiences for the users to develop a deep distaste for dealing with a lockdown.
I, on the other hand, am long a house and therefore short REITs. I think your position is smarter, but I do enjoy the intangible freedoms of owning my own residence.
With respect to your desired swaps, you might not be able to trade them as a retail investor, but if the more limited tenors are acceptable to you, you can almost certainly buy or sell exchange-traded Eurodollar futures and options.
codergeek42 writes
Then how will you print yellow stuff, stupid?
and justforaday writes (with less pointless hostility)
And then you can pay an assload for repairs and get a nice stern talking to from the repair guy about how you should never ever ever put the wrong color toner in the wrong bin
I think you're both missing the point. It doesn't matter if the printer is utterly destroyed and never prints again, since this needs to be done just once per model. For example, by an organization like EFF or Consumer Reports.
I understand the marking is done with yellow ink. It seems one would be able to expose a lot of these printers by replacing (or contaminating) the yellow ink with black.
As a member of the Globalized Corporate Oligarchy, let me just say that we have the U.S./Costa Rican/Chinese/Azerbaijanian Shadow Government under our complete control.
I can state unequivocally that 9-11 was not the work of our pawns.
However, our control of the Australian/South African/Polish Shadow Government has always been tenuous at best, and of course they have always had the motivation and resources for these sorts of things.
The rest of the world's Shadow Governments have generally been spending the last few decades implementing our labor reform acts and performing corporate audits, so of course they are beyond suspicion.
We in the Globalized Corporate Oligarchy seek to implement our vision statement across all of our puppets (shadow governments and media machines alike) in order to bring new exciting products that add value for all our stakeholders and customers. We apologize for the inconvenience of this attack.
program would produce a total of 330 pounds over 30 years at the Idaho National Laboratory
If we're running low, why not just buy some more from North Korea?
You would think if penny pinching is good enough for Bill Gates (look for the comment on the Rotary speech), pence pinching would do for the British.
Life could have been easier if they had supplied some Dijjer P2P links as well!
Especially for those of us at work stuck with no control of the NAT firewall.
I'm about to hook up a Python app to a PostgreSQL database. Last year when I checked, I found that psycopg was the best (fastest and most reliable) connector package. I decided that because it was the only one I tried that didn't choke on huge 500MB loan model runs.
What do you reckon is currently the "best"?
Although the conclusion of the Pythagorean theorem was known (long before Pythagoras) by many early civilizations, there's no evidence any of them made it into a theorem by proving it.
a friend of mine got a -3 on a question on a test. The girl sitting next to him got a -1 on the same question with a near identical response. He complained and the situation was resolved by giving the girl a -3 instead of a -1.
Heh. Back when I used to teach calculus, I did this (or at least threatened to do it a lot). I was a real BTFH.
Basically I would be careful never to err on the parsimonious side with partial credit. Frequently, two friends with fairly similar answers would walk up to me with their graded papers, having fairly similar answers but different scores. I would always out that the student with the higher score was the beneficiary of a generous mistake on my part.
"But, if you want me to, I could reduce your score to even things out." No one ever took me up on that offer, no matter how close their friends were. Usually, they left feeling relieved I had not in fact reduced the higher score, and perhaps a little peeved at their friend for putting them at risk.
Worse, it looks like the silhouette in the knockoff photo is an asian chick who clearly can't dance!
[Not that there aren't plenty of asian chicks who can...but the ones who can are mostly in North America]
PPS: Antibiotics, the true "medical" breakthrough of the 20th century, are primarily a tool of the surgeon, not the medical doctor
The may be the tool of the surgeon, but they are the refuge of the harried pediatrician!