How Perl and R Reveal the United States' Isolation In the TPP Negotiations

langelgjm writes "As /. reported, last Thursday Wikileaks released a draft text of the intellectual property chapter in the Trans-Pacific Partnership Agreement. Since then, many commentators have raised alarm about its contents. But what happens when you mix the leaked text together with Perl regular expressions and R's network analysis packages? You get some neat visualizations showing just how isolated the United States is in pushing for extreme copyright and patent laws."

Comments

In summary, what can we conclude from these data?

[DrPBacon]

"In summary, what can we conclude from these data? Canada, with by far the most sole-country proposals, seems like it is up to something." Those shifty Canadians. I knew it.

What a poor commentary

[Rosco P. Coltrane]

In summary, what can we conclude from these data? Canada, with by far the most sole-country proposals, seems like it is up to something.

Right, "Canada is up to something" is a great way to report on international negotiations. Okay, they've taken the geek approach of grepping through the drafts instead of reading it in full (fair enough), but at least they could have extracted whatever keywords appear after "Canada" and "oppose" / "propose", to figure out the something it's up to. It's not hard in Perl, gee...

Re:What a poor commentary

[Narcocide]

I can only assume that they must have meant it sarcastically. I analyzed the data they presented and came up with the conclusion that the US and Japan are "up to something," while Canada just seems to have a lot of friends and new ideas. But I didn't read the actual drafts either.

Re:What a poor commentary

[Sockatume]

Yes, this one-page article clearly represents the entirety of his knowledge on the subject, he's obviously not a political science professor or anything.

Perl? Why?

[Anonymous Coward]

Here is the original article with a little more technical detail. To those interested (like me) what was Perl doing there, it was just a single line script with regex. The rest is R.

Re:Perl? Why?

[Anonymous Coward]

From TFA:

print "$_\n" for @match = $txt =~ m/(?:(?:AU|BN|CA|CL|JP|MX|MY|NZ|PE|SG|US|VN)[0-9]*\/)+(?:AU|BN|CA|CL|JP|MX|MY|NZ|PE|SG|US|VN)[0-9]*/g;
It’s ugly, but it seems to get the job done.

I agree, it's Perl^Wugly. But Python wouldn't be much better. At least we can remove the redundancy in the regex:

countries="(?:AU|BN|CA|CL|JP|MX|MY|NZ|PE|SG|US|VN)[0-9]*"
print "\n".join(re.findall("(?:{0}/)+{0}".format(countries), txt))

Re:Perl? Why?

[biodata]

Why would you use something other than perl for parsing text? This is what perl was designed for and it's most likely faster than anything else scripted. I'm sure you could write a text parser in any language you happen to like, but if you have the skills then perl is the correct tool for this job.

Re:Perl? Why?

[bluefoxlucid]

I've written more Perl than I've written Python, but it was mostly in 2002 when Perl was still relevant. I've written more perl than I've written Awk and Bash and I use Awk extensively in my career as a Unix sysadmin.

It's a shitty language. Perl is full of mature networking modules, ORM frameworks, and other extensions that are absolute garbage despite being in active development for years. It's hard to write good code in perl--so hard that absolute veterans had to inform me of this, since I've only done a handful of mid-size projects in Perl amounting to maybe 30 or 40 thousand lines of code and not the hundreds of KLOC that some of these folks have been through over years.

You know what's funny about really good, highly experienced perl programmers? They complain about how shitty perl is, and especially how shitty other perl programmers are with stuff like "if $a && $b || $c { ... }" talking about how "it's well understood how that works, you don't need parenthesis." It's as much coding style as weird sigil bullshit and slow-as-fucking-balls precompilation, not to mention innocuous code sometimes causing Perl to lose track of its symbol table (so when you 'use MODULE' it works early on, but suddenly the functions aren't callable from the same code blocks later) and the piles and piles of absolute shit like DBIx.

I used to think Canada was quite European...

[Joining Yet Again]

...until I actually got to know it. While the political rhetoric is more even-handed and they do have a proper health service, the country is all about big business, just like the US.

Re:I used to think Canada was quite European...

[Nemyst]

I'd be curious to compare the political stance before and after Harper's rise to power. To me the difference is quite stark and I feel like things went very much American when he arrived. Anti-environment, anti-science, pro-oil, pro-business, anti-social policies, etc. Heck, we've had more debates on abortion, the capital sentence and other such social issues since he came around than we've had in the decades before.

Re:Go Canada

[Chrisq]

:Go Canada

I don't think it can. I looked at a map and its sort of suck between the USA and the arctic as far as North South moves go and between Greenland and Siberia East and West.

Don't worry, eh?

[smittyoneeach]

It's just Mayor Rob Ford's plan to go oot the hoose and take the world off to a the great, white, crystalline North. Beauty, eh?

Re:Go Canada

[somersault]

The US having the least correlation with other countries is a sign that there is either massive brainwashing/groupthink in the rest of the world as a whole, or in the US itself.. hmm :p

Re:Go Canada

[Anonymous Coward]

I would personally say it is because the USA has bet the farm on being the monopoly holder on intellectual properties. Look where Hollyweird is, where silicon valley is, etc. A goodly portion of the US's GDP is based on intellectual properties, and ventures related to or hinging upon, intellectual properties or intellectual property laws. (Hollywood, music, software, biomedical, pharmecutical, biotech, etc.)

Compare that with the economies of the other countries implicated, who have GDPs predominantly composed of the trade and sale of material goods.

Given its market position, NATURALLY, the USA would only sign on to an agreement like this, if it could leverage market dominance in that market niche.

How unsurprising

[vikingpower]

The US is being gently pushed ( nudged ) into a beginning of rrelevance. Has already been going on for a couple of years: computer technology, aerospace tech, politics. NSA scandal accelerated it. The sun is going down over US America.

Re:How unsurprising

[dkleinsc]

The US is being gently pushed ( nudged ) into a beginning of irrelevance

I'd consider it more the US government trying to do whatever it can for the 0.1% of the country that pays for the majority of political campaigning, and screw the other 99.9% of us. And whether the US government becomes irrelevant or not, as long as the corporate overlords are happy, the politicians will be kept comfortable.

In the words of Number Two: "But you, like an idiot, want to take over the world. And you don't realize there is no world anymore! It's only corporations!"

Re:How unsurprising

[VortexCortex]

Although it helps, you don't have to stop running to lose your position in a race; The others can simply speed up. When both factors are at work the rate of decline accelerates.

You can trace much of the change in position of USA and others via the amount of essential economies and state resources are privatized, and thus funding promised to them and thus the private interest in influencing politics (deregulation) increased. For instance: Solid Rocket Booster designs have had funding lobbied for based on the merit of bringing and keeping jobs in certain congressman's local economy instead of on the pros / cons of the various designs themselves. The same sort of thing ran amok in Chile in the 70's.

When progress is averse to profit you get stagnation in a private industry -- Like ISPs in the USA: Instead of spending on infrastructure to provide a better service they can simply charge more for less (oversell bandwidth) to make more money. Bits have never been cheaper to distribute and yet their cost doesn't reflect this.

The sun sets not upon one country, but around the world at different times. If we're not careful our country could be next.

Re:How unsurprising

[Sabriel]

It might not stay gentle. Do you remember the decline and fall of the Western Roman Empire?

Oh, wait, yeah, that was a while back. Here, some reading: https://en.wikipedia.org/wiki/Fall_of_the_Western_Roman_Empire

Hmm, that's a tad indigestible, I need a car analogy. No, a gorilla analogy!

Imagine a tribe of gorillas. Let's call the biggest, strongest, most heavily-armed gorilla "Sam". Luckily enough for the tribe, Sam was actually a fairly nice guy - so long as you purchased his stuff or at least used his bananas to purchase stuff, and didn't draw attention to his tendencies to vanity and his insistence on being in charge - and it really helped that he kept the more aggressive males in check (every so often one'd get nasty where Sam could see it, or even challenge him, and everybody else'd get a reminder of why nobody fought Sam).

When the second-biggest gorilla, a tyrant and almost as big as Sam, collapsed from steroid abuse, things were really starting to look up.

But as time passed, the other gorillas noticed Sam was changing. Now some folk go doddery and forgetful, but Sam, he kept poking through the tribe's stuff, peeking in on them all the time. It was like he'd spent so long keeping a lookout for what his old nemesis did, he couldn't stop doing it. And he started to care less and less about whether the other gorillas complained when he rode roughshod over someone. He even started hassling his own young, creating lots of rules about where they could go, what they could take with them, what they should report back to him, and his punishments got harder too.

Trouble is, it's not just Sam's young and his friends in the tribe that have noticed. Some of those aggressive gorillas, both the older ones who kept their heads down while Sam was in his prime and the younger ones who don't remember how bad it was before Sam became the tribe's silverback, they've noticed too. They've noticed the changes, and they've noticed he's having trouble holding his bananas.

Can you guess what they'll try to do if, some day, Sam can't hold his bananas anymore?

Re:How unsurprising

[ebno-10db]

Do you remember the decline and fall of the Western Roman Empire?

Not personally, no, but I have read about it. One of the things I read is that it took centuries. Perhaps in a few hundred years my descendants in North America will live like characters from "Monty Python and the Holy Grail".

Define negotiation

[Gothmolly]

A negotiation is possible only when both parties can benefit.

Re:Horrible news website

[VortexCortex]

I didn't know what you were talking about until I disabled ad-block for the page. Yeesh. Most of those "headlines" are ads.

Re:Go Canada

[Nerdfest]

Up until this point a lot of our copyright policy seems to have been dictated by the US. In the last couple of years, I think the US copyright lobby was actually found to be sending the exact text of what they wanted included to the people in Canada responsible for it.

Re:Go Canada

[ebno-10db]

brainwashing/groupthink

That's a very polite euphemism for corruption. The excessive influence of industries concerned about "intellectual property", was bought and paid for. In addition to locking up people, or fining them into penury, for sharing a few songs, it means the foreign and domestic economic policies of the US are distorted in favor of this handful of over-hyped industries, and to the detriment of the rest of the economy.

'Free Trade Agreement'

[king neckbeard]

Can we drop the nonsense that TPP is a 'free trade agreement?' A free trade agreement would be very simple. Don't bomb us or torture our citizens, and you can trade freely with us. TPP undermines free trade by forcing countries into even further support of anti-capitalism legal monopolies as a condition for not restricting trade.

Re:'Free Trade Agreement'

[king neckbeard]

It's not free of government interference, it is government interference. The uber-rich fucks are scared as hell of a world in which the government doesn't protect them.

Re:Go Canada

[ebno-10db]

I would personally say it is because the USA has bet the farm on being the monopoly holder on intellectual properties.

That's a bad bet - something more affected by corruption and propaganda (the wonders of our "post-industrial" economy!) than by any rational policy choice. The excessive and corrupt influence of the "intellectual property" and financial parts of our economy hurts the rest of the economy. There are limits to the potential value of "intellectual property". It's not as big a part of our economy as is often hyped, other countries can easily produce large parts of it (e.g. movie and music production don't have major barriers to entry), and does anyone really expect other countries to rigorously enforce IP laws that mostly benefit the US, regardless of what trade agreements say?

Re:Go Canada

[bill_mcgonigle]

The US Government is less concerned with the interests of its people than most countries. It's heavily controlled by monied corporate interests, which seek to control the power that comes with having the world's reserve currency and a printing press. Don't worry, this won't last too much longer (which will shock most Americans when their purchasing power falls by 60% or more when everybody else leaves Bretton Woods).

I'm actually more surprised that the interests that worked so hard against SOPA and PIPA are not raising a ruckus this time; most of the same provisions are in the US version of the TPP and it's not even 'just' a law that Congress can theoretically repeal - this is International Treaty, which effectively becomes permanent law under the US Constitution. What's worse, Congress is set to give the Executive Branch FastTrack approval on this treaty.

But, the US Government is less concerned with the interests of its people than most countries.

Where are the other countries

[fritsd]

When I looked at the map, I saw the following countries were missing from the list (plus lots of Oceania countries): Russia, China, Taiwan, Indonesia, Papua New Guinea, East Timor, Fiji, Ecuador, Colombia, Panama, Costa Rica, Nicaragua, Honduras, El Salvador, Guatemala.

Isn't it odd that at least Russia, China, Taiwan, Indonesia and Panama are excluded? I'd imagine they do lots of trade across the Pacific Ocean (for Panama I meant transport rather than production).

Re:In summary, what can we conclude from these dat

[ebno-10db]

Mod parent up, and ask him if he can also write haiku.

Re:Go Canada

[SuricouRaven]

That is why the talks are being conducted in secret. What the people don't know about, they can't protest over. Until it's too late.

Author here

[langelgjm]

Hi, I'm the author of the article. Thanks for reading it. Originally I thought I might extract the "oppose/propose" and attach it to country names, but I didn't for a number of reasons.

First, as you note, "oppose/propose" by itself tells us very little without knowing the content of what is being opposed/proposed. But even if we do know the content, without the context it may still convey little. E.g., we might find "[US propose: a]" or "[CA oppose: the]". I thought about using Perl's extract_bracketed (and actually did at first), but decided against it.

Second, anyone familiar with these issues already knows where the countries line up. The US is pushing extreme IP laws. Australia doesn't necessarily agree, but follows along in many cases. Canada often tries to do its own thing (e.g., they were one of the only countries to take advantage of a TRIPS provision allowing them to manufacture an on-patent drug and export it to a developing country without manufacturing capacity). So showing people this information wouldn't necessarily add much value.

Thanks

[langelgjm]

Author of the article here. You're right, I meant it to be a little funny. As I noted to the GP, most people studying these issues already know where the countries line up. Canada has a history of being different on IP issues than the US (much to the US's chagrin - it's why we put them on the Special 301 "priority watch list" in 2012).

Here's a link

[langelgjm]

Here's a link with more technical detail. I had to tone down the technical aspects for the Washington Post.

That link does not have full code, but if you want, I can e-mail it to you (I already have for two other people). I didn't post the code online because I wanted to keep track of who was asking for it. But I'm happy to share it.

Re:Here's a link

[fatphil]

Did you ever do a "countries on springs" analysis such that countries which concur with each other are pulled together, and those that disent are pushed apart? Similar to the 3rd quartile cutoff one, but with all countries, and with distance made more significant.

One way I've done this in the past is to stick all dots on the unit sphere, anneal them into a stable position on that sphere with a suitable repulsion law, and then to squash the sphere. Chose a random point on the surface, conformally map the sphere minus that point onto a unit disc, re-anneal, and chose the outcome with lowest energy. Random point selection will naturally find the biggest open areas, bordered by the most repulsive countries, ones which deserve to be on the peripheries of the flattened diagram.

Hehe, I like the idea of the US being the most repulsive country. ;-)

Author here

[langelgjm]

Hi, I'm the author of the article. I have read the text (well, not all of it, just the portions relevant to my research - mostly copyright and pharmaceutical data exclusivity aspects, as well as the traditional knowledge article). However, I'm not a lawyer, and many lawyers have already analyzed the legal aspects of the text. That's why I linked to Margot Kaminski, Michael Geist, etc. in the article.

I thought my analysis would be valuable if it did something the lawyers were not (and could not) do.

Re:Go Canada

[ebno-10db]

will shock most Americans when their purchasing power falls by 60% or more when everybody else leaves Bretton Woods

Bretton Woods was abandoned 40 years ago.

As for a reduction in the exchange value of the USD, 60% could only come from a panic, not an accurate readjustment. A panic won't be allowed to happen. If nothing else other countries have too much to lose from it.

OTOH a smaller reduction in the exchange value of the USD would be very good for us. It's idiotic that we've spent so much time and effort post-WWII to prop up the dollar, when it only gives us bigger trade deficits and destroys our industry. There have been a few exceptions, like the Plaza Accord, but that's been long undone, thanks in large part to the corrupt influence of the finance industry.

it's not even 'just' a law that Congress can theoretically repeal - this is International Treaty, which effectively becomes permanent law under the US Constitution

It's not a treaty, it's a "congressional-executive agreement". It requires simple majorities (like an ordinary bill) instead of 2/3 of the senate. It's a constitutional gray area, and it sounds like BS to me when applied to long term agreements like trade "agreements" (treaties in all but name), but they've been upheld. On the bright side they're easier to repeal.

Submitter/author here

[langelgjm]

Well, to be honest, I wanted to get it posted to /., so I thought I'd highlight the fact that OSS made it possible.

New visualizations with D3

[langelgjm]

Author of the article here. Michael Simeone from the University of Illinois asked for my data and code so that he could experiment with some D3 visualizations. He did a little bit last night, and I thought I'd share the results.

Normalization?

[Bill Dimm]

Canada, with by far the most sole-country proposals, seems like it is up to something.

Doesn't the raw number of sole-country proposals seem like the wrong metric? It seems more sensible to divide the number of sole-country proposals by the total number of proposals for that country to see what fraction of its proposals have no support from other countries. From the next to last graph, it seems that Canada has both a lot of sole-country proposals and a lot of joint proposals. If the fraction of Canada's proposals that are sole proposals is not particularly high, the large number of Canadian sole-country proposals would just reflect them making a lot of proposals in general -- you might conclude that they are just putting more effort into getting the treaty right (in their opinion) than other countries. I only skimmed the article -- did I miss something?

Anyway, interesting analysis. Unfortunate that the Washington Post didn't make the graphs available in a format that is large enough to read the labels.

P.S. I'm not Canadian.

Re:Normalization?

[langelgjm]

I'm adding a table showing the ratio of sole country proposals to total proposals from that country. It does change the order, though the main observations remain intact. Japan, the US, and Canada, come at the top, Malaysia and Brunei at the bottom.

Re:The US creates more than the rest combined

[Hairy1]

It isn't like its the wild west out there; we already have strong copyright legislation. What the TPPA is seeking is corporate control over the ability to exclude people from the internet at will, with no judicial oversight. To a large extent it already does; I dared to critique the Business Software Allience on YouTube and my account was closed. No comeback here - to challenge it I would need to agree to defending myself in California. Unless you are a U.S citizen there is no fair use or free speech on YouTube, Facebook, Google, Yahoo etc. You are there at their pleasure, and easily ejected.

The TPPA seeks to extend this power to your local ISP; to actually cut you off from the net totally if you are saying things they don't like.

It isn't about protecting works, it is about controlling the channel. The Internet was a danger to corporate control of how people got their entertainment and information. They are now getting the people back under control, subservient to their masters like they should be. The thing is that most are happy with having their entertainment and information fed to them, told what they should be angry about.

The risk to Hollywood isn't that we will steal their content - it is that we will discover their content is gilt covered crap, and that we can beginb to express ourselves without getting one hundred million dollars from a VC. What the RIAA and MPAA care about is making sue that they control the music we listen to and the movies we watch.

That is the focus of the TPPA. Control.

To be clear

[phorm]

The Canadian *people* are more vocal about IP issues. The government has been ramming US-style copyright down our throat whenever possible, even to the extent of secretly suggesting the US put us on the 301 list in order to induce stronger copyright enforcement in Canada...