Making Closed Software Act Like It's Open

The Installer writes "Researchers from the University of Washington have managed to add customization and accessibility options to proprietary software without ever touching the source code. Rather than alter program code, Prefab looks for the pixels associated with the blocks of code used to paint applications to a screen, grabs hold of them, and alters them according to whatever enhancements the user has chosen to apply. Any user input is then fed back to the original software, still running behind the enhanced interface."

Comments

The real question is-

[mxh83]

Are we really that desperate to continue using closed software.

Re:The real question is-

[WrongSizeGlass]

'Closed software' is a fact of life for most users. This attempt at 'expanding' the functionality isn't very impressive, though, and won't have very many real world uses. What if you resize your monitor, do your 'customizations' all go to hell?

I always liked using the plugin architecture for applications that provide it.

Re:The real question is-

[Yvanhoe]

A more interesting question is the legality of this. If I distribute a customization kit for a closed source software, when is it considered like a crack ?

Re:The real question is-

[tokul]

If I distribute a customization kit for a closed source software, when is it considered like a crack ?

What's the difference between hacker and cracker?

If your customization kit does not break closed source software licensing and you don't distribute it with software that you don't own, it is not breaking any copyright laws.

Re:The real question is-

[natehoy]

I have to imagine that, if this software is intercepting the outputs of legally-paid-for closed source software and altering them, this could never be considered a crack. Then again, I suppose if Hollyweird can sue someone for building a custom version of a movie with the swearing and naughty bits bleeped out, while including a copy of the original version of the movie to make sure the end consumer has actually purchased a license, who knows?

If this is considered a "crack", will software developers be able to stop me from purchasing a larger screen, or better speakers?

Re:The real question is-

[Hurricane78]

No we aren’t. But many (pretty dumb) people have the perception that we would, because they think that others would have that perception, because (1:) they said something alike, because they themselves think that others would have that perception, because... GOTO (1).

Or in other words: Monkey, see, monkey think, monkey parrot. ^^

What most people don’t know, is that it’s all in our head. Like fashion. Why did women think that wearing rubber boots with colorful flowers on them in the middle of the summer would be cool? Because “it was the fashion at the time“. And why was it that? Because some self-appointed “fashion expert” told them so. (Because he sold that stuff. But that’s another story.)

But I think in this case there wasn’t even really a starting point. There was some perception that “that’s how it is”, because the closed source people acted more secure, because, being business people, they were trained that way.

The thing is: Why would you let it just grow on us like a mind-virus, when you can change it just like that? After all, if it’s all in our heads, it only requires some of us to always securely act how it really is: We are gaining, winning, and rolling over them in an unstoppable wave!
Have you ever noticed how a big mass of people switched their mind sets, and started to become a raging mob, or something like that? It’s that exact thing. It only needs a seed (you) who is so secure, that they start to doubt themselves. Then the rest is only a matter of that self-amplifying mechanism above, and time (depending on how strong your reality is).

It’s a big war of psychology and social engineering, for the minds of people. And I don’t let the politicians, mass media or greedy multinational corporations win it. :)
Think like this: You’re a mind hacker. And the mind is by far the most complex computer ever. Isn’t that much cooler than doing it to such a (in comparison) ridiculously simple thing as a computer (network)?

Re:The real question is-

[fuzzyfuzzyfungus]

While the power of the almighty EULA and(in the case of software with embedded DRM/anti-tampering features, the DMCA) might well cloud the issue, there is an analogy worth looking at.

A while back, there was a company called "CleanFlicks" that operated a movie rental service, aimed mostly at Fundie Mormons and the like. They took DVDs, reviewed them, and produced bowdlerized versions, which they then rented out. They were sued, and lost, on the grounds that they were, indeed, producing and trafficking in unlicenced derivative works.(IRRC, they purchased as many DVDs as they rented out, so they weren't illegally duplicating, ie. 1 purchase to 1 edited rental copy, in any useful way; but they were still smacked down). They exist today with a much reduced catalog of movies that fit their standards without editing.

A similar company "ClearPlay" used a different tactic. They provided specially programmed DVD players that were able to interpret a control file(programmmatic mute/unmute, FF/play, etc.) and rented unedited commercial DVDs, along with matching control files that, when used with their players, automatically "edited" the DVD as you watched it. The MPAA threw a fit; but the company survived legal challenge(it helped that congress, tipping their hat to "the children" passed a law to explicitly clarify the scope of copyright on this point). Since, unlike "CleanFlicks", they weren't actually creating a derivative work, just a control file that modified the behavior of the DVD player during playback, they were judged to be in a different category.

Again, barring the sorts of tricks that can be pulled with even weak DRM+DMCA, this sort of "customization kit" tech would probably fall into the "ClearPlay" side of the analogy. Actually selling edited binaries, even if you purchased a legitimate copy for each edited one you sold, would almost certainly put you in the "CleanFlicks" camp, and get you smacked down; but selling a customization package that modifies the appearance of a binary only at the point of execution on the end user's computer, or even selling a bundle of "copy of commercial software + installer for customization kit" would probably pass legal muster.

The only complication, of course, is that court decisions are, in practice, driven by a mixture of the text of the law and a somewhat emotive sense of "intent" or "desirable outcome". Protecting the kiddies from corruption generally wins you warm and fuzzies. It isn't clear that modifying the appearance of dialog boxes would have the same cultural clout(unless you could, say, find a nice test case involving a bunch of blind kids who are able to use $SOFWARE_X with their screenreaders for the very first time*wipes tear* or something of that sort).

Re:The real question is-

[Anonymous Coward]

Oops, forgot to mention: Any precedent established by cases surrounding "Game Genie", "Gameshark" and the various other cartridge pass-through modding systems might be even more relevant.

Again, the DMCA can be used in some rather nasty ways; but my understanding (IANAL) is that "Game Genie" and friends, while strongly disliked by the console makers, generally survived legal challenge, because they were pretty clearly only useful for letting people who had purchased cartridges make fair use of them on their own personal systems. Had those passthrough-mod systems also included, say, cartridge ROM dump capabilities, they might well have been smacked down.

Re:The real question is-

[Animaether]

I can see many uses for this - but then, I've got AutoIt for the sorts of things I'm thinking of already.

I think the article title and summary are completely bunk, though. They're not making the software any more -open- than it was before; If you have a button that writes "Hello World" to a file, then you can replace that button with a contrast-rich enlarged version with excellent text-to-speech functionality, or make it a bouncing spinny glowing orb like something out of Kai Krause's mind... but pressing it is still going to write "Hello Word" to a file. It doesn't make it actually do anything differently from before.

It's fun that they can detect UI elements out of a bitmap, but there's so many non-standard UI elements in play that this is going to fail horribly on a lot of UIs. Maybe that means we should stop using custom UI elements, but sometimes those custom UI elements simply are more appropriate than the widgets that come with the OS/widget provider.

Re:The real question is-

[LordLucless]

No more Snow Crash for you

Re:The real question is-

[psnyder]

if this software is intercepting the outputs of legally-paid-for closed source software and altering them, this could never be considered a crack.

Here's Facebook threatening a Greasemonkey script developer for pretty much the same thing (altering the output after it's in the browser).

Re:The real question is-

[npsimons]

Closed software' is a fact of life for most users. This attempt at 'expanding' the functionality isn't very impressive, though, and won't have very many real world uses. What if you resize your monitor, do your 'customizations' all go to hell?

Not to mention the legal issues. Or trying to keep up with changes intentionally made to break your efforts (just ask the WINE, SAMBA or iPod-Linux compatibility devs about this).

The first time I saw this article in ACM links I thought "neat, but what a waste of effort; I should send them a note letting them know that open source welcomes this sort of innovation with open arms".

Re:The real question is-

[Exitar]

That's anyway better than being closed-minded and not using the best tools for the job.

Re:The real question is-

[cbiltcliffe]

It's not expanding the functionality. It's modifying the GUI.
Maybe adding an option checkbox that wasn't there, or rearranging some dialog somewhere.

Without the source code, and more specifically, modifying the source code, it's impossible to add features in any meaningful sense.

Sure, you could add a "show all images with red background" checkbox, or something like that, and modify the program display output appropriately, but I can't imagine any case where something like that would be called a feature.

But then, I'm not among the 95% of the population that considers "Ooooh, shiny!" to be a feature, either.....

Screen Scraper

[drrck]

This is called a screen scraper, and likes to break with updates to the underlying program, right?

Re:Screen Scraper

[BarryJacobsen]

If you RTFA or WTFV, you'd know that it's detecting the input elements using an algorithm and not hard coded to the specific application (they even demoed VNCing into an OS X machine and having it detect the UI elements there and applying the processing).

Re:Screen Scraper

[natehoy]

And if you've worked with screen scrapers, you'd know that most of them are based on UI elements, and an upgrade to the underlying software almost always causes problems because the UI elements frequently change when software is upgraded.

So, yes, this is a screen scraper, which means it will survive some small alterations to the UI, but you're usually looking at upgrading/rewriting the screen specs when the underlying screens change.

Re:Anybody got the equivalent for web pages?

[clone53421]

Yes, it’s called GreaseMonkey.

"Manipulating software at the pixel level"

[Yvanhoe]

This quote really made my day...

Stopgap at best.

[headkase]

Open Source as a development methodology has already won. It is more scalable: more features, custom features, localization, error removing all work better with it. It is also a lateral organization like a web which avoids some issues that the "Mythical Man Month" talks about which results from hierarchal organization. Close software will never go away but it's utility has already been greatly compromised by Open software. With Open software this extra translation layer is unnecessary you would just modify the source and have a more reliable program intrinsically. Code is Free, get over it.

Seriously?

[DarkKnightRadick]

That has to add a lot of overhead to the already running process and to what benefit? If it's reading the code "as many as 20 times per second" that is going to add tons of CPU and RAM usage to the system that just isn't needed. F/OSS ftw!

Re:Seriously?

[Chris_Stankowitz]

hat has to add a lot of overhead to the already running process and to what benefit? If it's reading the code "as many as 20 times per second" that is going to add tons of CPU and RAM usage to the system that just isn't needed. F/OSS ftw!

With i7 chips, SSD HDs, 64bit OSes that support 4+ gig of triple chan memory (any or all of those in one machine are huge improvements in desktop computing power) you'll still not push it to capacity with 20 such apps running. We are at a point where we have an abundance of CPU/memory to spare, I see nothing wrong with developing such apps (if only as stop gaps) until such time that a suitable replacement arrives. These apps very well may be the impetus for the development of those open apps once it proved that a user/market base wants it.

CS-

Re:Seriously?

[edbob]

This sounds like the same sort of attitude that software writers have had for ages. Just write bloated, inefficient code and let processing power eventually catch up to run the software. I think that this needs to be a legitimate concern or we will wind up back to the point where a new version of Windows would come out and barely be able to run on the technology available. Yes, this may not add much to a computer running a Core i7 with 6 gigs of memory, but that sort of system is pretty rare in the real world. Most people use computers that are a few years old.

Headline once again

[oldhack]

The headline seems completely disconnected from text of the summary

Copyright infringement

[hweimer]

Chances are that such "enhancements" constitute a derivative work of the proprietary user interface. In many jurisdiction even using such a thing is illegal and distributing the enhancements without permission from the copyright owner most certainly is. Which, of course, is very different from free and open source software, where people are encouraged to share and improve the code.

Re:Copyright infringement

[blueg3]

It's only a derivative work if you distribute the proprietary software along with your enhancement. If the enhancement simply requires that the user already have a copy of the proprietary software in order to use it, then it's not a derivative work.

Re:Anybody got the equivalent for web pages?

[natehoy]

Combine the two previous posts, and add NoScript in for good measure.

Greasemonkey + AdBlock Plus + NoScript = The Web, the way YOU want it to be. :)

And, just like the tool above, if a company changes their web page, you're looking at some redo on at least the Greasemonkey site. Be sure to add Greasefire in addition to Greasemonkey - lots of people have lots of great scripts that are at least good example code to pull from.

What's actually new here?

[time961]

Tools like this have been around for ages, although they are usually called "GUI test frameworks" or "automation assistants". Such tools provide a way of scripting interactions with an existing GUI. However, they really only focus on mimicking pre-recorded user interactions--it seems like it would be quite time-consuming and fragile to reverse-engineer an arbitrary program's dialogue boxes in a robust way that would allow control to be significantly enhanced.

Also, on Windows, at least, there are tools that enhance the operation of dialogue boxes (for example, adding history and options to the File Open dialog). Those tools work at a more abstract level than snagging pixels, which is a lot more efficient--but that means they are ineffective on applications that have already customized those dialogues.

It seems like the fundamental non-breakthrough here is that the application actually must already include the functionality that you want to express in your modified UI--otherwise, you can modify the UI all you want, but the app will only do what it's capable of. So if you want it to display a bunch of different renderings in sub-windows automatically (to use their example), it had better be capable of that display already.

wallhack, aimhack, maphax

[Khashishi]

this is nothing new

And next week...

[lxs]

We show prisoners how to paint the bars on their windows the color of the sky and pretend that they are free.

Scammers have done this years ago.

[SharpFang]

Exploits that do this are in the wild.

They hijack your client display when you access a banking page, fill in the account details and amount for you, while presenting you with interface to enter your target account, then replace the data on the confirmation page with whatever you entered while obscuring the data they entered. You sign the transaction with a token or OTP and instead of sending $15 to your aunt, $10k money gets transferred to the hijacker's account in Nigeria.