Slashdot Mirror
Wine vs Windows Benchmarks
PeterBrett writes "Tom Wickline recently posted to the Wine development list announcing that he'd done some benchmarks comparing Windows XP to Wine. They should be taken with the requisite dose of salt, but Wine has certainly come a long way."
We need real benchmarks! Get some Windows worms/viruses/trojans running on WINE and then we'll have some real-world benchmarks!
I say good day to you sir!
while i realise that postings to a dev list shouldn't be taken as gospel, why would a dev list posting of benchmarks be assumed to be doctored? of course i would expect this from a marketing dept, but a dev list?
chris
"Physics is like sex. Sure, it may give some practical results, but that's not why we do it." - R. Feynman
I've quite impressed with the performance of WINE, however these stats can be a little deceiving. These stats are based on a game that works. Getting the game to work in the first place can be quite a challenge. But for the part-time gamer that doesn't wanna be chained to Windows, this is a great alternative indeed!
http://religiousfreaks.com/The results aren't exactly surprising - Wine is excelling in what Linux is generally better than Windows in doing - memory management, hard drive speed, and related matters (stressing generally there, because of course different apps give different results). This is Gentoo after all, it's built for speed. Then the heavier the load on the video drivers the more the superiority of the Windows drivers takes hold, so for the graphical stuff things don't work as fast.
Congrats to the Wine devs!
To me this seems to be more a test of the linux implementation of teh video card drivers.. and NOT the wine system itself.
i think a wider suite of tests would be required.. and not just the preformance/gaming orinted stuff.
-- Shoe Lace
Speaking as someone who used to be a Wine-hater, Wine has definitely come a long way. My impression of Wine for years was that it a) was impossible to install and configure, b) didn't run anything other than solitaire, and c) caused major instability to desktops.
Then I tried Codeweavers' "Crossover Office," essentially a pre-configured Wine with graphical configuration and installation tools, and everything changed. I currently use all of the following under Fedora Core 4:
- Microsoft Office XP
- Wordperfect 12 (word processor only)
- Photoshop 6
- Framemaker 7
They all installed using the standard CD install, without my having to jump through any crazy hoops or type a single command, and they all run flawlessly and are great for serious work. They sit right in my KDE menu like all other applications and it's a real head-turner to be able to show up to work with my laptop running Linux and then pop into Word XP and Framemaker.
Wine works incredibly well after all, it's just more "raw material" than "finished product." Get someone to write a user-friendly front end for it (ala Codeweavers' Crossover Office) and it offers a very high level of Windows compatibility to Linux users.
STOP . AMERICA . NOW
Notice, however, that the 60 some tests that Wine leads on are synthetic through and through... and when you get to actual games it's XP all the way. While Wine's performance is impressive, the requisite dose of salt may be several kg for this article.
Enough with the gentoo bashing! These compiler optimization options would not be a part of gcc if they did nothing.
I don't make fun of your hair do I?! So stop making fun of my favorite distro!
A bad analogy is like a leaky screwdriver.
...it makes you feel relaxed, slightly fogged and, in sufficient quantities, happily drunk. Windows, on the other hand, just makes you feel angry and frustrated. Give me wine!
oh, wait, you were discussing software?
Seems a bit strange to me to do a current comparison by using a version of 3d Mark that is 5 years old. If you were going to test out a 6800 on Windows alone you would use 2003 or 2005, the fact they didn't use that one in their Wine comparison suggests to me it couldn't run the later versions at all. The fact that 2000 ran better than under XP, but 2001 ran considerably worse suggests this as well.
If this is the case, the results in regard to game performance are out-dated at best.
Blessed are the 1337, for they shall pwn the earth.
Anyone else notice the funny stuff going on with the statistics? All the statistics are reported as percentages of the XP value, with higher = better. That means that if wine is "+ 90%", it's performing less than twice as fast as XP. But if it's "- 90%", XP is performing ten times faster!
So whatever this is measuring (and I concur that it seems to be mostly Linux graphics drivers), it's not reporting the results particularly well.
Wine is significantly slower in nearly half of the tests. And getting faster results during memory and CPU tests don't make any sense. The OS shouldn't have anything to do with the results of these tests. Maybe the results are skewed by the Wine's timer implementations?
Well, my experience with Wine/Cedega has been that for the games and applications that work, the disk-access tends to be faster. Not necessarily because the actual disk is being accessed faster, but because the filesystem (in my case reiserfs) is speedier to read.
The other wonderful thing I've found about Wine is the graphics abstraction layer. My laptop has a GeForce FX5600 (mobile) card in it. It's actually rather spiffy for most games still, but sucked ass at Battlefield 2 in windows, popping up the warning that my graphics drivers were out of date. Well, it seems that the drivers are tied to the laptop in windows to co-habitate with the power-saving etc etc... so I couldn't update from the official NVidia ones. And of course, my laptop vendor doesn't offer updates for anything over a year old it seems.
In linux, however, the normal NVidia accelerated driver works. The game runs on that faster than in windows, and with better detail levels. I don't know if it's just that the Cedega HAL does a better emulation for the software bits, or if it's due to the more-up-to-date driver, but it's a much less painful experience in Linux.
Lastly, my soundcard. SB Live 5.1. Abit dated, but with livedrive still a very nice functional card, except that the windows drivers will eventually/randomly freeze in most directX intensive games. Running in linux... no problemo. That's actually why I switched to Cedega/Debian almost completely (too many losses in Warcraft from lockups).
Sorry if I'm a bit sensitive about this, but Slashdot is the last place I expected to get a hard time about my hair.
[Fuck Beta]
o0t!
The real question on everybody's mind is: how well does it run Counter-Strike 1.5? I didn't see that test on there.
The transition from Debian-based systems to Gentoo will be fairly painless. Just understand that it may be several days before you get a working system. My main computer is a dual-boot for WinXP and Gentoo. During the install, the family had to live without being able to access WinXP for almost 5 days. Of course, this was on a mid-range AthlonXP with 256MB RAM. I have (in boxes at the moment) a new dual-core Athlon. I expect it to take about 48 hours from fdisk to KDE.
There are precompiled packages avalible to speed things up, but where's the fun in that.
Yeah, Debian to Gentoo is quite easy. Just use "emerge" vice "apt-get".
I'd rather you do it wrong, than for me to have to do it at all.
The Linux Kernel has some graphics support but neither the Kernel nor the X Window System are geared for fast 3D graphics. DirectX is good at getting around using the slower Windows GDI. DirectX is one of the few things Microsoft does somewhat well. (Insert joke here about Directx 9 and taking nine trys to get DirectX right.)
I have a feeling that unless some major changes are made to the X Window System (and maybe Linux drivers) that WINE will not catch up with WindowsXP and DirectX, but that just means I would need a faster computer.
WINE doesn't need to be the fastest. As long as it will run my older games (which Windows 2000 does not always do well) it may be more useful to me than an actual install of Windows.
---
This is just my opinion, please don't flame me just because you like Windows.
Losing faith in humanity one person at a time.
I know this is repeat info for most people, but for the newbies...
There's actually an online application database where people have submitted their experiences/successes in getting Windows apps to run under Wine. If you want to see how well Office 2k3 works under Wine, this'd be the first place to look. Conversely, if you have success running a given Windows app, be sure to submit your experiences. Feedback to the App DB not only helps other Wine users, but is helpful feedback for Wine developers on outstanding compatibility issues.
The URL is: http://appdb.winehq.org/
Okay, I *like* WINE. It's a great piece of software, and very useful. But it doesn't make Linux a drop-in Windows replacement. If you decide "Gee, I think I should use Linux as a desktop" (I do), then it's icing on the cake if WINE runs something. It's not reasonable to simply expect a given piece of software to run flawlessly under WINE, though.
Any program relying on (nontrivial) preemptive multithreading will be buggy.
Nobody said they were doctored; the slashdot editor said "take it with a grain of salt". I see a lot of reasons to do so:
Honestly? The results probably aren't manipulated, but the presentation is very clearly set up with a number of tricks (perhaps without him/her realizing it) to give the impression that Wine "kicked some serious ass", when for the most part, it did horribly.
Please help metamoderate.
They often do things under very specific conditions and are useless elsewhere. If you're not a compiler expert (which I definately am not), it's unlikely you'll have any apreciable performance gain. It's quite possible that you'll see a performance loss if things aren't done correctly. Even an expert may or may not get a gain.
I love Gentoo myself, but I'm not delusional about performance gains. I do it for the customizability it offers. For instance, last I checked, you could get a Debian package that contained the Perl bindings for Vim, or the Python bindings for Vim, but not both. With Gentoo, I can easily instruct Portage to build in both.
Not a typewriter
Presumably, the memory tests deal with the various OS-level Alloc's (HeapAlloc, GlobalAlloc, LocalAlloc, VirtualAlloc, etc...), which include fault protection checking, SACL checking, and other safety features. The reason that Wine performs better is that either they have implemented a faster version of the WinXX memory management APIs, or that the underlying Linux memory management is faster and the cost of the Wine wrapping calls is negligible. Same for the CPU-related tests... Just as memory is a managed resource in the WinAPI world, so is the CPU (also having SACLs/DACLs to check, and threading/fiber management, etc...)
Several days was when there was still stage1. Now that you must start at stage 3 (and later recompile if you like) you are done in 1 day. After you finish kernel, you just emerge x and kde and next morning you are ready to go (just after you configure x). About speed, it is faster that kubuntu (which I used), but it's the other things that made me go back to gentoo.
*shrug* same site it was two years ago. Yes, Gentoo is swarming with clueless n00bies; I was a clueless n00bie once and so was everybody else. If it gives them something to play with, keeps their interests, and gets them learning about Linux, it's worth having to deal with "why doesn't -Os -f-unroll-loops work?" when I talk to one (and I try to help them, because plenty of people helped me back in the day -- that's the whole spirit of GNU, right?).
Seriously, what should n00bies do, then? Gentoo is a largely user-configured operating system with unbelievably simple and hand-holding documentations. Yes, #gentoo is always full of n00bs asking why they can't boot now that they disabled all block devices in their kernel. But then again, that means it's full of n00bs who have configured and compiled a kernel; other distros I've seen say "WARNING WARNING ELITE USERS ONLY" about that. Why? People point out (rightly) that you can install Gentoo and still be an ignoramus. However, if you're actually interested in learning, you can also learn from the installation procedure the commands for fdisk, the options to hdparm, how chroot works, what /etc/resolv.conf is, blah, blah, blah.
funroll-loops is half-right. Gentoo is not simply a ricer distribution; it's a hobbyist distribution. It's the kit-car of distros. There are plenty of people who are doing the software equivalent of bolting a huge spoiler on their Civic. But there are plenty of us who are just having fun. And, anyways, the point of free software is that we're free to do what we want with it, even if that means being a moronic jackass.
All's true that is mistrusted
As a longtime transgaming subscriber, I can tell you that wine really does work as well as pictured. However, it uses an absolutely offensive amount of ram while it does so. I don't knwo how closely related the branches are though.
Imagine if you weren't allowed to use roads because a bus company complained about your driving 3 times. --skunkpussy
It does not take a second look to figure out the stats made up.
1. Most of Wine's wins are in the 0-2% mark. Means nothing except _inconclusive_; otherwise where is the variance, num tests to justify this?
2. Wine's perf is bad in the tests it lost
3. Old test suites were used
4. As some one said, If Wine is 90% faster it means it is 90% faster. If it is 90% slower it means it is 10 times slower!!!
BUT, what is really impressive is that Wine actually managed to run all the tests. The compatibility is indeed impressive. This benchmark would have been very credible had it not played with the numbers and colors.
Maybe a troll, but here is my argument against Wine:
Windows is moving to WinFX. Then it makes more sense to emulate WinFX's API than Win32 API. (WinFX does use Win32 extensively underneath, but why emulate 2 API's??). In the longer term, the answer to Windows compatibility is not Wine, it is MONO.
To say that Wine developers have it easy is shamefully disrespectful to their efforts. (Unless you take the viewpoint that not having to work with MS code simplifies the work!) For Wine to work at all is commendable - to be (sometimes) faster is truly amazing, IMHO.
The latest LugRadio show ( http://lugradio.org/episodes/43 ) features a very interesting interview of Jeremy White about Wine.
These results aren't presented to try to make Wine look better, nor is the author, consciously or unconsciously, trying to make it appear faster. These results simply were not meant to be used to say that Wine is better than Windows, and only Slashdot would try to make it appear that way. The real point of these comparisons, as is apparent if you read the Wine weekly summaries, is to give the Wine developers an idea of what areas need to be improved, and what areas are adequate. Green obviously means "at least as fast as Windows", which means that it's good. There is no point in grouping them any other way, since they don't care if they are 50% better or 1% better. Also, your criticisms of why this benchmark doesn't give a good idea of the relative speeds of Wine and Windows are quite wide of the mark (though they are valid complaints). The real reason why this benchmark cannot be used to gauge relative speeds is that it doesn't cover real world work loads. They measure a very small number of very specific things, mostly related to gaming and 3D performance. The benchmarks they ran that weren't related to that were designed to test the *hardware* speed, not the speed of the API. The Wine developers know this, and that's what the comment about taking it with a grain of salt means. It's probably adequate to give a rough idea of what parts of Wine need to be improved, but it is nowhere close to a comprehensive comparison of the speed of Windows and Wine, and was never meant as such.
Looking at the results - "Grammer Check"
Shame it was not "Spelling Check" but on still quite amusing.
In all seriousness, interesting and makes me want to revist Wine as it looks a lot better than when I last tried it (given I run pretty low spec hardware, performance is key rather than stability).
Though I do think "Wine or XP aborted on 18 tests" was a bit cheeky as it was 3 XP aborts, 15 Wine aborts...
From what I know about Gentoo, as a whole, yes. I run KDE on a 2.4 with 512MB of RAM and have effectively 0 swapping happening (until I fire up a massive Java app, or do like 80 things at once). Gentoo's speed isn't the "-fmad-compile-optimize-h4x", it's just how the OS itself is built and configured by default
Um, the question was whether he'd see a marked improvement changing from debian/ubuntu -- and given that Debian uses the same kernel as gentoo, and the same apps, it's very unlikely that there will be any difference in performance unless it comes from the "-fmad-compile" options. [In my experience, the -fmad-compile won't make any difference in practice either unless you're talking about very specialized balls-to-the-wall code like scientific computing or whatever.]
The "0 swapping happening" is an attribute of the linux kernel, not the distro; I notice the same thing under debian (in fact I generally see no disk i/o at all unless I'm writing files, because of linux's excellent disk caching). If the OP is having problems with swapping it means that he doesn't have enough memory for the apps he's using; a distro change is unlikely to help with that.
We live, as we dream -- alone....
SGI. Microsoft stopped OpenGL development by interfering with the OpenGL ARB, in order to catch up with their own solution, DirectX.
Love the 'personal' theories...
Microsoft did not even have an alternative to OpenGL in development when Microsoft pulled out of OpenGL. Microsoft pressed for OpenGL to enhance low level hardware support with intention of doing more than cad/engineering and supporting 3D rendering environment conducive to gaming and directly access video card hardware for gaming.
OpenGL told Microsoft to go pound sand, and that OpenGL was not for games or going to support direct hardware features for gaming.
Microsoft started stringing together a set of technologies that were called WinG, mainly a 2D form of rendering with plans for a new model that was a 3D rendering solution with direct video access on par of what the current DOS based games were used to, but in the Windows environment.
If OpenGL would have not 'played catch-up' to DirectX, and instead took Microsoft's recommendations at the time Microsoft was a big OpenGL proponent, there would never have been a DirectX, as OpenGL would be what Microsoft would be using, and contributing to instead.
The vendor lock in, was just a bonus in the long run, it was others involved in OpenGL that made the choice to not go for gaming.
But you can say it was about paying to SGI or a diabolical plan to take control of the gaming industry, but the facts don't support it.
The second part of this topic is that DirectX evolved to be more than an alternative to OpenGL, as it encapsulates everything from input devices, networking, to sound and voice.
When DirectX first existed it was the only game in town for any standardized interface to video for accelerated graphics in gaming. Now it is more than just Video...
... the detailed reproduction of another manufacturers product following detailed study of it's function and composition.
.NET architecture would be another example of somebody coming up with an alternative implementation without it being reverse engineering. The components of Mono may actually work more efficiently or also much less efficiently than their .NET counterparts but they behave the same and have an identical interface but very different internals.
That sentence came from the Oxford American Dictionary. To me that sounds as if reverse engineering has more to do with cloning products than just coming up with an alternative implementation which is what Wine is. All that Wine does is create a blackbox that works like the Windows API from the point of view of a Windows application but using totally different internals. Reverse engineering would be if they had recreated the internals more or less exactly using components replicated as far as possible from the originals in which case they would have to recreate the Windows source code from binaries somehow. You could also point to the AMD processors that implement the x86 instruction set, internally they are radically different from the Intel originals but to Windows they function the same. If that was reverse engineering I somehow suspect AMD would have gotten it's pants sued of by now. Instead Intel has now adopted the AMD64 instruction set that AMD came up with which is basically just x86 extended to 64 bit. The 'Mono' implementation of the Microsoft
You did raise an interesting point about limited reverse engineering which is probably true. In order to clear up inconsistencies and undocumented features in the Windows API the Wine team probably did some analysis of Windows. Surprisingly enough I recenty found out that this is actually permissable under copyright laws, at least in my own country, if the manufacturer is reluctant to issue full and comprehensive documentation and it is vital to the success of your project and you are not attempting to replicate the internals of the undocumented product just determine how it works to fill in patchy documentation which is the case with Microsoft and the Wine team in this case, they are not making a bolt for bolt replica of Windows. From what I can tell Wine is not doing anything illegal or unethical if they were producing line for line copies of Windows system binaries by reverse engineering Windows source code that would be another matter.
Only to idiots, are orders laws.
-- Henning von Tresckow
Wine is most certainly a reverse engineering effort. The problem is that you don't fully understand what reverse engineering includes.
Reverse engineering also includes the following process:
This is black box reverse engineering. You treat some piece of software as a block box, write tests for it, figure out what the "box" is doing, and recreate that behavior. No decompilation required, no source code required, just lots of tests and ingenuity. This has the benefit that no copyright violations are required (since you never decompile the original program). This process is also used in clean room design, except step 3 is replaced with a documentation step -- instead of code being the result of the process a specification is the result. Compaq did this to reverse-engineer the IBM PC BIOS.
Wine is most certainly doing this, as it's the only way to determine undocumented connections between various APIs. Mono certainly does this ("what's this member supposed to do, and is Mono's version following that behavior properly?"). Another way to think of this is for bugs -- does this Mono code do what the .NET equivalent code does? If not, we'll get a bugzilla entry for it, and (eventually) fix it. This bug-report/fix cycle can also be considered as black box reverse engineering, since the bug-report is itself a test, through which we can determine what the actual functionality should be.
Decompilation is generally not legal, since it can lead to copyright violations. Black box reverse engineering is legal, and any attempt to limit black box reverse engineering would kill the interoperability market, since no compatible hardware/software could ever be created unless the original manufacturer permitted it.
Throughout most of my career, I've understood reverse engineering to be what you do when you DON'T have the source code. I think the wikipedia entry mentions that this is the most common interpretation. It can be extremely difficult and time consuming. I've done it on major projects a few times in my career. It is neither easy nor efficient, but sometimes the only choice you have.
It's also common to talk about reverse engineering hardware, where the innards of a chip may be deduced by observing its inputs and outputs. I worked on a project at a startup where we had to do that, because the original hardware developers were long gone, and no VHDL could be discovered, yet we had to write drivers for their (buggy) chips in order to make a deadline. At the same company we had to reverse engineer the workings of a (buggy) Fiber Channel PCI card, because the manufacturer would not give us the support we needed to make our deadlines. They were rather surprised when we talked to them about the details of their (proprietary, embedded in an ASIC) DMA engine that we deduced via logic analyzers and oscilloscopes.
Those projects were SO difficult compared to reading (even obscure) code, that I really think that using the one phrase for both activities is confusing and sometimes even deceptive.
Well, I don't agree with all of the Wikipedia entry, nor do I agree with your comment implying the necessity of decompilation.
However, if we're going to take Wikipedia to be gospel, then let's read all of it, including:
The Samba software, which allows systems that are not running Microsoft Windows systems to share files with systems that are, is a classic example of software reverse engineering, since the Samba project had to reverse-engineer unpublished information about how Windows file sharing worked, so that non-Windows computers could emulate it. The WINE project does the same thing for the Windows API, and OpenOffice.org is one party doing this for the Microsoft Office file formats. [emphasis mine]
I have no idea if the WINE engineers have disassembled (or as you say decompiled) Windows binaries or not. My point remains that this is not the determining factor in deciding if it is reverse engineering or not: meaning that disassembly is not necessarily a feature of reverse engineering.
I agree with you. When I was a kid we had 8 bit computers and you learned by doing. In those days you of fritzing your hardware by moving it too suddenly, causing a machine lockup with crappy code was the least of your problems.
Gentoo evokes that era for me, and you can't have a go at people for breaking something when they're busy learning. So what if they over-optimise compiler flags and break things? When they fix them up they're learning a valuable lesson, and that lesson isn't 'next time use a binary distro'
Anyway, today's school-age gentoo n00bs are tomorrow's crop of system admins.
Hal Spacejock: Science Fiction with Nuts
You missed the subtle point of the post. MS Flight Simulator was once the de facto IBM PC standard compliance test.