Slashdot Mirror
Wine vs Windows Benchmarks
PeterBrett writes "Tom Wickline recently posted to the Wine development list announcing that he'd done some benchmarks comparing Windows XP to Wine. They should be taken with the requisite dose of salt, but Wine has certainly come a long way."
while i realise that postings to a dev list shouldn't be taken as gospel, why would a dev list posting of benchmarks be assumed to be doctored? of course i would expect this from a marketing dept, but a dev list?
chris
"Physics is like sex. Sure, it may give some practical results, but that's not why we do it." - R. Feynman
The results aren't exactly surprising - Wine is excelling in what Linux is generally better than Windows in doing - memory management, hard drive speed, and related matters (stressing generally there, because of course different apps give different results). This is Gentoo after all, it's built for speed. Then the heavier the load on the video drivers the more the superiority of the Windows drivers takes hold, so for the graphical stuff things don't work as fast.
Congrats to the Wine devs!
To me this seems to be more a test of the linux implementation of teh video card drivers.. and NOT the wine system itself.
i think a wider suite of tests would be required.. and not just the preformance/gaming orinted stuff.
-- Shoe Lace
Notice, however, that the 60 some tests that Wine leads on are synthetic through and through... and when you get to actual games it's XP all the way. While Wine's performance is impressive, the requisite dose of salt may be several kg for this article.
Seems a bit strange to me to do a current comparison by using a version of 3d Mark that is 5 years old. If you were going to test out a 6800 on Windows alone you would use 2003 or 2005, the fact they didn't use that one in their Wine comparison suggests to me it couldn't run the later versions at all. The fact that 2000 ran better than under XP, but 2001 ran considerably worse suggests this as well.
If this is the case, the results in regard to game performance are out-dated at best.
Blessed are the 1337, for they shall pwn the earth.
While I assume that the results are correct, they are also substantially meaningless. First, Windows does a lot of work to handle backwards compatibility and boundary cases. WINE often breaks when it encounters really old code or boundary cases. It's easy to be faster when you are doing less work. Second, the functionality may not be exactly equivalent, just close enough. For example, when you have memory copies, one implementation may choose to optimize for very small allocations and be hitting the OS for new zero filled pages for 4k and 8k copies, while the other trades off higher memory use for faster fulfillment of these larger memory requests. Different design choices also mean that different amounts of work are done. Choosing the superior implementation involves looking at more than a small handful of benchmarks.
The Linux Kernel has some graphics support but neither the Kernel nor the X Window System are geared for fast 3D graphics. DirectX is good at getting around using the slower Windows GDI. DirectX is one of the few things Microsoft does somewhat well. (Insert joke here about Directx 9 and taking nine trys to get DirectX right.)
I have a feeling that unless some major changes are made to the X Window System (and maybe Linux drivers) that WINE will not catch up with WindowsXP and DirectX, but that just means I would need a faster computer.
WINE doesn't need to be the fastest. As long as it will run my older games (which Windows 2000 does not always do well) it may be more useful to me than an actual install of Windows.
---
This is just my opinion, please don't flame me just because you like Windows.
Losing faith in humanity one person at a time.
That's not unusual -- choosing the incumbent as a baseline is hardly an unreasonable choice if you are trying to do performance comparisons against a challenger.
Any program relying on (nontrivial) preemptive multithreading will be buggy.
Okay, I *like* WINE. It's a great piece of software, and very useful. But it doesn't make Linux a drop-in Windows replacement. If you decide "Gee, I think I should use Linux as a desktop" (I do), then it's icing on the cake if WINE runs something. It's not reasonable to simply expect a given piece of software to run flawlessly under WINE, though.
Any program relying on (nontrivial) preemptive multithreading will be buggy.
Nobody said they were doctored; the slashdot editor said "take it with a grain of salt". I see a lot of reasons to do so:
Honestly? The results probably aren't manipulated, but the presentation is very clearly set up with a number of tricks (perhaps without him/her realizing it) to give the impression that Wine "kicked some serious ass", when for the most part, it did horribly.
Please help metamoderate.
They often do things under very specific conditions and are useless elsewhere. If you're not a compiler expert (which I definately am not), it's unlikely you'll have any apreciable performance gain. It's quite possible that you'll see a performance loss if things aren't done correctly. Even an expert may or may not get a gain.
I love Gentoo myself, but I'm not delusional about performance gains. I do it for the customizability it offers. For instance, last I checked, you could get a Debian package that contained the Perl bindings for Vim, or the Python bindings for Vim, but not both. With Gentoo, I can easily instruct Portage to build in both.
Not a typewriter
As I understand it the essence of Wine is reverse engineering the Windows DLLs.
You might understand it that way, but you'd be wrong. All Wine does is implement the published API of Windows using Linux commands. Absolutely no reverse engineering is done.
What about higher end MS applications like VisualStudio or the .Net framework?
At some point you have to ask yourself why you are running Linux at all.
-matthew
"THERE IS NO JUSTICE, THERE IS ONLY ME." -Death
These results aren't presented to try to make Wine look better, nor is the author, consciously or unconsciously, trying to make it appear faster. These results simply were not meant to be used to say that Wine is better than Windows, and only Slashdot would try to make it appear that way. The real point of these comparisons, as is apparent if you read the Wine weekly summaries, is to give the Wine developers an idea of what areas need to be improved, and what areas are adequate. Green obviously means "at least as fast as Windows", which means that it's good. There is no point in grouping them any other way, since they don't care if they are 50% better or 1% better. Also, your criticisms of why this benchmark doesn't give a good idea of the relative speeds of Wine and Windows are quite wide of the mark (though they are valid complaints). The real reason why this benchmark cannot be used to gauge relative speeds is that it doesn't cover real world work loads. They measure a very small number of very specific things, mostly related to gaming and 3D performance. The benchmarks they ran that weren't related to that were designed to test the *hardware* speed, not the speed of the API. The Wine developers know this, and that's what the comment about taking it with a grain of salt means. It's probably adequate to give a rough idea of what parts of Wine need to be improved, but it is nowhere close to a comprehensive comparison of the speed of Windows and Wine, and was never meant as such.
Looking at the results - "Grammer Check"
Shame it was not "Spelling Check" but on still quite amusing.
In all seriousness, interesting and makes me want to revist Wine as it looks a lot better than when I last tried it (given I run pretty low spec hardware, performance is key rather than stability).
Though I do think "Wine or XP aborted on 18 tests" was a bit cheeky as it was 3 XP aborts, 15 Wine aborts...
Throughout most of my career, I've understood reverse engineering to be what you do when you DON'T have the source code. I think the wikipedia entry mentions that this is the most common interpretation. It can be extremely difficult and time consuming. I've done it on major projects a few times in my career. It is neither easy nor efficient, but sometimes the only choice you have.
It's also common to talk about reverse engineering hardware, where the innards of a chip may be deduced by observing its inputs and outputs. I worked on a project at a startup where we had to do that, because the original hardware developers were long gone, and no VHDL could be discovered, yet we had to write drivers for their (buggy) chips in order to make a deadline. At the same company we had to reverse engineer the workings of a (buggy) Fiber Channel PCI card, because the manufacturer would not give us the support we needed to make our deadlines. They were rather surprised when we talked to them about the details of their (proprietary, embedded in an ASIC) DMA engine that we deduced via logic analyzers and oscilloscopes.
Those projects were SO difficult compared to reading (even obscure) code, that I really think that using the one phrase for both activities is confusing and sometimes even deceptive.
I agree with you. When I was a kid we had 8 bit computers and you learned by doing. In those days you of fritzing your hardware by moving it too suddenly, causing a machine lockup with crappy code was the least of your problems.
Gentoo evokes that era for me, and you can't have a go at people for breaking something when they're busy learning. So what if they over-optimise compiler flags and break things? When they fix them up they're learning a valuable lesson, and that lesson isn't 'next time use a binary distro'
Anyway, today's school-age gentoo n00bs are tomorrow's crop of system admins.
Hal Spacejock: Science Fiction with Nuts
> To me that sounds as if reverse engineering has more to do with cloning products than just coming up with an alternative implementation which is what Wine is.
Reverse engineering is in fact almost always used for cloning. However, that's exactly what Wine is: a clone of various Windows components.
> All that Wine does is create a blackbox that works like the Windows API from the point of view of a Windows application but using totally different internals.
Reverse engineering isn't concerned with the internals, it's concerned with the effect.
> Reverse engineering would be if they had recreated the internals more or less exactly using components replicated as far as possible from the originals in which case they would have to recreate the Windows source code from binaries somehow.
No, that's not what reverse engineering means. Reverse engineering means "I'm want to make a thingy that works just like your thingy, but I don't have your blueprints, so I'm going to poke and prod your thingy as a black box, and mimic whatever behavior I observe."
> You could also point to the AMD processors that implement the x86 instruction set, internally they are radically different from the Intel originals but to Windows they function the same. If that was reverse engineering I somehow suspect AMD would have gotten it's pants sued of by now.
Intel did in fact sue whoever first cloned their x86 chips, but lost because the reverse-engineered chips did not violate any patents or copyrights.
One of the dangers of keeping trade secrets rather than patenting them is that someone can reverse engineer your goodies and market them without paying you anything.
Sheesh, evil *and* a jerk. -- Jade