Slashdot Mirror
How Snow Leopard Cut ObjC Launch Time In Half
MBCook writes "Greg Parker has an excellent technical article on his blog about the changes to the dynamic linker (dyld) for Objective-C that Snow Leopard uses to cut launch time in half and cut about 1/2 MB of memory per application. 'In theory, a shared library could be different every time your program is run. In practice, you get the same version of the shared libraries almost every time you run, and so does every other process on the system. The system takes advantage of this by building the dyld shared cache. The shared cache contains a copy of many system libraries, with most of dyld's linking and loading work done in advance. Every process can then share that shared cache, saving memory and launch time.' He also has a post on the new thread-local garbage collection that Snow Leopard uses for Objective-C."
snow leopard stories!
Also sounds like the prelink application in Linux.
Or the GAC
I take it since then shared library machine code has had to be patched in memory after it's loaded for a while now, thus preventing easy sharing among processes, and causing the page to need its own space in the swap file.
Sounds like this latest improvement effectively brings things back to the way they were, by effectively writing this patched version back to disk so that it can be mapped read-only as before, and not have to be patched every time the library is loaded into a process. It's odd, because I thought the OS already did this several versions ago when prebinding.
Did you even read the article ? Suppose not... this is slashdot after all.
The article states that prebinding (similar to prelink) was used in previous versions of OS X and has been replaced by a much faster shared cache.
The intent is to improve performance in situations where running an anti-virus scan or back-up utility would result in otherwise recently-used information being paged out to disk, or disposed from in-memory caches, resulting in lengthy delays when a user comes back to their computer after a period of non-use.
Everybody obviously knows that no viruses exist for the mac so your assertions are asymptotically false.
Once you start despising the jerks, you become one.
And reintroduced dll hell as well?
Well okay then, Apple were the ones who "popularised" it! ("Well I hadn't heard about Superfetch, but I heard about Apple doing it first, therefore, Apple did it first")
Or um ... they "integrated" it better. Yeah, that's it.
dyld - noun. A reminder that regardless of age, you'll always have an adolescent sense of humor.
no. we've had a variant but lesser hell for years. Which has led to a series of cargo cult like maintenance procedures for Mac OS.
Nothing in the world is more dangerous than sincere ignorance and conscientious stupidity.
It's nothing like Superfetch. Superfetch preloads applications into system memory and this shared cache doesn't do that instead from what I understand it preforms some of the work the linker would do on load in advance.
If all else fails, immortality can always be assured by spectacular error.
Sounds like they've just updated their dynamic (shared) library loader to be able to handle Objective C (aka Cocoa) instead of just plain C, and to be a little smarter about keeping track of what it's already got going on, so it doesn't duplicate things.
As a long-time UNIX and Linux (and other more esoteric OSes) geek, this alone doesn't impress me too much. The idea that they went through the whole OS and worked to get little efficiency/performance gains like this all over the place impresses me a little more.
Village idiot in some extremely smart villages.
What's the difference between this and sticky files?
I hoped my ridiculous words at the end would make people laugh, but I know most slashdotters lack a sense of humor. :) Tis life.
Once you start despising the jerks, you become one.
Well, the FP is claiming that they're the same thing, and your post seemed to be agreeing with him. But yea, that troll rating probably should have gone to the post you replied to.
I don't know anything about prelink, but Superfetch sounds completely different from dyld. Superfetch keeps frequently launched applications in memory to make them launch faster (much like Winamp Agent does for Winamp). dyld, OTOH, shortens application launch times by not reloading a shared library each time an application is launched. Keeping the shared library loaded in a shared cache also reduces the number of copies of that library you need loaded in memory. It doesn't sound like Superfetch does that.
Both a turbocharger and a cold air intake can improve car performance, but that doesn't make them the same thing.
Actually, it is interesting I was marked troll for quoting some text that the original poster linked. O.o A view into the minds of the moderators.
Once you start despising the jerks, you become one.
I do not wish to be a poo poo, but since dynamic libraries and shared libraries have been around for just about forever, when even a second year CS major would immediately notice this could be done, is such big news now?
The first thing I would have done is built a cache for the library system. LINUX has one, why not the Mac?
So certainly I congratulate the Mac community. But wow, DUH, a cache for the linkage editor. :-)
-Hack
Got Geometrodynamics? Awe, too hard to figure out? Too bad.
The shared libs are shared. What was not shared before is the linking table, that must be built for accessing that shared code. prelink precalculates that table, and this apple thing does more or less the same.
And maybe kdeinit does something similiar since 2003?
You phrased things much better than I could fine sir. The link to the wikipedia article that the original poster included read nothing like TFA. Though because of the glaring placement of the post toward the top of the page and how straight forward it is written, I think most mods probably modded it without checking against either of the articles. That is how human nature is :). It's nice that slashdot moderation tends to correct itself over time thanks to insightful replies lower in the tree.
Once you start despising the jerks, you become one.
Moderators, please mod the parent down -- it completely misses the point.
Objective-C selector uniquing caching is NOT the same as Windows Superfetch.
Objective-C uses a two-phase dispatch for method calls. When you see a call in the Objective-C source code that looks like:
the dispatch system:
The problem arises in the method dispatch table when you have multiple methods named "init" -- which is very common. When an application is loaded the dynamic loader ("dyld") needs to separately identify all of the methods named "init" (and any other methods with conflicting names) that apply to different classes. This is done by "tagging" each method in the dispatch table, a process called selector uniquing.
Now, this has to be not only for the application binary itself, but also for any Objective-C classes in shared libraries that are loaded. Almost all apps on Mac OS X load the libobjc.dylib library, which is cached to improve performance. As a part of the caching process, Snow Leopard now does the selector uniquing only once, and then stores the uniqued selectors in the cache. Thus, any application that links against libobjc.dylib (or any other library that is in the cache) only has to unique its own selectors, not those of the library as well. This significantly reduces the amount of overhead for launching an application compared to previous versions of Mac OS X.
This process does not attempt to retain application binary code in memory in the face of page-outs as Superfetch does. Selector uniquing caching speeds application launch times by reducing the amount of computation that has to happen at launch, not by pre-loading the application's binary.
Thread-local garbage collection is NOT the same as Windows Superfetch.
Thread-local garbage collection is a third phase of garbage collection added on top of the Objective-C 2.0 garbage collection system, which speeds up the garbage collection system even further. By concentrating GC to what has occurred in a single thread, the GC system can delay and reduce the cost of a slow global sweep even beyond the generational GC algorithm.
Windows Superfetch is a response to poorly written software.
To quote from the Wikipedia article:
The intent is to improve performance in situations where running an anti-virus scan or back-up utility would result in otherwise recently-used information being paged out to disk, or disposed from in-memory caches, resulting in lengthy delays when a user comes back to their computer after a period of non-use.
In my opinion as an experienced application developer the user should never run into the problem that Superfetch attempts to solve. Anti-malware scans or backups are generally limited by I/O transfer rates, not by CPU. In such situations, using lots of memory to pre-load data makes no sense. It is relatively easy to write a two-buffer, threaded, streaming system for situations that are constrained by disk transfer rates without consuming scads of memory.
In the bigger picture, Superfetch attempts to learn the times of day when apps are used and pre-loads their binaries. This is a nice concept, but I have serious doubts as to how useful it really is. The penalty for guessing wrong is fairly high, and users are more tolerant of consistent small slowdowns than they are of occasional long hangs (see the Mac literature on the spinning beach ball).
Mac OS X is less likely to need such anti-malware scans in the first place as the application binaries are now digitally signed by the developer. Any malware that attempts to insert itself into applications will run into problems. This is not to say that the Mac is immune -- I can think of a number of holes that could be exploited (such as the fact that unsigned binaries w
For reference, normally when a program is launched without prebinding, the program has to look into the symbol table for the shared library and "bind" it (basically, tell the program where it is). Prebinding basically does that in advance and saves the lookup table, but any time the library is changed, the bindings have to be regenerated.
The article says prebinding is actually quite efficient for C/C++ code, but objective-C (used by macOS X and iPhone) is structured more like Smalltalk or Java, and uses selectors, which I believe can't be prebound (for you java programmers, these are equivalent to interfaces - C/C++ does not have this concept and instead allows direct access to the classes using protected or public) to interface into classes and these are instanced once for every application accessing that shared library. According to TFA, by keeping a single cached copy of the selector, they avoid the memory overhead of keeping individual copies. Since the OS itself has over 30000 selectors, you can imagine this cuts overhead by quite a bit, especially with commonly loaded libraries like Cocoa.
For people comparing this to superfetch, it's not really the same thing - superfetch was pre-loading heavily used libraries into memory to avoid the delay in loading them during start time, and this is caching the library lookups onto disk that may or may not be memory resident at any particular time.
I'm not sure whether there are people out there writing Objective-C apps for the Mac without Cocoa, though. I guess there's always someone who won't use the nifty library and shortcuts and all that, because they're hardcore, efficiency nuts, or just masochists...
Or they use only a subset of Cocoa because they plan to port the app to GNU/Linux, *BSD, and Windows using GNUstep, an OpenStep-compatible toolkit that implements only some of Cocoa.
It's nothing like Superfetch. Superfetch preloads applications into system memory [microsoft.com] and this shared cache doesn't do that instead from what I understand it preforms some of the work the linker would do on load in advance.
The whole dyld sounds a lot like some of the basic features of the .NET runtime...
Or maybe some of the features in this advanced futuristic os:
http://blogs.technet.com/askperf/archive/2008/02/06/ws2008-dynamic-link-library-loader-and-address-space-load-randomization.aspx
mod parent : should be the article itself :D
SGI did this linking trick in the early '90s - they called it Quickstart.
Objective-C uses a two-phase dispatch for method calls. When you see a call in the Objective-C source code that looks like:
the dispatch system:
C++ follows the same steps. The big difference is that in Objective-C, the dispatch table is an associative array (C++ unordered_map, Java HashMap, Python dict) from strings to function pointers, not a plain array (C++ vector, Java ArrayList, Python list).
In my opinion as an experienced application developer the user should never run into the problem that Superfetch attempts to solve. Anti-malware scans or backups are generally limited by I/O transfer rates, not by CPU. In such situations, using lots of memory to pre-load data makes no sense. It is relatively easy to write a two-buffer, threaded, streaming system for situations that are constrained by disk transfer rates without consuming scads of memory.
But then you rely on the operating system to provide a method for applications to provide cache hints, and you rely on the antivirus software to provide such hints. SuperFetch tries to infer these even for applications developed prior to widespread knowledge of these hints or ported from systems that lack these hints.
In the bigger picture, Superfetch attempts to learn the times of day when apps are used and pre-loads their binaries. This is a nice concept, but I have serious doubts as to how useful it really is.
Having my applications ready to start at 08:57 when I'm about to grab the mouse at 08:58 improves my productivity. Consider that employees have sued their employers for requiring that employees be present during application startup time but not paid until the application has fully started up.
Mac OS X is less likely to need such anti-malware scans in the first place as the application binaries are now digitally signed by the developer.
But who signs the developer's certificate? And what keeps malware publishers from signing their trojans?
Any malware that attempts to insert itself into applications will run into problems.
Unless an application tries to insert itself as, say, an assistive technology using the accessibility API.
Windows had this in 1995. The whole idea behind PE style DLLs is that you can just read in the module from disk and you can just use it as is. If the default loading address is available, which is normally almost always the case, you can use all the prebindings and you don't even have to patch your I/E tables. The only reason I won't mention the DLL cache is that that was only added in 2000 or so.
A lot of people seem to be confused about operation of "shared" libraries.
Shared libraries on most platforms (mac,linux,windows..etc) are "shared" within a single processes memory space only. They are NOT "shared" between processes. Each separate process instantiates libraries separatly.
Apples change is to globally save some library state data to reduce each processes instantiation overhead when common libraries are used by multiple processes.
Dude, get over it. Sometimes mods are stupid. That's life. I'm sorry someone foolishly marked you as a troll, but you really need to move on.
Perhaps Obj-C has a few nice features but personally I don't see it. If they'd stuck to C or C++ like every other version of Unix then this would never have been an issue in the first place. Plus a lot more people would have been able to cross-code for OS/X without having to learn an obscure OO version of C which never caught on in the wider IT world and is still used on practically no other system.
"In other words, Apple just re-invented Superfetch" - by LO0G (606364) on Sunday September 06, @10:36AM (#29331275)
On that note? Well, I could say that Microsoft only "ripped off" ideas I've been using since the early 1990's (first on software ramdisks, & later on SSD's too, per how I use them noted in this post here on /., today (& long before THAT, many times here & elsewhere online)):
http://hardware.slashdot.org/comments.pl?sid=1359547&cid=29332071
Microsoft's only using a VARIATION OF MY IDEAS.
Ideas that took EEC Systems/SuperSpeed.com to a FINALIST position @ Microsoft's own Tech-Ed, 2 yrs. in a row, in its hardest category: SQLServer Performance Enhancement, albeit not for "home/end user oriented tasks" (which I list some of above & how I use it @ home @ least)...
That was for "back office app" perf. enhancement, & that url above shows proofs of that much for WebServers, File Servers, & DB Servers (from TechReport.com, in regards to Webserver, FileServer, & DB Server performance gains when using a TRUE SSD, like the IRAM).
E.G.-> AND? When that work showed improvements on SQLServer 6.5 & below, which used HDD space for various ops? Microsoft later "turned around", & made those ops take place in RAM...
(Which amounts to the SAME THING as doing it on a ramdisk in software really, & MUCH FASTER than HDD space could do it (MS uses a dedicated array/buffer in RAM now in SQLServer, because it works, for superior performance on DB work, bigtime)).
APK
P.S.=> More proof of my involvement here in this arena, goes back in WRITTEN PUBLICATION no less, as far back as 1996, here:
Windows NT Magazine (now Windows IT Pro) April 1997 "BACK OFFICE PERFORMANCE" issue, page 61
(&, for work done for EEC Systems/SuperSpeed.com on PAID CONTRACT (writing portions of their SuperCache program increasing its performance by up to 40% via my work) albeit, for their SuperDisk & HOW TO APPLY IT, took them to a finalist position @ MS Tech Ed, two years in a row)... apk
Also, so what if they're "different languages"? If they were the "same language" there'd be nothing to compare. Do you go around comparing your right eye to your right eye?
Over-the-top Response Guy! Giving "Over-the-Top Responses" since 1970.
selectors, which I believe can't be prebound (for you java programmers, these are equivalent to interfaces - C/C++ does not have this concept and instead allows direct access to the classes using protected or public)
I'm sorry, but this and most of the rest of your description is completely wrong. Selectors are nothing like Java interfaces. Interfaces are Java's version of Objective-C Protocols. Selectors are abstract method names (Smalltalk calls them symbols). Each Objective-C class has some data structure mapping these to function pointers. When you send a message (call a method) you look up the function pointer corresponding to the selector in the receiving class. To make this fast, all selector comparisons are done as pointer comparisons. To make this work, the runtime needs to make sure that selectors are unique. This process involves building a large hash table and inserting every selector referenced by every compilation unit into it. By making the linker handle this uniquing, you have several advantages. The first is that the resulting table can be shared more easily between processes, resulting in a memory efficiency gain. The second is that the runtime can first try doing pointer comparison when registering a new selector, and only use the hash if the linker didn't unique the selector.
I am TheRaven on Soylent News
Okay, whoever modded the above post as "Troll" is confused. Talking about the moderation system on a thread about some other topic (e.g. Snow Leopard) isn't a "Troll", it's "Off-topic".
"Convictions are more dangerous enemies of truth than lies."
So, Objective-C requires selectors (such as for message names) to be interned, and the old way was to intern all selectors at process startup time when the dynamic linker does its work; but the new way is to cache the interned selectors both on disk (faster startup) and in memory (even faster startup plus saved memory overhead). Is that correct?
Also sounds like the prelink application in Linux.
No, OS X has always done that. Except they call it prebinding.
It is cowardly, and a betrayal of whatever it means to be a Jew, to act as a white man
-James Baldwin
Do you always throw a fit when you get modded down? I'm gonna bookmark this and give you the "overrated" treatment next time I get mod points, be sure to post crying about it.
The intent is to improve performance in situations where running an anti-virus scan or back-up utility would result in otherwise recently-used information being paged out to disk, or disposed from in-memory caches, resulting in lengthy delays when a user comes back to their computer after a period of non-use.
In my opinion as an experienced application developer the user should never run into the problem that Superfetch attempts to solve. Anti-malware scans or backups are generally limited by I/O transfer rates, not by CPU. In such situations, using lots of memory to pre-load data makes no sense. It is relatively easy to write a two-buffer, threaded, streaming system for situations that are constrained by disk transfer rates without consuming scads of memory.
I don't think you understood it right: the perf problem is not for the anti-malware programs, but once they have run they have thrown everything out of the cache and subsequent applications have to re-populate it again, thus slowing eveything down. There used to be the same kind of problem under linux after the 'locate' cronjob.
Almost. The old way was to do it at module load time, in the Objective-C runtime library (libobjc), the new way is to do it in the loader (dyld). The loader caches the result of everything it does, including uniquing symbols (two symbols with the same name are resolved to one or other instance), while the runtime library does no caching.
I am TheRaven on Soylent News
Thread-local garbage collection is a third phase of garbage collection added on top of the Objective-C 2.0 garbage collection system, which speeds up the garbage collection system even further. By concentrating GC to what has occurred in a single thread, the GC system can delay and reduce the cost of a slow global sweep even beyond the generational GC algorithm.
Do you know how they detect that a pointer has escaped? Is there some sort of write barrier? How does this work with the somewhat unsafe base language?
Dyld's (Dynamic Loader) have existed before Snow Leopard. They are extensively used in all Mac OS X versions since they are at the base of the system. They also appear in BSD and Linux under slightly different names. This just explains how they found out a way to do caching and preloading better than previously. It's like Microsoft finding out a way to automagically load all necessary dll's and correctly find out the dll amongst several different versions of the same dll for a program and preload them before the program even needs them. It does so correctly and consistently every time, they just sped it up now. I don't know if Windows DLL's even do versioning but from what I remember (latest experience was last week in XP) if one program writes over the other programs dll, you're shafted (I'm looking at you Aladdin USB/DRM Keys).
Custom electronics and digital signage for your business: www.evcircuits.com
Depends on how you look at it. I was talking about how my post was rated troll which could and usually does turn into a talk about the content itself (which is on topic). A number of posts from other people on this article have been inaccurately moderated "Troll" for some reason though.
At his point, I do believe we may be both off topic (thanks to the mods). It will be interesting to see if you get rated off topic, and this post here gets rated troll, too. Here's hoping for helpful meta-moderation.
Once you start despising the jerks, you become one.
I understand, and Apple did a great job with it. It just really disturbs me when Apple (and their ilk) keep rehashing work previously completed on other platforms and claiming that they somehow invented it. I am yet to see a feature that's really *new* in Snow Leopard, yet I can't stop hearing about how many amazing new technologies Apple has created.
Apple did an intelligent new optimization of their dynamic loader on their platform. This is a good thing. They did not invent the concept of a dynamic loader.
Why don't they (any OS) just add something onto the generic filesystem caching layer to keep executable bits in RAM as long as the input files stay the same? If it was done that way you could theoretically reuse it for interpreted code as well.
Sir, I believe you are mistaken. There obviously must have been at least one virus at some point for a Mac otherwise why would Apple include virus and malware protection in Snow Leopard. Only now will none exist since Apple has deemed it so by not allowing the rogue virus to infect their hardware. :P
Perhaps Obj-C has a few nice features but personally I don't see it. If they'd stuck to C or C++ like every other version of Unix then this would never have been an issue in the first place.
But you can use either of those perfectly well mixed with ObjC calls.
ObjC is a relatively small set of additions to standard-C, so it really doesn't take that long to pick up the syntax changes if you've encountered C before, while at the same time it allows for some very nice dynamic behavior and things like introspection. The language has evolved to support things like garbage collection (though that specific feature is not available on the iPhone due to performance constraints).
What you are really overlooking though is that Objective-C that Apple uses, has a very rich and diverse set of foundation classes (some inherited from the NeXT days) - just as wide in scope as Java. Any modern language simply has to have a giant toolbox to help get common tasks done, and that's going to be the thing that takes the most time to learn. Happily, Objective-C has a fairly consistent set of tools and conventions, that make learning new parts easier once you have learned a few others.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Apple II had this in 1979. Back then we called it a "jump table". :)
I work for the Department of Redundancy Department.
There is a write barrier. Every pointer assignment is turned (by the compiler) into a call to the GC. You can find the code for this in the clang repository in lib/CodeGen/CGObjCMac.cpp (it's quite easy to read, in spite of being C++). If a pointer is assigned to a global or to an object that is marked as belonging to another thread, then it is treated as having escaped. The same is true if you call CFRetain() on it (which increments the ref count and tells the GC not to free it until the CFRelease() has been called.
I am TheRaven on Soylent News
Gretchen, stop trying to make Superfetch happen. It's not going to happen.
Green Monkey
I'm not so sure about that. Typically, code in shared libraries is re-entrant and code pages are loaded once, then mapped into the address space of the process using them. I don't know of any modern OS that wastefully makes a copy of the library's code for each process.
I don't know about other platforms, but I wouldn't be surprised if they do something to share the memory as well.
Dude, I think you're hearing things that people aren't saying. The article describes how apple has improved their dynamic loading. The author doesn't claim they invented it. Apple doesn't claim they invented it. Nobody said anything about amazing new technologies Apple created.
Check out DRM-free movies at http://www.bside.com
They probably just rolled in the 2.6.31 kernel.
Using OS X/Mac since 10.2.8, I haven't seen another abused tool like "update_prebinding" even while it is a very risky process in pre 10.5 systems since it deals with actual binary headers.
Also thanks to uninformed IT blogs etc, people always considered prebinding a thing which will go away in next release. Like, Apple is really stupid to do such thing. They basically misunderstood the added flexibility to prebinding scheme where tools without (or broken) prebinding will continue to run.
Anyway, want to see how much Apple users are abused by some shareware developers? Just watch for a applescript to basically issue this command and asks for money or donation. I don't have 10.6 but on 10.5, in its FIRST line of man page (man update_prebinding), Apple states "normally, there shouldn't be any need to issue this command manually" or something equivalent to that. On pre 10.5, like 10.4.11, it can get catastrophic if you keep doing it, as explained on article http://unsanity.org/archives/mac_os_x/shock_and_awe.php
Unless you lived a power loss in middle of a OS X/Quicktime update or kernel crash, there shouldn't be ANY reason to manually update prebinding. In fact, it can lead to a horrible cache fragmentation which may slow things down. Don't fix a working thing.
What the fuck? Nobody claimed Apple invented a dynamic loader. Nobody said this was a new technology. It's just a technical article about how Apple improved app launch time in Snow Leopard.
Applesoft BASIC on the Apple II mapped keyword tokens directly to a jump table of ROM entry points.
Other than the fact that it was not object-based, didn't reference system or external libraries, and all of the token offsets were fixed and predetermined...
It's exactly the same. :)
Any sect, cult, or religion will legislate its creed into law if it acquires the political power to do so.
The intent is to improve performance in situations where running an anti-virus scan or back-up utility would result in otherwise recently-used information being paged out to disk, or disposed from in-memory caches, resulting in lengthy delays when a user comes back to their computer after a period of non-use.
In my opinion as an experienced application developer the user should never run into the problem that Superfetch attempts to solve. Anti-malware scans or backups are generally limited by I/O transfer rates, not by CPU. In such situations, using lots of memory to pre-load data makes no sense. It is relatively easy to write a two-buffer, threaded, streaming system for situations that are constrained by disk transfer rates without consuming scads of memory.
This part right here makes no sense.
Small talk????
Obj-C is a union of smalltalk and C.
Just my 5 sents even if i have no clue.
The intent is to improve performance in situations where running an anti-virus scan or back-up utility would result in otherwise recently-used information being paged out to disk, or disposed from in-memory caches, resulting in lengthy delays when a user comes back to their computer after a period of non-use.
In my opinion as an experienced application developer the user should never run into the problem that Superfetch attempts to solve. Anti-malware scans or backups are generally limited by I/O transfer rates, not by CPU. In such situations, using lots of memory to pre-load data makes no sense. It is relatively easy to write a two-buffer, threaded, streaming system for situations that are constrained by disk transfer rates without consuming scads of memory.
I don't think you understood it right: the perf problem is not for the anti-malware programs, but once they have run they have thrown everything out of the cache and subsequent applications have to re-populate it again, thus slowing eveything down. There used to be the same kind of problem under linux after the 'locate' cronjob.
In these situations it's usually not the disk cache that's the problem, it's what binaries the OS has loaded in RAM. If app A is inactive and app B requests a lot of memory, the OS will swap app A's binary out to the VM backing store. When app A becomes active again there is a delay while its code is reloaded from the VM backing store. There is no reason for an app such as an A/V program or backup program or locate (all of which are I/O-bounded) to require huge amounts of memory other than the developer not realizing how to scale to handling large volumes of data.
What's happening is that the A/V app is just loading the entire application that it wants to scan into memory, then scanning it. This is unnecessary, especially if the app is really big and has lots of non-code resources (e.g., graphics). It's easy for the programmer, but bad for performance; indeed, it may cause problems for A/V performance as well if the app binary is larger than the amount of available memory, causing the A/V program to thrash in the VM system directly. Instead, create two buffers (each one around 1 MB as a good starting point, tune according to available memory, disk transfer rates, and system loads) and two threads. Thread 1 loads the buffers from disk -- its conditions are:
Thread 2 does the actual A/V scans:
You can do more improve efficiency by more threads, blocking on semaphores, and waiting on locks, but you get the idea. Note that there is no point in more than two buffers, since Thread 2 will always be done long before Thread 1. If the malware scan requires comparing widely separated parts of the target binary you may need to cache portions of it, but there's still no reason to hold in RAM the vast majority of the target binary.
--Paul
This line is posted with BOLD text and one word randomly capitalised! Have you ever been so far as to want to look more like that? I was wearing an Onion on my belt, which was the style at the time...have you ever... ever felt like this? More random bold TEXT.
Have I ever told you ABOUT the time I held Bill Gates TOWEL?
P.S =>_|_|_|_|_+-+-+-+-+-+-!>!>!>!>! Whoop whoop whoop whoop! Nurse!
I just checked and a 1-year code signing cert from Comodo is $179.95, with discounts for multi-year certs. Other vendors also seem to have pretty reasonable prices.
That's at least on the order of $100 per platform. The certificate for Windows is $179.95 per year, and the certificate for a secure web site from which to distribute copies of the software is another $99 per year. It gets even more expensive to target more than one platform: the certificate for XNA is $99 per year, the certificate for iPod Touch is $99 per year, and by the time one has ported an application to all the platforms that his audience uses, he'd be out of his hobby money.
Anyone who has the time to put together a serious app (even for freeware) can afford that amount.
Say I develop a video game and want to distribute it to the public. How would I recover the nearly $480 per year across three platforms without selling either copies or advertisements? It's almost enough to push someone toward a less expensive hobby.
See subject line above, & rinse/lather/repeat.
selectors, which I believe can't be prebound (for you java programmers, these are equivalent to interfaces - C/C++ does not have this concept and instead allows direct access to the classes using protected or public)
I'm sorry, but this and most of the rest of your description is completely wrong. Selectors are nothing like Java interfaces. Interfaces are Java's version of Objective-C Protocols. Selectors are abstract method names (Smalltalk calls them symbols). Each Objective-C class has some data structure mapping these to function pointers.
Although I will agree with you that GPP is somewhat misinformed I take issue with your statement that selectors are nothing like Java Interfaces.
It is true that the class structure of Objective-C (one root NSObject class, at least in common practice) and the class structure of Java (one root Object class) are virtually identical. And it is true that an Objective-C protocol has feature parity with a Java Interface and when you think of formal interfaces in Java the equivalent to that in Objective-C is a protocol.
So Java has anObj instanceof SomeClass which will indicate that anObj is an instance of SomeClass or an instance of some other class that derives from SomeClass. The Objective-C equivalent to this is [anObj isKindOfClass:[SomeClass class]]. And Java has anObj instanceof SomeInterface where the equivalent in Objective-C is [anObj conformsToProtocol:@protocol(SomeInterface)].
But then Objective-C also has this nifty thing [anObj respondsToSelector:@selector(doSomething:)] which does exactly what it says and allows you to see if the object will respond to the doSomething selector that takes one argument. Java has no analogue to this. I mean, you can sort of fake it using reflection to find methods but it isn't quite the same thing.
The bottom line is that when a unique selector can be looked up like this each selector functions almost as if it were its own Java-style interface. There is clearly a parallel between Java if(anObj instanceof DoSomethingInterface) ((DoSomethingInterface)anObj).doSometing(1); and Objective-C if([anObj respondsToSelector:@selector(doSomething:)]) [(id)anObj doSomething: 1];
From a coding standpoint where I would think to use a respondsToSelector: in Objective-C I wind up making an interface containing exactly 1 method in Java. Sometimes it's the right choice to add the extra lines and make an interface (and if so, then you should add all the extra lines and make a protocol in Objective-C). But often times I find the required formality of Java to be a distraction.
For thoroughness, would you mind defining 'module load time' in the Objective-C context for me? I am familiar with the language but much less familiar with its runtime support components.
When you compile any C-family language, you get an object code file. This contains symbols which will be resolved by either the static or dynamic linker and various sections. One of these sections contains an array of pointers to functions which are called by the loader when that object file is loaded. These are not used in C. In C++, they are used for static constructors. In Objective-C, there is one that issues a call to __objc_exec() with a pointer to the struct objc_module created for that compilation unit as the argument. This registers all of the selectors, protocols, classes, and categories declared in that compilation unit with the module.
When you statically link object code files together, these sections will be concatenated. When you then load the resulting object code file (either an executable or a binary), the loader will iterate through the combined section and call each function in turn. This is how you get code executing before main() in a program.
I am TheRaven on Soylent News
Thanks. The part about Objective-C using that mechanism to register, on a per-compilation-unit basis, selectors, classes, and the like is news to me. :)
See subject and lay off the profanity and coffee.
With Objective-C, a method may not even exist at compile time. It is impossible to define formal interfaces in many cases. respondsToSelector allows you to ensure that any arbitrary object walks, swims, and quacks like a duck without strict class definitions.
...an array of pointers to functions which are called by the loader when that object file is loaded...In C++, they are used for static constructors.
There are no static constructors in C++. Your other posts indicate that you're well-informed, so I assume you wrote that as shorthand for objects declared at namespace scope and static data members, but readers unfamiliar with C++ might have gone away misinformed. Also, IIRC nothing in the C++ standard requires static initialization to be handled by the loader (some platforms may not even have anything identifiable as a "loader"), but maybe you were referring only to Mac C++ implementations (I have no experience there).
- T