Register Log in

Infinite Loop / The Apple Ecosystem

Mac OS X 10.3 Panther


Finder performance

Previous versions of the Mac OS X Finder suffered from a bad case of "spinning rainbow disc" disease. That multicolored spinning cursor appears when an application has stopped processing events for a few seconds. In Jaguar and earlier, the Finder would become totally unresponsive in a variety of common situations, most involving access to network resources. Panther substantially reduces, but does not quite eliminate this problem.

Network volumes can be mounted via the "Connect To..." menu item, as in Jaguar, but Panther also allows network browsing through the previously underused "Network" icon at the top level of the Finder hierarchy. Windows networking is now based on the more capable Samba version 3. Sadly, the Finder's half-hearted implementation of FTP still does not allow FTP uploads and still performs like a dog when compared to third party FTP applications. On the WebDAV front, Panther includes an option to create a local copy of your iDisk that is automatically synchronized in the background, presumably as a sop for continuing poor network performance in this area.

Most complaints about the Mac OS X Finder's performance have revolved around the topics listed above, particularly networking. But there's another, simpler aspect of a file manager's performance that seems to have been eclipsed by these more serious problems. Back in the days when Mac OS X was just a series of colored boxes on a slide and a few thousand rumors on the web, the Finder in the next-generation Mac operating system was imagined to be a file shuffling machine, able to tear through folders and volumes with thousands upon thousands of files without breaking a sweat.

The reality of the original Mac OS X Finder was that simply trying to open a window containing a few thousand files resulted in a total application lock-up and some more quality time with your old rainbow friend. Eventually, the action would resume, but God help you if that folder was on a network volume. You might as well go get a cup of coffee.

Similarly, selecting and then acting on large numbers of files and folders was fraught with lock-ups and long waits. Right-click on a selection of a few thousand files and you're in for a nice little vacation. Try to drag those same files and you will be holding that mouse button down for a long time as you wait for the Finder to start responding again. This is not the advanced, ultramodern Finder we all envisioned.

Matters have improved steadily, and Panther is faster than its predecessor when dealing with large numbers of files. But steady improvement is not going to cut it, not at its current rate anyway. Manipulating, or even just viewing large numbers of files is still orders of magnitude too slow in the Panther Finder, particularly when dealing with network volumes.

For example, I recently tried to scroll through a list-view folder on an SMB share containing about 2,000 files using the Panther Finder on a dual 2GHz G5. The server was on the local 100Mbps LAN. It took me several minutes of patient hand-holding as I dragged the scroll thumb through small portions of the file list, waiting each time for the Finder to slowly display the icons for the newly revealed files before it would let me continue.

The Finder would literally not let me scroll to any of the "unknown" portions of the file list until it had finished doing whatever it was doing to display the icons of the newly revealed files. These files were all of the same type, and were plain, flat files with no resource forks and no Mac-specific metadata. Only after fighting with the window for several minutes was I finally able to drag the scroll thumb from the top to the bottom in one continuous motion.

Listing that same directory from the command line is nearly instantaneous. Some overhead for the GUI is expected, but this is ridiculous.

Unfortunately, making a lightning-fast GUI file browser is actually a very hard problem full of special-cases and in need of clever optimization. The Mac OS X Finder appears to use a lot of standard Mac OS X controls and views. Performance improvements in the Finder have partially been a result of the optimization of these OS-provided widgets.

But while optimizing common controls is an important task that can significantly improve the performance of all applications simultaneously, certain applications simply have needs that are far beyond what generic controls can provide. The Finder is one such application. Whether through completely new custom controls or through clever subclassing, the Finder needs to implement its views with an eye towards efficiently handling thousands of files and folders with ease and speed.

Even in situations where the underlying data is the gating performance factor (e.g. slow network disks), the Finder must do everything it can to remain responsive, showing as little or as much information as it has at the moment, but never blocking the user from interacting with the GUI. Controls and views that expect to be backed by in-memory objects and data structures are poorly suited to this task.

Independent of any other interface issues, I hope to someday view, scroll through, and manipulate vast numbers of files and folders in the Mac OS X Finder with the thoughtless confidence befitting a modern operating system running on a multi-gigahertz machine with gigabytes of RAM. Sorry, Panther Finder, but you are not yet that file manager.

Taking a poll

There's one final aspect of the Finder's performance that deserves some mention here, even though it's actually a usability issue at its core...or rather, it should be. As any software developer will tell you, "Polling is bad, m'kay?" Polling is the act of repeatedly rechecking some condition. An example that's relevant to the Finder would be repeatedly asking, "Are there any new files in this folder? Are there any new files in this folder? Are there any new files in this folder? Are there any new files in this folder?" and on and on at some regular interval.

This, needless to say, is not the best use of CPU cycles, since the answer will be the same nearly all of the time: "there are no new files in this folder, now leave me alone!" Okay, so I added that last part, but the sentiment is apt. Polling is indeed bad.

But if you think about the situation described above, the apparent responsiveness of the Finder seems to depend on the frequency of that accursed polling. Let's say you download a file to your desktop using a web browser. If the Finder only polls the desktop folder every ten seconds, that file won't show up for an average of 5 seconds. That's hardly "responsive."

Another alternative is to simply update Finder views when the user (or another program) tells the Finder to do so. This eliminates costly polling entirely. Even better, if applications faithfully notify the Finder of any changes they make to the file system, there will be no lag at all between the creation of a file and its appearance in the Finder. This is the system that Apple has chosen for the Mac OS X Finder, and it sounds like a good one.

In practice, however, not all applications appropriately notify the Finder after making changes to the file system. A command-line utility from the FreeBSD world is the worst-case scenario. It certainly can't be expected to have Mac-specific code for sending notifications to the Finder. Files created by one of these "uncooperative" (from the Finder's perspective) applications simply wouldn't show up in the Finder at all. Yikes.

The Mac OS X Finder has worked around this problem by polling just once when the user starts to do something with a particular folder. For example, a file copied to the desktop with the command-line utility "cp" will not appear until the user activates the Finder by clicking on the desktop, a Finder window, the Finder's Dock icon, etc.

Now it seems like we've covered all the bases. There's no polling at all, and the Finder is reasonably aware of any kind of changes to the file system. But issues remain. Most frustratingly, sorted list-view windows in the Finder have an annoying habit of updating just as you try to grab a file from them. If you're not paying enough attention, the window will re-sort its contents (due to a newly added, deleted, or modified file) and replace the file you meant to grab with another file. It is all too easy to really hose yourself this way, especially when deleting files.

This whole situation—polling vs. responsiveness—is somewhat indicative of what I have always viewed as a problem with the direction of Mac OS X's development. Granted, polling is bad, but its existence in classic Mac OS was indicative of a value system that put the user experience first. "What is the best possible user experience?" is the primary question, only then followed by "How can we make that happen?" In Mac OS X, it seems to me that the first priority has been to do what is most efficient, and then try to provided the best user interface that can feasibly be implemented on top of that foundation. The shoe is on the wrong foot in such a system.

It may seem contradictory to swear up and down about poor performance on the one hand and then ask that the user experience take precedence over all other concerns on the other, but performance is part of a good user experience. Focus on the user and performance will mostly take care of itself...yes, possibly meaning that you will have to implement new OS features in service of the user experience that you hope to provide. But the end result will be a better overall system. Quartz is one example of this principle in action, but the Mac OS X Finder seems to be a result of the opposite value system.

So let's look again at the responsiveness of the Finder when dealing with file system changes. Let's start with the premise that the Finder should always provide the user with an absolutely consistent view of the file system. This is essential to maintaining the illusion of direct manipulation. Obviously polling still sucks, especially if we need to poll several times every second in order to meet our responsiveness goals. What can we do that's better?

What if the responsibility for tracking file system changes and notifying any interested parties was moved from the applications to the operating system kernel itself? Since all local file I/O goes through the kernel anyway, surely it knows when, where, and how every change is made. This is all without polling, of course. There's no need to poll when you're the one making all the changes.

Well, it's Apple's lucky day. Faced with a similar problem—how to track state changes without costly, inefficient polling—those industrious FreeBSD developers have created a generic OS-level facility for asynchronous event notifications. It's called kqueue, and you can read all about it in this PDF file. Since Panther has synched with FreeBSD 5, it includes kqueue support. Now all the Finder has to do is register for notifications on all open Finder windows (and the desktop) and just sit back and wait for the operating system to inform it of any interesting events. Problem solved, and in a way that puts the user first!

Unfortunately, the Panther Finder does not do this. This feature is not absent due to the difficulty of its implementation. After a few days of effort, a handful of Ars Technica readers were able to whip something up that does the job. You can find the source code and some instructions for compiling it in the forum thread. The code is only about two pages long, and the most difficult part is finding a way to ask the Finder which windows are open. The code has to (sadly) poll the Finder periodically to gather this information. Apple's Finder developers would not have this problem.

So, why is there (apparently) no kqueue support in the Panther Finder? My guess is that Apple plans to add a new (or reimplement an existing) higher-level notification facility on top of kqueue, with APIs in CoreFoundation, and then Carbon and Cocoa on top of that. That's much more in keeping with Mac OS X's API structure than asking the Finder team to use the (decidedly Unix-y) kqueue APIs directly. But I could be wrong, and it could just be a lack of time that kept kqueue out of the Finder. Either way, in Mac OS X 10.4 the Finder team should proclaim "kqueue or bust."

Expand full story

You must to comment.

You May Also Like

    Need to register for a new account?

    If you don't have an account yet it's free and easy.