Debugging a weird I/O problem on Mac OS X – Spotlight or something else?

I noticed an odd problem the other night when I was using my Mac.   I had Activity Monitor open and was on the Disk Activity tab.  Every couple of seconds there would be a spike in disk writes.  Unexplained disk writes because I wasn’t doing anything myself saving documents, etc.  I guess having Activity Monitor open is geeky enough because most regular people probably don’t even know it exists.   Being the geek I am I wanted to know why these regular spikes in activity were happening.   It turned out to be an adventure in learning some new tools and techniques as well as an eye opening experience in how one issue could so easily mask another.

I’ve been using Mac OS X for years now.  I’m pretty familiar with the Terminal application and the UNIX underpinnings of the OS.   I’m also familiar enough with it to know what some of the typical culprits are when these types of odd resource issues happen.   I’ve had similar problems in the past and the wonderful Spotlight search feature was the issue.

Spotlight is responsible for maintaining a search index of files on the Mac.   It is updated in real time as things happen on the system.   It is also responsible for searching that index when a user hits Command-Space and brings up the search box.  Spotlight has several programs it uses to index changes happening on the system:

  • mds – this is the metadata server.  Any program that wants access to the index mentioned above uses mds to do so, including Spotlight.
  • mdworker – this is the program actually responsible for indexing the system.   Several of these are typically launched, controlled by the current user account as well as various system-level accounts such as “_spotlight”.

These applications are run by another system process called launchd.  launchd is the second process started on the system at boot time. It is started by the Mac OS kernel and is responsible for launching all other applications.

As I said earlier this whole investigation started when I had activity monitor open.  When I saw the spikes I switched to the “Active Processes” view.   I also filtered the list using “md*” which basically got me the mds process and mdworker.  Sure enough every time I saw a spike in writing activity two mdworker processes would flash on as active then disappear until the next spike in activity.

So great.  It looked like Spotlight was the culprit here.   The last time this happened I forced the system to rebuild the Spotlight index which fixed the problem.  So I did the same thing this time using a great system maintenance utility I have called Cocktail.

%d bloggers like this: