Musings on Photography

This morning I connected my new GPS receiver to my Mac Pro, fired up ImageIngester Pro, and downloaded and geotagged photos I took yesterday.

It worked perfectly and seamlessly. Each image is tagged with a location, and the locations seem to be correct. All I had to do was tell ImageIngester to read the tracklog out of the GPS unit, and it did all the rest.

Especially interesting to me was that I just turned the GPS receiver on when I started the walk, stuck it in my pocket facing out, and then took it out at the end of the walk. No camera mount, nothing.

Pretty slick, if you ask me.

The photo above is the Mac Mini based fileserver described in this post. The square white thing under the disk drive to the right is a bit of styrofoam packing material I stuck under the disk drive because the vibrations of the disk drive were making the countertop resonate and filling the room with a loud hum. The big off-white thing to the left is a UPS (uninterruptible power supply) that can run the server for about fifty minutes from the battery. (I could probably have gotten by with a smaller UPS; it only has to keep the server alive for the 30 seconds before our backup generator kicks in. If the power is out for more than about 40 seconds it means the generator failed to start, so the server is actually configured to shut down gracefully after three minutes on battery.)

Several folks have asked what I do for offsite backups. Since I just ordered new gear to revise my offsite backup scheme and make it easier to keep up, I’ll detail the new scheme.

What I bought were two one terabyte Western Digital MyBook external disk drives. Western Digital offers a confusing array of Mybook models which vary in case color, shape of the status lights, and a minor change in case dimension, all of which are of little importance to me. They also differ in the format of the filesystem the disk ships with; I always reformat a disk before using it (after reading about disk drives shipping from factories pre-infected with viruses), so that part doesn’t matter to me either. And finally, the important variation - the variety of interfaces the external disk can support. Nearly every combination of USB 2.0, Firewire (aka IEEE 1394a), Firewire 800 (aka IEEE 1394b), and eSata is offered. In general I choose the disk model with the largest number of interface options, to preserve my choices into the future. This most recent purchase, though, I bought the MyBook Home Edition, which gives me USB 2.0, Firewire 400, and eSata, but not Firewire 800. That’s because a) the Mac Mini doesn’t have Firewire 800, and b) I was able to exploit a price break.

So here’s my basic plan:

1. To start, I copy everything onto one of these disk drives. I then give this disk to my friend Bryan, who generously stores it in his house.
2. Then, once a month or so, I take the disk I still have, copy everything onto it, and exchange it for the one at Bryan’s house. The new disk goes back to Bryan’s place, and the one that had been at Bryan’s comes home, to be used the next time around.

This way, if some calamity results in the loss of every copy of data stored on the redundant servers at my home, I can go, get the disk from Bryan, copy it off that (now very precious) disk, and I’m up and running without any data loss. What sort of calamity might cause me to lose everything? The two big risks are probably a fire (my home was threatened by a forest fire just a few days after we moved in), or a burglary where the burglars are very thorough and make off with ALL the disks that comprise my redundant system of servers and workstations.

You can reduce the risk even further, by having multiple offsite copies. You don’t need to add two disks for every offsite copy. If you want N offsite copies, you’re fine with N+1 disks; you stagger the replacement and so you only need one extra disk beyond the disks stored offsite.

Finally, the reason why I don’t have one disk for each offsite backup, and go and get the disk, write a fresh backup onto it, and then replace it offsite - that’s because I know that what will happen is I’ll retrieve the disk, then it will sit around for a while before I remember to write a fresh copy, and then more time will elapse before I remember to put it back at the offsite location. In the end, if I do this, the offsite disk is actually offsite a small fraction of the time - and thus doesn’t offer any increased protection.

I’ve looked into doing offsite backup on a offsite server, transferring the data via my internet connection. Since I pay for the bandwidth I use, this ends up being an expensive option for me.

A final note: in the past, my offsite backups were written by the FreeBSD server, and thus used the native filesystem for FreeBSD. This caused me some anxiety (I can’t read those disks on any other of my computers) and some moderate amount of hassle. In the future the offsite backups will be written by the Mac Mini, and will be in the Mac native filesystem (HFS+) so that I can just plug the disk into any Mac and get at the data. This is one more reason for shifting away from the FreeBSD server. And although in theory you can plug a USB disk into the ReadyNAS NV+ and write a backup to it, I’ve never been able to get that to work. Beyond the fact that I had the keyboard, mouse, and monitor, it’s another reason why the Mac Mini based server system isn’t headless - having a keyboard, screen, and mouse makes it easy to do things like write the offsite backups when needed.

I’ve mentioned my mass storage solution previously, in this post. In that post, I indicated that the Freebsd fileserver was aging, and that I’d bought an Infrant Readynas NV+, and filled it with 320GB disks in a RAID configuration, and it was providing a redundant file server.

Well, it turned out that that plan didn’t go quite as smoothly as I hoped. Infrant, the company that was selling the Readynas line, was bought up by Netgear. There have been a few reliability issues with the Readynas NV+, but the big problem hasn’t been reliability, it’s been performance. Performance from a machine using the ReadyNAS as a file server was slower than the Freebsd server. The difference in performance was enough that instead of doing what I’d planned, and making the Readynas the main fileserver, and having the Freebsd machine serve as a backup/hot spare to the Readynas, I ended up keeping the Freebsd machine as the frontline server, and the Readynas was kept in sync with the Freebsd machine to provide EXTRA redundancy and to serve as a hot spare should the Freebsd machine lose its cookies.

This was all fine, but the Freebsd machine has really gotten old. It’s built from commodity parts, assembled by me (and my friend Rob), and during 2007 I was becoming increasingly nervous about it. Performance was good - not good enough that I didn’t keep things on local disks on the various computers around the house, but good enough that it didn’t bother me too much. The big problem, really, was that keeping the ReadyNAS in sync with the Freebsd server was a pain, mostly because the ReadyNAS just doesn’t have enough cpu horsepower to make the synchronization tool I wanted to use (rsync) work well. Add to that the disturbing fact that it’s come to light that the Readynas is somewhat prone to power supply failures, and it was becoming increasingly clear that I needed some way out.

In essence, what I needed was:

• a solution with filesharing performance (using either the Windows CIFS protocol, or the Apple AFS protocol) as good as or better than the Freebsd server.
• Something quieter than both the Freebsd server (which sounds like a jet taking off) and the Readynas (which has developed an annoying buzz).
• Something that minimized the amount of software I had to stay current with
• Preferably something physically small
• Something that doesn’t consume much power, particularly when idle (which is most of the time)
• Something where, if something happens to it, I can pretty much go out, buy the hardware off the shelf, bring it home, plug it all together, and having it running quickly

Around the middle of 2007 when I switched to Macs, I was already leaning toward putting together something based on a Mac Mini and one of the Western Digital MyBook dual external disks in mirror mode. That setup provides networked redundant disk storage, just like the firebreathing Freebsd server, and the Readynas.

Not long ago, I reported that I’d bought the external disk. I played around with it some, found it satisfactory, and planned on buying the Mac Mini soon. Well, just after Christmas, I did just that. It took surprisingly little time to set it all up, and in less than an hour I had it running as a file server, with one big filesystem on the Western Digital external disk.

Performance wise, this setup beats the (constructed specifically as a high performance file server) Freebsd machine handily by a pretty significant margin - initial benchmarks have it between 1.25x and 8x as fast as the Freebsd machine depending on task. It’s so much faster than the Readynas that I haven’t even bothered to actually measure the difference. And rsyncing to the Freebsd server it’s very fast.

So my assessment now is that the Mac Mini setup hits every single one of my requirements: fast, quiet (nearly silent, in fact, except when the MyBook Pro fan runs), it runs Mac OS X just like all the other Macs in the house, it’s tiny, consumes little power, and I can drive to the Apple store and actually buy the Mac Mini and the external disk right off their shelf.

I’m just so impressed by the Mac Mini - such a great performer in such a small package at such a competitive price, and it seems very nicely made. (I bought the 1.83Ghz version with the 80GB disk drive and 1GB of memory, $600 plus tax) The whole thing is just so simple - no muss, no fuss.

So now, I’ll retire that old FreeBSD server. The Readynas will continue in it’s duty of being a backup to the Mac Mini fileserver and being a hot spare. If the Readynas fails, it will get replaced with another Mac Mini.

While I was traveling in China, on several occasions I had big photography days where I ended up with one or more really full memory cards. Recall that I was using 4GB Lexar 133x SDHC cards, and a little USB SDHC reader that came with the cards.

This gave me an opportunity to learn an important lesson. Copying 4GB of raw files off one of those SDHC cards was fast. Here, you may interpret ‘fast’ as meaning ’so much faster than copies off of my 80x CF cards using my USB CF reader that I was actually worried that I hadn’t gotten all the data off the card.’ I haven’t benchmarked the speed of the SDHC cards/SDHC reader against my CF cards and the CF reader but I expect the difference is a factor of at least four, and perhaps quite a bit more.

The difference is certainly a whole lot larger than comparing the 133x and 80x would lead you to expect. The enabling feature, apparently, is that the SDHC cards and the reader for them are UDMA capable.

Now, I don’t think this is a breakthrough that’s going to revolutionize photography. But the speed improvement is enough that, when I was copying images off the card onto my laptop, I noticed that it was only slightly slower to copy them off the SDHC card than copying them to my external hard disk to back them up. That’s pretty darn fast - and the speed means that you can sit and wait, as opposed to starting the copy, and then heading off to some other task while the copy runs.

I haven’t yet popped $120 a card for new CF cards to go in the EOS-5d, and another $40 or so for a UDMA capable reader. But I’m real, real close.

My plans for this winter include a renewed focus on printing (especially color), both older images and newer work. I’d like to spend some of the dark dreary months playing with new papers. I have a lot of plans for keeping my spirits up during the Great Darkness that is winter in the PNW by filling huge trashcans with duffer prints.

My Epson 9600 is now two generations back from the leading edge, and dealing with new papers means sending test prints out for profiles, one week turnaround, etc. And I’ve been impressed by the print quality I’ve seen from the HP z3100.

So, just five minutes ago, I ordered a 24″ z3100. John at JVH Technical tells me I should have it real darn quick. I’ve been pondering this particular move for months now - basically, ever since I went over and helped my friend Rob set his up and saw the prints that came off it.

I’m looking forward to having it arrive.

What I’ve come to think of as the Big Switcheroo is now winding slowly down to a close. Essentially all of the household computing is now done on Macs. We still need to replace one Windows machine with an iMac, but that will come soon enough. To my delight, Paula has cottoned on to the Macs without any pain at all.
Some amount of the struggle was finding replacements for various tools I’d accumulated in Windows world, especially with regard to backup and synchronizing files on multiple machines.

On all the Windows XP machines, the important stuff off each machine was backed up to one of the file servers daily, using SyncBack. So far, the replacement I’m using for the same function with the Macs is Chronosync, which does much the same thing. Chronosync seems sufficiently flexible for my needs.

For email, Microsoft Outlook has been replaced with the standard Mac mail.app, which we are finding to be pretty nice. Moving the HUGE archive of email from Outlook to mail.app was an epic struggle, involving drinking of mystic potions of dubious origin, much girding of loins, and winding myself up like a Berserker. Oh, and the help of O2M, an application that runs on windows, grovels over your Outlook folders, and spits out files in a format that can be imported by mail.app. Well, it does that, unless your folder is really big. More than, say, 4 thousand messages and all bets are off. A large portion of several days was spent in battle with the email stored on various Windows machines, breaking up large folders into smaller folders and moving the email. Of all the things involved in the switch, moving our enormous archive of email (think hundreds of thousands of email messages) was by far the biggest hassle. All the hassle was on the Windows side, by the way.

The composition window on wordpress.com, which is WYSIABNQWYG (what you see is almost but not quite what you get) if you happen to be using a browser other than Safari, reverts to HTML mode only if you’re using Safari. Since we use Safari, this was the goad needed to force me to investigate offline blog software. After a brief, abortive try with Qumana, I seem to have settled down fairly happily with Ecto which I picked because, like Qumana it runs on both the Mac and Windows platform, and at the time I picked it I was still suffering from the delusion that I might still run Windows in some places. Recently MarsEdit has come to my attention and I’ll probably check it out when I get to the fabled time when things settle down somewhat.

On Windows, I used the RSS features built into Windows Explorer. On the Mac, I dabbled with the RSS features in Safari and promptly concluded that “that way madness lies, let me shun that”. After evaluating both Newsfire and NetNewsWire, I settled on NewsFire. After a few weeks with that, I complained about it here on the blog, and several readers pointed me back to NetNewsWire, so I gave it another try. At this point, I’m sold, mostly because of its ability to display blog posts in the context of the blog, and not just text on a page.

One of our file servers is an aging machine running FreeBSD, with a large RAID filesystem. It’s getting to an age where I’m starting to think about replacing it, so some thought has been given to what I might replace it with. As it currently stands, I could replace this huge, firebreathing (the server is fairly noisy) system with a Mac Mini and one of these Western Digital MyBook Pro II external disk drives configured in mirror mode for redundancy. At current street prices and choosing the least expensive Mac Mini, this would give me a fileserver which would outperform the FreeBSD server and would be relatively inexpensive for what you get - one terabyte of fairly high performance redundant networked storage. One big advantage is that, if the building catches on fire, you grab the external disk and run. The fact that it runs a commercially supported OS and I can replace the Mac Mini just by driving to the Apple store and forking over bucks rather than by assembling the machine myself is a plus. The fact that it would be almost silent and consume about 1/5 the power the BSD machine does is also a plus. Folks who are lusting after NAS that performs a bit better (and is more capable and configurable) than the current crop of fire-and-forget NAS units might want to give the Mac Mini approach a ponder.

For photo editing, I’m using Photoshop CS3. It’s better than CS2, which is not to say that I think it’s good. I’d like someone at Adobe to explain to me why, on a machine as fast as my Mac Pro, it STILL takes forever to start up Photoshop. What in the world can they be doing?

Managing the collection of photos is being done (with some regret) with Adobe Bridge. For me, it works. It’s rather like Churchill’s comment about democracy being the worst possible form of government, except for all the others. Bridge is the worst possible tool for this job, except for all the others, which all seem to suffer from bizarre, deal-breaking defects. All of my comments about Bridge on Windows can be ported to the Mac without alteration.

For word processing, spreadsheet, etc. we’re using Microsoft Office. When Office 2008 for the Mac finally ships we will get it, although I have gazed upon Apple iWork and wondered how long before it is sufficiently refined that I can sever ties with Word and Excel. Perhaps never, but one can hope.

All of this acquisition of software tools has run up some expense, although none except the Adobe stuff is even slightly expensive. On the other hand, I haven’t had to buy, install, configure, curse at, and be annoyed by any antivirus software. This puts me very slightly ahead on cost and way ahead on muss, fuss, and general botheration.

Just a quick note to comment on the speed of the Mac Pro I bought. It’s a four core machine (dual dual core), 3.0GHz, with 4GB of memory.

Mostly out of curiosity, I decided to compare the speed of some common operations in my digital workflow on the Mac Pro and on the old Windows machine (2.4GHz Athlon dual core, 3GB of memory). I was expecting things to be roughly 2.5-3x faster, but since some of the speedup was from increasing the number of CPUs, I would not have been surprised to find a less radical improvement.

On wide radius unsharp mask, the new machine is roughly eight times the speed of the old one (that is, the old machine took eight times as long to perform the task as the new one).

Running my noise reduction software of preference (Noise Ninja) on an image from my EOS-5d, the Mac Pro is roughly 4 times faster.

I was startled to see that both Photoshop and Noise Ninja were quite capable of parallelizing these two tasks. Maybe the software world (or at least the image processing software world) is more multi-cpu ready than I thought.

There’s a limit to the utility of speed, of course. Reducing the time to do the unsharp mask from 10 seconds to just over a second makes the work go more smoothly, but cutting it from 1.2 seconds down to .15 would not provide nearly the same benefit. Since my switch from scanned large format film to the EOS-5d for most of my photography, the speedups I’ve seen so far are nice but not earthshaking. But if I were working entirely with really large images (scanned 8×10 film, say, or images generated by stitching) the speedups would make the difference between operations being brief enough to sit and wait to see the results, instead of starting the operation, getting up and getting another cup of tea, and coming back.

Even better, the speed does not come at the cost of noise. When I built the dual processor Athlon machine, I took special care to build a machine which didn’t generate noise. It took a bit of work. Unlike the Mac G5 that my friend Rob loaned me (which makes noise rather like a fighter jet with afterburners on when the CPU usage ramps up) this machine is utterly silent all of the time. As someone who is afflicted with a sensitivity to environmental noise, I really appreciate this.

My now somewhat ancient (but still soldiering bravely on) Epson 9600 is up for replacement.  As readers here know, I’m very impressed by the HP Z3100 printers.  But, like so many photographers, I was kinda/sorta holding back, waiting to see what Epson’s response to the Z3100 would be.

Well, the wait is over.  Epson will formally announce the new printer models on August 13, 2007.  In the meantime, though, Epson UK have apparently jumped the gun and blabbed.  You can read interesting coverage at PhotographyBlog.  (hat tip: Luminous Landscape)

Based on what I see there, I’ll be going with the HP z3100.

In this post on superstition and the color accuracy of laptop screens, Rosie Perera commented

With all this talk about color accuracy, it seems poetic justice that the photo you posted today was black & white.

Rosie’s razzing me, here, but her comment reminds me of an interesting point which I think is often either misunderstood or else overlooked.

We tend to think of black and white photos as just that - black in the dark bits, white in the light bits, and a mixture of black and white in varying proportions for the bits in between.  It’s a reasonable thought - when we shoot with black and white film, the film essentially records luminance, and all of the color information (what would fall in the a and b channels in Lab space) is just thrown away.  At that point, we really have produced a black and white photo - the color gamut of the image is essentially nothing more than a straight line between Dmin and Dmax - no volume to the gamut at all.

The problem occurs when we want to turn this black and white photo into a print, either one that we can hold in our hands, or one that is displayed on the screen.  When we do this, we’re taking this image that has an incredibly narrow gamut (just a line, remember?) and we’re going to translate the image into the color space of that display medium.  And when we go to do this, we get some rather big and unpleasant surprises.

The first surprise is that, in the color space of the display device (the screen, or the printer, etc.) our ability to represent colors that lie on the L axis of the LAB color space is not as good as we naively expected.  We’re up against two problems, here.  The first is that the color accuracy of the device is not perfect, so when we specify ‘give me a perfectly neutral 18% grey’, what we get instead is the best job the device can do at a perfectly neutral 18% grey, which only in the rarest of cases is going to be exactly a perfectly neutral 18% grey.  It’ll be just barely off the axis.  Sad, but true.

The second, and closely related suprise is this: the human visual system is incredibly, fantastically good at detecting very slight variations from neutral.  You can take a perfectly neutral grey, add an homeopathically small quantity of blue to it, show it to another human, and that human will tell you that the grey is slightly cold.  The upshot is that when we take our conceptually perfectly neutral black and white image, and we print it on our inkjet printer (or display it on our screen) what we are expecting to see is a range of perfectly neutral greys ranging from Dmax to Dmin.  What we see instead is a range of color, ranging from the darkest the display device can produce up to the ‘base’ color, and in between we have a range of colors which are tantalizingly, frustratingly, achingly close to neutral but which our amazing vision tells us are ‘blue’ or ‘magenta’ or ‘yellow’ or ‘green’.  Needless to say, this is not the desired effect.

The upshot is that one of the most demanding tests of color accuracy is to use a device to display a perfectly neutral black and white image.  The human visual system will ruthlessly expose every inaccuracy, every spot on the image where some shade of grey is not perfectly neutral.  An image like the one above, with the greyscale stretched out considerably, is a particularly difficult challenge.

You can spend quite a lot of time getting your device tuned up to the point where it passes the ‘very long perfectly neutral gradient doesn’t show any color shifts’ test.  It takes a really good profile, a device with no drift over time, and attention to all the color management settings. 

The ultimate irony is that you didn’t really want a perfectly neutral ‘black and white’ print anyway, because they look peculiarly dead and lifeless.  Perhaps in another post I’ll tackle why.

Michael Reichmann and his pals have gone and compared prints from an HP Z3100 and an Epson 9800 (along with an older Epson 9600 loaded with Piezography K7 inks), over one Luminous Landscape.

Naturally, this makes for interesting reading.  Unfortunately, there’s a little gotcha that’s almost not mentioned and is difficult to spot (if you, as I do, tend to read such things by skimming rather than plodding).

Reichmann goes to great lengths to tell us about which version of printer firmware he’s loaded into the HP Z3100, practically telling us what brand of laptop he’s used to download the firmware update.  We are told exactly what sort of paper is used - all very good, although I think it would be better to compare prints on more than one paper, especially since the red gamut problem with the z3100 appears to be linked mostly to matte papers.  But that’s for another article, perhaps.

But that’s all fine, as far as it goes.  The gotcha is hidden further down, where we learn that the Epson 9800 is being driven by the Studioprint RIP.  Ok, well, that’s fair, I suppose.  But to go this long into the comparison without divulging this pretty major detail about the setup is more than a bit screwy.

This highlights a major point - when you’re reading comparisons of gear on the web, it’s often the stuff which seems minor in the text that has a major impact.  I don’t know much about the Studioprint RIP, but at least in marketing theory, it’s supposed to improve the quality of the output over the stock manufacturer’s printer driver.  You might think that it’s being used on the 9800 would be something they’d mention up front.

That doesn’t mean that we should ignore such comparisons (I’m addicted to them, myself).  But it does mean that we should have a huge quantity of salt handy during reading, just in case.  And it also means that, when we’re making purchasing decisions ourselves, it makes sense to try to arrange our own comparisons that reflect our own patterns of use.