You are here: Home > Articles > Compare Not!

Compare Not!

Comparative shots of the same subject using different digital cameras is a popular way to attempt to determine which digital camera gives better image quality. The principle is simple enough: keep everything constant except the digital cameras. The results should speak loud and clear for themselves. We all do that. But is this a valid way to compare the image quality of different digital cameras? Read on, you may be surprised!

First of all, the idea makes intuitive sense -- but there's a flaw in the thinking. It's the "keep everything constant" part that is not quite so simple to achieve, especially when taking pictures outdoors. See, in a studio, with controlled lighting and environment, it is quite possible to "keep everything constant." But, outdoors, light is always changing; though the changes may be too small for the human eyes to notice, they can make a huge difference to a digital camera's image sensor.

To prove this subtle point -- I know most of you are still unbelievers at this instant ;o) -- I have taken two shots of the same subject, one after another in rapid succession. Camera A took the first picture, and Camera B took the second. Both cameras were set on Programmed AE mode, Spot metering and ISO 50. The only difference is that I took a step back and changed the composition ever so slightly for the second shot to simulate the fact that when reviewers come back days later (or even months later) to take the same shot, they rarely take it at exactly the same spot (I wonder how many etch some kind of markings on the ground?).

Frozen Stream by Camera A
5.8mm, Programmed AE, Spot, 1/320, F5.6, +0.3EV, ISO 50

Frozen Stream by Camera B
5.8mm, Programmed AE, Spot, 1/200, F5.6, +0.3EV, ISO 50

Reviewers take the same shots all the time, and readers love it "because I can compare image quality this way." Is it a valid way to compare the image quality of different digital cameras? No! Yet many will do the very comparison as above, comparing images of same subject taken on different days in different weather conditions and with slightly different composition, to base their decision of which digital camera is better.

Why not, you ask? Why should we not use comparative shots to compare digital cameras?

Well, OK go ahead, based on those two shots, which digital camera do you prefer, A or B?

Take a minute to study the two images, then choose a camera.

OK, time's up. Now, I can reveal to you that the above two pictures were taken with the same digital camera! (Sorry for the slight deception used here to make my point ;D.)

But don't just take my word for it, try it yourself. Put your digital camera on a tripod and take a number of shots of the same subject in Auto mode or Programmed AE mode. Then compare the shots and see if there are any differences. Take a shot today, then come back and take the same shot the next day, and compare. The differences may not be as obvious as my examples above, but sometimes they will be there and sometimes they will not. All depending on the light and the subject matter, and a zillion other subtle factors.

Though comparative shots of outdoors subjects are interesting, they should not be used to compare digital cameras. As my examples above show, the same digital camera can sometimes produce different results with only a small change in the composition (remember, these two pictures were taken only seconds apart).

Reviewers (including yours truly) sometimes / often take the same subject matter for their image samples section (not that we are a lazy lot, but we run out of photogenic material after a while). You can use the pictures to compare the amount of detail captured by the different cameras, but not which image "looks better." For that, you need to look at all the image samples.

What most of us do is single out one picture to compare with the same one taken by another digital camera. Be careful of the cnclusion you draw from that comparison. You might well just be fooling yourself and perhaps unwittingly disqualify the digital camera that is right for you.

You should read the whole review. Using the image samples only (without reading the review) to draw your own conclusion is a surefire way to draw the wrong ones. In this instance, a picture is unfortunately not worth a thousand words. The weather, the reviewer's ability to take a good picture, and myriads other factors may conspire to make a photo come out worse than the camera is capable of taking in capable hands. Selecting the digital camera that is right for you does take some elbow grease. Our Digital Camera Buyer's Guide will help you do so.

A Word About Ratings

Assigning a numerical number to a digital camera sure makes it easy to compare digital cameras! The one with the higher number wins! But unless you understand what the rating means, you may be drawing the wrong conclusions.

A rating is always assigned to a digital camera based on the other digital cameras in the same category. [By the way, some reviewers incorrectly use megapixel resolution as a category. I'll let you think this one through.] So a digital camera is rated "Excellent" or "Good" (or "Recommended", etc.) depending on its competitors. So far, so good.

The rating is correct... until new and improved digital cameras are introduced into that category. The new digital cameras are again reviewed and rated one against another.

A problem now arises because a new digital camera may be rated "Good" (or assigned a number which means "Good") and be better than a previous model that was rated "Excellent." No reviewer in his or her right mind will go back and readjust the ratings of past reviews.

As long as you understand the rating is only valid compared to the other digital cameras that are 1) in the same "generation," and 2) in the same category, you'll do OK. It is when you compare ratings of digital cameras of different generations and in different categories that ratings lose all meaning.

If this article has been helpful to you, why don't you send it to a friend? Please give us your feedback on how we can make it better, or what articles you would like to read next.