How best to test a new lens?

Roger Ramjet

Suspended / Banned
Messages
177
Edit My Images
Yes
I've just received a Sigma 10-20mm f/4 - f/5.6 DC EX lens and I know many people have said they needed to return theirs a few times to get a "good" one. Hence I'm keen to check it out thoroughly as soon as I can.

Trouble is, I don't know how to test a lens. The obvious answer is to take some photos and I'll certainly be doing this, but I don't know what to look out for in the resulting images or what best to photograph to show up any errors.

Help?!

(Ps. did search for this subject but couldn't find the answer in previous threads, although I'm sure it's "out there" somewhere)
 
People may suggest test charts etc, but the best test is just to use it in the realworld! Unless you are a test chart photographer by trade...this isn't how you will be using it! You will soon notice if it isn't right, with spotting misfocuses etc when reviewing images.
 
This is it, if you take pictures and they are in focus and you are happy with them then it's right! The test charts etc are only really to test if they are wrong once you have an idea that there is a particular problem. eg if you think that the lens is focusing short every time then use a test chart and if that shows the same thing it can be sent off with the lens to show the problem otherwise I wouldn't bother.
 
as has been said just get out and shoot you usual stuff. The examples I've seen of problems with this lens have been pretty obvious and usually consist of a pronounced difference in sharpness bettween the left hand and right hand sides of the image. Pixel peeping and test charts only ever lead to disapointment and confusion.
 
Don't go looking for faults. You'll notice pretty quickly if the lens is faulty.
 
I had same concerns when I bought mine. I think Id read somewhere that the lens was soft on the left but had no problems with mine. Just took a few test shots as soon as poss.

Have fun with it!
 
Okey doke, thanks all. I'm off out with it this afternoon (if the rain holds off) and I'm expecting great things from what is officially my most expensive lens :wave:
 
yup ... as long as you can test it and if something problem you still have the warranty attached to it ... happy shooting
 
I always test my lenses, but before I buy them, not after ;) I phone ahead and make sure the shop has got two samples of the lens I want, then check them side by side at the store. I've now got this down to a fine art, and it's always nice to know that you've spent your hard-earned on a good un. Having said that, and having done this with the last five lenses I bought (so that's ten lenses in total, eight Canons two Sigmas) I would be quite happy to take any of the samples I checked. Differences were there in a couple of them, but miniscule and irrelevant. There are always manufacturing tolerances, and perfection is very rare, but equally, duff lenses are also quite rare (despite what you hear on internet forums!).

The overwhelming problem with lenses is poor assembly, meaning that not all the glass elements are precisely alligned and set absolutely square and central in the barrel. If this is the case, then then one or more of the corners of the image will be less sharp than the others, and it's very easy to test this.

Do not use a test chart. They are unreliable for various reasons, especially with wide-angles. The main problem is that they are flat, two dimensional, and you have to shoot them at an artificially close distance. Real photography is not like that.

Select the weakest focal length, in your case probably 10mm, but it doesn't matter much. What is vital is to use the widest aperture (brightest, lowest f/number) where any problems will always be most evident.

Choose a distant subject to eliminate any possible focusing errors, like a car number plate. If you use AF, and it is often more accurate than most people can focus manually, switch to centre-point focus, and make sure that the focusing spot is more than covered by the subject, to ensure that there is no possibility that it can lock on to anything except the target at a different distance. Use IS/VR if you have it. Once focus is set, switch the lens to manual so that it cannot change.

Choose a high shutter speed, so there is no chance of camera shake spoiling the test. Push up the ISO if necessary to get this - it won't matter to the test. Use the 1/focal length rule, don't forget to multiply by the crop factor if that applies, eg 1.5/1.6x, and then add an extra stop for luck.

For example, if you set your lens at 10mm, then the shutter speed should be about 1/15sec minimum in normal hand-held photography on a crop camera, always, and for this test make it 1/30sec or shorter, preferably 1/60sec. You do not want any camera shake skewing the results. For longer lenses like a 70-300mm zoom, you'll probably need at least 1/500sec, or 1/1000sec.

Position the target in the corner of the frame and holding the camera as steady as you can squeeze off a picture. There's no need for a tripod if you've got a decent shutter speed, but if you haven't, testing a long lens for example, then use a tripod. Now do the same for the other three corners, and it's job done.

Check the target subject by zooming in on the camera's LCD. What you are looking to see is not amazing sharpness, because almost certainly that is not what you'll see. What you want to see is an equal level of sharpness (or unsharpness!) in all four corners.

If the lens is bad, you can see this clearly enough on the LCD, but just to make sure, pop your card into the photo booth in Jessops or Boots, and print out a 6x4 of the target, at maximum magnification. Take a look at the prints. If one or more is clearly much worse than the others, then something is not right with the lens, or maybe it's been dropped (this is a good quick test when buying second-hand lenses). I say again, don't look for perfect sharpness because you won't find it - this is a severe test of the lens at its very weakest point. Even the best lenses will show some image deterioration in this test. What you want is all four corners to be equal.

A good lens will always pass this test. However, it is possible for a duff zoom lens to pass this test, although very unlikely. It could be that while the lens is ok at the focal length tested, when it is zoomed out it could change if the mechanism was faulty, or was extremely badly worn, or if the lens had been dropped. This doesn't happen with new lenses, but you can check this too by shooting at three focal lengths - minimum, middle and maximum. I will stress again that what your looking for here is equality in the four corners at each focal length, not equality between focal lengths. Don't look for that, because you won't find it! All zooms vary throughout the range.

The other test you might like to try is for AF accuracy. I'll explain that if you like later, but don't shoot a test chart - especially not that one where you download a sheet of A4 paper and shoot it from an unrealistically short distance!
 
Excellent reply Richard and if the OP doesnt mind, i'm going to nick it and try out the method on my lenses (hope they are all ok but will be interesting to test them anyway.

This should be a sticky!!

At the risk of hijacking the thread, when you do the "how to test the AF capabilities of a lens", please let me know :thumbs:
 
Thanks nigpd :) You're welcome. The test described above is very simple - all you need to know is what the problem is with 'bad copies' (nothing usually ;) ) and then devise a test to identify it. The only time you need test charts and stuff is when you test different lenses over a period of time, and need to repeat each test absolutely accurately so the results can be compared. Test chart type tests have the advantage of being measurable and repeatable, which is obviously kind of useful if that's what you need, but to check if a lens is screwed together properly or not, you just don't need a test chart - and indeed there are better and more relevant tests to be made in the real world.

I'll post about AF focusing later then. A'll I'll say now is "cerial packets" :D
 
Thanks Hoppy that's some useful info and I'll try it out over the next few days. :thumbs:
 
Okay then here's how I would test for AF focus accuracy. I've modified this in the light of recent debate on here, and have attempted to address the two potential shortcomings of that infamous A4-print-out-shot-at-45degs test that folks download and use, and then complain about their cameras. I can see why people use that test, because in theory it should work fine, but for various reasons it often creates more problems than it solves. I know for sure that manufacturer’s camera technicians hate it because it generates a steady stream of cameras and lenses in to them that the customer insists don’t AF properly yet they can’t find anything wrong with them. So they return them, and then the customer re-tests them, gets irate and sends it back again, and so it goes on.

But first a bit of waffle.

One problem with that test is it is generally shot at very close distance, deliberately so, in order to reduce depth of field to very narrow levels. This is so you can see any focus shifts more easily, but in doing so it pushes the lens right to the limit of its range. Lenses (and cameras) are analogue devises and they are optimised for normal focusing distances, plus they work within a tolerance range set within acceptable limits for depth of focus. When the camera detects focus within tolerance, it stops, even though focus might not be absolutely at the technical optimum. It's within acceptable range, and you'll never notice the difference in practise. There are photographic operational reasons for this, and it's not just sloppy design or manufacture - it makes the camera focus faster and more decisively under normal situations.

The second potential problem of shooting a target at an angle is that you sometimes cannot be certain exactly where on the slope of the target the camera is actually focusing. This is because the focus sensor often reads a slightly larger area than that depicted by the focusing spot in the viewfinder. So to be certain of avoiding this, shoot a flat target square on.

Some people have done AF tests, decided that their camera was fractionally off optimum, and had it adjusted by the manufacturer. They have then found that at more normal focusing distances it is out, when before the test it was fine. This is very rare, but can happen with equipment testing which is designed to provoke errors where often none really exists, by pushing it to unrealistic or irrelevant extremes – it makes you think something is wrong, when in fact it’s fine. Have you noticed in regular picture taking that the focus is out, and consistently out, with sharp focus either just in front of where it should be, or just behind? If there is a fault, then it will always show itself in the same way every time. And this test will confirm it.

Of course, it is possible that either the lens or camera could be slightly out. It happens, but before you go to the considerable trouble and expense of sending it back for calibration, make sure there really is a problem. If a lens is out of tolerance, then it will show exactly the same problem on different camera bodies. If a camera body is out, then it will also demonstrate the same fault with different lenses. It’s not too hard to cross-check like this, either with friends with similar equipment, or at the local camera shop if you ask nicely. You can surely devise a similar test to this one even in store. It’s got to be easier than sending your kit back unnecessarily.

Now the test. Nothing too tricky or technical, it just needs to be a flat target shot at a sensible distance – nothing more. If that distance is too far, then you’ll never notice any error as depth of field will cover it, so it has to be reasonably close up, just not to extremes.

I took three cereal boxes, and lined them up side by side on the kitchen worktop. I think this is a big enough target to be realistic, yet small enough to show up any significant errors. Position one box square to the camera in the middle, and one to the side and set at a slight angle so that part of the box is just behind the middle one, and part of it is just in front. Put a third box on the opposite side, square to the camera and lined up so that the front is in line with the back of the middle one. This third box serves no real purpose other than being something to lean a 12in ruler up against, standing vertically on end, right up close to the middle box. Lean it so that the top is behind the front of the middle box, and the bottom is a couple on inches in front. I hope that’s clear!

Set the lens to the widest aperture (brightest, low f/number) for minimum depth of field. Make sure you have an absolutely shake-free shutter speed and wind up the ISO as far as you need to get it, and use IS/VR if you have it. Flash is fine, in fact flash is good, so long as you don’t get nasty reflections off the boxes or the ruler, which is actually quite likely in this situation unless you use bounce flash. Stand back so that the three boxes just fill the frame (* see footnote) and make sure the camera is square to the target – in other words, the camera must be at exactly the same height as the centre of the middle box. Using just the centre focus point, focus on part of the middle box that has plenty of good detail for the AF to lock onto positively, and take a snap.

Zoom in on the LCD at max magnification and check the markings on the ruler. Scroll up and down it and decide where best focus is, and where you think it runs out, both in front of the middle box, and also behind it. Look carefully, because unless you have a fast lens like f/2.8 or wider, the focus fades very gradually and it’s hard to tell where ‘acceptable’ sharpness fades into unacceptable blurring. Ideally, the part of the ruler that is in line with the plane of the box you focused on should be perfectly sharp, and the parts in front and behind should be gradually fading equally out of focus.

I’ve just done it with a couple of lenses and both were spot on as far as I could tell. With one lens (Canon 100-400mm L, 180mm at f/4.5 and 2m, on a 40D) I estimated acceptable depth of field to be about 2cm, and when I checked this against what it should be ( www.dofmaster.com ) it was exactly right! Not bad just by blowing it up on the back of the camera.

To see what this looks like on something a bit more pictorial than a ruler, then you really need to make a big print and look at the box set at an angle. Look at the type, or even the tiny printing dots that make up the image detail. You will see the focus gradually fading towards the front and the back of the box face, with the sharpest point being in line with the plane of the middle box. This will give you a better idea of what you can expect to see in real picture taking.

* NB Don't rely on this test if you have to focus closer than 2ft from the target - see post #17 below.
 
Brilliant write up yet again Richard. Thanks for taking the time to do such a comprehensive reply

I've printed both sets of instructions and will be trying them out this weekend on my lenses.

If mods are looking in, this should be made in to a sticky or a tutorial. :thumbs:
 
Brilliant write up yet again Richard. Thanks for taking the time to do such a comprehensive reply

I've printed both sets of instructions and will be trying them out this weekend on my lenses.

If mods are looking in, this should be made in to a sticky or a tutorial. :thumbs:

Thanks again nigpd, and once again your're welcome.

That's kind of you to suggest a sticky, but let's see what the TP massive think about it first. There's no such thing as a perfect test, and no test will work for everything all the time, but this one does answer some common criticisms.

I will however say that I'd not be happy using this test with an ultra-wide angle lens, such as the 10-20mm Sigma of the OP. At those focal lengths, to fill the frame here you end up too close to the target when the point you have focused on is not the same distance from the camera as the ruler, due to triangulation. It's only a few mm further away but if you're being critical that could make the difference between acceptable and unacceptable.

There's also field curvature to consider and while there are ways of minimising the affect of both these things - by moving the focus point next to the ruler, or moving the ruler close to the focus point - this begins to defeat the object of a simple real world test. Basically, I don't like the idea at all, of testing any lenses (except macros) at unusually close distances. End of, really.

I would much rather say, don't use this test as in any way definitive if you have to get closer than 2ft - which means don't use it for focal lengths less than 28mm on a crop camera. In fact, I will say that - don't use this test at focusing distances under 2ft - and if you are under 3ft, then be aware of the concerns raised here before concluding your equipment is faulty.

And by the same token, very wide aperture lenses like f/1.8 or f/1.4 with inherently very shallow depth of field will also be on thin ice when shooting at close distance. But there's an easy fix for this - just move the camera back to 3ft or 4ft distance and because the depth of field is so narrow, it will still be possible to see it fading in and out, up and down the ruler, so you can make an assessment of accuracy.
 
Back
Top