Many of the lenses in use today date back to the film era when people shot 35mm film and might typically print at up to 12x8. The accuracy of the AF systems between camera body and lens served that era perfectly well. Of course, in the film days there was no such thing as pixel peeping and viewing 100% crops and then complaining about image (pixel!) softness.
Even when digital SLRs came onto the scene the low pixel density meant that the camera didn't really have the resolution to make AF performance too critical. A 6 megapixel DSLR file viewed at 100% would display at a virtual size of approx 27"x18" when viewed on my laptop screen. That is a big step up in magnification from a 35mm negative going to a 12" print. In fact it is an enlargement 3.6X greater, assuming the DSLR in question has an APS-C sensor, and far more likely to reveal problems with AF accuracy, but the tools were not available to us mere mortals to improve things, even if we needed to.
Now that we have APS-C cameras with pixel counts of 15MP and 18MP, viewing those images at 100% simply makes the cracks too obvious to ignore. Perhaps on a slow, possibly soft, consumer zoom lens the DOF is large enough to make hyper-accurate focusing less important, but with some lenses and shooting styles you will need mm perfect focusing.
A 7D file, viewed at 100% on my laptop, is equivalent to a virtual image of 39"x26". If those pixels are to add an advantage then the captures have to be focused accurately. What was acceptable for film, and the D30, and even the 30D, may no longer be acceptable for a 50D or 7D. The AF needs to be more accurate, but the lenses are still (mostly) the same old lens designs of yesteryear. It is therefore no longer the case that mere engineering tolerances will satisfy the most meticulous pixel peeper. The manufacturers have now given us a tool so that we can tighten the tolerances for ourselves, in our own homes.
Some may say that AF microadjustent is a cop out. Perhaps it is. However, I think it is a way for the manufacturers to manage manufacturing costs down to an acceptable level, while still allowing the most fastidious photographers to squeeze the maximum performance from their gear.
As an example, here's a shot I took today with my 7D and 100-400. (apologies for the IQ - the website that hosts the file has resized it)...
If we zoom in to 100% to take a look at the DOF we see that it is probably no more than 2-3mm....
If my AF was not accurate to the mm then a shot like this would fall short of the best that one would wish for. If my AF had been off by more than 1mm, or 2mm at the most, this shot would have been ruined. As it happens, I have checked the focusing of this lens and it has not needed adjustment, but I do have other body/lens combinations that do benefit from some tweaking.
If you want to realise the maximum potential of your high resolution camera, whether a 50D, 7D or something else, it is worth making sure your focusing is as accurate as possible, and not simply "close".