Mine is the same way. The bushings that I use have more play than the new ones that I don't. I don't consider them to be worn out. More like being broke in.
As for the alignment not being perfect, folks can say what they want, but in my humble armature opinion, any bushing that supports the drill bit at only one point will have some wiggle room. I consider it to be like any other tool or instrument in that it is how the craftsman learns to use it, that makes the difference.
DD4H put out a video showing that a slight pressure on the drill motor towards the front of the car could help gain more consistency, but I wasn't able to get that 'race ready' drill that I was looking for.
Personally, I will never drill enough blocks to become an expert, without some additional help. So for the last few weeks, i have been working on a jig to put the Silver Bullet Extreme in that I call my 'jig jig'. It will support the drill bit along the length of the shaft with (what i expect to be) perfect alignment to the single point of contact in the bushing. If I am successful I will be able to drill race ready blocks consistently. If I am not successful I will drill them as close as I can and perhaps become more proficient in altering the holes that need adjustment after drilling.
Just my 2c.