Canon EF 70-200mm f/2.8 L IS II USM Lens Image Quality
ISO 12233 Chart 100% Crops - it does show significant improvement in terms of sharpness.
I think the response will differ from person to person. For a 5D2 and a 5D3 comparison, let's not talk about other systems, the 5D3 did make the 5D2 looked a little embarrassed, at least in my own opinion, because if the largely acknowledged AF weakness of the 5D2.
But for the 3 versions of 70-200 f/2.8L, personally I don't think that way. The non-IS version is still a great lens itself, where it's sharpness is still top notch amongst the EF L series lens.
It really depends on what you are comparing on and what's your concern. If you are comparing the 28-70 f/2.8L VS 24-70L II, that would be some significant difference, or the 16-35L and 16-35L II
I guess it ultimately comes down to what matters to do, for you to determine whether the previous models are truly outdated or obsolete or not.