Bottom line - IS works. Even if you are not hand holding it helps to offset any vibrations even when mounted on a tripod which you'll encounter at the super-tele range (400mm equivalent and up). As someone pointed out; if you constantly shoot at high shutter speeds IS's usefulness is marginal.
Deciding factor - Is the significantly higher price you pay above the non-IS version worth it? Only you can decide. Can you live without IS? - Certainly. Its a luxury not a necessity.
Other consideration - I have to point out as with all electronic products you'll have to factor in wear and tear. IS gyroscope = more moving parts = more prone to wear and tear. For this reason I suspect that the 1st generation Canon superteles will last much longer than the IS versions by virtue of this (and we have living proof of the 20 year old superteles still functioning well up to today). People seldom mention this but its a real issue if you intend to hold onto your IS lens for the long term and beyond its production cycle.
Personal thoughts on IS on 70-200mm: My thoughts is that its something I can live without (and I have for the last 10 years). Quite frankly with good fundamentals 200mm is very handholdable and unless you really need it for your livelihood, the savings between the current MkII and a used 70-200 non-IS means that you can purchase a 24-70/2.8L to compliment it.