Depends on where you like to shoot.
For me, the 70-200mm focal lengths come in handy for outdoor shots, be it buildings, candid street shots, etc, during the day. So f/4 is quite adequate. I literally can't imagine walking around town taking shots with a boat-anchorish f/2.8 IS. It's an oxymoron actually. The appeal of Image Stabilization is to reduce camera shake in handheld shots, but the 3.2lbs weight simply isn't a huge fan for handholdability/portability.
Indoor use, I wouldn't even consider a 70-200mm focal length, unless I'm paid to do indoor event coverage. Even then, the 135mm f/2 would be a better candidate for serious indoor photography. For informal ones (like *urgghh* a colleague's wedding), I wouldn't even bother lugging multiple lens, just a Sigma 18-200mm would do.