Thanks guys for all your inputs.
I got the HTC desire. So far the impression has been positive. I'd say it is a worthy competitor for iPhone.
Everything is snappy, but granted, I haven't installed anything yet. If there is a downside compared to iPhone, is that it tends to be economical on animation, and zooming isn't as smooth and pretty. Understandably so as it lacks a dedicated graphic processor like the iPhone. The upside is its raw processing power shows when loading webpages, even when it does some flash while the iPhone do not. And once Android 2.2 is out the speed and flash performance should improve further.
I tried the Milestone XT, there is a bit of lag when scrolling the through the apps, the Desire does feel snappier, otherwise I do like the form factor.