Sorry for the long post below, this thread really struck a chord... so just wanna share my manual portrait lens journey...
So finally laid my hands on the lenses that are always on my camera now - the Contax G Carl Zeiss series
Super sharp wide open (both the 28mm F2.8 and 45mm F2), only slightly bigger than my oly pen f, the pair I got were simply my favourite lens to date. However again, they are not without downsides:
a) expensive ($500 upwards)
b) short focus throw (not really that smooth to focus, as they are autofocus RF lens, not SLR lens made for manual focusing)
So well, it is up to you to try out different lenses and decide what you want.. think what Anthony Lee said is very true :