it's not the lack of focusing points that slows down the AF speed. it's more of the camera's ability to track, and the default time interval set for the camera to refocus each time when the subject moves out of the focus plane.
for speed, the AF module plays a part also, since it determines how fast the module is able to confirm the focus. if the AF module is more sensitive to light, it will be able to confirm its focus faster. however, the biggest factor of AF speed is still how fast the lenses r able to perform AF. for screw drive lenses, the in-built AF motor of the camera body is the one of the determinants for their AF speed, while for AF-S lenses, it's the motors in the lenses that matter most. if the lens has long focus throw, it will take a longer time to focus from infinity to minimum focusing distance. it will be noticeable if u shoot a distant object firstly, n then shoot a subject which is alot closer to u. this is also why the 80-200 f/2.8D comes with a focus limit switch, to make the focus throw shorter n reduce the time of "hunting" if u shoot at a subject at any distance above 3m from u. u shld also see how fast the 80-200 f/2.8D focuses on bodies like d3s, even for still shots.
