I don't think Air Asia flys to anywhere near Port Dickson.
Too much hassle getting from Sg to Senai, fly Senai to KLIA, and come back down to Port Dickson.
I suggest to take the overnight train to Seremban, and take taxi from there to Port Dickson.
You can check out the train schedule and...