I think the uniform and his experience (didn't you say he's been with the fire dept. for a long time?) probably makes people assume he is older and in that case, I would take it as a compliment. He is respected.

Sounds like you have a great guy and I hope that he doesn't let this get him down.