Without being there, none of us can know. It sounds though that both of you flew off the handle a bit - you can only speculate on why he did, but could be worth thinking over why you did?
There was already a guy who was touchy that day (perhaps for a very valid reason), and had shown he was upset with a camera being pointed at him. The choice of a sarcastic comment......was it not pretty much inevitable to set him off again> I'd have thought so - and that in turn was hardly likely to make your day better, no? To me it seems a bit misleading to say he ruined your day, but you made the choice of prodding the sleeping dog with a sarcastic quip, and making it likely you'd go off at each other.
He doesn't like having his photo taken? Apologise and move on - there's many other things in life which can be photographed. If I'm interpreting rightly you're saying he should have swallowed his dislike of being photographed and tolerated images being taken of him? Is that not the same as "you should swallow your belief that you can photograph him, and you both move happily on your way"? Why does he have to do what we photographers want, and he doesn't have to do what we want?
Yep he was perhaps unexpectedly miserable, but we can choose to smooth past that situation, or apologise and move on in peace. I'd have said sorry for surprising him, beautiful evening, beautiful boat, chat for a while, then would he mind..? If he says no, it's his life, his boat, I'd say goodbye and move on. In the end do I want an image to remind me in future of a miserable git?
