If you're taking photos of a child at a park, and the parent (be it man or women) question you, IMO that's expected. After all, you're stood in a park, with a professional looking camera/lens (90% of the time), taking photos of their child on the swings.
It's an expectation for them to ask questions, and that's perfectly fine, because in this day & age you can never be too carefull. However, it's not perfectly fine for them to kick off over it, and even get violent. If a father of a child kicks off, and gets in your face about taking photos of his kid, he's in the wrong. He's not in the wrong to question what you're doing, be it perfeclty legal to shoot in public or not, but he IS in the wrong to give it large about it.
However, I think the next step of the situation is down to how you respond to him/her. If they ask what you're doing, and you be polite, sensible, and just explain to them that you're a budding photographer who's looking to become skilled at taking portrait photos, maybe show them the photo, and see what they say. That's one way to handle it, and personally, the best way IMO.
If, however, they come over asking what you're doing, and you straight away jump into the 'i'm doing nothing wrong, F'off' routine, they're going to be even more ticked off, because you've been rude to them.
As silly as it sounds, I think it also depends on what you look like, how old you are etc. It's sad, but I believe it makes a difference.
If you're a 19yr old lad taking photos of some kids in a park, and get confrunted, you're going to be looked at differently than a grown, middle aged man, IMO. That's just down to stereotype. Well all know sicko's can come in all shapes, sizes & ages, male or female, but like I said, it's down to stereotype.
I was confrunted a few weeks back whilst taking some photos of a building that was in the middle of being built, it was still a steel structure at the time, with a few cranes around it etc. It was 4am in the morning, just coming light, and it looked like a good shot to get. I stood over by a car garage, set my tripod up etc, and started snapping away. After a few minutes, I was confrunted via a tannoy, and then by a security guard, followed by a passing police van. I calmly explained that I was out taking some photos of the sunrise, and thought some shots of the building would make a nice photo for a project I was doing. I was then smiled at nicely, and left to my own devices. I wasn't doing them for a project, it was for a competition, but project sounded more formal.
I think it's all about how you interact with people, on first impressions.
But I agree, it's getting a bit silly now, and eventually you'll have to watch what you photograph due to certain laws etc. It's a pssssst take for sure, but what can we do?