Hitting in relationships bothers me. Oh wow, what a courageous and novel stand, I know. But I'm not just talking about blatant abuse. There's two kinds of hitting I see a lot that don't really reach the level of domestic violence but still bug the shit out of me.
1) Girl hitting. "Tee hee, I'm just a little girl and he's a big strong boy, when I get angry it's just a cute angry and when I flail my fists at him it's just cute little blows!" I know an unfortunately large number of girls who think it's just fine when they get frustrated in an argument--generally not a serious emotional argument, but a disagreement--to sort of ineffectually whack at the guy and think it's funny. Like a goofy cartoon "snap out of it, sillypants!" head-bonk. Only real.
First, you're discounting your own power. The idea that girl-hitting is okay hinges on the idea that girls are harmless. Maybe you're not as strong as him, but even if you're tiny (tiny girls do seem to have particularly poor hit-inhibition), you could do some damage if you tried. You could leave a mark. Respect that. Your fists, even small untrained fists, are weapons, and you don't goof around with weapons.
And second, you are, in anger, touching someone in a way they don't want to be touched. The fact that it doesn't do real damage doesn't make it all okay and adorable. Violence isn't just about injury, it's also about violation, and cute little "ooh you rascal" swats, when unwanted, are a cute little violation. It's not okay for him to hit you as long as it doesn't really hurt--so it's not okay to hit him. Ever.
2) Dom hitting. I've experienced this one. He's so used to hitting you during sex and giving playful swats or full-on "punishments" for "infractions," that you're out of role and you genuinely displease him and he spanks you. (Or you're just walking around and he sneak-attacks, which is not morally offensive but is very annoying.) The problem here is pretty self-evident, I think: a sub's consent isn't carte blanche, and sexy-hitting should have absolutely nothing to do with real hitting.
Like the girl hitting, dom hitting in my experience is usually meant to be silly and not physically dangerous. He's not deliberately being cruel, he's just failing to realize that the difference between a little spank for being such a dirty slut and a little spank for sass mouth is huge. It may hurt the same or less, but it's crossing the consent line. And it's scary. A lot scarier, I think, than he realizes. I've ditched guys over this; if I can't trust a guy when we're out of role just messing around, how can I trust him to tie me up?
The bottom line is basically don't hit people even a little bit unless they have very specifically and explicitly asked for it. It's a pretty easy rule to remember I think.