I assume by this statement that your father was probably in a few scraps. Admittedly immature, but I enjoy getting in a fight from time to time. I've had the shit beat out of me by way bigger dudes a couple time (being 5'8", 170lbs, its not very difficult to run across someone more imposing than I), but I've always found that in the moment you don't feel each punch. You're too focused on the task at hand; pain is not something to fear, and fear will only make you a terrible fighter. What hurts is the next day when your face is swollen and bruised, and you can't so much as blow your nose without wincing.And you don't worry about the pain when he hits you back
My father had a harder time of it. He was born an RH mismatch with his mother in deep rural NM and as a consequence had his growth permanently stunted. He barely made it to 5'7. I, on the other hand, am the product of recessive genetics and am the tallest person on either side of the family for at least three generations. So while he had to be a tough and eager scrapper in order to keep his nose clean, I generally didn't have to worry about anyone who wasn't at least a grade ahead. Which made adolescence somewhat delicious. There's nothing quite like having a former bully stare up at you from chest level.