I've heard the following argument many times, many places:
"Men can impregnate many women at once, which means that they'll be naturally driven to increase their evolutionary fitness by fucking around!"
The problem with this argument is that fitness isn't measured in conceptions. It's measured in descendants. Spreading your genes as widely as possible doesn't institute them in your species unless the carriers of those genes live to adulthood and produce grandchildren and great-grandchildren. Maybe you can impregnate and dump five single mothers in the time it would take to get your wife pregnant once, but if a single mother is ten times more likely to miscarry or have a child die or have the kid grow up too unhealthy to reproduce prolifically--the real stud is the one who stands by his woman. (And the smart woman, having some choice of her own in the matter, doesn't make babies with a man who doesn't seem like he plans to stick around.)
Or not. Maybe the single mom is pretty damn self-sufficient and she's only a little more likely to screw up. That would make it worthwhile for men to spread their seed. How much benefit fathering provides varies by species, and in humans it varies tremendously by era and culture. I don't really know which side of the equation early humans were on. My point is only that it isn't a given that causing the most pregnancies necessarily implies the most fitness. Many times, as in seahorses or wolves or penguins, the dad who stays and works spreads his seed further than the dad who just throws sperm around.
Also, whether fathering matters or not, having the most children doesn't always lead to the most fitness; having twenty kids miserably fighting for scraps of food may get you fewer grandchildren than raising five fat and happy little critters. A pregnancy isn't the determiner of fitness and neither is a baby. A great-great-great-great grandchild is.
This is all really theoretical. I haven't done the research. Maybe in humans (and more importantly, historical humans) fatherhood value is low and optimum family size is high, in which case spreading the seed really is the right strategy. (I doubt it, though; enough human civilizations seem to have independently developed and stuck to systems of faithful pair bonds that I'm guessing that's what's natural for people.)
But you know what? We're not strategizers. We're not slaves to instinct. We're fucking people.
Maybe you could raise sixteen children well enough for them to reproduce, but very likely you'd rather raise two or three with the resources for them to be happy and educated as well as fit. And maybe you could ditch a pregnant woman and breed again, but likely you'd rather find a woman you can love and be happy with. It might be an accident of evolution, as misguided as a bee fucking an orchid, but our brains are too big and our emotions to complicated to run on instinct.
We're lowering our fitness and increasing our humanity.