Your children should belong only to you. You should not have to share them with people out of your own home.
So, be happy with your own wife. Enjoy the woman you married while you were young.