Yes, what you and your present wife did was definitely wrong (it was sin), but, if I may, I would like to share with you something written about marriage by Douglas Anderson:

"Clearly, therefore, the Bible teaches permanence as the ideal; but unfortunately, human hearts are still hard; and divorce for various reasons still happens.  The Gospels are filled with examples of how Jesus dealt with persons who were struggling with guilt and failure (Luke 19:1-10), (John 8:2-11), including one woman who had been married five times and who was living with a man who was not her husband (John 4:1-42.)  Where guilt was involved, Jesus did not minimize it; but in every case He acted redemptively. That is, His goal was not to condemn people but to help them begin anew with God's grace and strength."

When we find ourselves in a situation as you've described above, what we need to do is trust in God's forgiveness, determine in our mind not to make the same mistake again, continue to move forward in our life with the Lord, and determine to be the best husband or wife, in Christ Jesus, that we can be.¹  Jesus still loves you and He's forgiven you; now you must forgive yourself.²

Just a suggestion, but you might also want to apologize to your first wife for what you did (you might just send her a letter letting her know that you are sorry for not being the husband you should have been) and you might also want to ask your present wife's forgiveness for causing her to stumble.³  Let your present wife know that you love her, that you love the Lord, and that what happened before will never happen again.  You should not leave your present wife.

Bible References:

¹ (1 John 1:9; John 8:10-11; Hebrews 12:1; Ephesians 5:22-33)

² (Romans 8:38-39; 1 John 1:9)

³ (Romans 14:21)