Condom use and sensitivity

My wife and I are newly into the LS. In our own sex life we haven't used condoms in 15+ years. We are in our early days of swinging and we use condoms to play with others (as only makes sense). However, it turns out I have a hell of a time cumming with a condom on, if I can at all. Since condom use is non negotiable for us in LS, are there any tricks of the trade that the fellas out there could share that might help?