Should You Always Take Off Your Shoes at Someone’s House?

We visited some friends for dinner the other night and started to take off our shoes once inside their house. They said, “oh no, you don’t need to take off your shoes. We don’t.” My wife took off her shoes anyway but I hesitated and then just left mine on. It did feel strange as we always take off our shoes inside. What’s the appropriate etiquette in this situation?