Developing for users all over the globe means understanding users from diverse cultural backgrounds, different gender, etc. How much do those differences affect our understanding and does, for example, visiting the users’ country help? To answer these questions, we invited 16 designers to interview 48 users on new technology for semi-autonomous vehicles. We used a quantitative empathic accuracy measure to explore the influences of interpersonal factors on designers’ empathic understanding. The results highlight that designers’ empathic accuracy is dynamic, incomplete, and inconstant. The same designer achieves different levels of accuracy when empathizing with different users, and different designers have a different empathic understanding of the same user. National cultural differences, visiting experiences, gender differences, and types of users’ mental content influence designers’ empathic understanding significantly.