I responded the same way each time.

“Oh, let me update your contact entry in my phone real quick.”

Then we go back to what we were doing.