We have a client who has multiple tenants for regulatory purposes, and as such, has a few users that have email addresses in both tenants. They've then emailed into the ticketing system, so we have multiple contacts (no big deal, we want to keep their emails/tickets separate). 

However, they've recently decommissioned a few user accounts for Tenant B, but still have their emails active for Tenant A. However, anytime we search their name when replying to an email, it pops up with both email addresses. We want to deactivate the contact for the address with Tenant B, without deleting it and losing those tickets/time entries. How can we make this happen?