Is there a way to edit the CONTACTS section that is highlighted in grey at the top of every Account page?

I like that it lists the contacts for the account, but I would like it to not display contacts that we have labeled as inactive inside the contact details.