You can export just the contacts that you want to download. Ensure you have the Export permission enabled for your profile.
To download a specific contact
Go to the Contacts module.
Select the contact you want to download.
Click the more icon and then select the Export option from the dropdown.
Click Download in the Export Completed pop-up.