Spam - The Shell Game

I have contacts saved as contacts and their emails still go to spam. I have had a saved contact send me an email and the email goes through. Later on the same day I have had the same contact send me a different email - no thread, and the email goes to spam. I dont get it ;-(