question about spam

I have a zoho e-mail and another e-mail with at&t. I was getting spam in my at&t e-mail from a spam-bot using a friend's email address. I forgot about this problem this particular friend was having and accidentally e-mailed them from my zoho spam-free address. They replied back to me and so now I'm getting spam here as well.

My question is... can whatever is using my friend's e-mail now get into my contacts and start sending out spam in my name?