I recently setup a business mail account for 1 user for our custom domain (celebiozkaramete.com)

I noticed 3 issues:

1) Sending mail from user@celebiozkaramete.com works fine and i can sent it to different addresses on different servers (needless to say celebiozkaramete.com is hosted on ZOHO)

2) Sending mail to user@celebiozkaramete.com fails. I tried sending mail from hotmail to user@celebiozkaramete.com and instantly hotmail returns undeliverable mail error.

3) Sending mail to user@celebiozkaramete.com fails. This time i tried sending mail from gmail to user@celebiozkaramete.com multiple times. Most of the time mail delivery is successful with a great delay (approx. 50 minutes) and most of the time mail is never received.

What might be the cause? Any help is greatly appreciated. 

By the way our dns records are setup as suggested.