Segment is not pulling all tagged contacts

Segment is not pulling all tagged contacts

So I am trying to create a segment in campaign that pulls all the contacts with specific tags from CRM.

When I create the segment it is only pulling about 80% of the contacts with that specific tag.


I have isolated a few contacts that this happening for. When I tag that individual contact with a new tag in the CRM they still do not appear when I update the segment for that specific tag. At this point I believe that the problem is with a setting for the specific contact in the CRM. 

Any idea why this is happening?