Customer Portal - All Portal Users

For some reason when I edit a report, restrict visibility changes and it keeps adding all Customer Portal User group to the visibility group....not good. Is there a setting I need to change? I don't want to check visibility every time we edit a report.