Security flaw with Attachments, Notes, and Activites

Security flaw with Attachments, Notes, and Activites

There's no way for me to keep other users that have "Read Only" access to my records (ie. Potential) from adding, editing, or deleting Attachments, Notes, and Activities that I have added to my record.  See the attached picture.

There are no settings under "Data Sharing Settings" or "Field Level Securities" to protect or restrict access.

Has anyone else found a way to protect these items while allowing other users to view the record?