Special Character Limitations On Fields

I am having issues with the not being allowed to use special characters in certain fields. As I cannot use symbols such as ' and & I am having to make deliberate spelling mistakes.

For example:

companies with "&" in their official name

addresses with an apostrophe,

surnames with an apostrophe,

Is there a way around this?