Add Custom Fields only in Customer module and not on supplier module!? Is not there a way to do that!?

Add Custom Fields only in Customer module and not on supplier module!? Is not there a way to do that!?

I am trying to create custom fields on clients module but it also gets created on suppliers module; which of course does not make sense at all as a lot of custom fields are client or supplier specific but never both.

I am missing something? This seems like such a basic feature that should be implemented.