Send invite only to added attendee(s)

Send invite only to added attendee(s)

When I add someone to an existing meeting, I can send it to everyone or send+me. This means that everyone who has already accepted receives another invite. 

When I have an existing event, and I invite someone new, the invite should only go to the person I added. 

How can I make this happen?