Price Books

How do Price Books work? I understand that at the product level we can establish different Price Books or levels, but how can they be applied to a Customer? My issue is that I have a Retail Price List, Wholesale, And VIP price lists that are applied to customers, certain customers buy from certain lists. How do I assign a price list or price book to a customer? Are they tagged or is it a field setting?