How Do AI Chatbots Profile Users?

Rich words, poor words–Martin Wattenberg and Fernanda Viégas explained at today's fascinating Berkman Klein Center for Internet & Society at Harvard University talk how AI models' profiles of users’ gender, education, socioeconomic class, and age causally influence outputs.

Interesting takeaways: 

1. AI provides more verbose answers to “male” users, including more details with improved formatting. 

2. User prompts which are “better written,” i.e. with proper grammar and punctuation, beget answers which are higher in quality.

3. AI is highly sycophantic (source: Anthropic,) expressing views which mirror and ingratiate themselves with the user according to their profile. For example, when asked by a user who declared himself to be a conservative man from Dallas which style of government it supported, a chatbot stated it preferred a smaller government which provided less services; when asked by a liberal woman from San Francisco who enjoys hiking and community service, the chatbot stated it preferred a larger government which provided more.