Studies suggest chatbots can influence users’ values and beliefs

In short: A small group of companies and researchers decide many of the “values” built into popular chatbots, and studies suggest those choices can influence what users think after even brief chats.

What's going on

Chatbots do not have their own morals. Instead, people design how they act by choosing training data, writing safety rules, and setting goals for what the bot should treat as helpful, harmful, acceptable, or true. You can think of it like a customer service script, where the script writers decide what counts as a “good” answer.

Anthropic, the company behind the Claude chatbot, says it shapes Claude’s values on purpose using a method called Constitutional AI. In simple terms, this means the company writes a set of guiding rules (like a rulebook) and trains the chatbot to follow them, along with extra training that encourages a certain tone and personality.

Researchers also describe “alignment” work as a process where human values are collected, combined into a target, and then used to train the model toward that target. That matters because it means a relatively small number of designers and paid raters can strongly influence the chatbot’s moral framing, which is the way it talks about right and wrong.

Recent studies add to the concern. One paper reports that even brief chatbot conversations can produce lasting changes in political beliefs and even core moral values, at least for some people. Other research on chatbots that use value-focused framing found they can significantly influence participant attitudes.

What to watch

The evidence so far shows influence, not that chatbots permanently change everyone’s values. Researchers say effects likely depend on which chatbot is used, how people talk to it, how long they use it, and whether a user is especially vulnerable. As chatbots become more common in daily life, expect more scrutiny of who sets their rules, how transparent those rules are, and how bots handle sensitive topics like politics, sexuality, harassment, and self-harm.

Source: NYTimes