Microsoft sets new AI privacy and safety rules for schools

In short: Microsoft has agreed to new, enforceable rules for how its AI tools can be used in schools, with limits meant to protect students and teachers.

What happened

Microsoft signed an agreement with the American Federation of Teachers (AFT) and the United Federation of Teachers (UFT), its New York City affiliate. The deal lists 10 safety and privacy principles that school districts can add to their contracts with Microsoft.

A key promise is that Microsoft will not use student or educator data to train its AI models. Training an AI model is like teaching it by showing it lots of examples. The agreement also says Microsoft should collect less data in the first place and explain to families, in plain language, how its tools work.

The terms also prohibit “AI companions,” meaning chatbots designed to act like a friend or personal buddy. For certain “high-risk” uses, the rules require a human to review decisions instead of letting software decide on its own.

The timing matters. The agreement comes about a week after New York City and Los Angeles announced one-year bans on many student-facing AI tools while they figure out rules and safeguards. Starting in November, school districts can choose to add these new terms to new or existing agreements without renegotiating their whole contracts.

Why it matters

Schools are under pressure from parents and educators who worry about too much screen time and about companies collecting data on children. This agreement gives districts a clearer set of guardrails they can enforce, and it may influence how other tech companies set rules for school AI.

Source: The Verge AI