Generative AI creates an accountability gap for schools: districts may authorize students to use systems that shape instruction, yet lack meaningful control over what those systems produce. That gap matters for liability and insurance because schools’ traditional duties center on supervising students and protecting them from foreseeable harm.
The concern is not simply that AI can make mistakes. It’s that some systems generate new material in response to student prompts, making their outputs difficult to predict, trace or constrain at school-wide scale. When harmful content reaches a student before an adult can intervene, a district may face questions about whether authorizing access met its duty of care.
Why system design matters
Schools use different kinds of artificial intelligence, and the article argues that their risk profiles are not interchangeable. Rule-based systems operate within defined parameters. A math program that guides a student through a set of problems, for example, can produce predictable results tied to its programmed structure.
That predictability makes rule-based tools easier for districts to review before use, monitor during use and audit afterward. Their outputs can be assessed against curriculum and school policy, and traced to the people or institutions responsible for designing or deploying them. Those qualities allow schools to apply familiar oversight practices and, if a problem occurs, assess responsibility among the parties involved.
Generative AI works differently. Systems such as ChatGPT and Gemini create novel responses in real time, drawing on a broad range of possible outputs. Their operation can be difficult to explain, and similar prompts may not produce identical responses. In the authors’ view, those characteristics limit a district’s ability to exercise meaningful control over content as it is generated.
That distinction puts pressure on standard governance tools. An acceptable-use policy can set expectations for student conduct, but it does not necessarily constrain what a system generates. A vendor contract can establish obligations between the provider and the school, but contractual language alone does not give educators control over each response. The authors argue that policies and contracts built around adult users may also fit poorly when minors access AI through school-authorized environments.
Instructional role, uncertain controls
For students, a generative AI system may explain a concept, guide reasoning, give feedback and influence how learning develops. The authors contend that these activities amount to an instructional role in practice. Teachers who perform comparable work are licensed, supervised and evaluated, and they operate under professional and legal standards. AI systems do not face those same requirements.
Calling a system a “tool” in district policy does not, in the authors’ analysis, settle the question of its function. If students rely on it for instruction, claims may focus on the effect of that role and the school’s decision to make the system available. The label a district uses may therefore offer little protection if the system’s output is central to an alleged harm.
Some vendors offer channels through which schools can report harmful content. Whether a report requires investigation, corrective action or steps to prevent recurrence depends on the vendor agreement; the authors say some contracts make no such commitments. Reporting after an incident may also do little to address the school’s duty if a student has already encountered harmful material and the process does not reduce the chance of another exposure.
Insurance and liability exposure
These limits have prompted coverage concerns. The authors report that some insurers have concluded generative AI may not satisfy insurability criteria when schools cannot reliably predict or trace outputs or prevent exposure to harmful content. Yet districts continue to provide access, including through school-issued devices.
Potentially inappropriate output could include pornographic or discriminatory material. The article notes that existing technical safeguards do not eliminate that possibility, while parents may reasonably expect schools to protect students from such exposure. The authors also point to lawsuits alleging psychological harm from AI-generated content, including allegations involving encouragement of self-harm or violence. Those are allegations described in the source, not findings about every system or incident.
Responsibility may be contested among students, schools and vendors. If a student follows district rules and the harmful material comes from the system, the student may not have misused the technology. Vendor contracts often include liability limits and indemnification provisions that the authors say may shift risk toward districts. They note that students might still pursue design-defect claims against vendors, but that route remains untested and would not, by itself, resolve a school’s potential exposure.
Districts retain duties to supervise students and protect them from foreseeable harm. Plaintiffs could argue that those duties extend to content produced by a system the school chose to authorize. The authors’ central point is that compliance with a district’s own policy may not answer that argument if the policy cannot constrain the system’s outputs and the district knew of that limitation.
Decisions districts and insurers face
The resulting liability question is likely to turn on the decision to grant access: could a school reasonably have foreseen the risks, and did its controls address them? The authors suggest that current conditions may make breach-of-duty claims difficult to dismiss, whether evaluated by a judge or jury. That is their assessment of potential litigation, not a settled legal outcome.
Districts and insurers can examine the exposure before an incident occurs. That review may include state laws, district policy, vendor contract terms, system safeguards and the specific ways students use the technology. Schools can also ask whether contractual provisions meaningfully allocate responsibility and seek clarity from insurers on which AI-related risks, if any, fall within existing coverage.
The authors suggest that districts may consider pausing student access while they assess these issues and await clearer guidance on risk and insurability. The practical decision is not simply whether generative AI has educational uses. It is whether a school can supervise the system’s instructional role, manage the foreseeable risks of its outputs and establish a credible account of who bears responsibility when those controls fall short.
Emily Fernandez is a New York-based partner and trial attorney at Wilson Elser. She is also a member of the Distraction-Free Schools Policy Project. She can be reached at [email protected]. Jill Coleman is the founder of the Digital Childhood Council and a child safety governance specialist focused on institutional risk, accountability, and structural gaps in digital environments. She can be reached at [email protected]. Jana Slavina Farmer is a White Plains, New York-based partner on Wilson Elser’s IP & technology team and one of the chairs of the firm’s data use and AI governance practice. She can be reached at [email protected].