Introduction

At some point in the spring of 2026, the largest school district in the United States drew a line. New York City's Department of Education, which serves 1.1 million students across more than 1,700 public schools in five boroughs, released its preliminary guidance on artificial intelligence use in classrooms and embedded a requirement that will reshape how every edtech company approaches product development: no AI tool enters the system without first clearing a bias and equity review. The message to vendors was direct, and it was not phrased as a suggestion. Governance — not speed, not innovation, not competitive feature sets — is now the primary pressure point at the most important education procurement gateway in the country.

The guidance, released on March 25, 2026, and accepted for public comment through May 8, represents the opening move in a longer regulatory sequence. A comprehensive AI policy playbook is scheduled for release in June 2026, and everything that is currently outlined in preliminary form will be formalised, expanded, and made into the fixed terms against which every AI vendor seeking access to New York City's classrooms must be judged. For edtech companies, the June playbook is not a guidance update. It is a compliance deadline.

What the Vetting Process Involves

The framework New York City Public Schools, or NYCPS, uses to approve AI tools is called ERMA — the Educational Resource Management and Approval process. Every tool that processes personally identifiable student information must go through ERMA before it can be used in any school, in any borough, with any student. That requirement is not new. What is new is what ERMA is being expanded to evaluate.

NYCPS has been direct about the current state and the direction of travel. The ERMA process currently evaluates tools for data privacy and security. It does not yet fully evaluate algorithmic bias, equity impact, or instructional effectiveness. Those expanded review criteria are what the June 2026 playbook is designed to establish. The district has committed publicly to building centralised capacity to review for algorithmic bias and equity impact, and the preliminary guidance makes clear that expanded evaluation will be a requirement, not an option.

What the expanded vetting will involve, based on the guidance and public statements from the 76-member Central AI Task Force that is building it, includes evaluation of a vendor's model training data — examining what datasets the AI was trained on, whether those datasets represent the full demographic range of the student population, and whether any demographic groups are underrepresented in ways that could produce systematically worse outputs for those students. It also includes testing for disparate outcomes across student demographic groups — assessing whether the AI performs measurably differently for students of different racial backgrounds, English language learners, students with disabilities, or students at different socioeconomic levels. Cultural responsiveness is a third dimension of the evaluation, alongside developmental appropriateness standards for the three distinct grade bands the district has identified: K-5, 6-8, and 9-12.

The governance structure around that vetting is more than a review checklist. A Central AI Task Force with 76 members drawn from across the Department of Education's divisions, a Data Privacy Working Group, and an AI Advisory Council that includes education technology partners from Google, OpenAI, and other companies all participate in ongoing implementation. The district's AI Advisory Council composition is itself a point of legitimate discussion, given that it includes the very companies hoping to contract with the school system, but NYCPS has made clear that community input, through the May public comment window, webinars, and direct family engagement, is a foundation of the process rather than a formality. Schools Chancellor Kamar Samuels wrote directly to parents: "AI is here, and our responsibility is to put strong systemwide safeguards in place."

The Traffic Light Framework and What It Permanently Prohibits

Alongside the bias review requirement, the preliminary guidance established a "traffic light" framework that divides AI applications into three categories: green for encouraged uses, yellow for uses requiring careful human oversight, and red for uses that are absolutely prohibited and are explicitly stated to not be subject to change in the final June playbook.

The red list is unambiguous and consequential. AI cannot make any decision about a student's academic placement, graduation, or programme access. AI cannot make disciplinary decisions. Individualised Education Programmes and 504 accommodation plans must remain with qualified human professionals — not because AI tools are currently incapable of generating text that resembles an IEP, but because the district has determined that the decision itself requires human accountability that cannot be delegated to an algorithm. Grading stays with the teacher of record. Counselling and crisis intervention are red. Biometric and behavioural data collection faces strict oversight requirements that are still being finalised.

What is green is, by contrast, deliberately practical. Teachers can use AI for brainstorming lesson ideas, drafting non-critical communications, exploring unit planning approaches, and scheduling support. AI can be used to find trends in student data, to generate translation support for bilingual learners, and to help adapt materials for students with disabilities — but in both of those yellow-category applications, a trained professional must review the AI's output before it reaches a student. The principle running through every category is the same: AI supports educator decision-making. It does not replace it.

Naveed Hasan, a member of the Education Department's Data Privacy Working Group, described the pre-guidance landscape with a comparison that landed: "Just like TikTok was unregulated until school networks blocked it, so are these free AI products." The Microsoft 365 contract that covers millions of NYC students did not originally include AI chatbots. It now does, because the underlying products evolved faster than the contract anticipated. The guidance is a retroactive attempt to impose order on a deployment reality that arrived before governance did. The June playbook is the attempt to ensure that does not happen again.

Why the Shift From Voluntary to Mandatory Matters

The significance of the NYC DOE framework is not its novelty in isolation. Most US states have issued some form of AI guidance for education. As of January 2026, state departments of education in 31 states had released AI guidance for K-12 public schools. The significance is the shift from guidance to requirement, and from privacy-only vetting to equity-inclusive vetting, at the scale of the country's largest school system.

Voluntary guidance has a known failure mode in education technology: it creates a compliance posture without creating compliance. A vendor that believes its product does not have a bias problem does not test rigorously for bias, because there is no external requirement making the test a prerequisite for market access. A district that receives a vendor's self-attestation that its AI is "equitable" has no mechanism to verify it. The result is a market in which tools that carry genuine risks reach classrooms because nobody with enforcement authority has required the evidence that would reveal those risks.

NYCPS's ERMA requirement changes that calculation. A vendor that cannot demonstrate compliance with ERMA data privacy standards cannot access New York City's schools, regardless of how good the product is on every other dimension. When the June playbook extends ERMA's scope to include algorithmic bias, equity impact, and instructional effectiveness, the same enforcement mechanism will apply to those dimensions. The tool passes, or it does not enter. The conversation about bias stops being a values discussion and starts being a procurement discussion.

The pattern echoes what has already happened in financial services and healthcare, where algorithmic accountability requirements moved from voluntary frameworks to enforceable standards as the risks of unreviewed AI became visible in practice. Education is following the same arc, and New York City's framework is the clearest single expression of that transition at school district scale.

What Other Jurisdictions Are Doing

New York City is the most prominent example, but the governance push in education is broader than a single district, and understanding what other jurisdictions are doing reveals both the direction of the consensus and how far most places still have to travel to reach it.

Ohio went the mandate route at state level. Under House Bill 96, every public district, community school, and STEM school in Ohio is required to have a formal AI policy in place by July 1, 2026. The state released a template policy that districts can adopt directly or customise, producing guaranteed coverage across every district in the state at the cost of depth and nuance. Vermont published 50 pages of detailed grade-band guidance in January 2026, one of the most specific frameworks available: no AI chatbots for PreK-2, curriculum-embedded AI only for grades 3-5, structured education-specific chatbots for 6-8, and broader AI fluency development for grades 9-12. Texas passed the Texas Responsible AI Governance Act in June 2025, effective January 2026, with civil penalties of up to $200,000 for violations — one of the few US examples of education AI governance backed by meaningful financial enforcement.

California established an AI Working Group under SB 1288, meeting publicly between August 2025 and February 2026 to develop statewide guidance, with a participatory stakeholder process that mirrors NYC's consultative approach at state scale. Chicago Public Schools, serving 330,000 students, published an AI Guidebook through its Office of Teaching and Learning and IT department, designating 2024-25 as a learning year with full GenAI integration planned for 2025-26. Houston Independent School District, serving 210,000 students, took a different direction, converting schools into Future 2 Schools focused on AI skills and critical thinking rather than leading with AI governance.

Boston announced in June 2026 that it would become the first major city school district to make AI fluency a graduation requirement, backed by a $1 million seed grant, with a curriculum developed by UMass Boston grounded in ethics and critical engagement rather than passive tool use. The Philippines' Department of Education issued Department Order No. 003, Series of 2026, officially sanctioning AI use in public schools while drawing a clear distinction between high-risk applications requiring strict human oversight and low-risk uses such as grammar correction — a framework that aims to reach approximately 1.05 million students as part of Project AGAP.AI. None of these frameworks require the combination of mandatory pre-deployment bias review and demographic equity testing that NYCPS is building toward in June. That remains distinctive.

What a Global Framework Should Look Like

The Brookings Institution's global study, which reviewed more than 400 studies across 50 countries and is cited in the NYCPS guidance itself, identified algorithmic bias, unequal access, insufficient educator preparation, and differential developmental effects across age bands as the four most consistent challenges in AI-in-education deployments worldwide. Those are not American problems. They are structural features of how AI tools interact with demographically diverse student populations, and every school system that deploys AI at scale will encounter them.

What the evidence from Brookings, the American Academy of Pediatrics' 2026 policy statement, and the NYCPS framework collectively points toward is a governance architecture that several elements must compose. The first is mandatory pre-deployment review, not voluntary disclosure, because voluntary disclosure produces only the evidence that vendors choose to generate. The second is demographic equity testing as a standard component of that review, not an optional add-on, because an AI tool that performs well on average while producing significantly worse outcomes for a specific student subgroup is not an acceptable tool for a public school system whose mandate is to serve every student equally. The third is grade-band differentiation, because a child in kindergarten and a student in eleventh grade are not the same audience for AI assistance, and governance frameworks that treat them identically are not actually governing.

The fourth element is transparency — publishing which tools have passed review, what the review found, and what conditions were attached to approval — so that families, educators, and researchers can see what is in classrooms and raise concerns when outcomes diverge from what the review predicted. The fifth element is human override requirements: clear rules about which decisions AI can inform and which decisions must remain with qualified humans regardless of what the AI recommends. The NYCPS red list — no AI grading, no AI discipline, no AI IEPs — is a workable starting model for that principle globally.

What is missing from most current frameworks, including the NYCPS preliminary guidance, is post-deployment evaluation: systematic measurement of whether an approved tool produces equitable outcomes in practice, with a clear mechanism for revoking approval if it does not. Approving a tool based on its training data and pre-deployment testing is necessary but not sufficient. The only test that finally matters is what happens to students when the tool is actually running in classrooms at scale.

Conclusion

New York City's requirement that every AI tool pass a bias and equity review before reaching its 1.1 million students is, in the current landscape, the most concrete example of what mandatory AI governance in education looks like rather than what it aspires to look like. The June 2026 playbook, when it arrives, will set the specific criteria, the testing methodology, the consequences for non-compliance, and the standards that every edtech vendor serving the largest school system in the country must meet.

The district's own documentation is honest about where the process currently is: ERMA reviews privacy and security. The expanded capacity to review algorithmic bias and equity impact is under construction. That honesty is itself an important signal. A governance framework that acknowledges what it cannot yet do, and commits to a timeline for building the capacity to do it, is more credible than one that claims comprehensive oversight it cannot deliver.

What happens in New York in June will be watched by school administrators in Chicago, London, Sydney, Mumbai, and São Paulo, because the question of how a public system governs AI tools deployed to children from wildly different backgrounds, with wildly different needs, is not a question that any single school district or country has fully answered. New York is trying to answer it, at the scale of more than a million students, in public, with a fixed deadline. The framework it produces is not the final word. It is the most serious attempt yet to write one.