Evidence-Informed Leadership in the Age of AI

A Practical Aotearoa New Zealand Framework for School-Owned Data, Intervention Intelligence and Improved Student Outcomes

Tony Gilbert

Discussion Paper Version 1.4

Prepared for New Zealand school leaders, boards, educational organisations, policy makers and technology providers.

Executive Summary

Artificial Intelligence (AI) is rapidly becoming part of the educational landscape in Aotearoa New Zealand. Schools are increasingly exploring tools such as Gemini, NotebookLM, Microsoft Copilot and a growing range of AI-enabled platforms. Much of the current discussion understandably focuses on productivity gains, administrative efficiency, lesson planning and classroom applications. While these uses are important, they may not ultimately represent the most significant opportunity AI presents to education.

The greater opportunity lies in helping schools make better use of the evidence they already possess. New Zealand schools collect vast amounts of information about learners through attendance records, curriculum progress data, assessment results, intervention histories, participation information, wellbeing indicators and teacher observations. Collectively, these sources provide a rich picture of student learning and development. Yet despite this abundance of information, many schools continue to grapple with a deceptively simple question: how do we know which actions are making the greatest difference for learners?

The challenge facing schools is not the absence of data. Rather, it is the difficulty of transforming data into meaningful evidence that can support decision-making. Information often exists across multiple systems, is recorded inconsistently, lacks common definitions, is disconnected from intervention records and is rarely analysed longitudinally. As a result, schools can find themselves surrounded by information while still struggling to identify patterns, evaluate effectiveness and learn systematically from their own experience.

At the same time, advances in AI have created new opportunities to analyse information at a scale and speed that would previously have been impractical for most schools. However, AI introduces its own challenge. Artificial Intelligence will only ever be as useful as the information provided to it. Poor-quality data produces poor-quality insights. Incomplete information produces incomplete conclusions. Inconsistent information produces inconsistent results. AI does not solve these problems and, in many cases, can amplify them by presenting weak evidence with a level of confidence that appears persuasive.

This paper argues that schools should begin their AI journey by strengthening the evidence practices that sit beneath it. It proposes the development of a school-owned Evidence Architecture that enables schools to organise, qualify, govern and analyse their own information in ways that support educational decision-making while maintaining appropriate privacy, professional oversight and local ownership. Within this framework, schools retain control of their data, sensitive information is protected appropriately, intervention evidence is strengthened and AI-generated findings are validated through historical evidence, human expertise and ongoing review.

Central to the framework is the concept of Intervention Intelligence, which is developed fully in Section 5 as a way of helping schools understand the relationship between interventions and outcomes.

The framework outlined in this paper is intentionally practical, scalable and independent of any particular Student Management System or AI platform. It can be applied across primary, intermediate and secondary settings and adapted to local contexts regardless of whether schools utilise Hero, KAMAR, eTAP, Edge or future systems. Most importantly, it treats AI as one part of a broader approach to evidence-informed leadership. The schools most likely to benefit from AI in the coming decade will not necessarily be those with the most sophisticated technology. They will be the schools that develop the strongest evidence cultures, the clearest understanding of their interventions and the greatest capacity to learn from their own experience in service of better outcomes for learners.

Foreword

For much of my professional life, I have been fascinated by a deceptively simple question:

How do we know whether what we are doing is making a genuine difference for learners?

It is a question that has followed me throughout my career in education. As a classroom teacher, I wanted to know whether my teaching was helping students learn. As a middle leader, I wanted to understand which initiatives were genuinely improving outcomes across a department and which were simply consuming time and energy. As a deputy principal, I became increasingly interested in understanding the relationship between interventions and outcomes: what was actually making a difference, for whom, and under what circumstances.

Later, through my work supporting schools across New Zealand, I discovered that this challenge was remarkably consistent regardless of school size, location or context. Schools were collecting more information than ever before. Student Management Systems had matured, assessment practices had become more sophisticated, attendance tracking was increasingly robust and digital technologies had dramatically increased the amount of information available to leaders. Yet despite this progress, many schools continued to wrestle with the same fundamental problem. They could often tell you what had happened, but they could not always tell you why.

Outside education, I have also had the privilege of working within high-performance environments, particularly in sport. One of the observations that struck me early was that the most successful organisations were rarely obsessed with data itself. They were obsessed with learning. Data was simply a tool that helped them understand performance, test assumptions, evaluate interventions and make better decisions. Coaches and analysts were not collecting information for the sake of producing reports. They were collecting information because they wanted to improve outcomes.

The same principle applies in education.

Schools do not exist to generate data. They exist to improve outcomes for young people. Data only becomes valuable when it helps educators make better decisions, allocate resources more effectively and understand whether their efforts are making a meaningful difference.

Over time, I became increasingly convinced that one of the greatest challenges facing educational leadership was not the collection of information, but the conversion of information into organisational knowledge. Schools are often rich in data but poor in insight. Information sits across multiple systems. Definitions vary between staff. Interventions are recorded inconsistently, if at all. Sensitive information creates legitimate governance and privacy challenges. Longitudinal analysis can be difficult, and valuable institutional knowledge is frequently lost as staff move on or leadership changes. As a result, schools often find themselves making important decisions with far less clarity than they would like.

The emergence of Artificial Intelligence has made this challenge both more urgent and more exciting. For the first time, schools have access to tools capable of analysing large volumes of information, identifying patterns and supporting inquiry at a scale that would previously have required significant specialist expertise. Tools such as Gemini, NotebookLM and Copilot have the potential to transform the way schools interact with their evidence.

However, I believe there is a risk in the way these conversations are often framed.

Many discussions about AI begin with the technology. They focus on platforms, functionality and capability. While those things matter, I believe the more important conversation starts with the conditions that make those tools worthwhile. Before asking how AI can help schools, we should first ask how schools can create evidence frameworks that allow AI to be used responsibly, effectively and meaningfully.

That distinction matters because AI cannot compensate for weak evidence. If the underlying information is poor, AI will not improve it. If intervention records are inconsistent, AI will not make them meaningful. If schools have not agreed what success looks like, AI cannot define it for them. Technology may speed up analysis, but it cannot create sound conclusions from weak foundations.

Conversely, when schools have strong evidence frameworks, AI becomes something remarkably powerful. It can help leaders identify patterns, evaluate interventions, challenge assumptions, surface organisational knowledge and support more informed decision-making. In this sense, AI is not the destination. It is an accelerator for work that must still be grounded in professional judgement and disciplined inquiry.

This paper is therefore not a proposal for a national student intelligence system. It is not an argument for centralising learner data, nor is it a suggestion that AI should make decisions about young people. Rather, it is an argument that schools should be empowered to use the information they already possess more effectively, more consistently and with greater confidence.

At its heart, this paper is about helping schools answer three enduring questions:

What do we know?

What should we do?

How do we know if it worked?

These questions are not new. They sit at the centre of effective teaching, effective leadership and effective school improvement. What is new is our ability to bring together information, evidence and technology in ways that may help us answer those questions more effectively than ever before.

If we can do that, we move beyond compliance, reporting and data collection. We move towards something much more valuable: a culture of inquiry, learning and evidence-informed leadership.

For me, that is where the greatest opportunity lies. Not in Artificial Intelligence itself, but in the possibility that it may help schools become more deliberate, more reflective and ultimately more effective in improving outcomes for every learner.

.

1. The New Zealand Context

The emergence of Artificial Intelligence in education is occurring at a time of significant change across the New Zealand education system. School leaders are navigating curriculum refreshes, increasing expectations around attendance, growing concerns regarding student wellbeing, workforce pressures, evolving approaches to assessment and heightened scrutiny around educational outcomes. At the same time, schools have access to more information about learners than at any point in their history. Student Management Systems, digital assessment platforms, attendance systems, learning support records and wellbeing tools are generating increasingly rich datasets about learners and learning. This combination of growing complexity and growing information presents both a significant opportunity and a significant challenge.

The opportunity lies in the potential for schools to make better-informed decisions about teaching, learning and student support. The challenge lies in ensuring that information becomes meaningful evidence rather than simply an expanding collection of data points. While the language of "data-driven decision making" has become common within education, the underlying concept is not new. New Zealand's educational research has long recognised the importance of leaders engaging critically with evidence to improve outcomes for learners.

Perhaps the most influential contribution to this discussion remains the School Leadership Best Evidence Synthesis undertaken by Robinson, Hohepa and Lloyd (2009). The synthesis demonstrated that leadership practices focused on teaching, learning, evaluation and professional inquiry have a significant impact on student outcomes. Importantly, the research did not suggest that leadership effectiveness was derived from intuition alone. Rather, it highlighted the importance of leaders actively engaging with evidence, questioning assumptions, evaluating effectiveness and creating cultures of continuous improvement. One of the strongest findings from the synthesis was that leaders who participated in and promoted professional learning and development alongside their staff had a measurable influence on learner achievement. Underpinning this work was an implicit understanding that effective leadership requires more than experience; it requires disciplined inquiry into what is and is not working.

Similar themes are evident within the Education Review Office's School Evaluation Indicators (ERO, 2016), which place evaluation, inquiry and knowledge-building at the centre of school improvement. These indicators encourage schools to move beyond the collection of information and instead focus on understanding the impact of their actions. ERO's work consistently highlights the importance of schools developing the capability to evaluate their own effectiveness, identify areas for improvement and use evidence to guide decision-making. This emphasis on internal evaluation reflects a broader shift in educational thinking, from accountability through reporting towards improvement through inquiry.

The growing focus on attendance across New Zealand provides a useful illustration of why this distinction matters. Attendance has become a significant national priority, with the Ministry of Education and schools alike seeking to improve rates of regular attendance. Attendance data provides valuable information, but attendance itself is not the outcome. Rather, it is a signal. It may indicate engagement, belonging, wellbeing, family circumstances, school culture or a combination of multiple factors. Attendance data tells us that something is occurring; it does not necessarily tell us why it is occurring or what interventions may be most effective in addressing it. The same principle applies across many aspects of educational leadership. Assessment data can indicate achievement levels and progress, but it rarely explains the underlying causes of those outcomes. Behaviour information may identify concerns but not the conditions that contributed to them. Wellbeing indicators may highlight emerging challenges without revealing the most effective responses.

This challenge is becoming increasingly relevant as schools seek to create more coherent understandings of learner progress and success. Recent work associated with the New Zealand Curriculum refresh, the Common Practice Model and increasing attention on structured approaches to literacy and numeracy all point towards a system that is seeking greater clarity about progress, improvement and effectiveness. At the same time, schools are being encouraged to strengthen their use of evidence to support decision-making and demonstrate impact. The question is no longer whether evidence matters. The question is how schools can organise, understand and utilise that evidence in meaningful ways.

Artificial Intelligence enters this landscape as a potentially powerful enabler. For the first time, schools have access to tools capable of analysing large volumes of information quickly, identifying patterns across multiple datasets and supporting inquiry processes that would previously have required considerable time or specialist expertise. Platforms such as Gemini, NotebookLM and Copilot offer the possibility of helping schools explore relationships between attendance, achievement, wellbeing, engagement and intervention data in ways that were previously difficult to achieve.

However, the emergence of AI also exposes a significant weakness within many existing information systems. Most school systems were not designed for sophisticated evidence analysis. Student Management Systems were primarily developed to support administration, reporting, communication and operational management. They were not designed to answer complex questions about intervention effectiveness, longitudinal learner outcomes or organisational learning. As a result, many schools now find themselves confronting a new question:

Is our data ready for AI?

For many schools, the answer is not yet.

This is not because schools lack information. On the contrary, most schools possess substantial amounts of information about their learners. The challenge is that information is often recorded inconsistently, stored across multiple systems, lacks common definitions and is rarely connected systematically to interventions or outcomes. Before AI can provide meaningful insights, information must first be qualified, validated, categorised and governed appropriately. Without this foundation, schools risk generating sophisticated analyses built upon weak evidence.

This is why the central challenge facing New Zealand schools is not primarily technological. It is organisational. Adopting AI tools is only one part of the task. The deeper work is developing Evidence Architectures that enable schools to learn systematically from their own experience. Schools need frameworks that help them determine what information matters, how it should be recorded, how it should be interpreted and how it can be used ethically and effectively to improve outcomes for learners.

The schools that benefit most from AI over the coming decade are therefore unlikely to be those with the most advanced technology. They are more likely to be the schools that invest in evidence quality, develop clear intervention frameworks, establish strong governance processes and foster cultures of inquiry. These schools will be able to use AI not as a substitute for professional judgement, but as a tool that strengthens it. Technology may accelerate insight, but it cannot replace the leadership behaviours that underpin effective school improvement.

For this reason, the remainder of this paper focuses less on AI itself and more on the foundations required for AI to be useful. Before schools can benefit from sophisticated analysis, they must first address a more fundamental challenge: understanding why many existing school data systems struggle to generate meaningful evidence and how a different approach can help schools learn more effectively from their own experience.

2. The Problem with School Data: Why Most Schools Are Data Rich and Insight Poor....

What to read the whole thing? email: tony@agfox.ai

© 2026 Tony Gilbert. All rights reserved.