This white paper – Assess. Analyse. Intervene. From E-Assessment to Personalised Learning – was written to help Ministries of Education, Local Education Authorities and prospective suppliers understand how to build on E-Assessment and E-Examination to create personalised learning experiences. Taking the three key building blocks of Assessment, Analytics and Intervention, the paper defines a Personalised Learning Platform and its interfaces within a broader schooling ecosystem – the Schooling Enterprise Architecture.
The central proposition to this paper is that using data generated by the growing use of E-Examination and E-Assessment process offers significant value for increasing the effectiveness of the schooling systems.
Schooling system needs to constantly innovate and evolve. This paper sets out a vision for how schooling leaders can make learning even more effective by personalising the learning experience for all school students – without introducing unmanageable complexities.
The implementation of the key recommendations of this paper should deliver the following benefits:
Effective learning – Intervention is about developing virtuous cycles of learning, tailored to individual needs
Deep insights – using deep analytics, new and unpredicted patterns can be found that can help inform decision makers about where to focus investments
Timely intervention – whilst E-Assessment takes essential “rear view mirror” snapshots of learning performance, predictive analytics can be used to constantly steer students in the right direction, maximizing the chances of doing well in assessment and examinations
Three interdependent processes combine to deliver a personalized learning experience:
Ongoing assessment from a range of sources is used to gather data about how individuals and groups of students are learning. This data is analyzed to help target students with tailored learning, and to make decisions that lead to increased effectiveness. Using data, interventions can be set up do deal with issues such as reducing drop-out rates; selecting the most effective ways of improving reading and mathematics; and dealing with risks before they become a problem. Ultimately interventions can be tailored for individuals and groups of students.
Each of these processes are interconnected in multiple ways –
The white paper explores these processes and how they integrate and can be implemented.
There are four distinct phases through which technology in schooling evolves. The first phase is characterized by access. In the next phase, technology is used to enhance existing processes. The third phase is characterized by using technology strategically. No longer is technology considered a “bolt-on”, or “veneer” on top of existing processes – it now helps drive schooling towards strategic goals such as significantly improved learning and better return on investments. In the final phase, leading edge schools use ICT to transform their operations, using it to personalize learning, integrate deeply with the wider community, run extremely efficient administration systems and develop a culture of performance.
‘Strategic’ Phase Vision
In the Strategic phase, technology becomes a key asset in achieving the strategic goals of an organisation. It’s about restructuring work and processes and doing things differently.
Typically in this phase, the strategic goals of an organisation would include raising standards and improving performance, and technology is a strategic tool for achieving these strategic goals by enabling:
Intelligent intervention – data driven support for learners
Connected Learning Communities – fully exploiting all available resources, and integration with the local community
Monitoring, analysis and planning – data driven decision making
This is essentially about using data to make well informed decisions about what students need to learn or do next. To fully personalise the learning experience students should be constantly assessed as they move through their schooling, and their learning pathways should continuously evolve. This relies on highly effective feedback loops and systems which dynamically adapts to the twists and turns of the learning process, and sets challenging learning goals and tasks. This is extremely difficult to do within a paper-based setup, but can be made a lot easier through using IT systems that provide analytic and workflow capabilities. Intelligent tutoring systems, and managed learning environments, are becoming more commonplace and increasingly sophisticated.
Monitoring, Analysis and Planning
To manage an organisation strategically, as opposed to fighting fires, the ability to monitor performance, analyse results and plan for improvements is fundamental. Organisations wanting to manage strategically must have three key capabilities:
This capability provides managers with the ability to know “what is happening” and “what has happened.” Organisations implement dashboards, scorecards, or reports to monitor their performance. These visual applications allow managers to keep an eye on important indicators of their organisation’s health.
This capability provides managers with the ability to know what is happening and why. To analyse performance, organisations implement solutions that are often very interactive in nature and allow managers to investigate the root cause of issues they see in their dashboards, scorecards, or reports.
This capability provides the organisation with the ability to model what should happen. Organisations develop processes and tools to conduct the essential planning, budgeting, and forecasting exercises. These processes allow managers to align groups and individuals around the metrics that drive the organisation—for instance: “what are our examination result targets?” or “what is our spending versus our revenue?”
Connected Learning Communities
Whilst there may be elements of learning that require independent work, learning only really acquires meaning in a social context, and the most immediate and direct social context for schooling is the local community.
ICT can be used to connect together all those who can make a contribution to students’ learning – e.g. local business, community resources (e.g. museums/libraries), parents and 3rd party learning services. It can connect students to inspiring individuals and inspirational speakers; promote debate and engagement between collaborators in face-to-face or virtual groupings; and provide mentoring opportunities. Connecting stakeholders together in a Connected Learning Community has enormous benefits such as engaging parents more deeply in the learning process, speeding-up processes and improving students’ connections with the outside world. The core of a connected learning community is a portal that can be accessed from anywhere.
In the Strategic phase, students have continual access to their own learning devices. These devices need to enable a range of learning scenarios (not just content consumption), be rugged, easy to repair and support, manageable on a network.
Devices should be available to students so they can learn anytime anywhere, access content, learning management and communication and collaboration tools via the Connected Learning Community Portal.
Having access to their own devices enables students to experience a wide range of learning scenarios:
Classrooms need to accommodate an increasingly wide range of learning styles, and equipment needs to be laid out in quite different ways according to the demands of each different learning task, for example:
BECTA provided the following guidance to UK schools on different classroom layout options:
Pods – separate circular / hexagonal / octagonal benches with workstations
No corners with 2 computers, so no dead spots that cannot be used
No extra space required for 2 pupils to share a computer
Can support collaborative work as users working around ‘one pod’
Pods – squares with computers on two sides only
No corners with 2 computers, so no ‘dead spots’ that cannot be used
No extra space required for 2 pupils to share a computer
Can support collaborative work as users working around ‘one pod’
Bays built along walls
Teacher can more or less see all computer screens from the centre of the room
Provides opportunity to use the centre of the room for tables enabling work away from the computer, and to gather groups for discussion
Cabling and electrical work is cheaper and easier than ‘pod’ designs as along the room edge.
In the Strategic phase, IT has become a strategic asset to schools. With the infrastructure optimised in the Enhanced phase, we now turn our focus on workloads delivered by servers.
The following services are core in the Strategic phase:
Optimised Infrastructure – including File and Print, Database Services, Directory Services, Security, Device Management, and Data Protection and Recovery
MIS – Management Information Systems
Virtualisation – centralizing computing tasks to improve scalability and system performance
These, typically, will be delivered through three layers:
On-Premises – the school hosts key functions on their own servers
Data Centre/Private Cloud – the Local Education Authority (LEA) delivers services to schools from their servers
Public Cloud – the school receives services from the LEA, Ministry of Education and private suppliers from Public Cloud Services
The Strategic phase is characterised by the Connected Learning Community, the core of which is a portal that can be accessed from anywhere. For it to be effective it needs to be “role based” i.e. present users with information and tools relevant to their role and to them as individuals. In other words a teacher in the community sees the information relevant to all teachers, their fellow subject specialists, and also information specific to their particular group of students, their particular HR information, and their particular teaching content, tasks, calendar, e-mail etc.
A portal should give students, parents, managers, teachers, their own “spaces” and deliver to them the resources that are important individually to them through a single web page. It aggregates information from diverse systems into one interface with a single sign-on ID – and organisation-wide search capabilities so that users can access relevant information quickly. Teaching and administration staff can use the portal to distribute information to students based on their enrolment, classes, security group or other membership criteria, while enabling them to personalise the portal content and customise the layout to suit their needs.
A great Portal reference architecture is Twynham School. Twynham is a 1600+ Secondary school in Christchurch UK, built a powerful collaboration platform – “Learning Gateway” – which allows students, staff and parents to work efficiently; develop independent and inter-dependence in their learning strategies; and support children in achieving their full potential. Twynham School won the BECTA ICT Excellence Award in 2008 for learning Beyond the Classroom and the schools works with over 400 schools internationally to support the development of their Learning Platforms.
Mike Herrity at Twynham has published a detailed e-book explaining how the Learning Gateway is used: http://bit.ly/qJohiL
Microsoft have also published a full architectural guide explaining how Twynham built their Learning Gateway – http://bit.ly/qORAW5
Enabling many of the functions in the portal are 2 sub-systems – Content Management and Unified Communications & Collaboration.
Content Management Systems (CMS)
When ICT is fully implemented, vast amounts of content gets created. In order to get maximum efficiencies from ICT, this content needs to be organised and managed in a way that means that people don’t replicate one another’s work.
A content management system in a connected learning community helps education institutions organise and facilitate the collaborative creation of documents and other content. They enable the full life cycle of content – from initial creation to delivery to end users. CMS comprise document and records management, web content management, forms, search, library systems, curriculum frameworks, curriculum systems, curriculum exemplars and resource assemblers.
Unified Communications (UC) & Collaboration
Today it is typical that people will have multiple contact addresses – direct line phone number; mobile phone number; e-mail; Instant Messenger; home number; personal mobile number; home e-mail, etc. Unified Communications (UC) takes identity and presence and then has all of these other ways of interacting simply connect up to that.
A single integrated identity can simplify how you find and communicate with others. One integrated desktop application can provide easy access to all the ways users are likely to want to communicate. Another key advantage to UC is that in using Voice over IP (VOIP) for telephone calls, it has the potential to significantly reduce communication costs.
UC enables students, teachers, parents and other stakeholders to confer and consult in the way that suits their work style by switching seamlessly between videoconferencing, telephone, email and instant messaging.
Also within UC are task and calendaring functions.
Data Driven Decision Making
In a schooling system, data driven decision making is supported by a huge number of information systems. Any process that involves the creation and transmission of information can be considered an information system – even informal discussions.
The collective term for the information systems in schooling is Management Information Systems (MIS).
Functions Supported by an MIS
The functions that a Management Information System need to support are:
Improving Student Performance
Parents Engagement In Learning
Better Teaching Decisions
Make Better Management Decisions
Monitor, Analyse and Plan
Tactical Decision Making
Manage Resources More Effectively
Planning and budgeting
Accountability and Alignment
Performance and Assessment Data
KPIs, Scorecards, Dashboards and Reports
Key Performance Indicators (KPIs)
Drive Administrative Efficiencies
Management Information Systems – Functional Architecture
In this context, an information system really means an organised hierarchy of information sub-systems. Management Information System (MIS) is a term used as a container for all of the electronic information systems within a schooling system. These systems vary in size, scope and capability, from packages that are implemented in relatively small organisations to cover student records alone, to enterprise-wide solutions that aim to cover most aspects of running large multi-site organisations.
In the Strategic Phase, the goal of service provision at Local Education Authority level is to deliver those services which when aggregated improve in quality and price.
Local Education Authorities can use their scale to negotiate the best prices for content, communication, support services etc. Many of the services requiring the most maintenance and management – e.g. learning services, system management, business intelligence, and administrative tasks such as payroll and HR, are delivered more cost effectively from a centralised point. Other benefits include the use of greater amounts of data for decision making – an LEA with data from many schools can perceive more patterns than a single school with its limited pool of data.
Many LEA services are delivered through data centres built on top of optimised infrastructures. Increasingly data centres will become Private Clouds – essentially Infrastructure as a Service (IaaS) within the data centre. The large scale and pay-as-you-go economics of Public Clouds aren’t available in typical Private Clouds. However, Private Clouds offer at least some of the scalability and elasticity benefits of Public Cloud but with additional control and customisation. Increasingly many of these services will be also be delivered from Public Clouds.
The services delivered by the LEA can be split into two main categories:
Schooling Enterprise Services
Monitoring, Analysis and Planning
Student Relationship Management
Ministry of Education
Some of the Schooling Enterprise Services delivered by LEAs to their schools and communities could be provided at National level from the Central Ministry of Education. Services such as strategy, policy, budgets, and curriculum are usually set and delegated at national level.
Computing functions at Ministry of Education level can be grouped into three main categories:
Internal departments – Curriculum, Policy, Research etc.
Regional Services – Resources and BI
National Services – Content (information services) and infrastructure – e.g. national level schooling enterprise internet backbone
One of the most important functions at Ministry level is to have a “clear line of sight” of the performance of the schooling system. This enables BI analysis and for resources to then be focussed on the areas where they will have most impact.
Fitts and Aziza (Joey Fitts and Bruno Aziza, 2008) talk of a “line of sight” from strategic to operational to tactical decisions as the discipline that drives aligned execution. “Line of sight” means clear visibility of goals, and progress towards them at executive (strategic), management (operational), employees (tactical) levels.
“Clear line of sight” is about performance metric alignment across organisational layers. This can be thought of as an organisation chart for performance metrics, indicating how the various levels of the organisation’s performance metrics relate to one another. At school level, classroom teacher’s metrics roll up to their Head of Department, which in turn roll up to Deputy Principals, which in turn roll up to the Principal. In turn, and depending on the mode of operations, performance metrics for Principals should roll up to those of Local Authority Directors, which in turn finally roll up to the Ministry of Education.
Technology Building Blocks
Finally, pulling these building blocks together we get the following high level architecture:
Moving from the Enhanced Phase to the Strategic Phase is as much about management as ICT. In this phase, the technology is used a tool for getting better allignement between strategy set at MoE level to exectution at school level. At all levels, there are strategic decisions that ICT can help monitor, analyse, plan and execute.
In the next article in this series, we will explore the final phase – Transformation.
Data is fundamental to operating schooling systems. Without data schooling systems would grind to a halt – teachers wouldn’t get paid; students wouldn’t get transported; taught and fed; and essential services would cease to operate.
As the value of good data for decision making is becoming more widely understood, the quantity of data in the world’s schooling systems is ballooning. But how much data are we talking about, how fast is it growing, and how can it be better managed.
To get a sense of how big the issue is, let’s start by looking at Charlotte Mecklenburg in the US – a School District that has paid a lot of attention to its data and information systems recently. According to David Fitzgerald, Vice President of the Education Group at Mariner, Charlotte Mecklenburg School District in the US plans to use 70 Terabytes for a system with 140,000 students – 524.3MB per student.
The US and Western Europe account for ~10% of the world’s school students population – 0.12Bn. So, assuming similar levels of consumption across these regions, we can estimate that in these areas alone there is 60,000TB of data in schooling systems. 1TB = 50k trees worth of paper and print, so we’re looking at 3bn trees worth of data. Imagine that every student on the planet used the same amount of data as Charlotte Mecklenburg – that would add up to 30bn trees.
Whilst it’s currently unlikely that the amount of data in schooling systems adds up to this amount yet, there are several factors pushing it hard in this direction.
For example, major countries such as Russia, Mexico and Brazil are developing and running massive student data operations, increasing both the quantities and sophistication of data used.
UNESCO (2003) state that most countries develop education databases, and they also specify the optimal datasets that should be maintained. Let’s suppose that this adds up to a minimum of 1/2 a typewritten page on each of the student population living outside the USA and Western Europe, roughly 1 Kilobytes each. Rounding-off, we can estimate that 1bn students x 1Kb = 954GB. It’s interesting to think that this could be kept on a single external hard drive no bigger than a paperback book. However, add other data, say a single low-resolution image per student, and that rises by a factor of 8. Add digital work produced by students and this number grows exponentially.
Also, there is a sharp increase in the rate at which data is used in developed countries. Take New South Wales for example. Last year, New South Wales Department of Education and Training – which has 1.3m students – used 280TB of storage space – but this has been doubling every year for last five years!
The amount of data used in schooling can only increase as governments around the world recognise that it is core to improving effectiveness.
WHY IS MANAGING DATA CORE TO IMPROVING SCHOOLING EFFECTIVENESS?
Driven by the need for better accountability for how public funds are spent, and the widespread use of international benchmarks such as PISA, there is a sharp increase in the number of governments and private companies that are investing in solutions for data driven decision making. These investments aim to use data to:
Improve student performance: Give students, parents, teachers and administrators a clear picture of student performance at an individual or group level so they can adjust and personalise learning accordingly
Make better management decisions: Inform routine decisions and strategic planning across all enablers and disciplines with accurate, readily-available data
Increase accountability: Quickly and easily understand performance across organisations
Manage resources more effectively: Gain a better understanding of projected revenues and expenditures; keep track of financial health; compare costs against those of other organisations
Drive administrative efficiencies: Improve time and effort taken to report information. Improve quality and presentation of information.
SO WE HAVE TO TALK ABOUT DATABASES THEN?
Why is it that peoples’ eyes glaze over when you start talking about databases? Most web pages that you will experience – including this one – are driven by databases. For most people databases are “black boxes”, and few care about how they work or what they do. However, a basic understanding of databases and how they work is essential to understanding how ICT can make schooling more effective – so let’s take a quick database 101:
WHAT IS A DATABASE?
Databases arrange data as sets of records, and these records are arranged as rows. Each record consists of several fields which are arranged in columns. The rows and columns combine to form a table.
Most large scale databases are Relational, which means that they can connect data from two or more tables.
Forms are a main way to enter data into a database
Queries are used to get data out of a database.
Reports format and display data from the database.
Indexes improve the speed of data retrieval operations by querying a unique key which in turn uniquely identifies each row in a table. Metadata – data about data – can include tables of all tables, their names, sizes and number of rows in each table; or tables of columns, what tables they are used in, and the type of data stored in each column.
At the heart of a database is the Database Engine – software for storing, processing and securing data; providing controlled access and processing capabilities. The structure of the database is described in a Schema, and this is usually written in a language called “Structured Query Language” SQL. This language determines how data is inserted, queried, updated and deleted. Different database vendors have different extension to SQL – T-SQL is Microsoft’s extension to SQL.
A Data Warehouse is a database that extracts data from operational systems for reporting. It can aggregate data from different sources, and ensure that the integrity of operational data isn’t compromised by the processes associated with analysing it.
Integration Services are the means by which data from various sources can be integrated, extracted, transformed, and loaded into data warehouses.
OLAP – or Online Analytical Processing – enables data to be manipulated and analysed from multiple perspectives. Eg a Longitudinal analysis could involve the study of student progress over time, and take advantage of an OLAP Cube to interrogate a number of different dimensions over a given period.
Analysis Services supports OLAP by allowing the design, creation, and management of multidimensional structures that contain data aggregated from a range of data sources, such as relational databases.
Data Mining – is about extracting patterns from large sets of data, to yield Business Intelligence (BI) for example, high achievement correlated with the number of books in the family home, or low reading ability impacting examination results. Data Mining Services enables the design, creation, and visualisation of data mining models.
Reporting Services – enabling reports to be published in various formats drawing on content from a variety of data sources. They also centrally manage security and subscriptions. Portal Integration – it’s crucial to for end-users to work with operational data – in ‘dashboard’ format ideally – through a portal site.
To be able to manage databases is crucial and several key tools are used for this. Master Data Services is the means by which all applications across the organization can rely on a central, accurate source of information. Replication – copying and distributing data and database objects from one database to another, and synchronizing between databases to maintain consistency. Automated compression and backup are also key tools.
WHAT HAS THIS GOT TO DO WITH THE CLOUD?
With massive growth in the amount of data used in schooling comes questions about sustainability, cost and management. The Cloud offers some major advantages here:
Having data in the cloud makes it easier for authorized users with internet access to access that data from almost anywhere.
In an enterprise architecture where resources are distributed, organisations usually have a single SQL Server back-end with WAN links and/or multiple distributed SQL Server installations that replicate data with each other. Maintaining this kind of environment is time consuming and expensive. With the cloud, replication, backup, compression etc are all taken care of.
As with other Cloud services, you only pay for what you use. During the peaks and troughs of schooling system operations, one can expect to see varying amounts of data storage requirements.
SQL Azure is Microsoft’s Cloud Database solution, and it offers the following benefits:
No physical administration required – software installation and patching is included, as SQL Azure is a platform as a service (PAAS)
High availability and fault tolerance are built in
Simple provisioning and deployment of multiple databases
Scale databases up or down based on business needs
Multitenant – i.e. a single database can provide services to multiple organisations
Integration with SQL Server and tooling including Visual Studio®
Support for T-SQL-based familiar relational database model
Option for pay-as-you-go pricing
The SQL Azure suit currently comprises of the following offerings, some currently on limited availability:
SQL Azure Database – a Platform as a Service (PaaS) relational database. Highly available and scalable .
SQL Azure Data Sync – allows organisations to extend their current sets of data into the Cloud. It provides synchronisation between an organisation’s current SQL on-premises databases and SQL Azure Databases in the Cloud. Currently available in Community Technology Preview.
SQL Azure Reporting – a complete reporting infrastructure that enables users to see reports with visualizations such as maps, charts, gauges, sparklines etc. Currently available in Community Technology Preview.
The Windows Azure Platform Appliance under limited trials, this will eventually enable organisations to deploy their own Cloud Services from within their own datacentres. The Windows Azure Platform Appliance consists of Windows Azure, SQL Azure and a Microsoft-specified configuration of network, storage and server hardware.
TAKING ADVANTAGE OF CLOUD DATABASE SERVICES
Taking full advantage of the Cloud is not something that is going to happen overnight. Besides careful analysis and planning for migrating existing services, Cloud computing opens up a whole set of questions around what new services could be offered. For example, the rise of virtual schooling across the world – as brilliantly analyzed in the US by Clayton Christensen in his book “Disrupting Class” – will be a major beneficiary of cheap, ubiquitous database services at massive scale.
As pointed out in the Cloud Watching #1, moving to the Cloud is not without effort and risk. David Chappell, in his excellent paper “The Benefits and Risks of Cloud Platforms: A Guide for Business Leaders“ points out that storing data outside their organization makes people nervous. Many countries have regulations about where certain kinds of data can and can’t be stored, so before putting data into the Cloud platform, it’s important to ensure compliance.
A key question is to ask whether any given data centre is more secure than those of the major Cloud service providers. A significant data breach for a Cloud services provider is likely to mean a huge financial loss, so there’s a very strong incentive for them to keep the data they hold secure.
David Chappell also advises – “as with any new technology, starting small can be a good approach. Perhaps your first cloud application should be important, for instance, but not truly mission critical”. The same can be said for data.
Whilst its early days for Cloud based database services in Education, we’re beginning to see interest turning to into plans and action. For example, Curtin University in Perth, Australia, has started to move some of its services to the Cloud and intend to take advantage of SQL Azure.
“The abundance of resources and relationships made easily accessible via the Internet is increasingly challenging us to revisit our roles as educators in sense-making, coaching, and credentialing”.
Cloud will no doubt change how data is gathered, manipulated and interrogated, and by making vast amounts of storage available at extremely low prices we can look forward to seeing innovative organisations build completely new services to reach growing numbers of learners in completely new ways.