Automatic Diagnostics with Intermediate Steps in Mathematics Education

Project details:

Start: September 1, 2016

End: August 31, 2019

Duration: 36 months

Funding: € 442.895

Programme: Erasmus+

Action: Strategic Partnerships



  • Open University of the Netherlands (coordinator)
  • Utrecht University
  • Cito
  • Saarland University
  • Université Paris-Est Créteil

Contact information:

Project summary

Competence in mathematics has been identified at EU level as one of the key competences for personal fulfilment, active citizenship, social inclusion, and employability in the knowledge society of the 21st century. Concerns about low student performance, as revealed by international surveys, led to the adoption in 2009 of an EU-wide benchmark in basic skills, which states that by 2020 the share of 15-year-olds with insufficient abilities in reading, mathematics and science should be less than 15%. Low achievement in mathematics is a common concern for all European countries. It is an issue associated not only with the effectiveness of teaching and learning, but also with providing an equitable system of education.

An extensive review of research evidence on what works for children with mathematical difficulties has concluded that interventions should ideally be targeted towards an individual child's particular difficulties. Individual support has proven to have significant impact on children's performance, and reporting progress on achievement is important for improving mathematics skills. ICT can be of crucial help here. Digital learning environments used for stepwise solving homework are just as effective as individual tutors, and have the added advantage of giving teachers information about progress of individual students.

The objective of this project is to develop flexible support for detailed diagnostics of mathematical competences of pupils, and to use this in four existing digital testing and practicing environments in mathematics education. The diagnostics summarise the knowledge, skills, and competences of a pupil, and can be used by teachers to identify underachievers. With this information, teachers can direct their attention to help pupils overcome particular difficulties. Pupils can use diagnostic reports to get an accurate understanding of their own performance and progress. Also the system itself can use the diagnostic information for adapting the environment to the level of an individual.

Most digital testing and practicing tools in mathematics education review only a final answer, and do not look at the steps taken to reach the final answer. However, intermediate steps provide essential information for a detailed assessment of student skills. Assessing intermediate steps also corresponds to the pen and paper practice, and is a wish from the mathematics education sector. This project studies the automatic assessment of intermediate steps in mathematics education for domains such as ‘Numbers’ and ‘Relationships’. We develop general feedback and assessment services that can be used by existing digital learning environments. These services give accurate diagnoses of intermediate steps and use these to determine mathematical knowledge, skills, and competences.

The expected results of the project are threefold. (1) We develop innovative technology for calculating detailed diagnostics in mathematics education, offered as an open, reusable set of feedback and assessment services. The diagnostic information is calculated automatically based on the analysis of intermediate steps. (2) The detailed diagnostics are fully integrated into three fullblown, advanced digital environments for practicing mathematics: the DME, Math-Bridge, and Pepite. Also, the Dutch test player Facet, which is developed for the national intermediary diagnostic test for all secondary schools, will be connected to the services in a proof of concept implementation. (3) We design and execute pilot and evaluation studies for assessing the effectiveness of the detailed diagnostics offered by the services in the three environments. The studies involve approximately 375 pupils and 25 teachers at different schools in different countries.

The desired impact is to improve math education and the state of the art in digital testing and practicing environments. The detailed diagnostics let pupils learn mathematics more effectively. The diagnostics support teachers in helping their pupils efficiently, identifying underachieving pupils, and providing individual support where this is most needed. This project can accelerate the introduction of digital tools in mathematics education, and start off a process in which more digital environments for testing, practicing, and learning mathematics make use of a shared set of (feedback and assessment) services. The collaborative design and development of such a shared set of high-quality services, and the innovation that such a collaboration brings, can help to tackle persistent problems in math education across Europe.

The project’s longer term benefits can be substantial because of the already established professional networks of the partners. In particular, we will involve policy-makers, national testing agencies, publishing companies of math textbooks (e.g. a textbook that uses the DME), developers of other learning environments, and associations for mathematics teachers (e.g. Sesamath).


Feel free to contact the project coordinator, dr. Bastiaan Heeren, about the Advise-Me project.

Project kick-off meeting

Project kick-off meeting

September 30, 2016

Heerlen, the Netherlands