Dysgraphia, a writing disorder, hinders many children in their academic learning.
To address this issue, it is essential to detect it reliably and on a large scale. Research on writing acquisition has led to the development of digital pens that integrate motion micro-sensors similar to those found in smartphones. These sensors collect a vast amount of rich and complex data, requiring artificial intelligence (AI) for analysis. These pens represent a new form of Human-Machine Interface.
Writing: A Complex and Difficult Skill to Acquire
For most of us, writing is the most complex skill we are capable of mastering. However, it requires a tedious learning process over several years to become a proficient writer. During this learning period, some children exhibit a writing disorder known as dysgraphia. Their written production is challenging, or even impossible, as severe forms of this disorder can be highly debilitating. The website monparcourshandicap.gouv, in its publication “Dys Disorders: What School Accommodations?” highlights the extent of the issue. It is estimated that dysgraphia affects approximately 5% to 10% of a given age group.
Detecting Writing Learning Disorders: The BHK Test
The BHK test, or Brave Handwriting Kinder scale, was created in 1987 by Dutch researchers Hamstra-Bletz, De Bie, and Den Brinker.
Although reliable, this test is difficult to implement on a large scale in school settings. While the actual test takes about five minutes per child, organizing and administering it to an entire class becomes a significant constraint. Additionally, the results must be processed by a qualified healthcare professional. Analyzing handwritten production is a tedious and time-consuming task. Consequently, without digital tools, research aiming to estimate the prevalence of dysgraphia remains challenging to conduct on large cohorts.
Digital Tablets
For several years, researchers have been using digital tablets. These tablets, which allow writing with a stylus, have made it possible to record handwriting data that is then processed using AI and numerous test recordings. A key reference is the research conducted at the École Polytechnique Fédérale de Lausanne, including Thibault Asselborn’s dissertation (2020) Analysis and Remediation of Handwriting Difficulties, and at the Parisian hospital Pitié-Salpêtrière with child psychiatrist Thomas Gargot’s dissertation (2021) Algorithms and Robotics: Understanding How We Learn Handwriting and Helping Children with Writing Difficulties.
Digital Pens
Today, digital pens offer a new generation of tools. Several versions have emerged in the past five years, including:
A Low-Cost Grip Pen Sensing Tool to Detect Handwriting Disorders
Ana Phelippeau, Wim Poignon, Adrien Husson, Clément Duhart, Marc Teyssier, Joël Chevrier
CHI 2023: The ACM CHI Conference on Human Factors in Computing Systems
Session: Late Breaking Work (LBW)
- A pen developed by Stabilo in collaboration with the universities of Munich and Erlangen, presented in 2021 in a paper titled Towards Online Handwriting Recognition Using an IMU-Equipped Pen.
- The SensoGrip pen from the University of Vienna, featured in a 2022 paper: Participatory Design and Needs Assessment for a Pressure-Sensitive Pen and Mobile App (SensoGrip) for Children with Writing Difficulties.
- Ana Phelippeau’s dissertation at Université Paris-Cité (2024): Sensitive Pen: A Digital Tool for Characterizing Graphomotor Gestures.
According to the SensoGrip website:
“As part of the interdisciplinary research project ‘SensoGrip,’ a pen for therapy of children with graphomotor weakness has been developed in collaboration with therapy centers and schools.”
The structure and technology of these pens are quite similar. In all three cases, data processing relies on AI strategies already implemented in tablet-based research.
Smartphones and Digital Pens: The Same Technologies
The development of these pens has greatly benefited from the availability of micro-sensors and digital technologies found in smartphones. These devices incorporate the same motion sensors within an IMU (Inertial Measurement Unit), consisting of an accelerometer, a gyroscope, and a magnetometer, as well as touch surfaces that detect finger position and pressure. Ultimately, studying pen movements and finger positions on the pen involves adapting smartphone sensors and technology into a pen, removing the screen and keyboard. This brings us back to the traditional pen and notebook in the classroom.
Psychomotricity and Design Shape the Digital Pen
Psychomotricity can now influence the digital pen, ensuring it remains, above all, a pen, defining its functions and performance. For children in particular, this digital pen must still feel like a pen, implying minimal changes in weight, size, and diameter. This is where designers come in. All the necessary technologies are readily available. The digital pen can be shaped by the requirements and constraints of psychomotricity, educators, clinicians, and child development researchers. Designers collaborate with these experts to refine the form and integration of the technology underlying these new functions.
Artificial Intelligence at the Forefront
In the French version of the BHK test (2003), researchers M. Charles, R. Soppelsa, and J.-M. Albaret identified 13 factors to analyze for detecting potential dysgraphia. Their list highlights the complexity of this manual analysis, including features such as large handwriting, rightward margin tilt, cramped words, broken letter connections, incorrect relative height, letter distortions, hesitations, and tremors. The goal of handwriting research is to automate screening using AI.
Florent Imbert’s 2024 dissertation, defended in Rennes, is titled Design of a Deep Neural Network Architecture for Handwriting Synthesis Using Kinematic Sensors from a Digital Pen.
The general method involves recording children’s handwriting in real time using a digital pen and then administering the traditional BHK dysgraphia detection test to the same children. The data acquired by the pen is then processed using a machine learning approach. The ultimate goal is to determine whether handwriting is dysgraphic without knowing the BHK test results.
A New Human-Machine Interface (HMI) and Even an AI Assistant
The pen is linked to one of the most complex, controlled, and precise movements most of us can produce. It is fundamentally independent of any digital infrastructure—it is, first and foremost, a pen. However, its integrated sensors also make it a new creative instrument based on hand and finger movements. This allows for tracking pen movement, rhythm, and amplitude and transforming this data into sound or music, similar to electric musical instruments like guitars or keyboards.
This pen could also serve as an AI assistant for teachers. One could envision continuous, individualized monitoring of handwriting learning. It could become a daily alert system, providing personalized exercises and guidance. Of course, ethical considerations and child protection measures will need to be anticipated.
Authors:
Joël Chevrier – Professor of Physics, UGA
Ana Phelippeau – Psychomotor therapist and Doctor of Cognitive Sciences
Adrien Husson – Designer
Teaching, exploring, learning with students... 