Clock Sketch

Summary

Clock Sketch is representative of a new wave of software within the medical community. With Clock Sketch, a doctor can analyze and view a true copy of the patient’s sketch. The UI allows a doctor to playback a movie of the sketch, or display analysis information overlays. Vestal designed and implemented all aspects of the application with guidance from Professor Randall Davis in the Design Rationale Group within MIT CSAIL.

Problem

In the initial stages of diagnosing people with neurological disorders, such as Alzheimer’s, doctors will ask the patient to draw a simple sketch of a clockface which conveys information about cognition, mental representations, and fine motor control.

Unfortunately, at the end of the test, all that is left to the doctor is a final drawing and observations they were able to make on the fly as the patient drew. The goal is to allow doctors to use this sketch as a more thorough test for detecting neurological disorders faster and more accurately, instead of as an initial screening procedure.

To achieve this goal, we must digitize the patient’s sketch, so a movie of the sketch can be played, analyzed, and quantitatively compared with the norm. The application receives uploaded patient data via a special pen which records a user’s strokes spatially and temporally.

Approach

Clock Sketch was designed to simultaneously interact with two different groups of users: novices which were simply conducting the patient experiment and only needed to upload sketch data, and experts who were interested in analyzing the sketches. Appropriately, the novice user experience must be tightly controlled so data is not lost. On the other hand, the expert interaction must minimize the amount of redundancy and maximize the amount of information presented.

Solution

A GUI was designed which allowed doctors to play, pause, scale, and change the speed of a patient’s sketch. After attempting to automatically classify the strokes within the drawing (as digits, hour/minute hands, basic shapes, etc), Clock Sketch then provides overlays of the analysis results. Doctors can also edit classification of a stroke through a drag and drop interface and tree hierarchy. As another example, after determining strokes compose the circle outlining the clock face, Clock Sketch calculates an ideal ellipse based on the aggregate data points in those strokes. The ellipse and its orientation is then superimposed over the original drawing. In another case, the individual data points, which are regularly sampled, can be shown, which gives doctors an approximate view of the pen speed through a stroke. Bounding boxes are displayed around recognized digits, which allows one to quickly and easily discern if the AI’s interpretation was correct. Accompanying this purely visual display, doctors can generate Excel spreadsheets of the exact numbers and statistics so they can easily compile data in studies.