Notate: A Pen-Based Interface that Bridges Handwritten and Textual Programing

Sketches and handwriting can now be turned into code using a new programming tool
Cornell University researchers created an interface which allows users to sketch and handwrite within computer code, a challenge to traditional coding that relies on typing.

Notate is a pen-based interface that allows users to open drawing canvases in their digital notebooks and write diagrams inside lines of traditional computer code.

The interface is powered by a deep-learning model that bridges the gap between handwritten and textual contexts. Notation in a handwritten diagram may refer to textual code, and vice versa. Notate, for example, recognizes handwritten symbols like \”n\” and links them to their typewritten counterparts.