Image to LaTeX

Reading handwritten math and turning it into LaTeX

  • ClientAn ed-tech company building a math-solving app
  • TypeNew idea

The problem

Retyping handwritten homework symbol by symbol is slow and error-prone, especially on a phone. Handwriting adds another trap: the same stroke can mean different things, so an "x" can be a variable or a times sign.

What we built

  • Led the deep-learning project: a model that reads a photo of pen or pencil math and returns clean, editable LaTeX
  • Built a training set of about 1 million handwritten formula images with LaTeX labels
  • Added about 100,000 machine-printed formulas so typed math is recognised too
  • Taught the model to tell look-alike symbols apart from context: x and ×, 1, l and |, 0 and O
  • Output that renders straight away, so students can check the result and edit it

The result

Students can photograph a formula instead of typing it. The model converts handwritten formulas correctly about 90% of the time.

Screenshots

Handwriting samples are illustrative.