Logo image
Open Research University homepage
Surrey researchers Sign in
SRD: A Tree Structure Based Decoder for Online Handwritten Mathematical Expression Recognition
Journal article   Peer reviewed

SRD: A Tree Structure Based Decoder for Online Handwritten Mathematical Expression Recognition

Jianshu Zhang, Jun Du, Yongxin Yang, Yi-Zhe Song and Lirong Dai
IEEE transactions on multimedia, Vol.23, pp.2471-2480
2021

Abstract

attention decoder Decoding Grammar Handwriting recognition Handwritten mathematical expression recogni- tion Image recognition Mathematical model Task analysis tree structure Vegetation
Recently, recognition of online handwritten mathe- matical expression has been greatly improved by employing encoder-decoder based methods. Existing encoder-decoder models use string decoders to generate LaTeX strings for mathematical expression recognition. However, in this paper, we importantly argue that string representations might not be the most natural for mathematical expressions - mathematical expressions are inherently tree structures other than flat strings. For this purpose, we propose a novel sequential relation decoder (SRD) that aims to decode expressions into tree structures for online handwritten mathematical expression recognition. At each step of tree construction, a sub-tree structure composed of a relation node and two symbol nodes is computed based on previous sub-tree structures. This is the first work that builds a tree structure based decoder for encoder-decoder based mathematical expression recognition. Compared with string decoders, a decoder that better understands tree structures is crucial for mathematical expression recognition as it brings a more reasonable learning objective and improves overall generalization ability. We demonstrate how the proposed SRD outperforms state-of-the-art string decoders through a set of experiments on CROHME database, which is currently the largest benchmark for online handwritten mathematical expression recognition.

Metrics

Details

Logo image

Usage Policy