Efficacy of Deep Neural Embeddings based Semantic Similarity in Automatic Essay Evaluation

Abstract

Semantic similarity is used extensively for understanding the context and meaning of the text data. In this paper, use of the semantic similarity in an automatic essay evaluation system is proposed. In this article, different text embedding methods are used to compute the semantic similarity. Deep neural emebeddings have been extensively used in the different natural language processing tasks such as, general language understanding, question answering, next word/sentence prediction, language translation, word sense disambiguation and many more. Recent neural embedding methods including Google Sentence Encoder(GSE), Embeddings for Language Models(ELMo) and Global Vectors(GloVe) are employed for computing the semantic similarity. Traditional methods of textual data representation such as TF-IDF and Jaccard index are also used in finding the semantic similarity. Experimental analysis of an intra-class and inter-class semantic similarity score distributions shows that the GSE outperforms other methods by accurately distinguishing essays from the same or different set. Semantic similarity calculated using the GSE method is further used for finding the correlation with human rated essay scores. Correlation of semantic similarity scores with different essay specific traits given in the ASAP++ dataset provided by the Center for Indian Language Technology (CFILT), IIT Bombay, is also performed in this article.