One of the most interesting methods of linguistic analysis is "the analysis of the tonality of the text" or the Sentiment Analysis (SA). The essence of SA is to determine the emotional color of the text, as well as the "heaviness" of that weight.

Areas of use

From the definition, we can draw several conclusions about where, theoretically (and practically, in fact), the concept of tonality analysis of the text might find application and clarify some of its details.

Firstly, textural tonality analysis can help to understand the laws that govern natural language and teach a computer to perceive it at a level close to human beings. Until recently, the machine included texts at an abstract level - mainly through lexemes (words), which for it had a form (a set of letters) and a content (meaning). This concept proposes to introduce another function - the so-called lexical tonality of the text (in the simplest case, it will be defined as the sum of the lexical tones of each lexeme).

Secondly, tonality analysis can greatly improve the quality of machine translation. It is known that machine translation is the result of the translation of a text by a professional translator. For more than 50 years, researchers have been evolving in this field and they are convinced that it is only possible to teach a machine to "think like a translator" taking into account all the considerations that a professional uses, translating this or that text. Naturally, translation cannot do without the primary analysis of text and individual words - including the analysis of tonality as such.

Thirdly, the purpose of analyzing the tonality of the text may be a certain opinion of the author or the author himself. This is the most interesting field of application, because it shows not only a means of delegating some powers from a scientist to a machine (for example, a philologist who studies the work of one or the other author), but also an attempt to bring the thought of the computer to the man. From this point of view, tonality analysis is probably one of the most important and promising steps in the development of artificial intelligence.

Methods

Rule-based methods and dictionaries

Within these approaches, the text is analyzed on the basis of pre-compiled tonal dictionaries. However, the process of creating these "folios" takes a lot of time; the main problem is the fact that the same word, in different contexts, may have a different tonality. This means that for the system to work properly, a large number of rules must be compiled. Therefore, in most cases, text tonality analysis systems are created with reference to a specific domain.

Methods based on graph-theoretic models

In these methods, the text is represented as a graph based on the assumption that some words have more weight and, therefore, have a greater influence on the tone of the whole text. After classifying the vertices of the graph, the words are classified according to the dictionary of tonality, to which each word is assigned a specific characteristic ("positive", "negative" or "neutral"). The result is calculated as the ratio between the number of words with a positive score and the number of words with a negative score.

Methods based on machine learning - with and without teacher

Big data can be of great help in learning neural networks, which are also used for tonality analysis of text. In addition, the accuracy of the tonality assessment in this way reaches 85% - at least this figure has been reached by Stanford scientists. The principle of the program is simple: it builds a tree with an assessment of the tonality of each word, each sentence and the whole text. The most interesting thing: the program understands that changing the word order changes the tone of the text. Presumably this is what ensures such a high accuracy of text evaluation and allows us to consider neural networks as a promising tool for such an analysis.

AZNResearch uses sentiment analysis based on neural networks of IT giants such as Google or Microsoft to create statistical models of the feedback from users, business customers and commentators on social networks.