DEVELOPMENT OF SYSTEMS FOR REAL-TIME RECOGNITION OF HUMAN PSYCHOPHYSIOLOGICAL STATE AS PART OF THE GROWTH OF INFORMATION HEALTH-PRESERVING TECHNOLOGIES
DEVELOPMENT OF SYSTEMS FOR REAL-TIME RECOGNITION OF HUMAN PSYCHOPHYSIOLOGICAL STATE AS PART OF THE GROWTH OF INFORMATION HEALTH-PRESERVING TECHNOLOGIES
Abstract
The ability to manage emotions, notice them in time, and also cope with them can serve as an incentive for human development. On the contrary, the inability to recognize emotions and control them can cause long-term stress, as well as have a negative impact on human life and health, and cause cancer.
The purpose of the work is to create an algorithm for developing a system for automated recognition of the psychophysiological state of a person in real time by analyzing biological signals (electroencephalogram (EEG), electrocardiogram (ECG), photopletismogram and body temperature) from the side of physiological human health. In the course of the study, it is assumed to measure the emotional and physiological responses of an individual using wearable devices in various conditions, highlight significant changes in biological signals during emotional states, recognize standard patterns for each emotional response, use a fast Fourier transform to extract spectral information from signals, as well as artificial neural networks (ANN) as an adaptive method for solving the problem.
The developed technology will help to cope with the problem of "emotional burnout", better understand people and their behavior, and develop intelligence. The results of the study can be used for medical purposes as an assistant for making a correct diagnosis to the patient, and the algorithms tested by the experiment will serve as the basis for creating a mobile system for automatic assessment and monitoring of the psycho-emotional state of the individual.
1. Introduction
The acceleration of the rhythm of life and the exponential increase in information noise causes severe stress loads acting on a modern person. To cope with so much information coming from outside, the human brain is increasingly forced to resort to methods of saving internal resources: artificially put barriers to incoming information, resist learning, automate responses to stimuli. This, in turn, provokes people to unconscious actions, causes apathy, leads to depression and interferes with the development of the student. In this regard, the problem of the development of the emotional intelligence of an individual is acute. Increasingly, a person wonders: "How to maintain your health and the health of others in the conditions of accumulated stress?" The technology of automated recognition of a person's psychophysiological state will help to cope with the problem, as well as better understand people and their behavior.
Emotions are a person's psychophysiological or mental responses to objects and events of the world around them. Emotions affect various aspects of human life, including attention, learning, development, goal achievement, communication, mood, and motivation
.The ability to manage and cope with emotions can be an incentive for human development and a source of energy. On the contrary, inability to recognize emotions and control them can cause long-term stress, as well as have a negative impact on human life and health, be the cause of cancer
. According to experts, emotions themselves are neutral. Our reaction gives them color. It depends on it how the emotion will affect our well-being and health .With the development of technologies and wearable gadgets (smartwatches and other portable devices), with the improvement of sensors built into them or connected to them to record changes in the electroencephalogram (EEG), electrocardiogram (ECG), photopletismogram and body temperature, the most convenient way to recognize emotions in real time is to analyze biological signals from the human body
.The purpose of the study is to create an algorithm for developing a system for automated recognition of a person's psychophysiological state in real time by analyzing biological signals. This algorithm is supposed to be tested by conducting a number of experimental studies on volunteers.
2. Research methods and principles
Over the past decade, many research and development works have been produced to create new approaches and methods for recognizing emotions. Choosing an approach depends on how researchers analyze and classify emotions. Of the many approaches, two main ones can be distinguished: discrete and constructivist
.The discrete approach assumes the presence of stable patterns of emotions in humans as an object of study. According to this approach, there is a certain neural network structure of the brain and a corresponding pattern in biological signals for each emotion.
The constructivist approach assumes that emotions are constructed in the object of study depending on behavioral or value models. This approach involves several measurements of EEG or ECG parameters in order to create an effective basis for studying and classifying emotions in each individual subject. The most indicative model of this approach is the descriptive model (VAD – "Valence-Arousal-Dominance")
.Currently, EEG is a reliable and easy to implement technology used to measure brain activity, and ECG is a technology that shows the effect of brain activity on the functional response of the rest of the body. Emotion detection using EEG and ECG signals is carried out in several stages, performed sequentially. Traditionally, these steps include removing interference from signals, extracting temporal or spectral features from oscillograms or signal spectrograms, and then developing a multi-class classification strategy.
Traditional methods for recognizing emotional responses involve extracting emotion traits from a fixed group of the same EEG and ECG channels for all study subjects. However, in some studies, it has been recorded that with the same emotional response, different brain regions can be activated in different study objects, that is, the emotional response can change from person to person and from one emotional state to another
.The algorithm developed by the authors of this article for recognizing the psychophysiological state of a person in real time includes the following steps:
1. Measurement of emotional and physiological responses by EEG and ECG in several subjects under different conditions;
2. Use of fast Fourier transform methods
, and wavelet analysis to extract spectral information from EEG and ECG signals;3. Rationale and choice of methods for analyzing EEG and ECG signals demonstrating significant changes in brain activity during emotional states;
4. Highlighting standard patterns for each emotional response.
3. Main results
The implementation of the first stage of the algorithm depends on the used technical means for fixing signals, as well as on the method of identifying the emotional state of the individual. This stage is currently under development and testing.
Pilot experiments were carried out with the participation of a group of conditionally healthy students in the amount of 55 people using a device for fixing photoplethysmograms MAX30102 I2C
, which showed that the signals of the photoplethysmograph cannot help solve the tasks set in the work (since such experiments can only find out the pulse rate of the subjects). Funds are being sought for the purchase of equipment for taking digital readings of the ECG and EEG, the analysis of which, as will be shown below, allows you to diagnose the emotional state of the individual. It is planned to make a variant search for the best option for taking readings (single or combined use of ECG and EEG devices).To demonstrate the 2 and 3 stages of the software implementation of the ECG signal processing and decryption algorithm using the fast Fourier transform method with the created Python program, the heart beat sound from the Internet in mp3 format was taken, which was then converted into a wav file. Figure 1 shows the oscillogram of the ECG signal read from the wav format, which shows the dependence of its amplitude in conditional dimensionless units of time in seconds.
Figure 1 - Electrocardiogram read from wav format
Figure 2 - The result of obtaining an electrocardiogram spectrum
4. Discussion
When processing ECG or EEG signals using the selected method in real time, the frequencies and amplitudes of harmonics will differ depending on the psychophysiological states of the individual. By calculating the number of harmonics, their frequencies and amplitude ratios, it is planned to find a pattern of significant changes in the biological activity of the nervous system during various emotional states, as well as, perhaps, to identify deviations from the normal state for use in the computer program under development, which should be used as an assistant for diagnosing various diseases.
We consider the use of artificial neural networks (ANN) as an adaptive method for solving this problem to be scientifically justified due to the obvious change in the frequency and amplitude characteristics of the individual's cardiogram obtained in real time
. We will use the harmonics and frequencies obtained during spectral analysis as the initial network data. In this case, to use ANN, work out the following steps:1. Prepare a sample of data for training an artificial neural network and receive synchronized ECG signals
;2. Create a software package for displaying signals and calculating parameters that will subsequently be used to create a neural network;
3. Create a neural network to obtain emotion values according to the VAD model – "Valence-Arousal-Dominance";
4. Analyze the results obtained by the artificial neural network;
5. Recognize the psycho-emotional state of a person by the results obtained.
In order to identify various anomalies of the psycho-emotional state of a person, as well as collect statistical data, it is planned to continuously monitor the state of an individual using wearable gadgets.
5. Conclusion
The results of the study can be used for medical purposes as an assistant to make a correct diagnosis to the patient. At the same time, the algorithms tested by the experiment will serve as the basis for creating a mobile system for automatic assessment and monitoring of the psycho-emotional state of an individual. This system will consist of a body sensor for recording the state and a program with an algorithm for processing continuous measurement results for a smartphone. As a test of the use of software for the first stage of research, we have developed a layout of an application for a smartphone through which the user could view the measurement results of biological signals and, responding to them, adjust his emotional state. In this case, a photoplethysmograph built into a smart watch is used as a heart rate sensor. It is also planned to use a wearable ECG analyzer "stitched" by the software created by the authors.