РАЗРАБОТКА СИСТЕМ ПО РАСПОЗНАВАНИЮ ПСИХОФИЗИОЛОГИЧЕСКОГО СОСТОЯНИЯ ЧЕЛОВЕКА В РЕЖИМЕ РЕАЛЬНОГО ВРЕМЕНИ В РАМКАХ РАЗВИТИЯ ИНФОРМАЦИОННЫХ ЗДОРОВЬЕСБЕРЕГАЮЩИХ ТЕХНОЛОГИЙ

The acceleration of the rhythm of life and the exponential increase in information noise causes severe stress loads acting on a modern person. To cope with so much information coming from outside, the human brain is increasingly forced to resort to methods of saving internal resources: artificially put barriers to incoming information, resist learning, automate responses to stimuli. This, in turn, provokes people to unconscious actions, causes apathy, leads to depression and interferes with the development of the student. In this regard, the problem of the development of the emotional intelligence of an individual is acute. Increasingly, a person wonders: "How to maintain your health and the health of others in the conditions of accumulated stress?" The technology of automated recognition of a person's psychophysiological state will help to cope with the problem, as well as better understand people and their behavior.

Emotions are a person's psychophysiological or mental responses to objects and events of the world around them. Emotions affect various aspects of human life, including attention, learning, development, goal achievement, communication, mood, and motivation

The ability to manage and cope with emotions can be an incentive for human development and a source of energy. On the contrary, inability to recognize emotions and control them can cause long-term stress, as well as have a negative impact on human life and health, be the cause of cancer

With the development of technologies and wearable gadgets (smartwatches and other portable devices), with the improvement of sensors built into them or connected to them to record changes in the electroencephalogram (EEG), electrocardiogram (ECG), photopletismogram and body temperature, the most convenient way to recognize emotions in real time is to analyze biological signals from the human body

The purpose of the study is to create an algorithm for developing a system for automated recognition of a person's psychophysiological state in real time by analyzing biological signals. This algorithm is supposed to be tested by conducting a number of experimental studies on volunteers.

Over the past decade, many research and development works have been produced to create new approaches and methods for recognizing emotions. Choosing an approach depends on how researchers analyze and classify emotions. Of the many approaches, two main ones can be distinguished: discrete and constructivist

The discrete approach assumes the presence of stable patterns of emotions in humans as an object of study. According to this approach, there is a certain neural network structure of the brain and a corresponding pattern in biological signals for each emotion.

The constructivist approach assumes that emotions are constructed in the object of study depending on behavioral or value models. This approach involves several measurements of EEG or ECG parameters in order to create an effective basis for studying and classifying emotions in each individual subject. The most indicative model of this approach is the descriptive model (VAD – "Valence-Arousal-Dominance")

Currently, EEG is a reliable and easy to implement technology used to measure brain activity, and ECG is a technology that shows the effect of brain activity on the functional response of the rest of the body. Emotion detection using EEG and ECG signals is carried out in several stages, performed sequentially. Traditionally, these steps include removing interference from signals, extracting temporal or spectral features from oscillograms or signal spectrograms, and then developing a multi-class classification strategy.

Traditional methods for recognizing emotional responses involve extracting emotion traits from a fixed group of the same EEG and ECG channels for all study subjects. However, in some studies, it has been recorded that with the same emotional response, different brain regions can be activated in different study objects, that is, the emotional response can change from person to person and from one emotional state to another

The algorithm developed by the authors of this article for recognizing the psychophysiological state of a person in real time includes the following steps:

1. Measurement of emotional and physiological responses by EEG and ECG in several subjects under different conditions;

2. Use of fast Fourier transform methods

3. Rationale and choice of methods for analyzing EEG and ECG signals demonstrating significant changes in brain activity during emotional states;

4. Highlighting standard patterns for each emotional response.

The implementation of the first stage of the algorithm depends on the used technical means for fixing signals, as well as on the method of identifying the emotional state of the individual. This stage is currently under development and testing.

Pilot experiments were carried out with the participation of a group of conditionally healthy students in the amount of 55 people using a device for fixing photoplethysmograms MAX30102 I2C

To demonstrate the 2 and 3 stages of the software implementation of the ECG signal processing and decryption algorithm using the fast Fourier transform method with the created Python program, the heart beat sound from the Internet in mp3 format was taken, which was then converted into a wav file. Figure 1 shows the oscillogram of the ECG signal read from the wav format, which shows the dependence of its amplitude in conditional dimensionless units of time in seconds.

Figure 1 - Electrocardiogram read from wav format

DOI:10.23670/IRJ.2023.133.40.1

Figure 2 shows the output of the ECG signal spectrogram using the commands of the function library Matplotlib.pyplot obtained by the fast Fourier transform method. The amplitude of the spectral components of the signal is expressed in conventional dimensionless units.

Figure 2 - The result of obtaining an electrocardiogram spectrum

DOI:10.23670/IRJ.2023.133.40.2

The obtained ECG signal spectrogram shows a significant increase in amplitude at frequencies of 22 Hz, 44 Hz, 66 Hz, 88 Hz and 110 Hz. However, the amplitude of frequency components with a frequency greater than 110 Hz in the ECG spectrum is small. Thus, as a result of analysis by the fast Fourier transform method, it was possible to identify 5 basic multiples of each other harmonics.

When processing ECG or EEG signals using the selected method in real time, the frequencies and amplitudes of harmonics will differ depending on the psychophysiological states of the individual. By calculating the number of harmonics, their frequencies and amplitude ratios, it is planned to find a pattern of significant changes in the biological activity of the nervous system during various emotional states, as well as, perhaps, to identify deviations from the normal state for use in the computer program under development, which should be used as an assistant for diagnosing various diseases.

We consider the use of artificial neural networks (ANN) as an adaptive method for solving this problem to be scientifically justified due to the obvious change in the frequency and amplitude characteristics of the individual's cardiogram obtained in real time

1. Prepare a sample of data for training an artificial neural network and receive synchronized ECG signals

2. Create a software package for displaying signals and calculating parameters that will subsequently be used to create a neural network;

3. Create a neural network to obtain emotion values according to the VAD model – "Valence-Arousal-Dominance";

4. Analyze the results obtained by the artificial neural network;

5. Recognize the psycho-emotional state of a person by the results obtained.

In order to identify various anomalies of the psycho-emotional state of a person, as well as collect statistical data, it is planned to continuously monitor the state of an individual using wearable gadgets.

The results of the study can be used for medical purposes as an assistant to make a correct diagnosis to the patient. At the same time, the algorithms tested by the experiment will serve as the basis for creating a mobile system for automatic assessment and monitoring of the psycho-emotional state of an individual. This system will consist of a body sensor for recording the state and a program with an algorithm for processing continuous measurement results for a smartphone. As a test of the use of software for the first stage of research, we have developed a layout of an application for a smartphone through which the user could view the measurement results of biological signals and, responding to them, adjust his emotional state. In this case, a photoplethysmograph built into a smart watch is used as a heart rate sensor. It is also planned to use a wearable ECG analyzer "stitched" by the software created by the authors.