НОВАЯ ТЕХНОЛОГИЯ ИНФОРМАЦИОННОГО ВЗАИМОДЕЙСТВИЯ МЕЖДУ ЧЕЛОВЕКОМ И ДЕЛЬФИНОМ

DOI: https://doi.org/10.23670/IRJ.2018.75.9.018

НОВАЯ ТЕХНОЛОГИЯ ИНФОРМАЦИОННОГО ВЗАИМОДЕЙСТВИЯ МЕЖДУ ЧЕЛОВЕКОМ И ДЕЛЬФИНОМ

Научная статья

Иванов М.П.^1, *, Толмачев Ю.А.², Стефанов В.Е³.

¹ ORCID: 0000-0001-7334-1584;

² ORCID: 0000-0002-5439-6405;

³ ORCID: 0000-0002-7407-8236,

^{1, 2, 3} Санкт-Петербургский государственный университет; Санкт-Петербург, Россия

* Корреспондирующий автор (20mivanov[at]mail.ru)

Аннотация

Экспериментально показано перспективная технология с использованием видео и акустического оборудования с применением различных сценариев взаимодействия между человеком и дельфином белухой. Для демонстрации этих сценариев был разработан специальный подводный монитор. При демонстрации животному различных картинок и звукового сопровождения проявилось резкое увеличение времени удержания внимания дельфина у экрана (до 7 минут). Акустическая характеристика сигналов животного при демонстрации собственного изображения с использованием веб-камер показала, что дельфин использовал последовательности пакетов ультракоротких широкополосных импульсов различной длительности и количеством импульсов с время-импульсной модуляцией. Анализ записанных сигналов показал, что отчетливо различаются три основных временных структуры, состоящие из повторения и комбинации таких пакетов.

Ключевые слова: коммуникационные сигналы дельфина, пакеты импульсов, время-импульсная модуляции, модуляция позиции импульса.

 

NEW TECHNOLOGY OF INFORMATION INTERACTION BETWEEN MAN AND DOLPHIN

Research article

Ivanov M.P.^1, *, Tolmachev Yu.A.², Stefanov V.E.³

¹ ORCID: 0000-0001-7334-1584;

² ORCID: 0000-0002-5439-6405;

³ ORCID: 0000-0002-7407-8236,

^{1, 2, 3} Saint Petersburg State University; Saint Petersburg, Russia

* Corresponding author (20mivanov[at]mail.ru)

Abstract

We have shown experimentally the promising way to dolphin and man communication using video and acoustic equipment by the beluga dolphins reaction to various scenarios presented. For the demonstration of these scenarios, a special underwater monitor was engineered. The participated animals’ interest to different pictures and sounds revealed itself in a sharp increase of the dolphin retention time at the screen (up to 7 minutes). The animal's acoustic response to the demonstration of its own image with the use of a web camera consisted of the frequency wideband impulse packets sequences of varying lengths with internal time-pulse modulation. The analysis of the recorded signals made it possible to distinguish three main temporal structures, consisting of repetitions and permutations of such packets in time.

Keywords: dolphin’ communication signal, packages of impulses, time modulation, pulse position modulation.

 

Introduction

Modern hydroacoustic systems allow for a long-term monitoring of bioacoustic activity of naval animals. The development of monitoring systems operating in various parts of the spectrum, as well as the use of additional channels for the information transmission and reception, suggest the search for psychophysiological techniques to provoke communication between animals [1] in the conditions of a closed pool. Progress in such experiments can allow us to form a database of acoustic signals used by dolphins during the “verbal interaction” between animals. This database will be used further to create the acoustic communication channel for the man - dolphin dialogue.

The general purpose of the study presented in our paper is in organizing an acoustic channel for the development of the intermediary language for the dialogue system "man - dolphin" based on the study of acoustic communication behavior between animals. In this process, one needs to solve the following problems:

- development of methodic to provoke the acoustic contacts between dolphins;

- creation of the technology of impact on the dolphin’s visual and acoustic sensor systems;

- formation of communication signals database used by dolphin in the interaction with the feedback system via video and acoustic channels.

Being the secondary aquatic mammals, cetaceans twice passed from the aquatic environment to the gaseous habitat in the course of a long evolution. These changes gradually led to changes in the sensory system. In the process of adaptation, the change of sensors changed the behavior of cetaceans, which inevitably led to a change in channels of communication. The difference in the physical properties of the human and dolphin habitat has led to the fact that the main information channel for a man is vision and hearing, while for a dolphin the main means of perception of the environment is sound as the physical basis of "illumination" of space, while vision plays an auxiliary (but rather large) role. Both channels are used when establishing contact with dolphins in human actions, but when using an acoustic channel, a man immensely loses both in the speed of information generation and of its processing. The effective frequency range of "generated" and "received" sounds by a man is 400-4000 Hz, while for a dolphin it extends from 10 Hz to hundreds of kilohertz. Visual contacts with the dolphin are more effective for the man but also have limitations due to the peculiarities of light propagation from air to water and backwards. One must note also that the interval of the animal's head being above water is very limited, as well as the underwater staying of the man without special equipment.

Numerous attempts aimed at the active human interaction with the dolphin are usually built on the variation of methods of training based on the each other imitating behavior of animals [2] or on the copying of certain types of human activity by the dolphin [3]. For the interaction with dolphins and other cetaceans, trainers use visual and acoustic channels as the main ones, the tactile channel is also successfully used but it requires the direct contact with the animal.

We see the most natural development of expanding the human - dolphin communication means in two directions:

- compression in time of the acoustic signals generated by a person and the reverse transformation of the dolphins’ signals;

- the use of underwater sources of images (perhaps with appropriate sound effects) of a static or dynamic character.

The first direction requires engineering and assembling of a completely new park of acoustic devices. Ability to complex forms of imitation plays highly significant role in dolphins’ social life. It is assumed that their ability to "vocal " and motor simulation is the basis on which various forms of coordinated group activity develops: synchronized swimming, collective hunting, protective behavior [4], [5]. The great similarity of the identification signals of animals belonging to the same community gives them the opportunity to quickly recognize their congeners.

The second, at the first stages, at least, can be implemented with a small modification of existing computer monitors [6]. We choose this direction for a series of experiments with Beluga wales (Delphinapterus leucas).

Specific features of dolphins’ physiology and psychology lead to the fact that the interaction within the group (pack) of dolphins and humans-to-dolphins significantly differs from that of other mammals. Accordingly, for the study of cognitive functions and communication behavior of dolphins, the formation of useful skills is realized in the process of active learning. Thus, we must abandon the conventional methods of training with the use of food deprivation and the use of negative reinforcement.

Material and methods

The positive results of the visual contact between the trainer and the dolphin with the help of images on the screen of underwater TV were shown in some well-known experiments [6], [7]. In our study, we used a system of interactive communication channels, including underwater computer monitor, webcam, microphones and acoustic signal synthesizer, which allowed activating the attention of the animal to the subject. This system permits to control the feedback between the flow of echolocation signals emitted by Beluga whales when approaching the monitor and the nature of the presented images. The software allows also the animal to control itself the image on the underwater monitor depending on the terms set up by the operator. By changing the amplitude of the acoustic signals and their frequency, the dolphin was able to turn independently the underwater monitor on or off and to change the image on the screen [8]. During all time of the experiment, the continuous acoustic monitoring was effectuated. Permanent processing of the incoming objective information and implementation of various devices interaction were realized with the help of specialized software. It provided communication between different devices: reception of dolphin’ echolocation of signals, control of scenes shown on the screen, synthesis of acoustic stimuli, etc. The block diagram of the engineered complex is shown in Fig.1.

Fig. 1 – The scheme for the study of dolphin’s cognitive functions:

1, 2, 3 - three hydrophones, 4, 5, 6 - three pre-amplifiers, 7, 8 - two loudspeakers; 9, 10 - monitor inside the underwater hermetic box; 11 - web camera; 12 - general three-channel amplifier; 13 - analog-to-digital convertor; 14 – computer

The developed software allowed us to form the dolphin conscious activity on the basis of self-learning or on the imitative reflex of the animal. The software interface is shown in Fig. 2.

Fig. 2 – The interface of the software “feedback-monitor” for the experimenter:

1 – position of the dolphin’s head taken by webcam; 2 – record of the acoustic pulses sequence; 3 – software to control the monitor; 4 – the monitor picture

For the self-learning subjects or vicarious learning, the broadcast on an underwater monitor was used. In the video clips demonstrated, the trained beluga whales are actively nodding their heads and spinning around their own axis at the command of the trainer. One of tested dolphins after a long observation of the scene demonstrated on the monitor repeated the motor acts of nodding and turning around its own axis. At all period of this apprehending, there were no additional acoustic accompaniment signals, in addition to echolocation signals.

Results

Demonstration of the floating fishes on the monitor in the aquarium, when their movement was synchronized with echolocation pulses, led to a continuous lingering of dolphin's attention to the monitor up to 7 minutes. A similar reaction was observed during video demonstration of the moving toy synchronized with echolocation pulses, as well as the periodic change of scale and number of different simple geometry figures.

Transmission of acoustic stimuli in the form of different song fragments, music, narration associated with various images on the screen, even synchronized with the echolocation pulses, did not induce the interest of dolphin. He swam up to the monitor, began the location test and after the beginning of the acoustic and video broadcast turned around and left the monitor. At the same time, the acoustic stimulus synthesized of a long noise-like pulses (in the frequency band up to 10 kHz) together with the amplitude modulation by the voice of the announcer (at the modulation depth not more than 5%) caused an active response of the animal. This response consisted of a package of broadband pulses with the modulation of time interval between pulses, which were clearly recorded by the acoustic control system. The delay of reaction varied from 300 to 2000 ms.

In all the described experiments during the tests, there were two animals in the pool: male and female belugas. The duration of each experiment varied from one to four hours and depended solely on the animal's free will to work. Sometimes, the preparatory period for setting up the equipment was delayed, but under the stable dolphin’ “research mood” they remained nearby the monitor screen and waited for the start of work. Food reinforcement in the experiments was not used at all. As is well-known, the imitative reflex does not require immediate nutritional reinforcement, and the positive result plays the role of such the reinforcement itself, sometimes. In these tests, the preparatory period was clearly distinguished, during which the animals fixed their position at the monitor either in turn or together. At the same time, the period spent at the monitor was not more than 10÷20 seconds. Only on the 11-th and 12-th day of the study, the duration of the animals’ activity reached a maximum value of 60 and 76 minutes for the whole experiment.

Demonstration of self-video image of the animal obtained with the help of web camera synchronized with echolocation signals caused an intense reaction of emitting the acoustic pulses packets of different duration and frequency. Packets consisted of ultrashort broadband pulses with modulation the time interval between pulses. The evident response was observed in 41 of 43 tests fulfilled. In some experiments, in parallel with sound effects, various video fragments were displayed at the monitor in these tests; the first dolphin’s acoustic response to the video and/or sound signal was of the main importance.

The results of induction of motor simulation behavior show the consistency of vicar teaching in this group of beluga whales. We observed the clearly visible cognitive function, which provides targeted attention, the process of learning about the surrounding world and interaction with it. Subjects containing the video games synchronized with dolphin’s acoustic activity by the feedback, induced the increase of attention to the monitor, but they did not cause the expected acoustic communication reaction. Quite different was the dolphin’s motor and acoustic response to the image received with the webcam: there was observed the exact match between the image and the acoustic communication signal sequence, even without any acoustic and food reinforcement.

The recorded acoustic signals of the dolphin clearly indicated that this was a verbal reaction to the video and acoustic impact. The dolphin tried to come into verbal contact with the image on the underwater monitor. The spatially distributed system of hydrophones also has shown that the dolphin can control the characteristics of the radiation field. For further processing of hydroacoustic information, there are important fragments, which were recorded simultaneously by three channels (with the necessary delays) with a maximum signal on the second channel.

Analysis of the set of dolphin’ signals obtained at a good signal/noise ratio in various experiments made it possible to reveal the characteristic temporal structures (Fig. 3), which are repeated in different fragments when displaying the webcam image and during acoustic noise stimuli containing amplitude modulation. Some specific pulse packages were well distinguished by their modulation structure. We have singled out two packages of ultrashort pulses, which were presented and clearly recorded in practically all fragments of the acoustic response. The first packet is indicated as 1, 1’ in Fig. 3, and the second one as 2, 2’. The duration of packets of the first type can slightly differ or be very close to 803 ms or 789 ms, the number of impulses being 119 and 85, correspondingly. The minimum and maximum change of the interval between pulses can both vary significantly or be very close one to another, while the modulation coefficient varied insignificantly from 0.15 to 0.17 (for the first type signals) and from 0.92 to 0.93 (for the second type signals). Signals of type 2 can differ significantly in the duration  (221 ms in Fig. 3.1 A and 331 ms in Fig. 3.1’ A), while the number of pulses varies insignificantly: 37 and 43, and the time parameters remain constant t_min = 2,9 ms, and t_max = 2,9 ms.

Fig. 3 – Two main type of communication signals (A) and dependence of internal pulse frequency modulation (B) on the impulse number for two experiments numbered 1 and 2. N -  is the total number of impulses recorded in one experiment, T - is the moment corresponding to the last impulse of the pack. Pay attention to the stability of the internal pack form

Discussion and Conclusion

Thus, the information parameters of signals of the first and second types can be fixed as following:

- the duration of the packets,

- the number of pulses in the pack,

- the variation of time-pulse modulation,

as well as the repeatability of the packets in all parameters and the invariance of these parameters in azimuth. In addition, there are signals with a structure that can be represented as permutations and repetitions of signals of these two types, stable in parameters.

The pulse series we have recorded have some similarities with the results reported in [9]. There were no frequency-modulated long pulses that were used as communication signals for the formation of the mediator language in experiments with Orcinus orca  [10]. We would specially emphasize that the time parameters of isolated verbal signals are stable in form and invariant with respect to the angle of observation. The considered experiments have shown also that there is a channel for the information transfer in the high frequency region of verbal signals with time-pulse modulation of a sequence of ultrashort pulses in packets, which is quite suitable for the information encoding.

The recorded signals allow us already to form a database of communication sequences that are associated with the subjects formed by the experimenter. Further development of the experiment of this type should be aimed on the provoking of the communication interaction between two dolphins in a duplex acoustic and video communication system (Fig. 4) with accumulation of a database of acoustic interaction of animals.

Fig. 4 – General view of the pool arranged for the experiment aimed on the formation of database on the acoustic activity of dolphins when operating with the duplex three-channel communication system

Our interest is in the formation of an intermediate language of dolphin-to-human “verbal” communication on the basis of the resulting verbal interaction database. In the conducted experiments, we have recorded signals with minimal distortions that are usually unavoidable when recording sound in the confined space with broadband reception system.

The results of this work show the prospects of complex experiments with various combinations of video and acoustic scenes included. The developed technical support of our experiments and the obtained data allow us to see the future experiments on the study of cognitive functions and communications between dolphins and dolphins-to-man, which may make feasible creation of a language-intermediary between human and dolphin.

The fulfilled study opens up also the possibility of implementation of duplex communication between animals and realization of an Internet video conference between groups of animals, in the future.

Благодарности Авторы выражают искреннюю признательность за предоставленную возможность провести описанное исследование всей команде тренеров, а также лично Игорю Костову, директору Санкт-Петербургского дельфинария (Крестовский остров); всем тренерам Анапаского дельфинария и лично Людмиле Камаевой, директору; всем сотрудникам Дельфинария Евпатории и лично Игорю Масбергу и Руслану Павловскому, менеджерам.

Acknowledgement The authors express their sincere appreciations for the given opportunity to conduct the described research to all team of trainers, as well as personally to Igor Kostov, the Director of St. Petersburg Dolphinarium (Krestovsky Island); to all coaches of the Anapa Dolphinarium, and personally to Lyudmila Kamaeva, Director in Chief; to the entire staff of Evpatoria Dolphinarium and personally to Igor Masberg and Ruslan Pavlovskiy, managers.

Конфликт интересов Не указан.

Conflict of Interest None declared.

 

Список литературы / References

1. Ivanov M. P. The Technique of Physiological Experiments on Acoustic Communication of Dolphins // Doklady Biological Sciences, - 2008. - V. 423. - № 2. – Р. 375-378. DOI: 10.1134/S001249660806002 [ IN RUSSIAN]
2. Brown D.H. Observations on the behavior of wild and captive alse killer whales, with notes on associated behavior of other genera of captive delphinids / Brown D.H., Caldwell D.K., Caldwell M.C. // Los Angeles City Museum Contribution to Science – 1966. – № 95. – P. 1–32.
3. Tayler C. K. Imitative behaviour by Indian Ocean bottlenose dolphins (Tursiops aduncus) in captivity / Tayler C. K., Saayman G. S. // Behaviour . – 1973. - №44. –Р. 286–98.
4. Herman L.M. Cognitive characteristics of dolphins / In: Cetacean behavior: Mechanisms and functions, , L. M. Herman. Wiley and Sons. – 1980.
5. Herman L. M. Cognition and language competencies of bottlenosed dolphins / In: Dolphin cognition and behavior: A comparative approach, , R.J. Schusterman, J.A. Thomas, & F.G. Wood. Lawrence Erlbaum. – 1986.
6. Herman L. M. Bottlenosed dolphin and human recognition of veridical and degraded video displays of an artificial gestural language / Herman L. M., P. Morrel-Samuels, A. A. Pack. // Journal of experimental psychology: General. – 1990. -№119. –Р. 215–230.
7. Herman L. M. The communication systems of cetaceans / Herman L. M., Tavolga W. N. // In L.M. Herman (Ed) Cetacean Behavior: Mechanisms and functions. - New York.: Wiley & Sons, Inc. - 1980. – 149-209.
8. Ivanov M.P. A Study of Cognitive Functions in the Dolphin Delphinapterus leucas / Ivanov M.P., Tolmachev Yu.A., Tulub A.A. and others // Journal of Evolutionary Biochemistry and Physiology.  - 2017. – V.54. - №3. – Р. 250-252.
9. Belikov R. A. Communicative pulsed signals of beluga whales in the reproductive gathering off Solovetskii Island in the White Sea / Belikov R. A., Bel’kovich V.M. // Acoustical Physics. - 2008. - №54 (1). –Р. 115–123.
10. Dudok van Heel W. H. An experiment in two-way communication in Orcinus orca / Dudok van Heel W. H., Kamminga C., J.D. van der Toorn. // Aquatic Mammals. - 1982. - V.9. - №3. - P. 69-82.