JMERC
Owner
Janbazan Medical and Engineering Research Center (JMERC) is the owner of IJWPH.
0.3
Cite Score
SJR: 0.114 / SNIP: 0.090
Volume 14, Issue 1 (2022)
Iran J War Public Health 2022, 14(1): 65-74 |
Back to browse issues page
Article Type:
Subject:
History
Received: 2021/12/15 | Accepted: 2022/02/23 | Published: 2022/03/5
Received: 2021/12/15 | Accepted: 2022/02/23 | Published: 2022/03/5
How to cite this article
Krechetov I, Skvortsov A, Poselsky I. Tactile and Force Feedback Gloves for Virtual Reality and Telecontrol of Anthropomorphic Manipulators; an Analytic Review. Iran J War Public Health 2022; 14 (1) :65-74
URL: http://ijwph.ir/article-1-1111-en.html
URL: http://ijwph.ir/article-1-1111-en.html
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Rights and permissions
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
1- Office of Scientific Research and Development, Moscow Polytechnic University, Moscow, Russian Federation
2- Department of Dynamics, Strength and Resistance of Materials, Moscow Polytechnic University, Moscow, Russian Federation
2- Department of Dynamics, Strength and Resistance of Materials, Moscow Polytechnic University, Moscow, Russian Federation
Full-Text (HTML) (704 Views)
Introduction
Modern technologies allow users to "live" in a virtual space when the illusion of the user's presence is created by tracking and visualizing real movements. With certain equipment settings, the user can not only look around but also interact with objects inside virtual spaces and through the telecontrol of a robotic arm in avatar mode. In recent years, virtual and mixed reality systems have evolved significantly and become available to the mass user. Low-cost virtual reality headsets (such as the HTC Vive and Oculus Rift) have emerged that provide high-resolution images and a wide viewing angle, and their built-in head tracking systems provide a new immersion experience in the virtual environment. The set of these helmets includes the simplest three-dimensional joysticks that allow influencing the virtual environment by moving hands.
Nevertheless, for a more realistic immersion, it is necessary to use other senses, in particular, the tactile sensation of touch and the force of impact on a virtual object, which is confirmed by studies [1]. The tactile glove is the most demanded and complex among the devices for interacting with virtual reality. Indeed, each person has a unique hand size and shape, and even pairs of hands are not identical or symmetrical. In addition, the hand is one of the most sensitive parts of the body. It is not only able to perceive small details with high frequency but can also exert a strong force.
Applications
Consequently, to be effective, a tactile glove must adapt to its wearer. It should be lightweight and compact yet deliver a large amount of power with very low latency. The main areas of application of such devices:
Modern technologies allow users to "live" in a virtual space when the illusion of the user's presence is created by tracking and visualizing real movements. With certain equipment settings, the user can not only look around but also interact with objects inside virtual spaces and through the telecontrol of a robotic arm in avatar mode. In recent years, virtual and mixed reality systems have evolved significantly and become available to the mass user. Low-cost virtual reality headsets (such as the HTC Vive and Oculus Rift) have emerged that provide high-resolution images and a wide viewing angle, and their built-in head tracking systems provide a new immersion experience in the virtual environment. The set of these helmets includes the simplest three-dimensional joysticks that allow influencing the virtual environment by moving hands.
Nevertheless, for a more realistic immersion, it is necessary to use other senses, in particular, the tactile sensation of touch and the force of impact on a virtual object, which is confirmed by studies [1]. The tactile glove is the most demanded and complex among the devices for interacting with virtual reality. Indeed, each person has a unique hand size and shape, and even pairs of hands are not identical or symmetrical. In addition, the hand is one of the most sensitive parts of the body. It is not only able to perceive small details with high frequency but can also exert a strong force.
Applications
Consequently, to be effective, a tactile glove must adapt to its wearer. It should be lightweight and compact yet deliver a large amount of power with very low latency. The main areas of application of such devices:
- Medical rehabilitation [2-6].
- Entertaining game complexes of virtual reality [7-10].
- Professional controller for CAD 3D modeling of similar software suites [11-13].
- Virtual simulators of complex control objects (equipment for the mining industry, cabins of trains and airplanes, etc.) [14-16].
- Telecontrol of multi-link robots manipulators (avatars) [17-19].
- Animation and film industry [20-22].
Virtual reality gloves are divided into several categories according to the method of constructing structural and layout solutions:
- Textile. Cover the palm and fingers with an elastic cloth. The sensors can be either integrated inside the fabric or located in the form of patches on the corresponding fingertips. The main advantage is the ease of putting on the hand.
- Textile. Cover the palm and fingers with an elastic cloth. The sensors can be either integrated inside the fabric or located in the form of patches on the corresponding fingertips. The main advantage is the ease of putting on the hand.
- Consisting of separate strips of fabric and plastic. Such gloves do not completely cover the hand, but only the necessary structural elements in which sensors and actuators are placed.
- Gloves with open fingertips. A variety of striped gloves with fingertip access. Allows interacting with touch screens.
- Exoskeletons. They are a set of joints with rigidly connected nodes. Typically located on the outside of the user's palm. Controlling the rotation of the joint emulates a force resistance proportional to the effect of the forces of the virtual object. The main advantage is high functionality compared to textile ones: the ability to implement force feedback.
- Exoskeletons. They are a set of joints with rigidly connected nodes. Typically located on the outside of the user's palm. Controlling the rotation of the joint emulates a force resistance proportional to the effect of the forces of the virtual object. The main advantage is high functionality compared to textile ones: the ability to implement force feedback.
Functions
The main functionality of the virtual reality gloves is to solve two problems:
1. Measurement of hand positions. Based on tracking the angular position of the fingers and the orientation of the user's palm. Required for tracking movements and recognizing individual gestures. An important advantage of special devices in the form of gloves compared to external optical tracking systems is their stability and independence from "shading" or depth effect caused by changes in hand orientations, in which the optical system cannot identify observation of individual groups of fingers. The key parameters are measurement accuracy, several degrees of freedom, refresh rate, and range of motion typical of a healthy human hand with 22 degrees of freedom in hand.
2. Feedback. Designed for intuitive interaction with virtual or real objects (as part of robotic avatars). The work is based on the generation of proportional forces on the user's skin and joints:
a) tactile. Provides a sense of touch, shape, texture, and roughness to an object. The main technologies used are electromagnetism (vibration, linear actuators, solenoids, and voice coils) and electrostatics (piezoelectric actuators, elastomers);
b) power. Provides a sense of inertia, mass, and firmness to an object by generating and regulating resistance to movement of the user's muscles. Servos, linear actuators, or flex-rod systems can be used. Application example: sensing contact with a wall, identifying stiffness and deformations of a virtual object. The drive control method is based on the so-called malleable control used in collaborative robotics.
History
The production of virtual reality gloves has wide geography: the major companies are located in the United States, but there are a number of developments in the European :union:, China, and Russia. Improving the quality and ease of control of both virtual objects and real robotic systems is provided by feedback systems that provide the "immersion" effect (Figure 1). These systems act on the operator's senses and transmit information about the control object. In general, feedback can be divided into two main areas, according to the senses on which they act: force feedback affects the muscles of the operator's limbs, and tactile feedback affects tactile receptors in human skin. According to the methods of implementation, the design of feedback systems can be divided into portable and stationary. Portable systems are placed on individual fingers, on the palm, on the back of the hand, or on the hand as a whole.
In the course of the study, several commercial virtual reality gloves were identified (Figure 2), the classification of which by functionality and technology used is presented in Table 1.
Figure 1) Design features of gloves with feedback
Table 1) Classification of gloves by functionality and technology used
Figure 2) The appearance of commercial VR gloves
Diversity
The methods of force feedback implementation can be divided into two groups: active and passive feedback. The principle of passive feedback is that the structural elements of the glove resist the movements of the operator's fingers. The following means are used: a controlled damper, a brake, or an electromagnetic clutch. The return of the elements to their original position is carried out by a spring. The advantage of such systems is the simplicity of design and safety for the user. Even in an error or failure of control, the glove cannot harm the operator due to the absence of active drives in it. A disadvantage follows from its advantages – the absence of active drives makes it impossible to influence the operator's senses at rest. Passive systems can generate feedback only when the operator's fingers move. One of the options for implementing a controlled damper is using hydraulic cylinders filled with a magnetorheological [42] fluid [43; 44], which is a suspension of a fine soft magnetic powder in silicone oil. Under the influence of an external magnetic field, the powder particles line up along the lines of force of the magnetic field, blocking the cross-section of the damper and thereby changing the viscosity of the oil. In this case, the liquid resists the movement of the damper piston until it stops completely. Such liquid dampers have not found widespread use due to their low operating speed [23].
The second implementation of the passive feedback system is electromagnetic brakes and clutches. Electromagnetic clutches [45] have a highly developed force with a compact size and low weight, but at the same time, they can demonstrate discreteness [46] of the fixed positions. Electromagnetic brakes [47] are much larger and heavier than clutches, but they operate smoothly and accurately. An interesting option is the implementation of a passive feedback system based on dielectric elastomers [48]. The principle of operation of such elastomers is to change the physical dimensions of an element under the influence of an applied electric voltage. The actuator is a rolled sheet of dielectric elastomer with a compressed spring inside. Under the action of an electromagnetic field, the elastomer is elongated and releases the spring that holds the operator's finger. Active feedback systems are based on active drives: electric motors, piezoelectric actuators, and pneumatic and hydraulic systems. The advantage of an active system is the ability to form feedback not only in motion and resistance but also in the effort. At the same time, the active system can work even when the operator does not make any movements and their fingers are at rest. Tactile feedback transfers information to the operator about the control object through exposure to skin receptors. There are several basic methods for implementing tactile feedback: vibration motors, an active single-point joystick, and multi-point matrix actuators. The simplest way to implement feedback is to use miniature vibration motors that are placed on the fingertips of the glove. When the operator's virtual hand interacts with any object, a control command is generated and the vibration motor begins to vibrate, thereby signaling the operator about contact with the virtual object. Such systems have a simple design, but the quality of the feedback is low and the operator's experience of interacting with the object is very far from reality [16].
The next option is to use active single-point joysticks that act on the operator's fingertips. There are several different joystick designs available: miniature delta robot, air and hydraulic air cushion, dielectric elastomer air cushion. The principle of operation of single-point joysticks is based on the three-dimensional movement of a structural element and applying pressure to the operator's fingertips. Such systems can convey the sensation of touching, holding, displacing, and sliding an object. The most common method for implementing an active feedback system is [49] electric motors. Many experimental and serial commercial designs implement a system of levers and rods, with the help of which electric motors generate feedback forces. Less common are pneumatic and hydraulic cylinder actuators. Such devices have high power, and are capable of developing very high forces. Still, at the same time, they require a complex and heavy circuit of pumps and control valves, which makes it difficult to implement them in a portable form [9].
A promising area in the construction of an active feedback system is the use of so-called soft actuators. Soft actuators are flexible devices, usually made of silicone, polyurethane, or rubber, powered by pneumatic or hydraulic pumps. Soft actuators have the same advantages and disadvantages as conventional pneumatic hydraulic actuators. A feature of soft actuators is smooth operation, ergonomics, simplicity, and low weight of the actuator design. The glove itself does not require the manufacture and assembly of the complex lever system required for servomotors and other rigid drives. The general advantage of active feedback systems is a high speed of operation and a large amount of generated forces, however, in the event of a control error or failure of the controller, there is a possibility of harm to the operator. Therefore, many developers deliberately limit the maximum developed drive forces [12].
Tactile feedback is the transfer of information to the operator about the control object through exposure to skin receptors. There are several basic methods for implementing tactile feedback: vibration motors, an active single-point joystick, and multi-point matrix actuators. The simplest way to implement feedback is to use miniature vibration motors that are placed on the fingertips of the glove. When the operator's virtual hand interacts with an object, a control command is generated and the vibration motor begins to vibrate, thereby signaling the operator about contact with the virtual object. Such systems have a simple design, but the quality of the feedback is low and the operator's experience of interacting with the object is very far from reality. The next option is to use active single-point joysticks that act on the operator's fingertips. There are several different joystick designs available: miniature delta robot, air and hydraulic air cushion, dielectric elastomer air cushion. The principle of operation of single-point joysticks is based on the three-dimensional movement of a structural element and applying pressure to the operator's fingertips. Such systems can convey the sensation of touching, holding, displacing, and sliding an object (Figure 3) [40].
Figure 3) Active single point joystick based on a miniature delta robot
The most complex, but at the same time providing the tactile feedback closest to reality, is the array of influencing elements, it is also called "tactile display". Such devices represent a matrix of some actuators that form a movable spatial picture, similar to the Braille cipher. There are two main methods for implementing spatial impact: direct mechanical and electrical stimulation. In the first case, the tactile display is a matrix of moving elements driven by external drives, electric motors, piezo actuators, or pneumatic-hydraulic pumps (Figure 4) [38].
Figure 4) Tactile mechanical display based on electric motors
HaptX has developed a tactile display representing an array of air cushions powered by a pneumatic compressor through a valve system (Figures 5 & 6) [35].
Figure 5) Tactile display based on inflatable cushions
Figure 6) Matrix of moving elements based on piezoactuators
Tactile displays based on electric motors, piezoelectric actuators, and pneumatic cushions provide a high resolution and realistic operator experience of interacting with virtual objects. The second method of implementing distributed exposure is direct electrical stimulation of the skin receptors of the operator's fingers. The advantages of the display implemented by the electrostimulation method are high resolution, simplicity of design, compactness, and low weight. The main disadvantage limiting the widespread use of electro-stimulators is the instability of effects and their dependence on pressing the operator's fingers to the electrodes. Excessive contact can cause pain in the receptors of the skin [22].
Advantages and disadvantages
In addition to the way force and tactile feedback are implemented, an equally important element is the mechanical design of the levers and rods, which transmit the forces from the actuators to the operator's hands. There are several basic designs, their features, advantages, and disadvantages are presented in Table 2 [19].
Table 2) Comparison of construction methods
Interest in the development of human-machine interaction systems has been observed for more than 30 years, i.e., the moment of mass introduction of home personal computers [1-7, 11, 48, 49]. At the same time, devices that allow using the functionality of natural hand movements are of the greatest interest. As a result of the study, the main technologies used to implement various functionalities of the virtual reality glove were identified: measuring hand positions and feedback generation. However, it is possible to assess the current state as not mature enough for mass use due to the high cost of devices for end-users, among which the most promising for the mass market are gloves with only limited functionality, both in terms of the number of degrees of freedom and the presence of only tactile feedback [25, 28]. The most promising applications are medical rehabilitation (in particular, restoring motor skills in patients after a stroke) and gaming [2-10].
The development of the glove design will be implemented using a kinematic scheme based on the kinematics of the human hand, considering the aesthetic and ergonomic requirements. As a sensor system, it is planned to use digital Hall sensors with a capacity of up to 14 bits, high speed, and at the same time, small dimensions, allowing them to be installed directly inside the swivel joints. To implement the subsystem of tactile influences, an analytical study of the methods of safe impact on the user's skin will be carried out [12, 34], with the determination of the permissible load ranges in terms of force and frequency to determine the optimal composition of the means for the generation of tactile influences. The main advantages of the planned development:
1. Ability to digitize up to 16 finger joints. As a rule, similar products are initially focused on performing the simplest actions in virtual reality, for which 6 finger positions are enough (2 sensors are used for the thumb). At the same time, the kinematic diagram of the human hand has 22 degrees of mobility. The developed glove is focused not only on interacting with objects in virtual reality but also on the remote control of anthropomorphic manipulators.
2. Tactile and force feedback. Inexpensive gloves use vibration motors as tactile feedback in the area of the fingertips [18]. This is a fairly effective approach for transmitting information about the tangency of an object, but it does not fully provide an opportunity to determine the stiffness and mass of an object [18, 33]. The developed glove uses a combined approach using actuators that provide flexible control (resistance to movement) of the glove links.
3. Reasonable cost, a benchmark for the mass market. Despite the rapid growth of virtual reality technologies, experiencing a rebirth, private home use of hand motion capture devices is still limited to ineffective optical tracking devices, due to the high cost of special gloves [22, 24]. One of the drawbacks of optical tracking is the effect of shading [20-22, 27], when, at certain orientations of the hand relative to the video camera, the fingers overlap each other, as a result of which the grip breaks and the tracking reliability suffers. In turn, virtual reality gloves with high-quality tracking of individual phalanges and feedback are still a niche product and are focused on specialized professional use – motion capture in film and animation production. The growth driver of capture devices is the balance of cost and functionality with a focus on the mass market of private users.
Conclusion
The scientific and technical groundwork planned to be obtained as a result of the project would allow creating of the so-called "end-to-end" technology, influencing several priorities of the Strategy of Scientific and Technological Development of the Russian Federation: 1) transition to advanced digital, intelligent production technologies, robotic systems, new materials and design methods, creation of systems for processing large amounts of data, machine learning and artificial intelligence; 2) transition to personalized medicine, high-tech healthcare and health-preserving technologies, including through the rational use of drugs.
Acknowledgments: None declared by the authors
Ethical Permissions: None declared by the authors.
Conflicts of Interests: The authors declare no conflicts of interest.
Authors’ Contribution: Krechetov IV (First Author), Introduction Writer/Main Researcher (40%); Skvortsov AA (Second Author), Methodologist/Statistical Analyst/Discussion Writer (30%); Poselsky IA (Third Author), Assistant Researcher/Statistical Analyst/Discussion Writer (30%)
Funding/Support: This study is funded by the Ministry of Science and Higher Education of the Russian Federation as part of the work under Agreement No. 075-15-2021-1340 dated 04.10.2021 on the topic "Research and development of a wearable device for digitizing the positions of the user's fingers". The study is conducted by the Moscow Polytechnic University.
The methods of force feedback implementation can be divided into two groups: active and passive feedback. The principle of passive feedback is that the structural elements of the glove resist the movements of the operator's fingers. The following means are used: a controlled damper, a brake, or an electromagnetic clutch. The return of the elements to their original position is carried out by a spring. The advantage of such systems is the simplicity of design and safety for the user. Even in an error or failure of control, the glove cannot harm the operator due to the absence of active drives in it. A disadvantage follows from its advantages – the absence of active drives makes it impossible to influence the operator's senses at rest. Passive systems can generate feedback only when the operator's fingers move. One of the options for implementing a controlled damper is using hydraulic cylinders filled with a magnetorheological [42] fluid [43; 44], which is a suspension of a fine soft magnetic powder in silicone oil. Under the influence of an external magnetic field, the powder particles line up along the lines of force of the magnetic field, blocking the cross-section of the damper and thereby changing the viscosity of the oil. In this case, the liquid resists the movement of the damper piston until it stops completely. Such liquid dampers have not found widespread use due to their low operating speed [23].
The second implementation of the passive feedback system is electromagnetic brakes and clutches. Electromagnetic clutches [45] have a highly developed force with a compact size and low weight, but at the same time, they can demonstrate discreteness [46] of the fixed positions. Electromagnetic brakes [47] are much larger and heavier than clutches, but they operate smoothly and accurately. An interesting option is the implementation of a passive feedback system based on dielectric elastomers [48]. The principle of operation of such elastomers is to change the physical dimensions of an element under the influence of an applied electric voltage. The actuator is a rolled sheet of dielectric elastomer with a compressed spring inside. Under the action of an electromagnetic field, the elastomer is elongated and releases the spring that holds the operator's finger. Active feedback systems are based on active drives: electric motors, piezoelectric actuators, and pneumatic and hydraulic systems. The advantage of an active system is the ability to form feedback not only in motion and resistance but also in the effort. At the same time, the active system can work even when the operator does not make any movements and their fingers are at rest. Tactile feedback transfers information to the operator about the control object through exposure to skin receptors. There are several basic methods for implementing tactile feedback: vibration motors, an active single-point joystick, and multi-point matrix actuators. The simplest way to implement feedback is to use miniature vibration motors that are placed on the fingertips of the glove. When the operator's virtual hand interacts with any object, a control command is generated and the vibration motor begins to vibrate, thereby signaling the operator about contact with the virtual object. Such systems have a simple design, but the quality of the feedback is low and the operator's experience of interacting with the object is very far from reality [16].
The next option is to use active single-point joysticks that act on the operator's fingertips. There are several different joystick designs available: miniature delta robot, air and hydraulic air cushion, dielectric elastomer air cushion. The principle of operation of single-point joysticks is based on the three-dimensional movement of a structural element and applying pressure to the operator's fingertips. Such systems can convey the sensation of touching, holding, displacing, and sliding an object. The most common method for implementing an active feedback system is [49] electric motors. Many experimental and serial commercial designs implement a system of levers and rods, with the help of which electric motors generate feedback forces. Less common are pneumatic and hydraulic cylinder actuators. Such devices have high power, and are capable of developing very high forces. Still, at the same time, they require a complex and heavy circuit of pumps and control valves, which makes it difficult to implement them in a portable form [9].
A promising area in the construction of an active feedback system is the use of so-called soft actuators. Soft actuators are flexible devices, usually made of silicone, polyurethane, or rubber, powered by pneumatic or hydraulic pumps. Soft actuators have the same advantages and disadvantages as conventional pneumatic hydraulic actuators. A feature of soft actuators is smooth operation, ergonomics, simplicity, and low weight of the actuator design. The glove itself does not require the manufacture and assembly of the complex lever system required for servomotors and other rigid drives. The general advantage of active feedback systems is a high speed of operation and a large amount of generated forces, however, in the event of a control error or failure of the controller, there is a possibility of harm to the operator. Therefore, many developers deliberately limit the maximum developed drive forces [12].
Tactile feedback is the transfer of information to the operator about the control object through exposure to skin receptors. There are several basic methods for implementing tactile feedback: vibration motors, an active single-point joystick, and multi-point matrix actuators. The simplest way to implement feedback is to use miniature vibration motors that are placed on the fingertips of the glove. When the operator's virtual hand interacts with an object, a control command is generated and the vibration motor begins to vibrate, thereby signaling the operator about contact with the virtual object. Such systems have a simple design, but the quality of the feedback is low and the operator's experience of interacting with the object is very far from reality. The next option is to use active single-point joysticks that act on the operator's fingertips. There are several different joystick designs available: miniature delta robot, air and hydraulic air cushion, dielectric elastomer air cushion. The principle of operation of single-point joysticks is based on the three-dimensional movement of a structural element and applying pressure to the operator's fingertips. Such systems can convey the sensation of touching, holding, displacing, and sliding an object (Figure 3) [40].
Figure 3) Active single point joystick based on a miniature delta robot
The most complex, but at the same time providing the tactile feedback closest to reality, is the array of influencing elements, it is also called "tactile display". Such devices represent a matrix of some actuators that form a movable spatial picture, similar to the Braille cipher. There are two main methods for implementing spatial impact: direct mechanical and electrical stimulation. In the first case, the tactile display is a matrix of moving elements driven by external drives, electric motors, piezo actuators, or pneumatic-hydraulic pumps (Figure 4) [38].
Figure 4) Tactile mechanical display based on electric motors
HaptX has developed a tactile display representing an array of air cushions powered by a pneumatic compressor through a valve system (Figures 5 & 6) [35].
Figure 5) Tactile display based on inflatable cushions
Figure 6) Matrix of moving elements based on piezoactuators
Tactile displays based on electric motors, piezoelectric actuators, and pneumatic cushions provide a high resolution and realistic operator experience of interacting with virtual objects. The second method of implementing distributed exposure is direct electrical stimulation of the skin receptors of the operator's fingers. The advantages of the display implemented by the electrostimulation method are high resolution, simplicity of design, compactness, and low weight. The main disadvantage limiting the widespread use of electro-stimulators is the instability of effects and their dependence on pressing the operator's fingers to the electrodes. Excessive contact can cause pain in the receptors of the skin [22].
Advantages and disadvantages
In addition to the way force and tactile feedback are implemented, an equally important element is the mechanical design of the levers and rods, which transmit the forces from the actuators to the operator's hands. There are several basic designs, their features, advantages, and disadvantages are presented in Table 2 [19].
Table 2) Comparison of construction methods
Interest in the development of human-machine interaction systems has been observed for more than 30 years, i.e., the moment of mass introduction of home personal computers [1-7, 11, 48, 49]. At the same time, devices that allow using the functionality of natural hand movements are of the greatest interest. As a result of the study, the main technologies used to implement various functionalities of the virtual reality glove were identified: measuring hand positions and feedback generation. However, it is possible to assess the current state as not mature enough for mass use due to the high cost of devices for end-users, among which the most promising for the mass market are gloves with only limited functionality, both in terms of the number of degrees of freedom and the presence of only tactile feedback [25, 28]. The most promising applications are medical rehabilitation (in particular, restoring motor skills in patients after a stroke) and gaming [2-10].
The development of the glove design will be implemented using a kinematic scheme based on the kinematics of the human hand, considering the aesthetic and ergonomic requirements. As a sensor system, it is planned to use digital Hall sensors with a capacity of up to 14 bits, high speed, and at the same time, small dimensions, allowing them to be installed directly inside the swivel joints. To implement the subsystem of tactile influences, an analytical study of the methods of safe impact on the user's skin will be carried out [12, 34], with the determination of the permissible load ranges in terms of force and frequency to determine the optimal composition of the means for the generation of tactile influences. The main advantages of the planned development:
1. Ability to digitize up to 16 finger joints. As a rule, similar products are initially focused on performing the simplest actions in virtual reality, for which 6 finger positions are enough (2 sensors are used for the thumb). At the same time, the kinematic diagram of the human hand has 22 degrees of mobility. The developed glove is focused not only on interacting with objects in virtual reality but also on the remote control of anthropomorphic manipulators.
2. Tactile and force feedback. Inexpensive gloves use vibration motors as tactile feedback in the area of the fingertips [18]. This is a fairly effective approach for transmitting information about the tangency of an object, but it does not fully provide an opportunity to determine the stiffness and mass of an object [18, 33]. The developed glove uses a combined approach using actuators that provide flexible control (resistance to movement) of the glove links.
3. Reasonable cost, a benchmark for the mass market. Despite the rapid growth of virtual reality technologies, experiencing a rebirth, private home use of hand motion capture devices is still limited to ineffective optical tracking devices, due to the high cost of special gloves [22, 24]. One of the drawbacks of optical tracking is the effect of shading [20-22, 27], when, at certain orientations of the hand relative to the video camera, the fingers overlap each other, as a result of which the grip breaks and the tracking reliability suffers. In turn, virtual reality gloves with high-quality tracking of individual phalanges and feedback are still a niche product and are focused on specialized professional use – motion capture in film and animation production. The growth driver of capture devices is the balance of cost and functionality with a focus on the mass market of private users.
Conclusion
The scientific and technical groundwork planned to be obtained as a result of the project would allow creating of the so-called "end-to-end" technology, influencing several priorities of the Strategy of Scientific and Technological Development of the Russian Federation: 1) transition to advanced digital, intelligent production technologies, robotic systems, new materials and design methods, creation of systems for processing large amounts of data, machine learning and artificial intelligence; 2) transition to personalized medicine, high-tech healthcare and health-preserving technologies, including through the rational use of drugs.
Acknowledgments: None declared by the authors
Ethical Permissions: None declared by the authors.
Conflicts of Interests: The authors declare no conflicts of interest.
Authors’ Contribution: Krechetov IV (First Author), Introduction Writer/Main Researcher (40%); Skvortsov AA (Second Author), Methodologist/Statistical Analyst/Discussion Writer (30%); Poselsky IA (Third Author), Assistant Researcher/Statistical Analyst/Discussion Writer (30%)
Funding/Support: This study is funded by the Ministry of Science and Higher Education of the Russian Federation as part of the work under Agreement No. 075-15-2021-1340 dated 04.10.2021 on the topic "Research and development of a wearable device for digitizing the positions of the user's fingers". The study is conducted by the Moscow Polytechnic University.
Keywords:
References
1. Kim M, Jeon C, Kim J. A study on immersion and presence of a portable hand haptic system for immersive virtual reality. Sensors. 2017;17:1141. [Link] [DOI:10.3390/s17051141]
2. Placidi G. A smart virtual glove for the hand telerehabilitation. Comput Biol Med. 2007;37(8):1100-7. [Link] [DOI:10.1016/j.compbiomed.2006.09.011]
3. Placidi G, Cinque L, Polsinelli M, Spezialetti M. Measurements by a LEAP-based virtual glove for the hand rehabilitation. Sensors. 2018;18(3):834. [Link] [DOI:10.3390/s18030834]
4. Besnea F, Cismaru SI, Trasculescu AC, Resceanu IC, Ionescu M, Hamdan H, et al. Integration of a haptic glove in a virtual reality-based environment for medical training and procedures. Acta Tech Napocensis Ser Appl Math Mech Eng. 2021;64(1-S2):281-90. [Link]
5. Draganov IR, Boumbarov OL. Investigating Oculus Rift virtual reality display applicability to medical assistive system for motor disabled patients. 2015 IEEE 8th Int Conf Intell Data Acquis Adv Comput Syst Tech Appl. 2015;2:751-4. [Link] [DOI:10.1109/IDAACS.2015.7341403]
6. Yates M, Kelemen A, Sik Lanyi C. Virtual reality gaming in the rehabilitation of the upper extremities post-stroke. Brain Inj. 2016;30(7):855-63. [Link] [DOI:10.3109/02699052.2016.1144146]
7. Zyda M. From visual simulation to virtual reality to games. Computer. 2005;38(9):25-32. [Link] [DOI:10.1109/MC.2005.297]
8. Kuei-Shu H. Application of a virtual reality entertainment system with human-machine haptic sensor device. J Appl Sci. 2011;11(12):2145-53. [Link] [DOI:10.3923/jas.2011.2145.2153]
9. Cheok AD, Haller M, Fernando ONN, Wijesena JP. Mixed reality entertainment and art. Int J Virtual Reality. 2009;8(2):83-90. [Link] [DOI:10.20870/IJVR.2009.8.2.2729]
10. Von Itzstein GS, Billinghurst M, Smith RT, Thomas BH. Augmented reality entertainment: Taking gaming out of the box. In: Lee N, editor. Encyclopedia of computer graphics and games. Cham: Springer; 2019. p. 1-9. [Link] [DOI:10.1007/978-3-319-08234-9_81-1]
11. Gallo L. A glove-based interface for 3D medical image visualization. In: Damiani E, Howlett RJ, Jain LC, Gallo L, De Pietro G, editors. Intelligent interactive multimedia systems and services. Berlin: Springer; 2010. p. 221-30. [Link] [DOI:10.1007/978-3-642-14619-0_21]
12. Gallotti P, Raposo A, Soares L. V-Glove: A 3D virtual touch interface. 2011 XIII Symposium on Virtual Reality, Uberlandia, Brazil, 23-26 May 2011. Piscataway: IEEE; 2011. [Link] [DOI:10.1109/SVR.2011.21]
13. Wong PYJ, Lau RWH, Ma L. Virtual 3D sculpturing. J Visualiz Comput Anim. 2000;11(3):155-66.
https://doi.org/10.1002/1099-1778(200007)11:3<155::AID-VIS225>3.0.CO;2-7 [Link] [DOI:10.1002/1099-1778(200007)11:33.0.CO;2-7]
14. Perrenot C, Perez M, Tran N, Jehl JP, Felblinger J, Bresler L, et al. The virtual reality simulator dV-Trainer® is a valid assessment tool for robotic surgical skills. Surg Endosc. 2012;26(9):2587-93. [Link] [DOI:10.1007/s00464-012-2237-0]
15. Ćwil M, Bartnik W. Physically extended virtual reality (PEVR) as a new concept in railway driver training. In: Chen J, Fragomeni G, editors. Virtual, Augmented and Mixed Reality. Applications and Case Studies. Cham: Springer; 2019. p. 230-42. [Link] [DOI:10.1007/978-3-030-21565-1_15]
16. Nainggolan F, Siregar B, Fahmi F. User experience in excavator simulator using leap motion controller in virtual reality environment. J Phys Conf Ser. 2020;1566(1):012093. [Link] [DOI:10.1088/1742-6596/1566/1/012093]
17. Mizera C, Delrieu T, Weistroffer V, Andriot C, Decatoire A, Gazeau JP. Evaluation of hand-tracking systems in teleoperation and virtual dexterous manipulation. IEEE Sens J. 2019;20(3):1642-55. [Link] [DOI:10.1109/JSEN.2019.2947612]
18. Lv X, Zhang M, Cui F, Zhang X. Teleoperation of robot based on virtual reality. 16th International Conference on Artificial Reality and Telexistence - Workshops (ICAT'06). Piscataway: IEEE; 2006. p. 400-3. [Link] [DOI:10.1109/ICAT.2006.124]
19. Hollerbach JM, Jacobsen SC. Haptic interfaces for teleoperation and virtual environments. Proceedings of First Workshop on Simulation and Interaction in Virtual Environments. Piscataway: Institute of Electrical and Electronics Engineers Inc.; 1995. p. 13-15. [Link]
20. Leite L. Virtual marionette. Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces. Piscataway: Institute of Electrical and Electronics Engineers Inc.; 2012. p. 363-6. [Link] [DOI:10.1145/2166966.2167049]
21. Geigel J, Schweppe M. Motion capture for realtime control of virtual actors in live, distributed, theatrical performances. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG). Piscataway: Institute of Electrical and Electronics Engineers Inc.; 2011. p. 774-9. [Link] [DOI:10.1109/FG.2011.5771347]
22. Kartiko I, Kavakli M, Cheng K. Learning science in a virtual reality application: The impacts of animated-virtual actors' visual complexity. Comput Educ. 2010;55(2):881-91. [Link] [DOI:10.1016/j.compedu.2010.03.019]
23. Dickens L, Edensor T. Entering the Fifth Dimension: modular modernities, psychedelic sensibilities, and the architectures of lived experience. Transact Inst Br Geogr. 2021;46(3):659-74. [Link] [DOI:10.1111/tran.12440]
24. Bickmann R, Tran C, Ruesch N, Wolf K. Haptic illusion glove: a glove for illusionary touch feedback when grasping virtual objects. Proceedings of Mensch und Computer, September 2019: pp. 565-9. [Link] [DOI:10.1145/3340764.3344459]
25. synertial.com [Internet]. Unknown city: Synertial; 2021 [cited 2021 Jul 28]. Available from: http://www.synertial.com/cobra-gloves. [Link]
26. dextarobotics.com [Internet]. Unknown city: DextaRobotics; 2021 [cited 2021 Jul 28]. Available from: http://www.dextarobotics.com. [Link]
27. Wang H, Tong X, Lu F. Deep learning based target detection algorithm for motion capture applications. J Phys Conf Ser. 2020;1682:012032. [Link] [DOI:10.1088/1742-6596/1682/1/012032]
28. Claverie L, Ille A, Moretto P. Discrete sensors distribution for accurate plantar pressure analyses. Med Eng Phy. 2016;38(12):1489-94. [Link] [DOI:10.1016/j.medengphy.2016.09.021]
29. Aretinsky VB, Telegina EV, State VLI. Restoration of motor function of the hand in patients with stroke using the "hand tutor" system. Уральский медицинский журнал. 2014(9):46-9. [Russian] [Link]
30. noitom.com [Internet]. Unknown city: Noitom Ltd; 2021 [cited 2021 Jul 28]. Available from: http://www.noitom.com. [Link]
31. Pons JL, Rocon E, Ceres R, Reynaerts D, Saro B, Levin S, Van Moorleghem W. The MANUS-HAND dextrous robotics upper limb prosthesis: mechanical and manipulation aspects. Autonomous Robots. 2004;16(2):143-63. [Link] [DOI:10.1023/B:AURO.0000016862.38337.f1]
32. Caeiro-Rodríguez M, Otero-González I, Mikic-Fonte FA, Llamas-Nistal M. A systematic review of commercial smart gloves: Current status and applications. Sensors. 2021;21(8):2667. [Link] [DOI:10.3390/s21082667]
33. nansense.com [Internet]. Unknown city: NANSENSE Inc.; 2021 [cited 2021 Aug 16]. Available from: http://www.nansense.com. [Link]
34. Sers R, Forrester S, Moss E, Ward S, Ma J, Zecca M. Validity of the Perception Neuron inertial motion capture system for upper body motion analysis. Measurement. 2020;149:107024. [Link] [DOI:10.1016/j.measurement.2019.107024]
35. Kim H, Lee A, Shin YI, Chang WH, Koo KH, Seong H, Kim YH. Effects of digital smart glove system on motor recovery of upper extremity in subacute stroke patients. Ann Phys Rehabil Med. 2018;61:e28. [Link] [DOI:10.1016/j.rehab.2018.05.061]
36. Moeslund TB, Hilton A, Krüger V. A survey of advances in vision-based human motion capture and analysis. Computer vision and image understanding. 2006;104(2-3):90-126. [Link] [DOI:10.1016/j.cviu.2006.08.002]
37. senseglove.com [Internet]. Unknown city: Sense Glove; 2021 [cited 2021 Aug 16]. Available from: http://www.senseglove.com. [Link]
38. Vidal-Verdú F, Delgado-Restituto M, Navas R, Rodrı́guez-Vázquez A. A design approach for analog neuro/fuzzy systems in CMOS digital technologies. Comput Electric Eng. 1999;25(5):309-37. [Link] [DOI:10.1016/S0045-7906(99)00008-7]
39. Darrah M, Humbert R, Finstein J, Simon M, Hopkins J. Are virtual labs as effective as hands-on labs for undergraduate physics? A comparative study at two major universities. J Sci Educ Technol 2014;23(6):803-14. [Link] [DOI:10.1007/s10956-014-9513-9]
40. Burdea GC. Haptic feedback for virtual reality. Virtual reality and prototyping workshop. 1999;2:17-29. [Link]
41. tautracker.com [Internet]. Unknown city: Tau Tracker; 2021 [cited 2021 Sep 2]. Available from: http://www.https://www.tautracker.com/devices_ru. [Link]
42. Cassar DJ, Saliba MA. A force feedback glove based on magnetorheological fluid: Preliminary design issues. Proc Mediterr Electrotech Conf. 2010;5476012:618-23. [Link] [DOI:10.1109/MELCON.2010.5476012]
43. Winter SH, Bouzit M. Use of magnetorheological fluid in a force feedback glove. IEEE Trans Neural Syst Rehabil Eng. 2007;15(1):2-8. [Link] [DOI:10.1109/TNSRE.2007.891401]
44. Blake J, Gurocak HB. Haptic glove with MR brakes for virtual reality. IEEE/ASME Trans Mechatron. 2009;14(5):606-15. [Link] [DOI:10.1109/TMECH.2008.2010934]
45. Guo Y, Yang X, Wang H, Xu W, Wang D. Five-fingered passive force feedback glove using a variable ratio lever mechanism. Actuators. 2021;10(5):96. [Link] [DOI:10.3390/act10050096]
46. Gu X, Zhang Y, Sun W, Zhou D, Kristensson PO. Dexmo: An inexpensive and lightweight mechanical exoskeleton for motion capture and force feedback in VR. In: Conference on Human Factors in Computing Systems - Proceedings. New York: Association for Computing Machinery; 2016. p. 1991-5. [Link] [DOI:10.1145/2858036.2858487]
47. Wang D, Wang Y, Pang J, Wang Z, Zi B. Development and control of an MR brake-based passive force feedback data glove. IEEE Access. 2019;7:172477-88. [Link] [DOI:10.1109/ACCESS.2019.2956954]
48. Zhang R, Lochmatter P, Kunz A, Kovacs G. Spring roll dielectric elastomer actuators for a portable force feedback glove. Proc. SPIE 6168, Smart Structures and Materials 2006: Electroactive Polymer Actuators and Devices (EAPAD), 61681T (22 March 2006). [Link] [DOI:10.1117/12.658524]
49. Koyama T, Yamano I, Takemura K, Maeno T. Multi-fingered exoskeleton haptic device using passive force feedback for dexterous teleoperation. IEEE Int Conf Intelligent Rob Syst. 2002;3:2905-10. [Link]