IoT Based Virtual Reality Game for Physio-therapeutic Patients

K. Martin Sagayam, Shibin D, Hien Dang, Mohd Helmy Abd Wahab and Radzi Ambar, "IoT Based Virtual Reality Game for Physio-therapeutic Patients”, Annals of Emerging Technologies in Computing (AETiC), Print ISSN: 2516-0281, Online ISSN: 2516-029X, pp. 39-51, Vol. 4, No. 4, 1st October 2020, Published by International Association of Educators and Researchers (IAER), DOI: 10.33166/AETiC.2020.04.005, Available: http://aetic.theiaer.org/archive/v4/v4n4/p5.html. Research Article


Introduction
Physical therapy is always a big part of rehabilitation, but no one wants to do the exercises. The purpose of this training is to improve muscles strength and facilitating fast recovery of injured areas. Traditional physiotherapy includes pain relief, various stretching, strengthening exercises, and aerobic training. In recent years, Virtual Reality (VR) technologies have begun to be used as a complementary tool in rehabilitation as a new form of interactive play. In the last few years, researchers have focused on exergames or active video games using VR in clinical research and practice. VR is not only fun but will continue to be a driver in the world of rehab. On the other hand, the Internet-of-Things (IoT) is a network in which electronic devices, software, sensors, actuators and connections are placed. This network consists of physical devices, tools, household appliances and other items that allow objects to bind and modify data. However, VR and IoT share the same principle and objective which to bridge the digital and actual physical domain, regardless of the contradicting approaches of the task. VR is about producing the digital domain look real using wearable devices. On the contrary, IoT is about making actual physical objects usable in the digital domain.
In order to experience VR environment, VR users are require to use necessary devices such as headsets, headphones and hand controllers to produce realistic images, sounds and experience of being physically in a virtual environment. A person using virtual reality equipment is able to look around the artificial world, move around in it, and interact with virtual features or items.
There are various studies that show the effectiveness of VR and IoT in improving social wellbeing. Kairy et al. had stated that tele-rehabilitation i.e., service from distance using internet contributed to the highest care of patients at night and also enhances the continuous care of patients after discharge [1]. Padilla-Castañeda et al. stated that virtual reality had been very useful in treating fractures by intensive training with VR environment [2]. This provided better feedback due to replication of their arm movements. According to Tagaldeen et al., VR games provide a correct restructuring of the arm [3]. It will also keep the patient busy and reduce the total cost of medical and health care for the patient. Pournajaf et al. in recent studies on virtual reality in rehabilitation, show the efficacy of VR on proprioceptive capacity, postural control, and walking [4].
Therefore, this study is aimed to provide an emotional and physically comfortable environment to the patient by developing an interactive game for physical rehabilitation by combining biofeedback and physical computer environments. In the developed application, an interactive game was developed for the basic treatment of Frozen Shoulder by using the IoT features of the web 3.0 and the VR platform which is frequently used in recent times. The rest of the article continues as follows: In the second part, there is a literature review that gives prominent results especially in the treatment and rehabilitation studies of VR. The proposed method is explained in the third chapter with details. In the fourth section, the obtained results were given and necessary comments were made. Finally, results and recommendations are given.

Literature Survey
Various studies have been carried out in researchers related to rehabilitation that utilize VR and IoT. Corbetta et al. executed a trial on adults with a clinical diagnosis of stroke by including virtual reality based rehabilitation to the usual rehabilitation method that had been used [5]. It had been observed that basic measures such as walking speed, balance and mobility was increased. It had been also found that the cost to add these extra VR based rehabilitation is small.
Mathews et al. stated that to improve the prospective memory of stroke patients a computer based technology had been developed to teach patients to remember real time events using VR environment [6]. Constraint based modelling had been used to track actions and provides feedback. It had been observed that the prospective memory skills of 15 participants improved significantly with the patients enjoying the VR environment. The improvement had been found to be stable.
Yang et al. had found that to reduce dizziness which is major consequence of imbalance and vestibular dysfunction balance training is desired over surgery [7]. To reduce the tediousness of exercises, an interactive VR game based rehabilitation had been adopted along with sensor based measuring system. It had been observed that patients who had completed training showed progressed balance.
Suyanto et al. observed that several people suffering from fear of height (acrophobia) face much difficulty [8]. It had been found that the VR or virtual environment helps them overcome the fear of height gradually based on level progression in virtual world. It had been observed that patients enjoyed the real time objects and sounds used in virtual reality.
Cuperus et al. investigated that manipulation in spatial distance in virtual reality can influence treadmill exercises performance [9]. Survey had been performed on participants on treadmill with and without virtual reality and told to stop on pain. Patients with virtual reality walked further.
Ravi et al. reported that 369 participants who participated in rehabilitation using VR demonstrated improvement in balance and motor skills in children and adolescents with cerebral palsy [10]. At the end of their studies, they said that the use of VR in motor rehabilitation of children with cerebral palsy was very effective.
Vieira et al. had stated that virtual reality environments had been used in cardiac rehabilitation aimed to analyse the effect of exercise program in virtual reality on body composition, eating www.aetic.theiaer.org pattern and lipid profile of patients with coronary heart disease [11]. They had been equipped with bio impedance sensor. It has been observed that VR has a positive effect on body composition, especially from waist to hip ratio.
Lloréns and his colleagues studied the effectiveness of rehabilitation system (bio-track) with VR system for patients with brain injury [12]. 20 sessions were conducted using Biotrak's balancing module. The results have confirmed that the use of VR for balance retardation gives better results.
Jongmin et al stated that VR had been used to reduce the falling in elderly people. They introduced a VR program consisting of complex exercises [13]. Patients over 65 years of age were studied and it had been found that implementation of VR exercises improved balance and muscle strength in elderly compared to balance exercise.
Kairy et al. found that tele-rehabilitation i.e., service from distance using internet contributed to the highest care of patients at night [1]. This task can be done by sitting in their place in comfort for shoulders and hands as shown in Figure 1. They used VR system allowing ongoing rehabilitation of upper extremity after a stroke with a doctor at distant. It has been found that this enhances the continuous care of patients after discharge.
(a) (b) Figure 1. Sitting exercise: (a) reaching tasks using shoulders, (b) reaching tasks using hands [1] Lkilic et al. aimed to present a rehabilitation system system based on VR for Parkinson and multiple sclerosis patients [14]. The problems such as balance, tremor and movement coordination had been solved by VR environment for physical rehabilitation. VR environment was implemented using unity 3d game engine and the detection of joint movement had been implemented through Microsoft kinet human machine interface.
Taneja et al. observed that several young people suffer from mental stress which leads to drug abuse, decline in academic performance [15]. Rather than contacting a psychatrist or social support group,the stressed patients had been subjected to VR based stress therapy that had included island and forest environment with calming instrumental music which soothed the patients.
Ghanouni et al. aimed at creating a tele-rehabilitation platform for children with autism syndrome [16]. Patients were tested and explored on design and parameter of the platform. It had been aimed to improve clinical features and increasing motivation in autism affected patients.
Wake et al. studied that amputees often perceive vivid awareness of their amputated limbs (phantom limbs) which leads to intense pain [17]. Neuro rehabilitation techniques using VR had given a visual imagination of a phantom limb to patients which had produced movement perception of phantom limb. It had been observed that 86% decrease in tactile feedback condition.
Padilla-Castañeda et al. stated that the combination of robotics and virtual reality had been very useful in treating fractures by intensive training with VR environment [2]. A robotic VR platform had been designed which provides VR scenarios with realism. It had provided feedback due to replication of their arm movements. It had been concluded that the system was highly acceptable.
Khotimah et al. observed that post stroke patients suffer from physical disorder or paralysis that they find it difficult to visit therapist and hence a VR system had been designed for sittingstanding-sitting ,moving leg aside and walking having tree difficulty level [18]. It had been observed that the patients' comments were positive and it had been told that it could be used for elderly also. www.aetic.theiaer.org Solignac et al. had designed a VR system for space, polar and submarine crews living in artificial habitats to indulge in daily physical exercise in natural landscapes [19]. It had been observed that this system benefitted crew living and working in confined environment during exercise.
Hoffman had studied that patients with severe burn wounds face a lot of pain during wound care [20]. VR had proved to be an analgesic using computer generated world to escape pain. Results had shown that the wide field view of VR goggles enhances analgesic effect providing distraction and reducing pain.
Vargas-Orjuela et al. presented the development of a VR prototype for cardiac auscultation training [21]. It had aimed on diagnosing the condition of a patient's heart which is done by interpreting heart sound which can be achieved only through extensive training. This system simplified the training process.
Intraraprasit et al. reported that memory declines with age or mild cognitive impairment [22]. To reduce this problem cognitive training or brain training with VR application that has a head mounted display with VR controllers was introduced. It also had different staged of difficulty. It had been observed that the patients were very satisfied with the VR based cognitive training.
Minyoung et al. proposed individualized feedback based VR exercise on elderly women [23]. 54 older women aged above 65 were exposed to this system three times a week for eight weeks. It had been observed that mental health, emotional strength had increased along with physical fitness.
Singh et al. designed a VR balance games to decrease the risk and fear of falls among women [24]. A test had been conducted among women with VR game and with conventional method. It was studied that practising VR balance games increases balance confidence and decreases risk of fall among women.
Lotan et al. found that individuals with intellectual and developmental disabilities needed effective physical training based on VR [25]. It had been tested on patients for 30 minutes session per week and had been observed a significant reduction in heart rate.
Jungjin et al. developed a VR based exercise program on hip muscle strength and balance control in older adults [26]. Hip muscle strength had been measured using multimodal dynamometer and ground reaction force. It had been observed that they showed significant improvement in hip muscle strength.
Bernades et al. observed that training using simulations is good to implement emergency training in case of fire evacuation [27]. It had been seen that workers under a high stress situation may not follow taught behaviour. VR used for training had been proven more efficient.
Harrington et al. developed the world's first fully interactive medical education simulator using the Oculus VR platform [28]. They have demonstrated its effectiveness by conducting tests. This study provides a foundation for future medical development in this platform. Some critical and non-critical decisions and diagnoses were made among patients. These included appropriate completion of the initial examination, responding to vital examination data, and diagnosing lifethreatening conditions. This diagnostic function has been enhanced with patient examination animations that reflect pathology in a given time period.
Johnsen et al. observed that there is a need to reduce the high rates of childhood obesity in the developed world [29]. So a VR game is developed by the researchers where the children can play with a virtual pet and train it. The researchers say that there will be more levels to make the children play and tame the virtual pet which makes the children exercise more, reach different goals and get interacted with the pet. When a survey was done by the researchers, the outcome was positive and this method was liked by many children.
Tageldeen et al. stated that conventional rehabilitation led to a bad outcome because of inadequate patient involvement and participation [3]. Therefore, a VR-based game was developed and tested for arm rehabilitation using a wearable robotic exoskeleton. The author says that such games are correctly reconfigured and that the patient will be busy and that the patient will reduce the total cost of medical and health care.
Javia et al. stated that they used many types of simulators in the otolaryngology [30]. Otolaryngology, like other surgical disciplines, is increasingly relying on simulators to train its www.aetic.theiaer.org trainers. In generally simulators are two types such as physical simulators and VR simulators. Physical simulators are systems that can be interacted physically. Whereas, VR simulators is used to create virtual environment is modelled and designed in the form of games. These are used to specially treat the disorders in head and neck related to ENT.
Sucar et al. proposed gesture therapy using VR-based system for rehabilitation of upper limb [31]. The author says that in these virtual reality platforms that are created is capable of assisting rehabilitation. It promotes repetition of the exercise, task-oriented training, appropriate feedback and a motivating environment. Gesture therapy uses probabilistic decision models with Artificial Intelligence and specifically designed to guide therapy. It has a 3D monocular tracking system and an adaptation algorithm for optimizing game performance according to patient performance.
In [32], Bernardo described that virtual neurosurgical training continued to mold the surgeoncomputer relationship by reducing the learning curve and difficulties, conceptually developing the complex anatomy of the human body, and developing Visio spatial skills.
Wiederhold et al. reported that in the place of where narcotics and other medicines used to treat pain and anxiety, now VR technology-based games are developed and used [33]. This method is used to eliminate certain phobias and anxiety disorders, and post-traumatic stress disorder and also to rehabilitate patients with chronic pain or other weakness.
Yang et al. reported that wearable devices using electromyography (EMG) signals had been rapidly developed for medical use [7]. This system is based on VR, and EMG signals are obtained and pre-processing, the properties are subtracted; a mathematical approach using a support vector machine (SVM) is applied to classify movements. These movements are sent to the virtual reality environment for the task of rehabilitation training. It is used by the stroke patients. It occupies less space and a convenient and motivational approach.
Braga et al. stated that flight fear is a real problem affecting most people in this world [34]. The authors state that in this study, a VR application has been developed in the computer environment that encourages systematic exposure to stimuli that cause a significant increase in anxiety levels related to fear of flight. This is a very effective treatment for those who hesitate to fly.
Oliveira et al. stated that acrophobia is an extreme or irrational fear of heights [35]. VR is increasingly used as a therapeutic solution for different phobias. The author says that the users are exposed to two virtual environments, which have features to reproduce different levels of anxiety. In these environments, the user can navigate and interact with avatars and objects where each and every action of the patient is monitored by the therapist.
Adjorlu et al. reported that the developmental disorder, characterized by difficulties in social interaction and communication and limited and repetitive thinking and behaviour patterns, is called Autism Spectrum Disorder [36]. To overcome this, the author says that a VR simulation of a supermarket was created and evaluated to improve the shopping skills of people with this disorder. After research by researchers, the results show some benefits of DLS training using VR.
Pfandler et al. stated that VR based simulators offer many useful benefits in the evaluation and training of surgical skills [37]. The use of VR-based simulators is standard in some surgical subspecialties. However, their actual use in spinal surgery is still unclear. The author says at present only technical review is available for VR based spine surgery. The main goal of the surgery is to reduce and relieve the pain that is caused by nerve root pinching.
Thomsen et al. stated that cataract surgery is done to remove the lens inside your eyes that have become cloudy [38]. So it is removed and replaced with an artificial lens. Using VR, the author says that cataract surgical skills can be improved and mid-level surgeons have improved. It is purely used for learning and practicing purpose in a 3D environment.
Oikonomou et al. presented a study called "Body Project", on the development of an interactive VR "Artistic Game" in the human body [39]. The main problem of this Artistic Game is the artistic simulation of the life transformation process which aims to reconcile the contemporary participation of interactive works with the feeling that life is always going on after the death of the human body.
Moglia et al. evaluated the contributions of education in VR for robotic surgery by examining the literature [40]. The authors reviewed the literature for studies on virtual simulators for robotwww.aetic.theiaer.org assisted surgery. They have used various simulation methods for validity studies. No evidence has been found for the transfer of skills from simulation to clinical surgery in real patients.
Ghanouni et al. reported that methods of stroke rehabilitation usually involve the application or recovery of motor activities [41]. In their study, they applied a balance rehabilitation therapy on fourteen patients with paralysis using Wii or Kinect. The results showed that it is effective in increasing balance and balance confidence in both methods.
Garcia et al. stated that the inner ear defect (Vertigo), which is characterized by sensation attacks like in the world, is stopped in the ears (Tinnitus), hearing loss and fullness [42]. This is also called ear Meniere's disease. The authors state that, according to the case group, subjects participated in 12 rehabilitation sessions in a VR environment with a Balance Rehabilitation Unit (BRU). The results show that VR-based balance rehabilitation effectively increases the limit of dizziness, quality of life and stability of patients with this disease.
Brun et al. reported using a system with an upper extremity exoskeleton and VR game platform to evaluate individuals during rest and active movement in complex regional pain syndrome patients [43]. They developed a new, leak-proof avatar-based virtual feedback system integrated with the exoskeleton.
Mayie M.Y. Tsang et al. (2013) [44] had found that a mental disorder which is characterized by abnormal social behaviour and failure to understand what is real is called as Schizophrenia. The general study on the psychological and stroke of the human, based on traumatic events, such as overweight or obese, medical risk factor like high blood pressure, high cholesterol, diabetes etc [45]. The detailed review report based on an adaptive virtual reality games for therapeutic rehabilitation with respect to different ages of male and female participants [46]. This work deals with the game based treatment for psychological and stroke patients. It can indeed be a valuable aid for motor rehabilitation techniques [47]. In addition to the therapeutic rehabilitation, it describes about the finding the intersection of neuro-plasticity, stroke recovery, and learning methods [48]. This treatment is done with tele-rehabilitation and promoting execution of the exercise to the patient without coming to the rehabilitation centre [49].
The author stated that the purpose of this study about this disorder was to investigate the efficacy and effectiveness of VR as a cognitive intervention for enhancing vocational outcomes.

Proposed Methodology
This paper presents the exploration and implementation of human-computer interaction (HCI) concepts to enhance the experience of patients undergoing a physical rehabilitation. Using various motions capturing techniques and strategically designed games, physical rehabilitation can be made a pleasurable experience to the patients. To start with this research, we initially conducted a couple of field visits and interacted with the professionals in the field of physiotherapy and occupational therapy. It was highly evident that although the concept of virtual rehabilitation isn't new to researchers, there was very little implementation of the idea amongst the Indian medical professionals and practitioners. Biofeedback therapy trains the patient to control voluntarily the involuntary process of their body. This non-invasive and non-drug treatment is also used as a means to rehabilitate the physical impairments that may follow a stroke, a traumatic brain injury or even in neurological aspects within occupational therapy.
Our idea behind using immersive gaming as a tool for physical rehabilitation combines the idea of biofeedback and physical computing to get a patient emotionally involved in a game that requires them to do the exercises in order to interact with the game. The physical motions are captured by the wearable ultrasonic sensor attached temporarily to the various limbs of the patient. The data received from the sensors are then sent to the game using serial wireless communication.
There are two main aspects to this research: Motion capturing and Game design. The current status is a single ultrasonic detector. This game aimed towards addressing the basic treatment of a 'Frozen Shoulder'. www.aetic.theiaer.org This proposed system is consist of seven basic components such as Arduino microcontroller, IoT module, ultrasonic sensors, Wi-Fi module, Pressure sensors, unity 3D and VR headset. Each component is described in the sub section in detail.

Arduino Microcontroller
It has hardware infrastructure to add many environmental components. The inputs and outputs of the microcontroller are easily controllable. It is provided as open source in the software required for the control of these environmental components. In order to perform these applications, Arduino programming language based on cabling or Arduino Software (IDE) based on Processing is used. For example, a speed control of a motor can be performed, or warning systems can be developed using heat light sensors.

Internet of Things (IoT)
The Internet of things (IoT) is a network in which electronic devices, software, sensors, actuators and connections are placed. IoT allows objects to be detected remotely via sensors on the internet infrastructure or automatically controlled remotely. It provides the necessary services to make people's work easier. It allows the creation of computer-aided applications in the world and automatically enables them with less human intervention. It is naturally used in healthcare systems such as IoT applications, remote health monitoring or emergency notification systems.

Ultrasonic Sensor
Ultrasonic sensors are electronic component that detect the distance of an object with sound waves. Figure 2(a) shows the actual image of the sensor. These sensors detect the distance of the object according to the return times of the sound waves they send at a given frequency using transmitter and receiver as shown in Figure 2(b). This process is based on the calculation of the distance between the object and the sensor, measuring the time elapsed between the transmission and the return of the sound waves.
The distance can be calculated with the following formula: Distance (L) =1/2*T*C (1) Where L represents the distance, T represents the time between the emission and reception C denotes the sonic speed. (The value is multiplied by 1/2 because T is the time for go-and-return distance.)

Wi-Fi module (ESP8266)
The ESP8266 WiFi Module is an interface that enables a microcontroller to communicate over a WI-FI network using TCP / IP protocols. Figure 3 shows an image of the module. This module can perform Wi-Fi network functions with programs on any hardware or microcontroller.
This low-cost module has a sufficiently built-in processing and storage capability that allows it to be integrated into sensors and other application-specific devices with minimum preload and minimum load during operation with GPIOs.

Pressure sensor (BMP180))
As a pressure sensor, a digital barometric pressure sensor based on BMP-180 as shown in Figure 4 is used. This sensor combines barometric pressure, temperature and altitude.
It can also measure unbalanced pressure and temperature values. After being converted from the microcontroller when sending a start-up message to perform a pressure or temperature measurement, the pressure or temperature result values can be read through I2C, respectively.

Unity 3D
Unity is a cross-platform game engine developed by Unity Technologies, primarily used to develop three-dimensional and two-dimensional video games and simulations for computers, consoles and mobile devices. The Unity app is a complete 3D environment suitable for editing levels, creating menus, animating, writing scripts and editing projects. The user interface is fully organized and fully customizable by dragging and dropping panels.

VR Headset
The VR headset as shown in Figure 5 is a start-up screen that allows users to interact with simulated environments and experience first-person views. VR headphones are replaced by virtual reality content, such as the pre-recorded 360-degree VR environment, which allows the user to look around their own natural environment, film, game or physical as well as around them.

Design Framework
The framework of this proposed work is shown in Figure 6. This flow graph shows that the virtual reality game for physical rehabilitation has designed using Arduino-ESP8266 and ultrasonic sensor (pressure). It can be incorporated with Unity 3D Game engine through VR headset. Simultaneous direct connection with actual physical environment provided by IoT enables interactive rehabilitation tools to be developed. The performance of the patient has measured through the treatment and therapy process. If the patient shows the improvement in the VR game, then he/she will go for next higher levels or else start from first level.

Results and Discussions
The game based physical rehabilitation has designed using pressure sensor into Unity 3D Game engine as shown in Figure 7. In recent years, research in clinical practice has generally focused on a new form of interactive game as a complementary tool in rehabilitation. Virtual Reality (VR) is not only fun but will continue to drive in the real world of rehabilitation. Simultaneously, a direct connection with an actual physical environments provided by IoT enables www.aetic.theiaer.org interactive rehabilitation tools to be adopted. The effectiveness of these platforms developed by incorporating enhanced VR components. Thus, the efficiency of the treatment and therapy processes of the patients increases, but can also shorten these times. Thus, while the costs of diagnosis, treatment and therapy decrease, the quality of life of patients is to be improved.

Figure 6. Flow chart of the proposed work
ThinkSpeak is an IoT data analytics service that describes about to visualize, analyze and aggregate the data streams in the cloud. Figure 8 shows the analysis of game for physical rehabilitation using ThingSpeak graph 1 . Through that time and distance has been measured using the above graph shown in Table 1. Based on the suggestion from the physiotherapist, Iot based virtual reality game is proposed. During the first visit to the hospital, the doctor stated that people in the age group of between 50 and below can recover completely and those between 60 and 70 can partially recover. Strengthening exercise is mostly prescribe to people of age group 50 and below whereas movement related exercise is given to aged people (60 and above). www.aetic.theiaer.org In conventional therapy there are many types of exercise being performed in hospitals. Some of them are tera band exercise, lock and key therapy, dropping ball exercise, .etc. For frozen shoulder patients the first step of conventional therapy is applying wax and heat. There are five levels of movements observed in patients as shown in Table 2.   During the 2nd visit, the physiotherapist identified the symptoms of the patients and provides the serious gaming to them. In addition to the conventional therapy, the angle of movement of shoulder and range of movement plays a major role. Presently, improvement of patients is noticed based on emotions and grades given based on the performance. Stroke patients won't be able to lift or move their hands, so even for a small movement the patient should be able to play the game. The results obtained by having the patients playing this developed game at the rehabilitation process. During this test, their status of the mood has been changed with respect to the different age group as shown in table 3.
During the 3rd visit, the serious IoT based virtual reality game is assigned to the patients as shown in the Figure 7. This technology is used by the physiotherapist in the rehabilitation centre. They tested and evaluated perfectly for the intended purpose, the conduction of a pilot study in order to familiar with real use cases. The different age group, gender, time, duration and status is survived, after experience this virtual game. It shows really the improvement to the patients based on their performance.

Conclusions
The game for physical rehabilitation has been designed successfully using pressure sensor into Unity3D Game engine and the resultant output has proven to be useful for the physiotherapy department for treating the patient for the specific problem treatment which is "Frozen Shoulder exercise". The serious IoT based virtual reality game is assigned to the patients. This technology is used by the physiotherapist in the rehabilitation centre. They tested and evaluated perfectly for the intended purpose, the conduction of a pilot study in order to familiar with real use cases. The different age group, gender, time, duration and status are survived, after experience this virtual game. It shows really the improvement to the patients based on their performance. This work has to plan for implementing in real-time for rehabilitation of unskilled operator who disordered with some issue in their body movement as a future enhancement.