Emerging information technologies for DCHC

APJCP 18(10): 2775

3. Emerging information technologies for DCHC

Sustainability of DCHC demands advanced information technologies to perform data modelling and analytics. Emerging information tech- nologies such as VR, AR, AI, ML, DL, DT, and BD have proven effective for data modelling and predictive analysis in other industries [6,14,131,

135]. The reviewed studies also suggest that characteristics of these

technologies are compatible and have potential to meet digital health- care needs for sustainability. However, implementation of these tech- nologies only for data modelling and analytics is not enough to achieve impact. The presence of other advanced information technologies for data connectivity, data storage and security are also required (due to technological interdependency) within the ecosystem. This section provides an analysis of research on emerging information technologies for data modelling and analytics. It includes a brief conceptual back- ground of the explored technologies and recent studies with implica- tions for DCHC. The inner workings and characteristics of VR, AR, AI, ML and DL are beyond the scope of this review.

3.1. Virtual reality (VR) and augmented reality (AR)

VR can be defined as computer generated interactive simulations where a user can engage themselves in the environment which appears to feel similar to the real-world [15]. It is an advanced form of human

Table 1

Highlights of the reviewed VR and AR research.

Study

Reference Highlights

[22] •VR-based home system that uses a serious games approach for elderly patients.

•Applies ML models for remote activity observation and exercise recognition

•Using a virtual world, patients perform kinesiology exercises, participate in quizzes and provide feedback

•System collects patients’ exercise data and interprets it to deliver models of personalized care, construct a user profile, adjust the game’s difficulty level (by modifying the language of questions), evaluate the progress of daily exercises, and provide feedback on performance.

[23] •VR based assistive training tool for dysphonia rehabilitation.

•Uses normal microphone of a tablet computer and/or Kinect2 (motion sensing input device from Microsoft) for voice input.

•Deploys serious games as an attractive logic part.

•Enables user to conduct voice training (as per the therapist’s guideline) and play the game without any interference simultaneously.

•Provides extraction of recorded voice for the evaluation of long- term rehabilitation progress.

[24] •An augmented reality-based visualization tool to determine human motion analysis.

•The tool was developed based on personalized anatomical reconstruction of joint structures and optical motions from medical imaging.

•Provides situ visualization of joint movements for healthcare professionals.

[25] •Proposed an augmented reality-based visualization tool for joint movements in sports medicine and rehabilitation.

•The tool comprises a recording mechanism to examine the acquired movements and various motion related information, which can be utilized for patient progress on kinematics throughout the rehabilitation phase.

[26] •“HealthVoyager”, a VR technology to assist in knowledge transfer between patient and physician.

•Provides interface customization.

•Fully operational with smartphones or tablets.

•Illustrates personalized procedural and surgical findings along with images of normal anatomy.

[27] •“VR4Neuropain” a VR based system to monitor real-time elec- trophysiological data.

•Comprises three components: Platform, VR interface, and a Glove (GNeuroPathy).

•Enables physiotherapists, occupational therapists, and physicians to develop innovative and interactive intervention procedures.

[28] •Presented a novel, specialized lightweight tele-presence and tele manipulation system for caregivers.

•Allows the caregiver to employ a sensor system with a VR headset, to collect sense related data in real-time with minimal latency (250 ms, 30fps refresh rate max).

[29] •Developed an AR application which incorporates care plan tracker (created by the physicians) to support patients in day-to- day activities without any supervision.

•Displays daily tasks (exercises), medications, and records health data.

•Allows physicians access to patient information on a regular basis, without any schedule.

[30] •Evaluates the efficiency of an RGB-D sensor (built-in Microsoft Kinect v2) in gait analysis.

•Compared sample results (each parameters) with real measurements.

•Findings show that an RGB-D sensor is efficient in patients’ walk ability assessment and potential to improve the overall rehabili- tation process.

[31] •Developed 3 types of mini game contents to train strabismus patients.

•Incorporates VR space, accessible with head mounted display (HMD).

[32] •Developed a VR-based multi-disciplinary system for assisting elderly patients in homecare and other related services.

•The system was developed to serve stroke patients.

•Compatible with institution-based rehabilitation programs.

(continued on next page)

International Journal of Medical Informatics 149 (2021) 104420

4

computer interaction [16] and has been evolved to augment reality (AR) [17]. AR can be defined as a phenomenon where real world objects (in an acknowledged context) are amplified with additional graphical in- formation in order to assist the process of augmentation [18]. An AR system typically provides three basic features: combination of virtual and real-world, interaction in real-time, and precise 3D registration of real and virtual objects [19]. The overlaid sensory information of the system can be either constructive (additive to the real-world environ- ment) or destructive (masking the real-world environment) [20].

Through this, AR modifies an ongoing perception of real-world context, whereas virtual reality (VR) substitutes a user’s real-world context with a simulated one [21].

Several research findings on VR and AR are tabulated below (Table 1), addressing their use in DCHC. It is evident that VR technol- ogies assist elderly and perhaps less digitally literate patients to benefit from DCHC. VR rarely provides sufficient utility as a standalone tech- nology but complemented by AI and IoT devices in order that the pre- scriptive analytics generated are sound. AR is in its early stage in DCHC and shows potential to provide precision and higher accuracy in disease diagnosis and surgical procedures with the facility for surgeons to train and practice in simulated environments [18].

Table 2

Highlights of the reviewed AI, ML and DL research.

Study

Reference Highlights

[52] •Developed AI-based interactive system for virtual healthcare to assist patients with chronic diseases (e.g. metabolic syndrome).

•System assesses regular check-up results, evaluates potential risks and provides personalized healthcare services to the pa- tients (activity and dietary guidelines).

•Applies interactive procedures to collect patient feedback (via any IoT device, smartphone, tablet or PC) on vital signs and provides instant suggestions on health management.

•Can be operated through computer and smartphone.

[53] •“Medical Sieve” program, which aims to develop “cognitive assistant” with reasoning capabilities, reasoning analytics, and a wide range of clinical knowledge.

•The developed “cognitive assistant” is capable of conducting analysis on radiological images with expedient manner and precision.

[54] •“ConfidentCare”, an AI-based CDSS.

•System learns personalized screening policies from an EHR.

•Operated by computing data for clusters of patients with similar features.

•Uses iterative algorithm for risk-based clustering of the women’s feature space.

•Learns “best” screening procedure for each cluster by applying a supervised learning algorithm.

[55] •“PRISM”, an AI based digital health platform. Includes mobile health technologies and data visualization tools.

•Offers risk evaluation for eight types of cancers, five types of chronic conditions and two types of psychiatric disorders.

[56] •Designed and developed an AI-based analytics framework applying NLP and ML techniques to determine intelligent anal- ysis and automated aggregation of patients’ information and conversation routes in online support groups.

•Able to identify, extract and analyse patient behaviour, interaction aspects, demographics, emotions, decisions, and clinical factors.

[57] •An AI empowered, context mining based mental health model.

•Accesses users’ profiles from depression index, weather index, and personalized context information to provide depression index service.

•Uses Personalized context information for data modelling to deliver guidelines to construct data model of depression.

[58] •An application scheme, based on ML and similarity calculation algorithm for primary and home healthcare.

•Obtains physiological data from the patient’s movement and can provide medical services including doctor’s recommendations.

[59] •A ML based method to filter patients’ electrocardiograms (ECGs).

•The method uses a ML classifier to detect cardiac health risks and assess severity.

[60] •An IoT-based Human Activity Recognition (HAR) system to monitor vital signs of user remotely. The system uses ML algo- rithms to determine the accomplished activity within four pre-set categories (sit, walk, jog, and lie).

•Provides feedback on activities (both during and after performance).

•Provides remote visualization and customizable alarms.

[61] •A machine learning based personalized healthcare system to assist diabetic patients in self-management of their chronic condition.

•Uses BLE (Bluetooth Low Energy) sensor device to collect vital sign data like heart rate, blood pressure, blood glucose, weight.

•Provides real time data processing (using Apache Kafka (an open- source stream-processing software) and MongoDB (a cross- platform document-oriented database program) for data storage)).

[62] •“DAIS”, a Data-analytical Information System which generates predictions on future BMI (Body Mass Index) changes before commencing any therapy.

•The system considers existing parameters (age, heart rate) through standard exercises.

•Uses ML models to analyse the collected physical data along with user’s RCT (randomized clinical trial) data and determines prediction on change.

[63] •“VASelfCare”, a prototype to assist in self-care of aged patients with type 2 diabetes mellitus.

(continued on next page) Table 1 (continued)

Study

Reference Highlights

•Capable of serving patients with dementia, Parkinson’s Disease (PD) or any other chronic disease.

•Capable of supporting community and telehealth services for elderly patients upon commercialization.

[33] •Incorporates sensing technology (eye tracking) with a VR simulator to enhance the performance of clinical decision support systems (CDSS) for mental disorder diagnosis and assessment.

•Developed VR modules based on a traditional patient simulator (scenario based).

•Uses data from eye tracking sensing technology to construct analytical models for predicting mental illness risks.

•Applies AI tools in VR-based healthcare training to accelerate learning and decision making.

[34] •Proposed a mobile app with CDSS to assist in-field operators in bedsore measurement, status classification, trace evaluation (with time), and decision-making regarding treatment.

•The mobile app is based on AR, supported by DL.

•Applies AR to develop an established environment (with support of operators’ cognitive process) for decision-making.

[35] •Developed a method and a platform for cardiac care.

•The platform uses 3D and 4D medical imaging data (Cardiac 3D CT or 4D Echocardiography data).

•The application includes an AR/mixed reality module. It visualizes 3D Cardiac CT and 4D Echocardiography data on AR and mixed reality devices (AR/VR glasses).

•Supports medical training and education with AR/VR environment.

[36] •“HoloPrognosis”, an AR-based serious game for mitigating the upper limbs motor impairment of early-stage PD patients.

•The game encourages patients to perform large amplitude movements gradually.

•Tracks patient performance and collects hand movement data for further analysis.

[37] •Developed an AR-based serious game for the rehabilitation of PD patients.

•The game allows PD patients to ambulate while following virtual cues in the real-world environment.

•The game incorporates multiple stages with increasing difficulty level.

[38] •Determines an AR-based simulation test with Body Integrity Identity Disorder (BIID) patients.

•The simulator virtually amputates participants’ alienated limbs, allowing them to feel their ideal selves.

•Test results suggest that AR has potential to improve existing BIID related therapeutics and diagnostics.

•Warrants further consideration for AR in real-world clinical settings.

A. Zahid et al.

3.2. Artificial intelligence (AI), machine learning (ML), and deep learning (DL)

AI, as a branch of computer science [39], aims to recognize the essence of human intelligence and to develop a new intelligence ma- chine which is able to communicate and respond similarly to human intelligence [40]. Research in this field includes image recognition, language recognition, natural language processing (NLP), robotics, and expert systems [41]. AI was first introduced in 1956 at the Machine Stimulated Intelligence (MAI) conference [41].

ML is a subset of AI which systematically uses algorithms to syn- thesize the fundamental relationship between data and information [42]. Distinct from AI in general, ML uses an adaptive approach (e.g.

neural network) to train on new data without any explicit programming of new rules (usual requirement of other Al algorithms like expert sys- tems) [43,44]. DL, in turn, is a specific subset of ML [45]. It learns from representative data with multiple abstraction levels using computational models that consist of multiple processing layers [46]. DL is able to explore complex structures in massive datasets by applying a back- propagation algorithm, which defines how a machine should optimize its inner parameters, deployed to compute the representative changes from layer to layer.

AI is among the most influential technological innovations shaping the healthcare system [47]. Its adoption by the industry is gradually increasing [48,40,49

–51]. ML is also widely adopted to develop com-

puter models and learning strategies for predictive analysis. It comprises the potential of exploring patterns and values; which empowers

Table 2 (continued)

Study

Reference Highlights

•The ML and AI techniques adopted by the prototype are found efficient in providing virtual assistance to patients in accordance with their emotions and preferences.

[64] •Proposed a real-time analytics approach based on seamlessly collected sensory data to monitor patients’ vital signs.

•Also developed a ML model to determine predictions based on the collected physiological data.

•Provides notification to patient if any symptom of risk is observed for cardiovascular disease.

[65] •Introduced project “Medic” which utilizes fuzzy logic, NLP and DL to ensure seamlessly evolving knowledge for accurate disease diagnosis.

•Delivers various services to physicians for decision making regarding treatments.

[66] •Proposed a DL based (end to end) medical diagnosis system (DL- MDS).

•Aims to deliver disease diagnosis to verified users.

[67] •Proposed “DeepReco”, an intelligent health recommender system (HRS) based on Restricted Boltzmann Machine-Convolutional Neural Network (RBM-CNN, a DL method).

•Provides an insight into Big Data’s implementation for an effective HRS engine.

[68] •Introduced a DL based wellbeing recognition system to deliver a non-invasive monitoring system.

•The system categorises wellbeing levels applying three features, which encompassed mood and stress.

•instructed to execute both personalized classification and genetic classification. The personalized approach is considered to deliver a personalized health DSS, which will increase awareness among users and enhance their behaviour towards better wellbeing.

•Provides real-time visualization of user’s behaviour along with the recognized well-being.

[69] •Proposed “ARA”, an Accurate, Reliable, and Active image classification framework and introduced a DL based Bayesian Convolutional Neural Network (ARA-CNN) to categorize histopathological visuals of colorectal cancer.

•The model produces exceptional classification accuracy in comparison with other existing models.

•The outputs from the network show uncertain measurements for every tested visual. It helps to identify mislabelled samples.

Furthermore, it can be deployed in an active learning workflow.

•The model also segments the whole-slide visuals of colorectal tissue and determines spatial statistics based on the segmentation.

[70] •Presented a deep CNN (convolutional neural network) and body- sensor-based robust activity recognition approach for smart healthcare.

•It analyses sensory data collected from body sensors (e.g.

accelerometer, magnetometer, ECG, gyroscope sensors).

•Extracts the salient features from the collected data and uses it to train the deep activity CNN (based on Z-score normalization and Gaussian kernel-based principal component analysis).

•The trained deep CNN is utilized to identify activities in testing data.

[71] •Ensemble DL and feature fusion based smart healthcare system for heart disease prediction.

•Applies feature fusion method to combine data (derived from sensors and EMR) and deliver valuable healthcare data.

•Applies information gait technique to: eliminate redundant, irrelevant features; select the important features; and improve system performance.

•Applies conditional probability approach to compute specific feature weight for every class and enhance overall system efficiency.

[72] •Developed a DL based pain intensity detection framework.

•Able to detect pain intensity by analysing facial extraction (image).

[127] •Proposed a model to incorporate ML techniques for abandonment prediction in pulmonary rehabilitation.

•Aims to detect abandonment trends of patients who are engaged in the treatment and to abstract knowledge which contributes to the application of pulmonary rehabilitation care plan.

•Developed a tool to integrate the proposed model and provide data visualization for the healthcare providers.

[73]

Table 2 (continued) Study

Reference Highlights

•“HiTANet” (Hierarchical Time-Aware Attention Networks), a DL based risk prediction model.

•Replicates the decision making process of doctors upon risk prediction.

•Models time information in 2 stages (local and global).

•Local evaluation stage includes time aware transformer to embed time information into visit-level embedding and executing local attention weight for every visit.

•Global synthesis stage adopts time-aware key-query attention mechanism for allocating global weights to different time steps.

[74] •Proposed a DL based model to predict complications of Type 2 Diabetes Mellitus.

•For predicting complications, the model follows feature extraction, pre-training, data collection, validation process, Deep Belief Network (DBN) and classification steps.

[75] •Proposed a ML based analytical framework for predicting hospital readmission.

•Initiates data driven approaches by utilizing inpatient administrative data, collected from a nationwide healthcare dataset.

•Developed a joint ensemble-learning model (validated) to combine the customized weight boosting algorithm with stacking algorithm.

•Applies ensemble learning in association with the customized weight boosting algorithm to tackle class imbalance problems and enhance predictability.

•Provides misclassification costs through different weight settings for every class in the course of model training.

[76] •Proposed a risk prediction model based on administrative data and ML techniques to determine risk of cardiovascular diseases for type 2 diabetes patients.

•The model was tested with a real-world dataset, contributed by two patient cohorts from Australia (patients with both type 2 diabetes and cardiovascular diseases, and patients with type 2 diabetes only).

[77] •Proposed a smart predictive healthcare framework for patients with chronic disease in homecare.

•Applies DL model to predict patients’ health status, delivering recommendations and assistance remotely.

•Uses BD to obtain vital signs, context data, symptoms, and medication information of patients.

International Journal of Medical Informatics 149 (2021) 104420

6

engineers, researchers, data scientists and analysts to explore covert insights for delivering trustworthy decisions and results. In recent times, DL has been dramatically advanced, providing visual object recognition, speech recognition, object identification, drug discovery and genomics.

Deep convolutional networks have made a breakthrough in processing speech, image, audio and video, where recurrent networks have excelled in sequential data (speech, text) [46]. Recent research of AI, ML, DL in DCHC are tabulated below in Table 2.

3.3. Digital twin (DT)

In healthcare, DT is a newly introduced approach for healthcare tracking and monitoring [78]. It is able to provide rapid, efficient and accurate healthcare service [78] using multi-physics, multi-science and multi-scale models [79]. The generic concept of DT was firstly proposed in 2003 [80]. In a typical use-case, DT consists of three parts: physical object, virtual object, and health data [78]. The physical object includes a patient, a wearable device, a medical device, an external factor (government policy, social behaviour, weather which may have an effect on a patient

’

s health) or a system containing all (or several) of the mentioned objects. In a healthcare application, the virtual object in- cludes digital person model, wearable device model, medical device model, digital system model or external factor model. Health data in- cludes real-time monitoring data (derived from wearable devices), detection data (derived from external systems or medical devices), medical records and historical data (derived from healthcare providers), simulation data (derived from digital models), and service data (derived from service systems or platforms which connect the virtual and physical spaces). Based on the given description, the following characteristics of DT are identified by Liu et al. [78]:

Table 4

Highlights of the reviewed BD research.

Study

Reference Highlights

[98] •Proposed a systematic data-processing pipeline framework for big healthcare data.

•Offers data capturing, storing, analysing, searching, sharing, and decision support.

•It collects genomic, behaviour, public health and EHR data.

•Uses ML algorithms and feature selection to perform analysis.

[99] •Introduced a smart health framework which provides data analytics for smart healthcare applications. It includes big data, cloud computing and integrated sensor technologies.

•Collects patient data from various resources (e.g. radiology and laboratory information systems, hospital information systems).

•The framework applies pattern matching and predictive modelling techniques to provide a data analytics service.

[100] •Introduced a data analytics and visualization framework for health shock-prediction.

•The system collects data from various socio-economic, geographic and cultural contexts.

•Uses Amazon Web Services (AWS) (a cloud computing service), combined with geographical information system (GIS) for big data collection, storage, indexing and visualization.

•Applies predictive modelling with summarized fuzzy rules to perform data analysis.

[101] •Proposed a cloud-based Big Data analytics framework for voice pathology assessment (VPA).

•The system uses MPEG-7 low-level audio and intertwined de- rivative patterns to process speech or voice signals.

•Applies a Gaussian mixture model as classifier and ML algorithms as support to a vector machine (an extreme learning machine).

[102] •Introduced “Health-CPS”, a cloud and Big Data analytics-assisted cyber-physical system to provide patient centric healthcare ser- vices and applications.

•The system comprises the followings to offer different smart healthcare services and applications:

o A unified data collection layer (for integrating public medical resources and personal health devices).

o A cloud-enabled and data-driven platform (for data storage and analysis of heterogeneous healthcare data from multisource).

o A unified API (for developers).

o A unified interface (for users).

[103] •Proposed “T-CPS”, a Big Data analytics-assisted and cloud ori- ented energy-aware cyber-physical therapy system.

•Includes smart devices and smart things in both the cyber and physical world to determine therapy sensing.

•Applies multimodal sensing for therapy sensing, annotation, visualization, therapy playback, and energy efficiency.

•Synchronizes multimedia data (audio, video) with the energy measurement sensors and the 3D depth therapeutic kinematic data.

[104] •Proposed a big data enabled smart healthcare system framework (BSHSF).

•The framework provides smart healthcare services by provisioning a service-oriented infrastructure.

•Collects biometric data, diagnosis reporting, surveillance data, social media data and EHR data.

•Combines a mechanism for healthcare knowledge discovery with smart service infrastructure.

[105] •Introduced “PopHR” (Population Health Record), a semantic web application which automates the extraction and integration of massive heterogeneous data (Big Data) derived from numerous distributed sources (survey responses, clinical records, and administrative data).

•The platform aims to support in monitoring of population health, providing measurement of system performance, and improving decision making in planning, deployment, and assessment of population health and system interventions.

[106] •Proposed a SLA (service-level agreement) based big data analytics and computing model.

•The model uses a Parallel Semi-Naive Bayes (PSNB) based probabilistic method for processing healthcare Big Data that is stored in the cloud to determine predictions on future health conditions.

•The model applied a Modified Conjunctive Attribute (MCA) algorithm to enhance the accuracy of the PSNB method.

(continued on next page) Table 3

Highlights of the reviewed DT research.

Study

Reference Highlights

[78] •Proposed “CloudDTH”, a cloud-based digital twin healthcare system to monitor, diagnose and predict different health aspects of an individual.

•Includes different wearable devices to collect data on an individual.

•Aims to provide personalized health management for individuals, especially for aged people.

•A generalized, novel, and extensible framework in the cloud.

[81] •Proposed “Cardio Twin”, an architecture for IHD (Ischemic Heart Disease) detection.

•The architecture classifies the myocardial and non-myocardial conditions with a convolutional neural network (CCN).

[82] •Presented a prospective DT concept for personalized healthcare (in developing process).

•The prospective model introduces three feedback loops for contextual monitoring, adaptive care management, and adaptive behavioural models.

[83] •Proposed DT frameworks (Better Community Healthcare, Intelligent Control and Emergency Planning in Hospitals, Strategic Planning of Hospital Services) for precision healthcare.

[84] •Proposed a clinical information integration architecture in the form of DT.

•Simulates patients with lung cancer under treatment.

[85] •Developed a DT model based on Finite Element Analysis (a simulation method).

•Uses ML algorithm for classifying categories of heart disease.

•Analyses IoT and sensor enabled heart pacemaker’s data to generate virtual instance using simulator.

[86] •Proposed and implemented an intelligent context-aware health- care system based on DT framework.

•The framework is found effective in improving healthcare operations.

•Incorporates ECG heart rhythms classifier model, developed by using ML.

A. Zahid et al.