MedPlus applications

R EFERENCES

3. MedPlus applications

Observing what other IoT, sensors, and applications have been built by scientists with the purpose of properly tracking the health state of patients remotely, we came up with a system that does this and a few more. The proposed solution entails reading real-time data from a Google Fit application installed on a patient’s phone which is account- able for assembling data from the phone’s default health app or added through associated devices and sensors such as smartwatches, weight scales, and others. After all the information from Google Fit is collected and synchronized in our application, it becomes available for the patient and his doctor, allowing prompt medical access, treatments and diag- nosis when required. Its capabilities were designed to help everyone in times of need of health monitoring by including daily symptoms by manual input or vocal commands, keeping an eye on measurements of heath rate, steps, sleep, and weight, but also to engage all age categories to improve their lifestyle, all due to a friendly and full-responsive in- terface available on all devices. It is enhanced with modern features like dark mode and internationalization (support for both English and Romanian languages). More concretely, our solution consists of a fully functional web appli- cation split in front-end implemented using React.js [13], respectively back-end written in Node.js [14]. React.js is a Javascript library for building user interfaces and Node.js is an open-source, cross-platform, back-end JavaScript run-time environment. The proposed method of patient monitoring system gathers health data from a patient’s mobile phone and analyses it. Regarding user privacy, our solution uses OAuth [15] which is an open protocol to allow secure authorization in a simple and standard method from web, mobile, and desktop applications.

MedPlus has three types of users: patient, doctor, and admin. Each user registers with a valid Gmail account and he’s asked for permission to collect his data. At registration time, each patient is assigned to a doctor that will follow the patient’s progress and health condition. As for the doctors and admins, the enrolment is performed with an additional step. An invitation is being sent to their Gmail address containing a link with a uniquely generated JWT [16] authorization token, thereby ensuring that only authorized people can have those specific roles.

3.1. Data source

Unlike the mentioned approaches and many others, Medplus is able to collect a broad spectrum of information from multiple devices due to the Google Fit [17] app that can associate with a lot of mobile apps and wearable devices such as Bluetooth scales, smartwatches, fitness apps. This makes our app more efficient because it has a single source of truth, without needing separate sensors for each body part that we would like to monitor. The way our application is collecting data is through Google Fit Rest API [18], so the only requirement for a user to be able to use Medplus is having Google Fit installed on their phone. Google Fit is responsible for collecting the information from the phone’s default health app, but can also show you details grouped from many of your favorite apps and devices to give you a holistic view of your health, so you’ll never lose track of your progress. These include Wear OS by Google [19], Runkeeper [20], MyFitnessPal [21], Sleep as Android [22], Withings [23], Xiaomi Mi bands [24], and more, but in essence, it is su ffi cient for Android users to use the Google Fit to collect records from wearable devices, or their phone sensors, while iOS users can import their records from the Health into the Google Fit.

3.2. Collected Data

The Google Fit REST API enables its users to store and read health and wellness data in the fitness store from apps

on any platform. Google Fit app congregates the information from all the linked apps and stores it in the cloud in the

Google fitness store.

3754 Andra-Elena Gîştescu et al. / Procedia Computer Science 192 (2021) 3751–3760

4

Andra-Elena Gˆı¸stescu, Teodor Proca, Camelia-Maria, Milut¸, Adrian, Iftene/Procedia Computer Science 00 (2021) 000–000

As shown in Figure 1, using the Google Fit Rest API, our application can obtain, parse and show datasets consisting of heart rate, steps, sleep, and weight data points in a user-friendly interface. CronJobs [25] are useful for creating periodic and recurring tasks and in our case, such a cronJob was implemented in Node.js with the responsibility of performing health information synchronizations every hour. Every data sync works as follows: the cronjob makes a request through Google Fit Rest API and retrieves from Cloud the aggregated information from the user’s health apps.

In addition, an admin user can complete syncs manually in case of any system failures or downtime.

Fig. 1. Manual and automatic health data synchronization flows

From the Google fitness store, the information returned by the Rest API is grouped into sessions that represent a time interval with associated metadata and once this information reaches the system, the data is parsed and stored in a non-relational database - MongoDB. In Figure 2, there is such an example of an entry for the cronJob, as a result of measuring the number of steps through a smartwatch and collecting the score from the Google Fit app using the Rest API.

Fig. 2. Steps measurements collected through Google Fit Rest API

On a daily basis, an enrolled patient can give his daily status so that the doctor is aware of any changes in health.

MedPlus comes with a set of already predefined symptoms from which the patient can choose, but he can also add his own as described in Figure 3. Besides this, a patient can add at any time a symptom if he starts feeling sick either by writing his own symptoms and by choosing from the predefined symptoms, or by giving a voice command which will trigger the action of adding a symptom.

For the speech recognition part, our solution is based on a React hook [26] which gives component access to a

transcript of speech picked up from the user’s microphone and it is mainly targeting elderly and disabled people. In

Figure 4, we can see an example of such a command.

Andra-Elena Gîştescu et al. / Procedia Computer Science 192 (2021) 3751–3760 3755

Andra-Elena Gˆı¸stescu, Teodor Proca, Camelia-Maria, Milut¸, Adrian, Iftene/Procedia Computer Science 00 (2021) 000–000

5

Fig. 3. Daily status form for symptoms Fig. 4. Voice command for collecting symptoms

3.3. Digitization of medical records

Sometimes at the doctor’s request and not merely, patients may be required to upload documents like X-rays, scans, prescriptions, or old diagnoses. When it comes to difficult diagnoses or rare diseases, this is one of the most helpful capabilities because it allows the doctors to give an opinion and help the patient with specialized pieces of advice. A common use-case of this feature is when a patient is not satisfied with the results from a medical exam he took or maybe he has faith that a medical error took place and he’s asking for a second opinion from another doctor. Another case could be when a patient might have had a full medical history with di ff erent problems that require constant observation and he would like to store all his X-rays, past treatments, test results, and so on within the application.

By virtue of our solution, patients can benefit from such a feature and store their documents so that they can also be analyzed by the doctors. Our system o ff ers support for file uploading and downloading, this way patients are able to digitize their medical history. Something else worth mentioning is that only the proprietary of a document and his assigned doctor have access to it in our application.

Fig. 5. Storage of health records in AWS S3

3756 Andra-Elena Gîştescu et al. / Procedia Computer Science 192 (2021) 3751–3760

6

Andra-Elena Gˆı¸stescu, Teodor Proca, Camelia-Maria, Milut¸, Adrian, Iftene/Procedia Computer Science 00 (2021) 000–000

The component responsible for document uploading uses AWS S3 [27] due to which data availability, security, and performance are ensured in our system. S3 is secure by default and upon creation, only the resource owners have access to the Amazon S3 resources they create. Also, resources are redundantly stored on multiple devices across multiple facilities in an Amazon S3 Region. Additionally, in order to maintain and guarantee confidentiality and privacy of users’ data and secure it from unauthorized access, our solution makes use of the encryption features of S3.

A patient can easily upload all types of medical records within our system. In this regard, for better security, we use AWS S3 enhanced with server-side encryption to store the documents on the backend side as can be seen in Figure 5. The encryption algorithm used is AES-256. First of all, a plaintext data key is generated by S3 and used for encryption. The outcome is an encoded version of the document which is then stored within an S3 bucket. The same plaintext data key is afterward encrypted with a secret and unique S3 master key, resulting in an encrypted key also stored on S3 which has an association with the encrypted data. Finally, the plaintext key is removed from memory.

Once the storage procedure was successful, the patients and their assigned doctor can visualize their documents based on which presumptive diagnosis can be provided.

3.4. Data analysis

The analysis of a patient’s health data is performed automatically every 24 hours on all the health data of a patient collected during that day so that the rate of false positives anomalies that are being reported is as low as possible.

Before a manual analysis from a doctor’s side, our solution uses an AI service that helps to foresee problems before they occur, namely Anomaly Detector [28] from Microsoft Azure. Its responsibility is to ensure easy embed anomaly detection capabilities into our app so that patients and doctors can quickly identify problems. Time series anomalies can offer information relevant to critical situations, especially medical domains. Given that our application already collects time series data, the Anomaly Detector can next identify if the data contains any anomaly and gives back a time window when the anomaly happened and a score indicating how severe the anomaly is compared to others in the time series [29]. Its anomaly detection processor finds anomalies for each single time series, while a smart alert processor correlates the anomalies from di ff erent time series and generates an incident report accordingly. Concerning the privacy of analyzed data, the service does not store customer data.

With all the collected and analyzed data, we build a profile for each patient. This is what the doctor is going to evaluate and provide input such as diagnoses and treatments in order to improve the patient’s health state.

Fig. 6. Notification menu with different types of alerts

The notification system is used for informing a patient and a doctor in the application about two possible scenarios:

automatic sync of data meant to keep the health data up-to-date or detected anomalies. The detection service performs

regular analyses on the collected data and scans such that possible spikes don’t remain unobserved. When these are

Andra-Elena Gîştescu et al. / Procedia Computer Science 192 (2021) 3751–3760 3757 6

Andra-Elena Gˆı¸stescu, Teodor Proca, Camelia-Maria, Milut¸, Adrian, Iftene/Procedia Computer Science 00 (2021) 000–000

The component responsible for document uploading uses AWS S3 [27] due to which data availability, security, and performance are ensured in our system. S3 is secure by default and upon creation, only the resource owners have access to the Amazon S3 resources they create. Also, resources are redundantly stored on multiple devices across multiple facilities in an Amazon S3 Region. Additionally, in order to maintain and guarantee confidentiality and privacy of users’ data and secure it from unauthorized access, our solution makes use of the encryption features of S3.

A patient can easily upload all types of medical records within our system. In this regard, for better security, we use AWS S3 enhanced with server-side encryption to store the documents on the backend side as can be seen in Figure 5. The encryption algorithm used is AES-256. First of all, a plaintext data key is generated by S3 and used for encryption. The outcome is an encoded version of the document which is then stored within an S3 bucket. The same plaintext data key is afterward encrypted with a secret and unique S3 master key, resulting in an encrypted key also stored on S3 which has an association with the encrypted data. Finally, the plaintext key is removed from memory.

Once the storage procedure was successful, the patients and their assigned doctor can visualize their documents based on which presumptive diagnosis can be provided.

3.4. Data analysis

The analysis of a patient’s health data is performed automatically every 24 hours on all the health data of a patient collected during that day so that the rate of false positives anomalies that are being reported is as low as possible.

Before a manual analysis from a doctor’s side, our solution uses an AI service that helps to foresee problems before they occur, namely Anomaly Detector [28] from Microsoft Azure. Its responsibility is to ensure easy embed anomaly detection capabilities into our app so that patients and doctors can quickly identify problems. Time series anomalies can offer information relevant to critical situations, especially medical domains. Given that our application already collects time series data, the Anomaly Detector can next identify if the data contains any anomaly and gives back a time window when the anomaly happened and a score indicating how severe the anomaly is compared to others in the time series [29]. Its anomaly detection processor finds anomalies for each single time series, while a smart alert processor correlates the anomalies from di ff erent time series and generates an incident report accordingly. Concerning the privacy of analyzed data, the service does not store customer data.

With all the collected and analyzed data, we build a profile for each patient. This is what the doctor is going to evaluate and provide input such as diagnoses and treatments in order to improve the patient’s health state.

Fig. 6. Notification menu with different types of alerts

The notification system is used for informing a patient and a doctor in the application about two possible scenarios:

automatic sync of data meant to keep the health data up-to-date or detected anomalies. The detection service performs regular analyses on the collected data and scans such that possible spikes don’t remain unobserved. When these are

Andra-Elena Gˆı¸stescu, Teodor Proca, Camelia-Maria, Milut¸, Adrian, Iftene/Procedia Computer Science 00 (2021) 000–000

7

spotted, a notification (Figure 6) is created and the system sends one to the patient in question and one to his assigned doctor. Next, the doctor is alarmed by the received message and can provide support for the patient.

3.5. Patient profile

With all the collected and analyzed data, we build a profile for each patient as shown in Figure 7. This is what a doctor is going to evaluate and provide input such as diagnoses, treatments, or just simple pieces of advice in order to improve the patient’s health. So every user benefits from a patient profile assembled based on the collected data until the current day: steps, heart rate, weight, and sleep.

Fig. 7. Patient profile summarizing the collected and analyzed health data

The mentioned profile contains multiple charts and statistics showing the progress from the last year of a certain

patient, as well as the last alerts and spikes detected by the anomaly statistics. This profile is shared with the assigned

doctor as information based on which he’ll be able to provide a presumptive diagnosis and a possible treatment. Given

the fact that the entire profile actually contains the previous diagnoses and symptoms, together with uploaded files

(e.g. X-rays, prescriptions, scans) and metrics based on the health conditions, this profile could be perceived as a

medical history of the patient.