Physics and Technology in Medicine Vol. 3 No. 1 (June) 2022, 14 - 22 http://myjms.mohe.gov.my/index.php/ptm
Physics and Technology in Medicine ISSN: 2710-7221, e-ISSN: 2710-723X
Original Article
A Preliminary Study: Chatbot Development As A Virtual Personal Assistant For Mobile Learning In Diagnostic Imaging Course
1
A
ZLAN,N
.A
., 1R
USLI,N
.A
.F
., 1,2,*M
OHAMAD,M
., 1,2M
OHAMADN
ASSIR,K
.&
1,2M
OHAMEDS
HARIF,N
.ABSTRACT
Although the quantity of resources available to students has expanded dramatically, the use of chatbots in diagnostic imaging courses remains limited. This study aims to develop an artificial intelligence chatbot called Dibot that can assist students by answering queries about diagnostic imaging studies. Dibot was implemented on the Snatchbot platform and later deployed on a Telegram channel and Facebook Messenger. This study was carried out in three phases. Phase 1 is the chatbot's development and validation. Phase 2 evaluates the student's perspectives on the chatbot, and phase 3 determines the impact of the chatbot on the students' knowledge. Data analysis used was mean, standard deviation, and ANCOVA. Experts gave a high overall mean ± S. D, 4.33 ± 0.94, with the recommendation of adding more content and interactive learning. The highest mean for students' perception of usefulness is 4.74. The post-test result showed that chatbot users have a higher mean score (20.71 ± 3.17) compared to the control group (p<0.05). Dibot is helpful in assisting students and can be used as an online learning tool to review diagnostic imaging modalities, including general x- ray, computed tomography (CT), and magnetic resonance imaging (MRI).
Keywords: Chatbot, digital learning tool, diagnostic imaging
1Diagnostic Imaging and Radiotherapy Programme, Faculty of Health Science Universiti Kebangsaan Malaysia, Jalan Raja Muda Abdul Aziz, 50300 Wilayah Persekutuan Kuala Lumpur, Malaysia
2Center for Diagnostic, Therapeutic and Investigative Studies, Faculty of Health Science Universiti Kebangsaan Malaysia, Jalan Raja Muda Abdul Aziz, 50300 Wilayah Persekutuan Kuala Lumpur, Malaysia
*Corresponding Author: Mohamad, M.
Email: [email protected] Tel: 03-92897293
Fax: 03-26914304
Received: 28 April 2022 Revised: 30 July 2022
Accepted for publication: 25 August 2022
Publisher: Malaysian Association of Medical Physics (MAMP) http://www.mamp.org.my/
https://www.facebook.com/MedicalPhysicsMalaysia
Copyright © 2022 Malaysian Association of Medical Physics.
All rights reserved.
INTRODUCTION
The number of educational resources available to students has increased significantly over the previous years (Wynter et al.
2019). Students increasingly use mobile technology and internet applications to support standard learning materials such as lectures, reference books, and tutorials (Wynter et al.
2019). Chatbots are now widely used in a wide range of applications, particularly in systems that provide intelligence support to users (Brandtzaeg et al. 2017). In today’s world, chatbots are extensively employed in various industries, including business, health care, and education (Adamopoulou et al. 2020). A chatbot is an artificial intelligence (AI) software or a conversational agent that can mimic human speech (Adibe et al. 2016). The existence of a chatbot provides the user with great flexibility. As a result, users can ask a question and have the answer at any time and anywhere.
Furthermore, humans are isolated in a home due to a Covid-19 outbreak. Thus, the usage of chatbots may help a human communicate without meeting each other.
Chatbot has also been implemented in education (Colace et al. 2018). Chatbot technology is a notable e-learning advancement, and in fact, it has been shown to be the most inventive option for finding the balance between technology and education (Clarizia et al. 2018). Chatbots provide a learning experience similar to the interactions made between students and teachers. The system can provide plenty of educational benefits by offering a question-answering function
Phys. Technol. Med.
and exchanging knowledge among the students (Palasundram et al. 2019).Den Harder et al. (2016) stated that the implementation of radiology e-learning in undergraduate medical education could help students gain better understanding of radiography.
Until now, the implementation of chatbots in education has been rather scarce. Although there are several chatbots developed to support e-learning such as Termbot which help students improve their learning of medical terminology (Hsu
& Yum 2021) and Marsh (2018) that focused on helping students name labeled anatomical structures on various radiological studies. However, literature research has not yet found chatbot specific for radiography procedures. In this study, we propose “Dibot”, which acts as a conversation agent that can help the students answer their questions and assist their theoretical and practical studies. This includes the learning areas of technical factors, patient positioning, and patient preparation of diagnostic imaging procedures. The usage of this chatbot is to ease students’ studies as it does not require students to go through pages of reference books to find an answer if they have a question. Furthermore, this chatbot also provides a link that connects to correspond diagnostic procedure video as a reference. This conversational agent should also be able to save students' time providing answers regarding the learning areas mentioned above in a short amount of time.
The objectives of this study are 1) to develop a chatbot that can help students by answering their questions regarding diagnostic imaging studies, especially in technical factors, patient positioning and patient preparation, 2) to measure the effectiveness of the chatbot from student perspectives and 3) determine the impact of the chatbot on student's knowledge.
EXPERIMENTAL METHODS
This study consists of three phases. Phase 1 is the chatbot's development and validation by two experts in the diagnostic imaging course. Phase 2 evaluates the student's perspectives on the chatbot, and phase 3 determines the impact of the chatbot on the students' knowledge. This study was approved by the Medical Research and Ethics Communities of Universiti Kebangsaan Malaysia (Ref No: UKM PPI/111/8/JEP-2021-857).
The feature of the chatbot covered the modalities within the diagnostic imaging course, the protocol of each diagnostic procedure, patient preparation and the procedure steps of each protocol. Table 1 below shows the list of modalities made as to content in the chatbot. The contents included are checked by experts before being transferred into the chatbot. The protocols are listed as shown in Table 2.
TABLE 1 List of subjects
No Subject
1 General X-ray
2 Computed Tomography (CT)
3 Magnetic Resonance Imaging (MRI)
TABLE 2 List of protocols
Subjects Part/region Protocol name
General X-Ray Chest PA Chest
Lateral Chest Oblique Chest
Abdomen AP Abdomen
Kub
Lateral Decubitus Abdomen Pelvis AP Pelvis
AP Frog-Leg AP Hip Unilateral Spine AP Cervical Open Mouth
AP Axial Cervical Lateral Cervical AP Thoracic Lateral Thoracic
Phys. Technol. Med.
AP Lumbar Lateral Lumbar AP Axial Sacrum AP Axial Coccyx Lateral Sacrum Coccyx AP Thoracolumbar Lateral Thoracolumbar AP Lumbosacral Lateral Lumbosacral
Cranium AP Towne
Lateral Skull Pa Caldwell SMV
Reverse Towne Facial Lateral Facial
Facial Waters Method Facial Caldwell Method AP Axial Zygomatic Arch Orbit Caldwell Method Orbit Waters Method Pa Axial Mandible AP Axial Mandible AP Axial TMJ Schuller
Computed tomography (CT) Brain CT Brain Plain
CT Brain with contrast
Thorax HRCT Thorax
CT Thorax with contrast Abdomen and pelvic CT 4 Phase
CT Urogram
MSK CT Knee
Cardiac CT Cardiac Magnetic Resonance Imaging (MRI) Head MRI Brain
MRI IAC MRI Sella
Spine MRI Cervical
MRI Thoracic MRI Lumbar
MSK MRI Shoulder
MRI Knee
Dibot was developed using the Snatchbot website. In Snatchbot, new interaction is added to create a welcome message and every reply sent by the users. Bot statement is used to write a welcome message and responses. Responses will include modalities under the Pengimejan Diagnostik dan Radioterapi (PDR) course, part of imaging and the protocols.
In the Bot Message section, welcome messages and responses are written as shown in Figure 1. Adding a connection will enable quick replies buttons to ease the user using the chatbot.
The user needs to tap the quick replies buttons to receive a response as shown in Figure 2.
The user interface acts as a medium to publish the chatbot. Dibot’s user interfaces are Facebook Messenger and Telegram applications. These apps are widely used in
Malaysia. They offer several benefits, including simple installation through Google Playstore for the android users or Appstore for iPhone users and accessible instant communication through text and voice messaging.
Furthermore, because students are familiar with the Facebook Messenger (Kumar et al. 2020) and Telegram interfaces, they will have no trouble conversing with our chatbot over instant messaging and gaining the necessary topic knowledge. The chatbot can be accessed through Facebook Messenger and Telegram applications for sending and receiving messages.
The developed chatbot was validated by two experts to determine its applicability and make suggestions for development and some modifications were made to improve students' understanding and provide better-personalised
Phys. Technol. Med.
learning help. The criteria used to determine our Dibot evaluators are necessary to have working experience in the diagnostic imaging field of at least 3 years. A structured interview in the form of a questionnaire has been done online.
Experts completed the questionnaire as an evaluation of the chatbot's suitability. The assessed aspects are divided into
three parts: Part A is knowledge content, Part B is the content organisation, and Part C is the chatbot application (Vanichvasin 2021). There were 12 items on a 5-point Likert scale (1=extremely low, 5=extremely high) with one open- ended question. This questionnaire was used to get expert comments on the chatbot’s applicability.
FIGURE 1 Welcome message from the chatbot
FIGURE 2 Response received from the chatbot
Phys. Technol. Med.
In this study, researchers will ask respondents to fill out a questionnaire that will be distributed through email and social media. Respondents will be asked to fill out a questionnaire in the form of a google form. For the second phase, the type of sampling was convenience sampling. The target population is fourth-year undergraduate students in PDR at Universiti Kebangsaan Malaysia (UKM) Kampus Kuala Lumpur. A census sample method was used for this study because the population of fourth-year undergraduate PDR students at UKM Kuala Lumpur is 28. The inclusion criteria for the participants of this phase are the fourth-year undergraduate students of PDR in UKM Kampus Kuala Lumpur, aged between 21 to 25 years old, and the previous education level includes matriculation, STPM or diploma. A questionnaire was constructed based on Vanichvasin (2021). Throughout this study, fourth-year PDR students evaluated the chatbot performance in answering their queries by answering a questionnaire that was given to them. There were 15 questions in 3 different parts (Part A until Part C). A= student interaction, B= perceived ease of use, C= perceived usefulness and one open-ended question. Five questions were asked in each part. The questionnaire uses 5-point Likert scale (1=strongly disagree and 5=strongly agree) as a score.
The target population for the third phase is third-year undergraduate students of the PDR program. Before starting the chatbot, students were given pre-tests. They were asked to use the chatbot for ten weeks. At the end of the semester (after 10 weeks), a post-test was given to the students. Inclusion criteria for this phase participants are the third-year undergraduate students of PDR in UKM Kampus Kuala Lumpur, aged between 20 to 25 years old, and the previous education level includes matriculation or STPM. The exclusion criteria are the previous education level is diploma.
The questions were created based on Dibot's information. The same set of questions were used as a pre-test and post-test.
The sample population were divided into the chatbot group (Group A) and the control group (Group B). This phase aimed to investigate the impact of the chatbot on diagnostic imaging students’ knowledge. Students cannot discuss or refer to any references during the test, including the chatbot. The data analysis performed included descriptive statistics and Analysis of covariance (ANCOVA) using SPSS V.27. Descriptive statistics were used to analyze the mean of student interaction, perceived ease of use and perceived usefulness while ANCOVA was used to investigate the differences between the students' research knowledge of the two groups.
RESULT AND DISCUSSION
The results for phase 1 are shown in Table 3. Table 3 illustrated that the validators agreed that the chatbot could be applied as a material for learning online with the mean of 4.33 (S.D.= 0.94). The validator stated that this chatbot is a good innovation in medical imaging education and the content given to help students in learning. However, a few information needs to be added such as the guideline to use chatbot, the topogram images for CT planning, and the content source need to be shown in the chatbot. Since this chatbot will be used for PDR student specifically, it is more useful to address the topics covered along with the course code based on the faculty’s proforma. The content of the chatbot has been refined and added according to the suggestions before moving to the next phase.
TABLE 3 Mean and standard deviation of the chatbot applicability
Chatbot applicability Mean S.D. Interpretation Part A: Content of knowledge
1. Content is related to diagnostic imaging knowledge 5.00 0.00 Strongly agree 2. Content covers necessary diagnostic imaging
knowledge
4.00 0.00 Agree 3. Content is suitable for students 5.00 0.00 Strongly agree 4. Content is easy to understand 5.00 0.00 Strongly agree Part B: Organization of Content
5. Organisation of content is typing through text messages
4.50 0.71 Strongly agree 6. Organisation of content is interesting 3.00 1.41 Neutral 7. Organisation of content is interactive 3.50 2.12 Agree Part C: Application of Chatbot
8. Chatbot is easy to use 4.50 0.71 Strongly agree
9. Chatbot can be used at anytime and anywhere 5.00 0.00 Strongly agree 10. Chatbot provides diagnostic imaging knowledge
students need
4.00 1.41 Agree 11. Chatbot provides information in a short time 4.50 0.71 Strongly agree 12. Chatbot provides correct answers 4.00 1.41 Agree
Total 4.33 0.94 Agree
Phys. Technol. Med.
Based on the validators’ questionnaire survey, we identified a few components of the Dibot that get the highest positive responses. The component that got the highest positive response from the validators is Part A: Content of Knowledge. From this result, we can conclude that the content of our chatbot has relevant and relatable information that can help and ease the diagnostic imaging students in their studies.
Chatbots can help students get more personalised information such as topics covered, assignments, tasks, assessments, and assistance (Okonkwo et al. 2021). The low-cost, quick, and easy educational technologies created play a significant role in providing and assessing educational material, spanning a wide range of topics using multimedia content. Accurate contents are essential in developing a chatbot as it will be used to deliver answers for the user. In a prior study, scripted answers were generated and put into the BOTCURATIVO prototype. It received positive feedback as it provided the correct and reliable solution to theispssr query (Roque et al. 2021).
Another component that gets a positive response is Part C:
Application of Chatbot, in which the Dibot can be used anytime and anywhere. In a recent systematic review (Okonkwo et al. 2021), the chatbot has been proved to be a conversational agent that immediately responds to the user.
The main findings from the previous study stated that immediate, content-related, and high-quality interaction could be beneficial through a chatbot. Although the overall response is highly positive, we received a total mean below 4.00 for points 6 and 7 in Part B: Organization of Content (3.00 and 3.50, respectively). This may be due to the specific content of the particular section, which needs improvement compared to the other section. In terms of chatbot usability, it is advisable to provide a simple guideline on using the Dibot. As for the content, the information resource needs to be added, so the user is more confident of its validity. As the MRI section had provided the planning images, it is likable to add topogram images in the CT section. This improvement had been executed before the chatbot was given to the students.
TABLE 4 Summary of student's perspectives on chatbot (Dibot)
No. Question n (%)
SA A N D SD
Part A: Student’s interaction
1. The chatbot shortens the time in waiting for responses
20 (71.4) 7 (25) 1 (3.6) 0 (0) 0 (0)
2. The chatbot provides instant responses after typing texts
23 (82.1) 5 (17.9) 0 (0) 0 (0) 0 (0)
3. The chatbot can be used via any device 0 (0) 28 (100) 0 (0) 0 (0) 0 (0)
4. The chatbot is easily accessible whenever needed
23 (82.1) 5 (17.9) 0 (0) 0 (0) 0 (0)
5. The chatbot interacts through text messages 20 (71.4) 6 (21.4) 2 (7.1) 0 (0) 0 (0) Part B: The perceived ease of use
6. There is no need to download any application 19 (82.1) 5 (17.9) 3 (10.7) 1 (3.8) 0 (0) 7. The chatbot is easily accessible whenever needed 19 (67.9) 9 (32.1) 0 (0) 0 (0) 0 (0) 8. The chatbot shortens the time in searching 23 (82.1) 5 (17.9) 0 (0) 0 (0) 0 (0) 9. The chatbot gives correct answers searched 22 (78.9) 5 (17.9) 1 (3.6) 0 (0) 0 (0) 10. The chatbot can be used at any time anywhere 26 (92.9) 2 (7.1) 0 (0) 0 (0) 0 (0)
Part C: The perceived usefulness
11. There is no need for technical skills or proficiency to use chatbot technology as it is very easy to use
22 (78.6) 5 (17.9) 1 (3.6) 0 (0) 0 (0) 12. The chatbot stimulates interest in learning 18 (64.3) 7 (25) 3 (10.7) 0 (0) 0 (0) 13. The chatbot stimulates engagement in learning 16 (57.1) 11 (39.3) 1 (3.6) 0 (0) 0 (0) 14. The chatbot provides diagnostic imaging
information in a fun, interesting and innovative way
14 (50) 3 (10.7) 1 (3.6) 0 (0) 0 (0)
15. The chatbot is effective when used as a digital learning tool to provide
personalised learning support
24 (85.7) 3 (10.7) 1 (3.6) 0 (0) 0 (0)
SA = strongly agree, A = agree, N = neutral, D = disagree, SD = strongly disagree
Phys. Technol. Med.
Table 5 shows the students' perspectives on the chatbot's effectiveness. Table 5 illustrates the mean value and standard deviation for Part A: Student’s Interaction, Part B: The Perceived of Use and Part C: The Perceived Usefulness. Based on Part A, respondents agree that the chatbot has a good interaction with the users, with mean 4.59 ± 0.54. The highest mean for Part A is that the chatbot provides instant responses after typing texts and is easily accessible whenever needed.
While the lowest mean is the chatbot can be used via any device which is 4.00. The mean value for Part B is the highest among the three component which is 4.74 with standard deviation 0.54. The highest mean for Part B is the chatbot can be used at any time anywhere which is 4.93 while the lowest mean is 4.50 for there is no need to download any application.
The overall mean for Part C is 4.62 with standard deviation 0.58. The respondent gives the highest value to the chatbot is effective when used as a digital learning tool to provide personalised learning support with 4.82 while the lowest is 4.46 for the chatbot provides diagnostic imaging information in a fun, interesting and innovative way.
Based on the findings from Table 5, we can conclude that the Dibot is effective in helping the students in their diagnostic imaging studies. The performance of the chatbot can be assessed through students' perceptions of whether the chatbot is user-friendly, works smoothly, and achieves the desired
goals. Some comments from the participants about Dibot are the chatbot help and ease them in finding information regarding diagnostic imaging as they did not need to browse the internet. In addition, Dibot is a great platform to use because it can provide the desired in a short time and it is user- friendly. Students nowadays prefer a quick and easy approach to receive answers to questions without spending hours scrolling through a website. These responses indicate that students valued a chatbot's more conversational tone and the fact that it was shorter and more to-the-point. The perceived ease of use has the highest mean of 4.74 and S.D. of 0.54 which show that Dibot has a good interaction with the users, is easy to use, and it is helpful for the students. The application of Telegram and Facebook Messenger are the media used to access Dibot. Thus, the chatbot does not use a specific application to be downloaded and is easily accessible whenever needed. Furthermore, A chatbot is one of the platforms used in mobile learning and has been an essential part of education in recent years (Sönmez et al. 2018). In addition, communication and interaction rely primarily on online platforms; hence chatbots are increasingly being employed in students’ daily life. Furthermore, chatbot systems can be implemented as mobile applications considering that most students in higher education own a smartphone (Okonkwo et al. 2021).
TABLE 5 Mean and standard deviation of students’ perspective effectiveness
Student’s perspective on the chatbot Mean S.D. Interpretation
Part A: Student’s interaction
1. The chatbot shortens the time in waiting for responses 4.68 0.55 Strongly agree 2. The chatbot provides instant responses after typing texts 4.82 0.39 Strongly agree
3. The chatbot can be used via any device 4.00 0.00 Agree
4. The chatbot is easily accessible whenever needed 4.82 0.39 Strongly agree 5. The chatbot interacts through text messages 4.64 0.62 Strongly agree
Total 4.59 0.54
Part B: The perceived ease of use
6. There is no need to download any application 4.50 0.84 Strongly agree 7. The chatbot is easily accessible whenever needed 4.68 0.48 Strongly agree 8. The chatbot shortens the time in searching 4.82 0.39 Strongly agree 9. The chatbot gives correct answers searched 4.75 0.52 Strongly agree 10. The chatbot can be used at any time anywhere 4.93 0.26 Strongly agree
Total 4.74 0.54
Part C: The perceived usefulness
11. There is no need for technical skills or proficiency to use chatbot technology as it is very easy to use
4.75 0.52 Strongly agree 12. The chatbot stimulates interest in learning 4.54 0.69 Strongly agree 13. The chatbot stimulates engagement in learning 4.54 0.58 Strongly agree 14. The chatbot provides diagnostic imaging information in
a fun, interesting and innovative way
4.46 0.58 Agree
15. The chatbot is effective when used as a digital learning tool to provide personalised learning support
4.82 0.48 Strongly agree
Total 4.62 0.58
Phys. Technol. Med.
Table 6 shows the difference between the two groups of students' research knowledge after the chatbot was implemented. The data was calculated by ANCOVA. Table 6 illustrated the post-test result with mean for Group A, 20.71
and the standard deviation was 3.17, which is higher than Group B with the mean value of 17.36 and standard deviation 2.65. The p-value for both groups are significant (p<0.05).
TABLE 6 Students’ knowledge after implementing the chatbot
Group Mean S.D. p-value
Group A 20.71 3.17
0.005
Group B 17.36 2.65
Based on Table 6, it can be concluded that the chatbot positively impacts the students' knowledge. The chatbot improved students' diagnostic imaging knowledge as shown in the mean score for the group of students that used the chatbot is higher than the control group. This happens probably because the students can instantly get the information they are seeking when using the chatbot as the chatbot is constantly available for the whole day unlike human who is not always available. Many students nowadays learn through online modalities of schooling as they expect an immediate response without having to read a lengthy document. They would rather read online, cruise the internet, and use mobile apps than read traditional textbooks. Traditional search methods, in particular, require them to examine several results and choose the best answer. This is consistent with a prior study, which found that the chatbot's design may cause users to take a more active role in their learning and have a good learning experience, resulting in improved learning outcomes (Chen et al. 2020), (Chan et al. 2019). Besides, students' diagnostic imaging knowledge improved when they used the chatbot as a digital tool to help recall, revise, and remember the knowledge learned before. The chatbot can act as a simplified textbook and replaces the thick textbook. This will make it easier for students to review the subject anytime and anywhere as the chatbot is already in their smartphones . In an earlier study, (Chen et al. 2020) and (Mahdi 2018) also concluded that the chatbot can be used in various ways to increase student performance and the efficacy of mobile devices in the classroom.
CONCLUSION
We developed a chatbot named Dibot to help diagnostic imaging students in this research. Three phases have been done to validate the Dibot, measure its effectiveness, and determine its impact of the Dibot. The findings explained that this Dibot could be used as online teaching and learning material. Contents based on students' needs ease the students
to refer and revise the diagnostic imaging subjects, specifically in general x-ray, CT scan, and MRI subjects. Furthermore, comments given for improvement have been employed in the Dibot.
However, few limitations have been identified when using this Dibot. Firstly, a flow chatbot is quite tricky to use compared to an AI chatbot. This is because the flow chatbot needs the same query as the script generated to get the answer.
Therefore, a guideline on how to use the Dibot has been added at the greeting of the Dibot. This guideline must be followed to avoid any confusion during the usage of the Dibot.
Secondly, the queries by the user are restricted according to the answer script made by the developer. Thirdly, too wordy answers have made the Dibot less appealing. The limitations mentioned above can be improved to provide the students with a better personalised-learning chatbot in future works.
ACKNOWLEDGEMENT
This study was supported Geran Pengajaran dan Pembelanjaran provided by Universiti Kebangsaan Malaysia (Grant No. PDI-2021-016). We would like to express our gratitude to all participants for their involvement in this study.
REFERENCES
Adamopoulou, E. & Moussiades, L. 2020. Chatbots: History, technology, and applications. Mach. Learn. Appl.
2(2020): 100006.
Adibe, F. O, Nwokorie, E. C & Odii, J. N. 2016. Chatbot technology and human deception. Rev. Art Soc. Sci. 5:
286 – 291.
Brandtzaeg, P. B. & Folstad, A. 2017. Why people use chatbots. 2017 International Conference on Internet Science, Thessaloniki, 22-24 November 2017, 377-392.
Chan, S. C. H., Wan, C. L. J. & Ko, S. 2019. Interactivity, active collaborative learning, and learning performance:
Phys. Technol. Med.
the moderating role of perceived fun by using personal response systems. Int. J. Manag. Educ. 17(1): 94-102.
Chen, H.-L., Vicki Widarso, G. & Sutrisno, H. 2020. A chatbot for learning chinese: learning achievement and technology acceptance. J. Educ. Comput. Res. 58(6):
1161-1189.
Clarizia, F., Colace, F., Lombardi, M., Pascale, F. and Santaniello, D. 2018. Chatbot: An education support system for student. 2018 International Symposium on Cyberspace Safety and Security, Amalfi, 29-31 October 2018, 291-302.
Colace, F., De Santo, M., Lombardi, M., Pascale, F., Pietrosanto, A. 2018. Chatbot for E-Learning: A case of study. Int. J. Mech. Eng. Robot. Res. 7(5): 528-533.
Den Harder, A. M., Frijlingh, M., Ravesloot, C. J., Oosterbaan, A. E. & Van Der Gijp, A. 2016. The importance of human–computer interaction in radiology e-learning. J. Digit Imaging 29(2): 195-205.
Hsu, M. & Yu, C. 2021. Gamified medical terminology learning through chatbot-based crossword puzzles: The termbot, 29 September 2021, (Preprint of version 1 is available at Research Square).
Mahdi, H. S. 2018. Effectiveness of mobile devices on vocabulary learning: A Meta-analysis. J. Educ. Comput.
Res. 56(1): 134-154.
Marsh, P. 2018. The Use of a Chatbot in Radiology Education. Electronic Presentation Online System (EPOS), European Society of Radiology, Poster ASM/R- 0095.
Okonkwo, C. W. & Ade-Ibijola, A. 2021. Chatbots applications in education: A systematic review. Comput.
Educ. Artif. Intell. 2(2021):100033.
Palasundram, K., Sharef, N. M., Nasharuddin, N., Kasmiran, K. & Azman, A. 2019. Sequence to sequence model performance for education chatbot. Int. J. Emerg.
Technol. Learn. 14(24): 56-68.
Roque, G. D. S. L., De Souza, R. R., Do Nascimento, J. W.
A., De Campos Filho, A. S., De Melo Queiroz, S. R. &
Santos, I. C. R. V. 2021. Content validation and usability of a chatbot of guidelines for wound dressing. Int. J. Med.
Inform. 151:104473.
Sönmez, A., Göçmez, L., Uygun, D., Ataizi, M. 2018. A review of current studies of mobile learning. J. Educ.
Technol. Online Learn. 1(1): 12-27.
Vanichvasin, P. 2021. Chatbot development as a digital learning tool to increase students' research knowledge.
Int. Educ. Stud. 14(2): 44-53.
Wynter, L., Burgess, A., Kalman, E., Heron, J. E. & Bleasel, J.
2019. Medical students: what educational resources are they using? BMC Med. Educ. 19(1): 1-8.