4. To know if international students use marginalization
3.4 VALIDITY AND RELIABILITY OF RESEARCH INSTRUMENTS
The study must be both valid and reliable. To be valid, an instrument's results must be suitable, understandable, accurate, and useful in making conclusions about a group or at the individual level (Fraenkel & Wallen, 2009). To be considered trustworthy, the instrument must give consistency of scores or answers from one administration of the instrument measuring the same construct to the next (Fraenkel &
Wallen, 2009).
3.4.1 Validity
In this study, content validity is employed to examine the validity. This validity instrument demands that the study cover all of the content and domain of the fairly and thoroughly (Louis, Lawrence & Keith, 2007). Using interview questions, questionnaires, and other tools to accurately assess what we want to know.
Prior to data collection, the researcher validates the scale's content validity by seeking expert opinion on the scale (Mills & Gay, 2016). The majority of specialists should have extensive personal intercultural experience, teach in English, and have worked or studied abroad. Expert judgment determines content validity, which is tested by relying on the expertise of experts who are familiar with the concept being measured (Louis et.al, 2007).
The validity of the questionnaire and interview questions in this study is verified by a team of professionals with education competence. The precise procedure is as follows:
3.4.1.1 First, the researcher distributed the prepared invitations to these specialists in order to invite them to join.
3.4.1.2 After all of the experts responded positively to the invitation, the researcher sent them a questionnaire and explained the content evaluation technique.
3.4.1.3 Request that each expert evaluate the significance of each item in the paper. Furthermore, the expert must determine if all items are relevant to the target group and the questionnaire's intended purpose.
Turner and Carlson (2003) state that all instruments must be verified using the Item-Objective Congruence (IOC). The IOC is used to evaluate questionnaire items based on score ranges ranging from -1 to +l. A grade of -1 indicates that the stated aims are incongruent, a rating of 0 indicates that the given objectives are acceptable, and a rating of +l indicates that the stated objectives are quite correct. An item-objective index of 0.7 or higher is consistent with the goal (Seok, Meyen, Aust, Fitzpatrick & Newberry, 2006).
The researcher sends the consent form, synthesis of the research proposal and IOC form of questionnaire and interview questions by email to three experts who are now in education filed to do the IOC, and receives the patient feedback from the three experts a week later. Regarding to the IOC of questionnaire, first, in the first part which is background information, Expert 1 points out the grammatical problem, such as "above 5 years" need to change to "over five years". Secondly, in the second part of the questionnaire, which is the “international students' perceptions of ICC”, Expert 1 points out the question of the tense use of the questionnaire, the original tense is past tense, while Expert 1 suggests to use the general present tense. After thoughtful consideration, the researcher adjusts the items of questionnaire to present tense. At the same time, Expert 1 and Expert 2 points out some problems in grammar and language use, and give reasonable suggestions in order to make the language more suitable and understandable for the subjects.
Furthermore, given IOC of interview questions, Expert 1 and Expert 2 make some suggestions on interview questions, for example, the question about separation strategy: "Are you proud of your national identity?”, Expert 1 believes that the outcome of this question is predictable, and suggests paying attention to some historical aspects, etc. Expert 2 thinks that some language usage can be adjusted, such as "Do you want to be a Thai" in the original text, suggesting it be changed to "Do you want to be a part of
this country" and so on to make the wording more accurate. Expert 3 basically has similar feedback and suggestions as Experts 1 and 2. Therefore, the researcher adjust the questionnaire and interview questions based on the 3 experts’ reasonable and remarkable comments and suggestions. Finally, the validity of this questionnaire is calculated to be 0.895, and the original validity of the interview question is o.688, however, the researcher deletes the two questions with the lowest expert evaluation scores to make the validity of this interview question reach 0.78, which can be acceptable by this research.
3.4.2 Reliability
3.4.2.1 Pilot Study
The pilot research was used in this study to evaluate their grasp of the questionnaire and get comments and ideas in order to establish the reliability of the questionnaire. The questionnaire was assessed on 30 international students who were not part of the study group. First, the participants were instructed to thoroughly read the questionnaire under the observation and guidance of the researcher. Second, the questionnaire was handed to each participant for 15 minutes to complete. Finally, they were invited to fill out a questionnaire with comments and ideas. The researcher then attentively reads the input and makes any required edits and adjustments.
The Alpha coefficient was applied in this study to determine the internal consistency dependability of the instruments. Cronbach's Alpha is a widely used, statistically based coefficient that measures the reliability of a collection of items and how they perform in a survey (Tavakol & Dennick, 2011). This sort of reliability establishes "how all things on the tool connect to each other and the tool as a whole"
(Tavakol & Dennick, 2011).
The Cronbach Coefficient Alpha value is as follows: o.9=excellent, 0.8=good, 0.7=acceptable, and 0.6=problematic. O.5=unacceptable, 0.5=poor. As a result, the Cronbach Coefficient Alpha of the questionnaire must be at least 0.7 in order for it to be considered trustworthy (George & Mallery, 2010).
3.4.1.2 Test Reliability of Questionnaire: The researcher used the questionnaire to try- out ( pilot study) on the 30 pilot samples who did not included in the actual target samples but had the same characteristics. Then the questionnaire was tested for Cronbach's alpha coefficient of 0.5. Which the confidence value of the whole questionnaire should have a confidence value of 0.7 or more (George & Mallery, 2010) The test results showed that the confidence of the questionnaire part 2 had the whole confidence of Cronbach's Alpha 0. 982 and the results of each items ( 41 items) tested of Cronbach's Alpha were between 0.981 - 0.983. This means that the questionnaire is reliable and can be used to conduct research. The analysis results of the whole set questionnaire and each item questions appear in Table 3.7 and Table 3.8.
Table 3.7 Result of Reliability of the whole set questionnaires 41 items
Part Questionnaire Cronbach's Alpha (α)
2 Perceptions towards ICC (ICC) total 41 items 0.982 Table 3.8 Result of Reliability of each item questions in the part 2 questionnaire
Item No. Questionnaire Cronbach's Alpha (α) 1 I know the essential norms and taboos of the
host culture.
0.982 2 I can contrast important aspects of the host
language and culture with my own.
0.982 3 I could contrast my own behavioural with
those of my hosts in important areas.
0.982 4 I can discuss and contrast various
behavioural patterns in my own culture with those in the host country.
0.982
5 I can recognize signs of culture stress and some strategies for overcoming it.
0.981 6 I know some techniques to aid my learning of
the host language and culture.
0.982
Table 3.8 Result of Reliability of each item questions in the part 2 questionnaire (Cont.) Item No. Questionnaire Cronbach's
Alpha (α) 7 I can describe a model of cross-cultural
adjustment.
0.982 8 I can cite various learning processes and apply
strategies for learning and adjusting to the host culture.
0.982
9 I can describe interactional behaviours
common among people in the host culture in social and professional areas (e.g., family roles, team work, problem solving).
0.982
10 I can cite important historical and socio- political factors that shape my own culture and the host culture.
0.982
11 I am willing to interact with host culture members.
0.982 12 I am willing to learn from the language and
culture of host country.
0.982 13 I am willing to try to communicate in the host
language and behave in appropriate ways.
0.982 14 I am willing to deal with my emotions and
frustrations with the host culture when experiencing cultural differences.
0.982
15 I am willing to take on various roles appropriate to different situations.
0.982 16 I am interested in learning and experiencing
new cultural aspects.
0.982 17 I am willing to try to understand differences in
the behaviours, values, attitudes and styles of my culture and those of the host.
0.982
Table 3.8 Result of Reliability of each item questions in the part 2 questionnaire (Cont.) Item No. Questionnaire Cronbach's
Alpha (α) 18 I am willing to adjust my behaviours to
communicate appropriately with people from different culture.
0.981
19 I am willing to reflect on the impact and consequences and ethical implications of my decisions and choices on my hosts.
0.981
20 I am willing to deal with different ways of perceiving, expressing, interacting, and behaving even in something that I was not accustomed or preferred.
0.983
21 I am willing to suspend judgment and
appreciate the complexities of communicating and interacting interculturally.
0.982
22 I demonstrate flexibility when interacting with persons from the host culture.
0.982 23 I adjust my behaviour, dress etc as appropriate
to avoid offending my host.
0.982 24 I am able to compare and contrast different
aspects of my culture and those of the host.
0.982 25 I use strategies for learning the host language
and culture.
0.982 26 I have the ability a to interact with people
appropriately in various social situations.
0.982 27 I use appropriate strategies to adapt to the host
culture and reducing stress in adapting process.
0.982 28 I use culture-specific information to improve
my style and personal interaction.
0.982
Table 3.8 Result of Reliability of each item questions in the part 2 questionnaire (Cont.) Item No. Questionnaire Cronbach's
Alpha (α) 29 I help resolve cross-cultural conflicts and
misunderstandings when they arose.
0.982 30 I use models, strategies, and techniques that
aided my learning of the host language and culture.
0.982 31 I monitor my behaviours and its impact on my
learning, my growth, and especially on my hosts.
0.982 32 I am aware of differences and similarities
across my own culture and the host language and culture.
0.982
33 I am aware of the need to adjust my interaction strategies in different situation in accordance with the host culture.
0.982
34 I am aware of diversity in the host culture (such as differences in race, gender, age).
0.981 35 I am aware of the dangers of generalizing
individual behaviours as representative of the whole culture.
0.982
36 I am aware of my choices and their consequences which made me more or less acceptable by the members of the host culture.
0.982
37 I am aware of my hosts’ reactions to me and the reason that reflected their cultural values.
0.982 38 I am aware of varying cultural styles and
language use, and their effect in social and working situations.
0.982
39 I am aware of my own and others level of intercultural development and the factors affected my solutions to overcome cultural frustrations.
0.982
Table 3.8 Result of Reliability of each item questions in the part 2 questionnaire (Cont.) Item No. Questionnaire Cronbach's
Alpha (α) 40 I am aware of my personal habits and
preferences and reactions or responses to
differences reflected the values and ethics of my culture.
0.981
41 I am aware of how I perceived myself and being perceived by others as communicator, facilitator, mediator, in an intercultural situation.
0.982
From Table 3. 8, the reliability result of the questions in part 2 questionnaire in the total of 41 items. It was found the Cronbach’s Alpha in between
0
. 981- 0. 983. Which was found all items have the confidence value over than 0. 70 (George & Mallery, 2010). Therefore, each item questionnaire can be used to conduct the research.
Through the pilot study of interview, the researchers summarized the following problems and came up with corresponding solutions:
1) The researcher found that the interviewee did not understand some conceptual terms, such as "identity and cultural values". Therefore, researchers need to add explanations of these terms in the interviews for the main research later.
2) Through the pilot study, the researcher found that there were not enough questions asked, and all the questions were asked in less than ten minutes.
Therefore, in the main research later, the researcher needs to add some related questions based on the interviewer's answers.
3) Through a pilot study, the researcher found that there was silence during the interview and the author did not know how to proceed. Therefore, researcher herself need to learn some interviewing skills, such as how to guide the interviewee to continue speaking, how to follow from their words, and how to naturally transition from one topic to another, so as to achieve better communication.