The development and psychometric properties testing of intolerance of uncertainty scale for Indonesian adults

(1)

Volume 6

Number 1 : April 2023 [In Press] Article 5

4-18-2023

The development and psychometric properties testing of The development and psychometric properties testing of intolerance of uncertainty scale for Indonesian adults intolerance of uncertainty scale for Indonesian adults

Divani Aery Lovian

Faculty of Psychology, Universitas Indonesia, divanialovian@gmail.com Dewi Maulina

Faculty of Psychology, Universitas Indonesia, dewi.maulina@ui.ac.id Hilma Ramadina

Faculty of Psychology, Universitas Indonesia Nathania Kusuma

Faculty of Psychology, Universitas Indonesia

Follow this and additional works at: https://scholarhub.ui.ac.id/proust

Part of the Clinical Psychology Commons, Other Psychology Commons, and the Personality and Social Contexts Commons

Recommended Citation Recommended Citation

Lovian, Divani Aery; Maulina, Dewi; Ramadina, Hilma; and Kusuma, Nathania (2023) "The development and psychometric properties testing of intolerance of uncertainty scale for Indonesian adults," Psychological Research on Urban Society: Vol. 6: No. 1, Article 5.

DOI: 10.7454/proust.v6i1.1118

Available at: https://scholarhub.ui.ac.id/proust/vol6/iss1/5

This Original Research Paper is brought to you for free and open access by the Faculty of Psychology at UI Scholars Hub. It has been accepted for inclusion in Psychological Research on Urban Society by an authorized editor of UI Scholars Hub.

(2)

ORIGINAL RESEARCH PAPER

Development and Psychometric

Property Testing of the Intolerance of Uncertainty Scale for Indonesian Adults

Psychological Research on Urban Society 2023, Vol. 6(1): 48-58

DOI: 10.7454/proust.v6i1.1118 proust.ui.ac.id

U

rbanization contributes to mental health issues and stress among populations due to high population den- sity and constant activity, which results in increased vulnerability to stressors and high risks of anxiety and depression (Gruebner et al., 2017; Sharifi and Khavarian-Garmsir, 2020). Daily life uncertainties, such as weather, job applications, grades, business, finding a soulmate, or the COVID-19 pandemic, which affected populations worldwide, exacerbate this situation (Graffeo et al., 2022; Gu et al., 2020).

Uncertainty is a natural part of life that can significantly influence mental and physical well- being. People may feel worried during uncertain times, but the proneness differs due to a personality characteristic known as intolerance

of uncertainty (IU; Buhr and Dugas, 2002; Car- leton, 2012; Carleton, 2016; Dugas et al., 2001;

Freeston et al., 1994). IU is the tendency to view the possibility of negative things as unacceptable or threatening regardless of the probability of the occurrence (Ladouceur et al., 2000; Mill- roth and Frey, 2021). It represents cognitive, emotional, and behavioral reactions toward uncertainty in daily life, which can lead to unobjec- tive perceptions, disrupt problem-solving abili- ties and decision making, and contribute to the development and maintenance of worries (Birrell et al., 2011; Freeston et al., 1994; Koerner and Dugas, 2006, Meares and Freeston, 2008;

Luhmann et al., 2011).

Understanding the concept of IU is crucial in recognizing its root causes and formulating effective treatment approaches. Hence, researchers are working to develop IU instruments that can be applied to various contexts, including adults in urban areas. In this regard, considering that cultural factors can impact the validity of Corresponding Author:

Divani Aery Lovian

Faculty of Psychology, Universitas Indonesia Kampus Baru UI, Depok, West Java—16424 Email: divanialovian@gmail.com

Abstract

Uncertain situations have further exacerbated great vulnerability to stressors and a high risk of mental health problems in urban populations. The adverse effects of uncertainty on well-being have been increasingly concerning and heightened the need to understand intolerance of uncertainty (IU) as a factor that influences responses to uncertainty. However, few scales have been developed to measure IU, especially in Indonesia. To address this concern, the current study aimed to develop a reliable and valid Intolerance of Uncertainty Scale for Indonesian Adults (IUS-A). The scale was developed and validated using data from 588 participants aged 20–65 years (M = 32, SD = 11.04) from various provinces in Indonesia. Confirmatory factor analysis confirmed that the unidimensional IUS-A fit the theoretical model. The final version of the IUS-A consists of 18 items with good internal consistency and a valid measurement of IU. Furthermore, the items exhibited good discriminatory power to differentiate individuals with high and low levels of IU. Based on these results, IUS-A is a reliable and valid instrument for measuring IU in the adult population in Indonesia.

Keywords

Intolerance of uncertainty, Uncertainty, Inventory, Urban, Adult Divani Aery Lovian^1*, Dewi Maulina¹, Hilma

Ramadina¹, Nathania Kusuma¹

1Faculty of Psychology, Universitas Indonesia

Received: November 9th, 2022 Revision Accepted: March 20th, 2023 p-ISSN 2620-3960

(3)

tests and their items is important, as discussed by Cohen et al. (2013). The validity of a test may differ across cultures due to diverse cultural influences on knowledge and perception. Thus, the current study aims to develop IU instruments tailored to the Indonesian population to ensure good reliability, validity, and norms that accurately reflect the IU levels of Indonesian adults.

As a transdiagnostic mechanism, IU contributes to a wide range of mental health issues and can manifest differently across individuals. For example, individuals who are intolerant of uncertainty tend to deem uncertain situations as

more troubling than those who are more toler- ant. For example, studies have demonstrated that adults with high levels of IU exhibited high levels of anxiety during the pandemic compared with those with low levels (Glowacz and Schmits, 2020; Huang and Zhao, 2020; Rettie and Daniels, 2021; Smith et al., 2020). As a cognitive scheme, IU contributes to the development of anxiety symptoms and is associated with other psychological problems such as social phobia, obsessive–compulsive disorder, panic disorder, agoraphobia, and depression (Berenbaum et al., 2008; Carleton, 2016, Hong and Lee, 2015;

Koerner and Dugas, 2006; Mahoney and

IUS (Freeston et al., 1994) IUS (Buhr and Dugas, 2002) IUS-12 (Carleton et al., 2007) As-

pects

Five aspects of the Intolerance of Uncertainty Scale

 Uncertainty is unacceptable and should be avoided

 Uncertainty reflects badly on a person

 Frustration is related to uncertainty

 Uncertainty causes stress

 Uncertainty prevents action

Four aspects of the Intolerance of Uncertainty Scale

 Uncertainty paralysis

 ^Distress

 Desire for predictability

 Inflexible uncertainty beliefs

Two aspects of the Intolerance of Uncertainty Scale

 Prospective anxiety

 Inhibitory anxiety

Measure- ment scale

French version 27 items

Five-point Likert-type scale Total score

English version 27 items

English version 12 items

Partici- pants

Undergraduate students Undergraduate students; majority female

Undergraduate students; majority female

Reliability ɑ =.91 ɑ =.94 ɑ =.83–.85

Validity Exploratory factor analysis (EFA) Cattell’s scree test Confirmatory factor analysis

Limitations  Using specific samples, such as college students in the testing (Freeston, et al., 1994; McEvoy and Mahoney 2011).

 Factor analysis relies solely on EFA so there may be a higher number of aspect estimates (Carleton et al., 2007; Field, 2018).

 Potential repetitive and unre- lated items (Carleton et al., 2007; Helsen et al., 2013;

McEvoy & Mahoney, 2011;

Norton, 2005).

 Potential generalization issue due to the homogenous samples in testing (Buhr & Du- gas, 2002; McEvoy & Ma- honey, 2011)

 Factor analysis relies on a Scree test which may de- crease the accuracy of the number of aspects (Carleton et al., 2007; Field, 2018;

McEvoy & Mahoney, 2011)

 Potential repetitive and un- related items (Carleton et al., 2007)

 Using specific samples, such as college students in the testing (Carleton et al., 2007;

McEvoy & Mahoney, 2011)

 Uncommon procedure and process of test development that might affect the content validity (Hong & Lee, 2015;

Sexton & Dugas, 2009).

Table 1. Review of the Intolerance of Uncertainty Scale (IU) Scales

(4)

McEvoy, 2012; McEvoy and Mahoney, 2011).

Previous studies illustrated that IU is a unidimensional construct consisting of several in- terrelated aspects. The first instrument, which is called the Intolerance of Uncertainty Scale (IUS), was developed by Freeston et al. (1994) in French and consists of 27 items and five aspects.

Buhr and Dugas (2002) later adapted it into Eng- lish and simplified it into four aspects while re- taining all items. Carleton et al. (2007) argued that IU only consists of two aspects and developed the IUS-12. These three existing IU instruments use a total score in which the higher the score, the more intolerant individuals are toward uncertainty. See Table 1 for a detailed review of the IU scales.

Second, IU is a personality predisposition that tends to remain relatively stable throughout life. However, several studies debate on the ex- istence of variations in the levels of IU among different age groups in which a few studies sug- gest that older individuals tend to have low levels of IU. Low levels of IU in older adults may be associated with a more stable living situation, which reduces the need to worry, or with emotional maturity due to certain life experiences (Basevitz et al., 2008). Thus, one may also argue that these changes likely occur as a result of ex- ternal and contextual factors. Additionally, Brenes (2006) demonstrate that older individuals experience anxiety differently than younger adults, that is, with less cognitive anxiety but no significant differences in somatic and affective anxiety. In general, the evidence for a correlation between age and IU remains inconclusive and inconsistent (Parlapani et al., 2020; Brenes, 2006).

Third, the use of an odd number of responses on the Likert scale in the previous scale may lead to the tendency for respondents to select the neutral option to appear favorable (Moors, 2008) or to represent a context-dependent answer (Simms et al., 2019). Furthermore, a recent study by Simms et al. (2019) illustrated that no advantage exists for odd- over even-numbered scales in terms of internal consistency or criterion validity. To prevent potential bias due to the midpoint option, the current study presents the items in the new IU scale with an even number of responses (six). The number of options will not exceed six to avoid confusion among participants, especially in perceiving differences between similarly worded responses (Simms et al.,

2019).

Fourth, the absence of unfavorable items in the IUS versions which may lead to response bias. If the scale only consists of favorable items, then the respondents may answer the questionnaire without understanding the context and meaning of each item (Gravetter and Forzano, 2016). Hence, the current study develops a new instrument that consists of favorable and unfavorable items to reduce response bias. Further- more, the new scale is designed to be briefer than the original IUS with less than 27 items across the four aspects.

Lastly, the limitations regarding the validity of the existing IU scales underscore the need for further research and refinement of these instruments. The IUS versions developed by Freeston et al. (1994) and Buhr and Dugas (2002) have been criticized for containing repetitive and un- related items (Carleton et al., 2007; Helsen et al., 2013; McEvoy and Mahoney, 2011; Norton, 2005). Additionally, the use of only exploratory factor analysis (EFA) by Freeston et al. (1994) and the scree test by Buhr and Dugas (2002) may lead to an inaccurate determination of the number of aspects or factors (Carleton et al., 2007;

Field, 2018; McEvoy and Mahoney, 2011). Car- leton et al. (2007) attempted to address this issue by selecting one factor each from the five and four factors and conducting a refinement. How- ever, the selection of these factors and the items were not based on theoretical considerations, and the uncommon procedure used in the test development by Carleton et al. (2007) may influence the content validity of the scale (Hong and Lee, 2015; Sexton and Dugas, 2009). To address these issues, the current study intends to ensure validity through expert judgment, peer review, CFA, and the correlation of the new scale with other scales.

We use Buhr and Dugas (2002) as the theoretical foundation for the IU construct because they reference the classical concept of IU put forth by Freeston et al. (1994) but simplify and clarify the structure. The current study aims to develop a new IU scale for adults (IUS-A) that incorporates four aspects, namely, behaviors, cognitive tendencies, and affective tendencies that contribute to IU. This scale will feature new items instead of adaptations from the existing IU scales. As IU is a unidimensional construct, an individual's level of IU is obtained from the total

(5)

measurement score and without subscale scores.

In general, the items in the IUS-A represent common situations in the daily life of adults, which makes the scale suitable for use in general uncertain situations that can induce IU. This new scale can be used as a tool for screening purposes and scientific research, and it is expected to help professionals in psychology evaluate levels of IU and provide appropriate inter- ventions.

Methods

The development of new IUS and psychometric testing were described in several sections and subsections.

Participants

Data were collected through online administra- tion with the following inclusion criteria: Indo- nesian adults between 20 and 65 years who agreed with the statements of consent. Data collection was conducted for five days (from No- vember 20 to 24, 2020). A total of 588 out of 626 data were further analyzed after removing du- plicates and incomplete data and after screening the participants using the inclusion criteria. In total, 67.35% of the respondents were female (N

= 396; SD = 11.04) with a mean age of 32 years (SD = 11.04). The majority were single (N = 319),

graduates of bachelor's degrees (N = 319), and private sector employees (N = 207).

IUS for Adults (IUS-A)

The IUS-A is a new scale, hence, we undertook the development process through five stages with reference to Cohen et al. (2013). These stages include test conceptualization, construction, and tryout, item analysis, and test revision. The process generally began by operationalizing the theoretical indicators of items, reviewing the items through expert judgment and peer review, conducting a readability and tryout test, analyz- ing the items, and revising the test. Expert judgment was performed by two experts in psychometric and research methods. The objective of expert judgment was to check the content validity of the scale, which focuses on the relevance and ability of items to represent all indicators of the IU construct.

Constructing the IU includes four aspects, namely, uncertainty paralysis, distress, desire for desirability, and inflexible uncertainty belief (Buhr and Dugas, 2002). Table 2 provides the definition and the indicators of each aspect.

We initially constructed 36 new items for trial purposes with the target of shortening the IUS-A to 18 items after reliability and validity testing. The items cover the general context of uncertainties in the daily life of adults without

Aspect Definition Behavioral Indicators

Uncertainty Paralysis

A sense of feeling unable to act when faced with uncertainty and lack of confidence in decision making (Birrell et al., 2011; Carleton et al., 2007; Einstein, 2014; McEvoy and Mahoney, 2011)

 Inability to response or act in uncertain situations

 A tendency to act slower in decision making or solving problems in uncertain situations

 Lack of confidence in decision making in uncertain situations

Distress A tendency to perceive uncertain situations as threatening and generate negative feelings like distress, frustration, and upsetting (Birrell et

 Negative feelings regarding uncertain situations

 Perceive uncertainty as a threatening situation

Desire for Predictabil- ity

An urge to avoid uncertain situations as it may disrupt one’s plan or other negative conse- quences (Birrell et al., 2011; Einstein, 2014;

McEvoy and Mahoney, 2011).

 Make a thorough preparation to minimize potential uncertainty

 Perform an anticipatory behavior or action to avoid uncertainty or the worst possibility from any situation Inflexible

Uncertainty Beliefs

Perception about uncertainty as an unfair experience; being uncertain about the future is unfair (Freeston et al., 1994; McEvoy and Ma-

 Feeling alone when facing uncertainty

 Negative perceptions about self and personal experience when dealing with uncertainty

Table 2. Description of Aspects of the IUS

(6)

reference to any specific situation or event.

Therefore, the IUS-A is expected to capture IU in uncertain situations, for instance, but not limited to, the pandemic. The items of the IUS-A were rated using a six-point Likert-type scale from 1 (not at all true) to 6 (exactly true). The IUS-A will compute one total score from the sum of the score of each item. Therefore, the possible score in the final version of IUS-A will range from 18 to 72. The higher the score, the higher the IU level.

After developing the items, we sought the input of experts to evaluate the initial version of the IUS-A, including content, relevance, and appropriateness. Following the recommendations, we revised the scale and conducted a readability test with a sample group of 13 participants aged 21–62 years. On the basis of feedback, we revised 14 items out of 36 with a focus on the wording and language structure. To ensure that each aspect of IU was adequately represented, we included nine items per aspect in the scale.

The IUS-A starts with instructions and a few examples of uncertain situations in daily life, such as future events, pandemic-related uncertainties (duration of the pandemic, potential infection, and the long-term impact of pandemic), exams, or job applications. Table 3 provides examples of items of each aspect.

Data collection was administered online using Google Forms. The form included a brief description of the study, an informed consent form, demographic data, the IUS-A, and the IUS -12 and GAD-7 for validity testing.

Psychometric Testing

The current study conducted a thorough psychometric evaluation of the IUS-A. We used Cronbach's alpha to measure internal consistency and employed CFA to assess validity. The objective of the CFA was to examine the factor structure of the scale, which is based on four in- terrelated aspects in line with the theoretical model proposed by Buhr and Dugas (2002). The model fit was evaluated using commonly accepted indexes such as root mean square error of approximation (RMSEA), comparative fit index (CFI), and the Tucker–Lewis Index (TLI). Addi- tionally, we compared the IUS-A with the Indo- nesian version of the IUS-12 by Bagaskara (2009), which was established as a reliable (α

=.74) and valid measure, to test for construct validity. Furthermore, we also tested criterion validity by comparing the scale with the GAD-7 by Jaya (2017, 2020) as a measure of anxiety. Addi- tionally, we applied various methods for item analysis such as the proportion of endorsement, discriminatory power analysis, correlation index method, and analysis of factor loadings. All data were analyzed using JASP ver. 0.14.1.

Results and Discussion

The 36-item IUS-A consisted of 10 unfavorable items and 26 favorable items. The results of the psychometric testing indicated that the scale exhibited a high level of reliability with Cronbach's alpha of 0.89. This finding is con- sistent with that of Kaplan and Saccuzzo (2009), that is, the IUS-A demonstrates good internal consistency. Additionally, CFA was conducted to further examine and refine the underlying structure of the scale. The CFA employed several indices for evaluating fit between the hypothesized model and the data (RMSEA < 0.06, CFI >

0.90, TLI > 0.90; Hair et al., 2019; Hu and Bentler, 1999; Schreiber et al., 2006). The results of the CFA demonstrated that the 36-item IUS-A is not a valid measure of IU according to the theoretical model proposed by Buhr and Dugas (2002).

Aspects Examples

Uncertainty Paralysis

“Dalam menghadapi situasi tidak pasti, saya akan menunda pekerjaan“

(When dealing with uncertain situations, I will procrastinate)

Distress “Dalam menghadapi situasi tidak pasti, saya tetap bisa tenang dan berpikir jernih”

(When dealing with uncertain situations, I keep calm and think clearly) Desire for

Predictabil- ity

“Saya menghindari berada dalam suatu kondisi yang tidak bisa saya prediksi”

(I avoid being in a condition that I can- not predict)

Inflexible Uncertainty Beliefs

“Menurut saya, tidak adil jika hanya saya yang mengalami kesulitan untuk bertindak dalam situasi tidak pasti”

(In my opinion, it is unfair if I am the only one who has difficulty taking action in uncertain situations)

Table 3. Examples of the Item in Each Aspect

(7)

Table 4 presents the conclusion of the model fit of the 36-item IUS-A.

According to the measurement of the Pro- portion of Endorsement (PoE), the majority of items on the IUS-A exhibited good PoE based on the criteria established by Millon and Bloom (2008). Specifically, 31 items have a PoE index greater than 85%, which indicates that the participants generally had a favorable response to the majority of the items (Table 5).

We then conducted the corrected item–total correlation analysis (CrIT) to analyze the discriminatory power of the items. According to Cristo- bal et al. (2007) and Nunally and Bernstein (1994), items with good discriminatory power should have a CrIT value >.30. Analysis revealed that 78% of the items had good discrimination power in distinguishing individuals with high and low levels of IU (Table 6). Furthermore, the

study conducted factor loadings analysis to evaluate the suitability of the items in representing their respective factors. According to Hair et al. (2019), items with factor loadings >0.50 are considered good. The results of the CFA indicated that 53% of items or 19 items had a factor loading index of >.50, which indicates that these items are well-suited for representing their related factors (Table 7).

Moreover, the study conducted an integrative analysis of the IUS-A to produce a final 18- item scale with good psychometric properties.

The analysis considered various factors, including CrIT, factor loadings, PoE, and modification indices related to the factor loading of each item.

In addition, the study conducted qualitative considerations such as the wording and relevance to the theoretical concept of the scale. The items were revised or omitted on the basis of their psychometric properties, response distribu- tion in terms of PoE, and overall quality. Among the 18 items retained, items 4, 6, 30, and 31 were revised due to their low discriminatory power and factor loadings, which were slightly less than.50. The revision was done by changing

RMSEA

(<.07)

CFI (>.90)

TLI (>.90)

Interpreta- tion 36-item .072 .730 .711 Model unfit

Table 4. Model Fit of the 36-item IUS-A

PoE

Index Category Item Percent-

age PoE

<.15

Item tends to be disapproved (an indication of social desirability)

– 0

PoE.

15 – .85

Item triggers evenly distrib- uted responses

1, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 28, 29, 30, 31, 32, 33, 34, 35, 36

86,1%

(31 item)

PoE

>.85

Item tends to be approved (an indication of social desirability)

2, 6, 24, 26, 27 13.89%

(5 item)

Table 5. Proportion of Endorsement (PoE) of the 36 -item IUS-A

Index Item Percentage

r <.30 2, 4, 6, 21, 24, 26, 27, 31

22%

(8 items) r >.30

1, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 28, 29, 30, 32, 33, 34, 35,

78%

(28 items)

Table 6. Corrected Item–Total Correlation (CrIT) of the 36-item IUS-A

Index Item Percent-

age η <.50 2, 3, 5, 6, 7, 8, 9, 11,

12, 13, 21, 26, 27, 30, 31, 34, 35

47%

(17 items) η >.50

1, 4, 10, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 28, 29, 32, 33, 36

53%

(19 items)

Table 7. Factor Loadings (Std Est. All) of the 36- item IUS-A

(8)

them from favorable to unfavorable items. For example, item 4, “It is important for me to make alternative plans as much as possible” (Penting bagi saya untuk membuat sebanyak mungkin rencana cadangan) was revised to “I do not think I should make too many alternative plans” (Saya merasa tidak perlu membuat terlalu banyak rencana cadangan). Table 8 provides details about item selection.

To assess the psychometric properties of the 18-item IUS-A, we reanalyzed the scale after item elimination. The results revealed that the 18 -item IUS-A has good internal reliability for measuring IU (α =.86). Notably, the performance of the participants during testing can also impact the reliability of the scale. Factors such as fatigue, motivation, indifference toward the test, and effort to maintain a good impression can play a role in this performance (Urbina, 2004).

Additionally, the conditions of the testing situation, such as noise levels, the personality of the examiner, and the testing time, can also influence the scores of the participants (Urbina,

2004). To minimize these effects, we employed several strategies during data collection. First, we ensured that the respondents voluntarily participated in the study by obtaining informed consent. Second, we collected data online to ena- ble them to fill out the questionnaires at a time that was convenient for them. Third, we ensured the appropriateness of wording in instructions and items through readability testing and peer review. These measures, namely, voluntary par- ticipation, online data collection method, and comprehensible questionnaire, likely contribut- ed to the good reliability of the IUS-A, because they optimized the performance of the participants and eased the testing process.

The construct validity of the 18-item IUS-A was assessed through CFA to confirm the factor structure and model fit of the scale. CFA revealed that the 18-item IUS-A is valid for measuring the IU construct. The scale displayed good factor loadings, and the data model fit well with the hypothesized one factor solution model. The items in the IUS-A represent different aspects of

Scale Selected Item Eliminated Item Revised Item

Intolerance of Uncertainty (IU)

11, 14, 15, 16, 18, 19, 20, 22, 23, 25, 28, 29, 33, 35

1, 2, 3, 5, 7, 8, 9, 10, 12, 13, 17, 21, 24, 26, 27, 32, 34, 36

4, 6, 30, 31

Total 14 18 4

Table 8. Integrative Item Analysis

RMSEA (<.07) CFI (>.90) TLI (>.90) Interpretation

18-item IUS-A .05 .94 .92 Model fit

Table 9. Model Fit of the 18-item IUS-A

Figure 1. Model Plot for CFA

(9)

IU and align with the original theoretical frame- work of IU, which comprises several aspects but only uses the total score. Moreover, the CFA results indicated that the model fit well with the data and was similar to the goodness-of-fit indexes from the first CFA of the 36-item IUS-A (Table 9 and Figure 1). In summary, the 18-item IUS-A aligns with the theoretical model of IU by Buhr and Dugas (2002) and is valid for measuring the IU construct.

In addition to the CFA, the study further evaluated the construct validity of the 18-item IUS-A by correlating it to the IUS-12. The results revealed a significant correlation between the two scales (r =.73; p =.00), which indicates that the 18-item IUS-A is a valid measure of IU (Table 10). Criterion validity was also tested by comparing the 18-item IUS-A to the GAD-7 as a concurrent criterion. The result suggested that the 18-item IUS-A significantly correlated with GAD-7 (r =.52; p =.00). The results also pointed to a significant correlation between the two scales (r =.52; p =.00), which implies that the 18- item IUS-A is a valid measure of anxiety. The positive correlation of the 18-item IUS-A with the IUS-12 and GAD-7 further strengthens the validity of the scale as a measure of IU.

**Correlation is significant at the 0.01 level (2- tailed)

The findings have significant implications for future research on the topic of IU among In- donesian adults. The reason is that, as per the literature review, studies on the development of IU measurement in Indonesia are scarce. The 18- item IUS-A developed in this study presents several advantages over the existing IU scales.

First, the IUS-A comprises 18 items less than the original scale (i.e., Buhr and Dugas, 2002). The model aligns with the prior theoretical formula-

tion of IU as a unidimensional construct that comprises several aspects, which results in a total score only (e.g., Buhr and Dugas, 2002; Free- ston et al., 1994). Second, the majority of the items in the IUS-A have good discriminatory power, which indicates that the items represent a content domain similar to that of Buhr and Du- gas (2002), which was the primary reference used in developing the items of the IUS-A.

Third, the majority of the items in the IUS-A also have a good PoE, which suggests the minimal possibility of social desirability bias Additional- ly, we included three unfavorable items in the final version of the IUS-A, which is in contrast to the existing IU scales that exclude any unfavorable items.

The present study has several limitations that need to be addressed in future research.

First, the IUS-A has a disproportionate number of favorable items compared with unfavorable items, which may impact the quality of the measurement by increasing the possibility of response style bias (Kaplan and Sacuzzo, 2009).

Future research should include more unfavorable items to achieve a balanced proportion of favorable and unfavorable items, which may prevent the possibility of an acquiescence response set. Second, items 4 and 6 exhibited the lowest quality, and both originated from the desire for predictability aspect. To improve the scale, the study recommends further investiga- tion and qualitative review of indicators related to the desire for predictability aspect. Third, the overall quality of items 4, 6, 30, and 31 was significantly lower than those of others and affected the psychometric properties of the IUS-A.

Future research should retest the revised items, and the overall quality of the IUS-A in measuring the level of IU among adults should be im- proved. Fourth, the participants were primarily young adults and women, which may have limited the generalizability of the results. To address this concern, future research should seek a more diverse sample, including a greater proportion of middle-aged adults and a balanced ratio of male and female respondents. Moreover, future studies may address the validity issue of IUS-A by correlating the IUS-A with additional constructs such as worry, neuroticism, extraver- sion, and alcohol consumption (i.e., Mahoney and McEvoy, 2012; McEvoy and Mahoney,

IUSA18 IUS12 GAD7

IUSA18 1 - -

IUS12 .730** 1 -

GAD7 .523** .438** 1

Table 10. Correlation of the 18-item IUS-A with IUS-12 and GAD-7

(10)

2011).

Conclusion

In conclusion, the present study has developed and validated a reliable and valid 18-item scale for measuring IU among Indonesian adults. The results demonstrated that the IUS-A has good psychometric properties and can differentiate individuals with high and low levels of IU. Pro- fessionals in the field of psychology can poten- tially use this scale as a tool for screening IU levels in Indonesian adults. The items capture individual reaction to uncertainty that may occur in daily life and provide information about uncertainty paralysis, distress, desire for predictability, and inflexible uncertainty beliefs. The IUS-A can aid professionals in psychology in formulating appropriate intervention plans or programs related to individual IU levels, particularly in urban living. Additionally, researchers can use this scale for conducting scientific research on IU topics in the adult population.

Acknowledgments. We thank the expert and those who provided helpful suggestions and recommendations regarding the development of the IUS-A.

Declaration of Conflicting Interest. In the present study, there is no conflicting interest in the authorship and/or the publication of the manuscript.

References

Bagaskara, S. (2009). Fundamentalisme dan Closed-Mindedness: Peran Religiusitas. In- tolerance of Uncertainty dan Need for Clo- sure Terhadap Fundamentalisme Agama.

[Master’s Thesis, Yarsi University].

Basevitz, P., Pushkar, D., Chaikelson, J., Con- way, M., & Dalton, C. (2008). Age-related differences in worry and related processes.

International Journal of Aging and Human De- velopment, 66(4), 283–305. https://

doi.org/10.2190/AG.66.4.b

Berenbaum, H., Bredemeier, K., & Thompson, R.

J. (2008). Intolerance of uncertainty: Explor- ing its dimensionality and associations with need for cognitive closure, psychopatholo- gy, and personality. Journal of Anxiety Disor-

ders, 22(1), 117–125. https://

doi.org/10.1016/j.janxdis.2007.01.004 Birrell, J., Meares, K., Wilkinson, A., & Freeston,

M. (2011). Toward a definition of intolerance of uncertainty: A review of factor ana- lytical studies of the intolerance of uncertainty Scale. Clinical Psychology Review, 31 (7), 1198–1208. https://doi.org/10.1016/

j.cpr.2011.07.009

Brenes, G. A. (2006). Age differences in the presentation of anxiety. Aging and Mental Health, 10(3), 298–302. https://

doi.org/10.1080/13607860500409898 Carleton, R. N. (2016). Into the unknown: A re-

view and synthesis of contemporary mod- els involving uncertainty. Journal of Anxiety Disorders, 39, 30–43. http://

doi.org/10.1016/j.janxdis.2016.02.007 Carleton, R. N. (2012). The intolerance of uncer-

tainty construct in the context of anxiety disorders: Theoretical and practical per- spectives. Expert Review of Neurotherapeutics, 12(8), 937–947. https://doi.org/10.1586/

ern.12.82

Cohen, R. J., Swerdlik, M. E., & Sturman, E. D.

(2013). Psychological testing and assessment (8th ed). McGraw-Hill.

Cristobal, E., Flavián, C., & Guinalíu, M. (2007).

Perceived e‐service quality (PeSQ): Meas- urement validation and effects on consumer satisfaction and web site loyalty. Managing Service Quality. Managing Service [An inter- national journal], 17(3), 317–340. https://

doi.org/10.1108/09604520710744326 Dugas, M. J., Gosselin, P., & Ladouceur,

R. (2001). Intolerance of uncertainty and worry: Investigating specificity in a non- clinical sample. Cognitive Therapy and Re- search, 25(5), 551–558. https://

doi.org/10.1023/A:1005553414688 Einstein, D. A. (2014). Extension of transdiag-

nostic model to focus on intolerance of uncertainty: A review of the literature and implications for treatment. Clinical Psychology Science and Science, 21(3), 280–300. https://

doi.org/10.1111/cpsp.12077

Field, A. (2018). Discovering statistics using IBM SPSS Statistics (5th ed). SAGE Publications.

Freeston, M. H., Rhéaume, J., Letarte, H., Dugas, M. J., & Ladouceur, R. (1994). Why do people worry? Personality and Individual Differ- ences, 17(6), 791–802. https://

(11)

doi.org/10.1016/0191-8869(94)90048-5 Glowacz, F., & Schmits, E. (2020). Uncertainty

and psychological distress during lock- down during the COVID-19 pandemic: The young adults most at risk. Psychiatry Re- search, 293, 113486. https://

doi.org/10.1016/j.psychres.2020.113486 Graffeo, M. T. Albano, G., Salerno, L., Di Blasi,

M., & Lo Coco, G. (2022). Intolerance of uncertainty and risk perception during the COVID-19 pandemic: The mediating role of fear of COVID-19. Psych, 4, 269–276.

https://doi.org/10.3390/psych4020023 Gravetter, F. J., & Forzano, L. A. B. (2016). Re-

search methods for the behavioral sciences.

Cengage Learning.

Gruebner, O., Rapp, M. A., Adli, M., Kluge, U., Galea, S., & Heinz, A. (2017). Cities and mental health. Deutsches Ärtzeteblatt Interna- tional, 144(8), 121–127. https://

doi.org/10.3238/arztebl.2017.0121

Gu, Y., Gu, S., Lei, Y., & Li, H. (2020). From uncertainty to anxiety: How uncertainty fuels anxiety in a process mediated by intolerance of uncertainty. Neural Plasticity, 2020, 8866386. https://

doi.org/10.1155/2020/8866386

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis.

Cengage learning.

Harris, M. A., Brett, C. E., Johnson, W., & Deary, I. J. (2016). Personality stability from age 14 to age 77 years. Psychology and Aging, 31(8), 862-874. https://doi.org/10.1037/

pag0000133

Helsen, K., Van den Bussche, E. V., Vlaeyen, J.

W. S., & Goubert, L. (2013). Confirmatory factor analysis of the Dutch intolerance of uncertainty scale: Comparison of the full and short version. Journal of Behavior Thera- py and Experimental Psychiatry, 44(1), 21–29.

http://doi.org/10.1016/j.jbtep.2012.07.004 Hong, R. Y., & Lee, S. S. (2015). Further clarify-

ing prospective and inhibitory intolerance of uncertainty: Factorial and construct validity of test scores from the intolerance of uncertainty scale. Psychological Assessment, 27(2), 605–620. http://doi.org/10.1037/

pas0000074

Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alter-

natives. Structural Equation Modeling, 6(1), 1 –55. https://

doi.org/10.1080/10705519909540118 Huang, Y., & Zhao, N. (2020). Generalized anxi-

ety disorder, depressive symptoms and sleep quality during COVID-19 epidemic in China: A web-based cross-sectional survey.

Psychiatry Research, 288, 112954. https://

doi.org/10.1016/j.psychres.2020.112954 Jaya, E. S. (2020). Validation of patient health

questionnaire in Bahasa Indonesia [Unpublished manuscript].

Jaya, E. S. (2017). A longitudinal mediation analysis of the effect of negative self-schemas on positive symptoms via negative effect. Psy- chological Medicine, 48(8), 1–11. https://

doi.org/10.1017/S003329171700277X Kaplan, R., & Sacuzzo, D. (2009). Psychological

testing – Principles [Application], and Issue (7th ed). Wadsworth Publishing.

Kazdin, A. E. (2016). Research design in clinical psychology (5th ed). Pearson.

Koerner, N., & Dugas, M. J. (2006). A cognitive model of generalized anxiety disorder: The role of intolerance of uncertainty. In G. C. L.

Davey & A. Wells (Eds.), Worry and its Psychological Disorders: Theory, assessment and treatment (pp. 201–216). John Wiley & Sons.

Ladouceur, R., Gosselin, P., & Dugas, M. J.

(2000). Experimental manipulation of intolerance of uncertainty: A study of a theoretical model of worry. Behaviour Research and Therapy, 38(9), 933–941. https://

doi.org/10.1016/S0005-7967(99)00133-3 Luhmann, C. C., Ishida, K., & Hajcak, G. (2011).

Intolerance of uncertainty and decisions about delayed, probabilistic rewards. Behav- ior Therapy, 42(3), 378–386. https://

doi.org/10.1016/j.beth.2010.09.002

Mahoney, A. E. J., & McEvoy, P. M. (2012). Trait versus situation-specific intolerance of uncertainty in a clinical sample with anxiety and depressive disorders. Cognitive Behav- iour Therapy, 41(1), 26–39. https://

doi.org/10.1080/16506073.2011.622131 McEvoy, P. M., & Mahoney, A. E. J. (2011).

Achieving certainty about the structure of intolerance of uncertainty in a treatment- seeking sample with anxiety and depression. Journal of Anxiety Disorders, 25(1), 112–

122. https://doi.org/10.1016/

(12)

j.janxdis.2010.08.010

Meares, K., & Freeston, M. (2008). Overcoming worry: A self-help guide using cognitive behavioral techniques. Robinson Publish- ing.

Millon, T., & Bloom, C. (2008). The million in- ventories: A practitioner’s guide to person- alized assessment (2nd ed). Guilford Press.

Millroth, P., & Frey, R. (2021). Fear and anxiety in the face of COVID-19: Negative disposi- tions towards risk and uncertainty as vulnerability factors. Journal of Anxiety Disor- ders, 83, 102454. https://doi.org/10.1016/

j.janxdis.2021.102454

Moors, G. (2008). Exploring the effect of a middle response category on response style in attitude measurement. Quality and Quantity, 42(6), 779–794. https://doi.org/10.1007/

s11135-006-9067-x

Norton, P. J. (2005). A psychometric analysis of the intolerance of uncertainty scale among four racial groups. Journal of Anxiety Disor- ders, 19(6), 699–707. https://

doi.org/10.1016/j.janxdis.2004.08.002 Nunnally, J. C., & Bernstein, I. H. (1994). Psy-

chometric theory. McGraw-Hill, Inc.

Parlapani, E., Holeva, V., Nikopoulou, V. A., Sereslis, K., Athanasiadou, M., Godosidis, A., Stephanou, T., & Diakogiannis, I. (2020).

Intolerance of uncertainty and loneliness in older adults during the COVID-19 pandemic. Frontiers in Psychiatry, 11(842), 842.

https://doi.org/10.3389/fpsyt.2020.00842 Rettie, H., & Daniels, J. (2021). Coping and toler-

ance of uncertainty: Predictors and media- tors of mental health during the COVID-19 pandemic. American Psychologist

[Advance online publication], 76(3), 427–

437. http://doi.org/10.1037/amp0000710 Schreiber, J. B., Stage, F. K., King, J., Nora, A., &

Barlow, E. A. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. Journal of Educa- tional Research, 99(6), 323–338. https://

doi.org/10.3200/JOER.99.6.323-338 Sexton, K. A., & Dugas, M. J. (2009). Defining

distinct negative beliefs about uncertainty:

Validating the factor structure of the Intol- erance of Uncertainty Scale. Psychological Assessment, 21(2), 176–186. https://

doi.org/10.1037/a0015827

Sharifi, A., & Khavarian-Garmsir, A. R. (2020).

The COVID-19 pandemic: Impacts on cities and major lessons for urban planning, design, and management. Science of the Total Environment, 749, 142391. https://

doi.org/10.1016/j.scitotenv.2020.142391 Simms, L. J., Zelazny, K., Williams, T. F., &

Bernstein, L. (2019). Does the number of response options matter? Psychometric per- spectives using personality questionnaire data. Psychological Assessment, 31(4), 557–

566. https://doi.org/10.1037/pas0000648 Smith, B. M., Thowy, A. J., & Smith, G. S. (2020).

Psychological inflexibility and intolerance of uncertainty moderate the relationship between social isolation and mental health outcomes during COVID-19. Journal of Contextual Behavioral Science, 18, 161–174.

https://doi.org/10.1016/j.jcbs.2020.09.005 Urbina, S. (2004). Essentials of psychological

testing. John Wiley & Sons, Inc.