Fokkema JPI, Scheele F, Westerman M, et al. Perceived Effects of Innovations in Postgraduate Medical Education:
A Q Study Focusing on Workplace-Based Assessment. Acad Med. 2014;89(9).
Copyright © by the Assocation of American Medical Colleges. All rights reserved. 1
Supplemental Digital Appendix 1:
Individual Q Sort Loadings for Five Factors Representing Principal Perceptions of Effects of Using Workplace‐Based Assessment (WBA) in Practice, as Determined From a Study of Users of WBA in Residency Training in the Netherlands, 2012aQ sort
Participant characteristic
Factor: Perception 1:
Enthusiasm
2:
Compliance
3:
Effort
4:
Neutrality
5:
Skepticism Role Hospital
1 R 1 0.30 0.33 0.40 0.11 0.16
2 A 1 0.14 0.66b 0.16 0.10 –0.09
3 A 1 0.69b 0.53 0.15 0.16 0.13
4 A 1 0.83b –0.01 0.22 0.09 –0.04
5 A 1 0.61b 0.26 0.26 0.27 –0.25
6 A 2 0.60 0.52 0.23 0.16 –0.11
7 R 1 0.31 0.53 0.45 0.11 0.07
8 A 2 0.33 0.56 0.61 0.08 –0.05
9 A 2 0.19 0.20 0.66b 0.06 0.29
10 A 2 0.14 0.39 0.11 0.08 0.71b
11 R 2 0.26 0.58b 0.06 0.02 0.05
12 A 2 0.73b 0.36 0.01 –0.15 –0.13
13 A 2 0.23 –0.25 0.22 0.66b 0.20
14 A 2 0.23 0.29 0.07 0.56b 0.25
15 R 2 0.15 0.75b 0.10 0.36 0.02
16 A 2 –0.23 0.17 0.11 0.09 0.69b
17 R 2 0.26 0.62b 0.11 0.21 0.24
18 R 1 0.44 0.69b 0.19 0.02 0.05
19 A 1 0.62b 0.25 0.17 0.45 –0.24
20 A 1 0.34 0.50 0.43 0.04 0.21
21 R 2 0.29 0.58b 0.32 0.23 0.11
22 A 2 0.35 0.36 0.43 0.37 –0.03
23 A 2 0.34 0.64b 0.10 0.07 0.31
24 A 2 0.34 0.27 0.24 0.59b 0.19
25 A 2 0.53b 0.04 0.35 0.26 0.02
26 A 1 0.01 0.21 0.54b 0.13 0.02
27 A 1 0.68b –0.03 0.15 0.24 0.10
28 R 2 0.20 0.54b 0.31 0.12 0.36
29 A 1 0.68b 0.35 0.14 –0.07 –0.35
30 A 2 0.33 0.54 0.44 –0.02 0.16
31 R 2 0.47 0.62 –0.02 0.26 0.30
32 R 3 0.23 0.67b 0.24 0.01 0.39
33 A 3 0.14 0.57b 0.42 0.22 0.08
34 R 3 –0.04 –0.07 0.07 0.04 0.46b
Fokkema JPI, Scheele F, Westerman M, et al. Perceived Effects of Innovations in Postgraduate Medical Education:
A Q Study Focusing on Workplace-Based Assessment. Acad Med. 2014;89(9).
Copyright © by the Assocation of American Medical Colleges. All rights reserved. 2
Q sort
Participant characteristic
Factor: Perception 1:
Enthusiasm
2:
Compliance
3:
Effort
4:
Neutrality
5:
Skepticism Role Hospital
35 A 3 0.20 0.60b 0.11 0.45 0.33
36 A 2 0.49 0.42 0.09 0.44 –0.06
37 R 1 0.36 0.37 0.48 0.05 –0.25
38 R 1 –0.10 0.49b 0.44 0.03 0.03
39 R 2 0.31 0.55b 0.35 0.01 0.11
40 A 4 0.27 0.47 0.57 0.19 0.14
41 A 4 0.38 0.37 0.19 0.60b –0.02
42 A 4 0.53 0.48 0.02 0.24 0.27
43 A 4 0.20 0.09 0.74b 0.36 0.19
44 A 4 0.29 0.46 0.21 0.43 –0.05
45 A 4 –0.14 0.08 0.13 0.70b –0.07
46 A 4 0.35 0.15 0.43 0.41 0.00
47 R 5 0.01 0.54 0.03 0.14 0.68b
48 R 5 0.28 0.51b 0.37 0.13 0.03
49 R 5 0.02 0.65b 0.46 0.08 0.11
50 A 5 0.34 0.69b 0.27 0.14 –0.09
51 A 5 0.44 0.62b 0.38 –0.02 –0.04
52 A 5 –0.14 0.07 0.07 –0.03 0.53b
53 A 5 0.58b 0.45 0.15 0.17 –0.13
54 A 5 0.19 0.23 0.25 –0.31 0.28
55 A 6 0.38 0.44 0.33 0.05 –0.38
56 A 6 0.22 0.07 0.52b 0.33 0.14
57 A 6 0.48 0.43 0.46 0.10 0.12
58 R 6 0.00 0.60b 0.24 0.36 0.17
59 R 6 0.11 0.48 0.24 0.43 0.13
60 A 6 0.70b 0.55 0.18 –0.14 –0.06
61 A 6 0.48 0.36 0.39 0.37 –0.10
62 R 6 0.46 0.67b 0.06 0.11 –0.12
63 R 6 0.57b 0.30 0.24 0.25 0.19
64 A 2 0.10 0.56b 0.36 0.00 0.32
65 R 5 –0.05 0.69b 0.26 0.07 0.29
aThe 65 participating users were residents (Rs), attending physicians (As), and program directors in departments of obstetrics–gynecology at 6 hospitals. In this table, program directors are labeled as attending physicians to preserve their anonymity (n < 5). The six hospitals are labeled 1 through 6.
bDefining Q sort for factor. A Q sort defines a factor if: (i) the Q sort correlates statistically significantly with that factor; the loading of a respondent on a factor should exceed the multiplier for the statistical significance level (P = .05) divided by the square root of the number of statements, in this case: 1.96 36 0.33; and(ii) the factor explains more than half of the common variance; that is, the square of the loading on that factor should exceed the sum of the squares of the factor loadings on the remaining factors.
