• Tidak ada hasil yang ditemukan

NOTICE

N/A
N/A
Protected

Academic year: 2023

Membagikan "NOTICE"

Copied!
26
0
0

Teks penuh

(1)

NOTICE: This document contains correspondence generated during peer review and subsequent revisions but before transmittal to production for composition and copyediting:

• Comments from the reviewers and editors (email to author requesting revisions)

• Response from the author (cover letter submitted with revised manuscript)*

*The corresponding author has opted to make this information publicly available.

Personal or nonessential information may be redacted at the editor’s discretion.

Questions about these materials may be directed to the Obstetrics & Gynecology editorial office:

[email protected].

(2)

Date: May 26, 2020

To: "Chi Chiung Grace Chen"

From: "The Green Journal" [email protected] Subject: Your Submission ONG-20-957

RE: Manuscript Number ONG-20-957

Establishing Validity for a Vaginal Hysterectomy Simulation-Based Assessment Dear Dr. Chen:

Your manuscript has been reviewed by the Editorial Board and by special expert referees. Although it is judged not acceptable for publication in Obstetrics & Gynecology in its present form, we would be willing to give further consideration to a revised version.

If you wish to consider revising your manuscript, you will first need to study carefully the enclosed reports submitted by the referees and editors. Each point raised requires a response, by either revising your manuscript or making a clear and convincing argument as to why no revision is needed. To facilitate our review, we prefer that the cover letter include the comments made by the reviewers and the editor followed by your response. The revised manuscript should indicate the position of all changes made. We suggest that you use the "track changes" feature in your word processing software to do so (rather than strikethrough or underline formatting).

***Due to the COVID-19 pandemic, your paper will be maintained in active status for 30 days from the date of this letter.

If we have not heard from you by Jun 25, 2020, we will assume you wish to withdraw the manuscript from further consideration.***

REVIEWER COMMENTS:

Reviewer #1: Review of Manuscript ONG-20-957 "Establishing validity for a vaginal hysterectomy simulation-based assessment"

Chen and colleagues have submitted results from a group of 50 surgeons that utilized a commercial vaginal hysterectomy trainer and then had "Experts" assess videos of the study participants in order to determine competency or lack thereof based on several previously evaluated, although modified in at least 1 case, scales. The authors provide a reasonable commentary on the importance of surgical simulation and why fidelity is needed in this important area of medicine. As a minor note, there are several stray and remaining track changes present in the document. I have the following questions and comments.

Title - Consider noting in the title some about a surgical training model - don't have to mention the model by name - and/or video review which more closely captures what you all did.

Précis - I think you could consider stating that it "… appears to be an effective tool…"

Abstract - Line 43 - Really this is just for vaginal hysterectomy skills unless data demonstrates otherwise that it is for all vaginal surgical procedures.

Line 49 - Do you identify later who the blinded reviewers were - authors/investigators on the study?

Line 55 - Appears to be a typo as the same p value is represented 2 different ways.

Line 57 - Why not include information re: the AUC here for competency or not?

Line 60-1 - Do you know if these skills translate to other vaginal surgical procedures? If not, should you state that this assessment applies to vaginal hysterectomy surgical skills?

Introduction - The authors have outlined issues related to why the study was performed and built on previous experiences.

Methods -

Line113-4 - Can you provide commentary on the fact that while each cadaver may be different, full anatomy, including lower limbs are present and while individuals are likely not performing a vaginal hysterectomy on a cadaver with the limbs in place, was thought given on how to at least mimic the presence of extremities?

Line 136 - Thank you for providing information on the approximate cost of the model. Since it appears that the inserts have to be replaced after each use, is there any cost and/or cost effectiveness information available?

Line 137 - Only the traditional more expensive insert was used for this study correct? If so should clarify that while another insert may be available it was not evaluated in this study.

View Letter

1 of 7 6/29/2020, 2:32 PM

(3)

Line 141 - How were residents identified for participation? Did their program director know if they did or did not participate?

Line 150 - Is data available regarding the applicability and/or appropriateness of using the modified versus the original VSSI global rating scale? How was it modified? What was utilized in the previous study you referenced?

Line 153 - See comment re: line 150 above in terms of information about the validity of modifying this scale.

Line 155 - How do we know the "experts" that reviewed the videos are in fact experts? Did they perform the procedure on the model and have someone else assess them? What is their background? If the entire basis of the paper is that experts can successful adjudicate and assess videos to determine competence and lack thereof, more information on why they are experts is needed.

Line 160 - Appears to be the modifications. Perhaps this should be moved up with the previous comments?

Results - Line 205 - Since you modified the VSSI, should you also make it clear in the results that this was a modified VSSI?

Line 206-7 - First portion seems to be more commentary/discussion, please remove and just note the data and differences observed.

Discussion - Line 236 - Sorry to harp on this again, but if the measure was modified you should note that it was modified throughout the manuscript.

Line 270-1 - You note that the participants had no commercial ties to the product, but it appears that at least 2 of the authors (Drs. Anderson and Myazaki) have a commercial tie/interest in the product, which is a potential cause of bias. How was this managed for the study? Did either of these authors review the videos - I presume Dr. Anderson did not as he is not a Medical Doctor?

Tables - Table 1 - is the column entitled "Competent" only referring to individuals that were fully competent per the assessments or more "experienced" surgeons? I think the heading may need to be "Novice Surgeons" and "Experienced Surgeons" since you are trying to determine competence based on the assessment of their performance on the model.

Table 2 - Can be supplementary in my mind as you note the key points in the manuscript already.

Table 3 - So I presume that the PGY1 and PGY 2/3 are the 33 novices from table 1. If this is the case, then perhaps for Table 1 it should be "Novice Surgeons (PGY1-3)" and likewise "Experienced Surgeons (PGY4+)"

Table 4 - Please adjust column headings as previously noted.

Figures - Figure 1 - Important and fine

Figures 2 and 3 - Perhaps could be combined into a Figure 2A and 2B?

Figure 4 - No comments.

Reviewer #2: This interesting paper brings into view the emergence of simulated clinical situations in the training and evaluation of clinicians. The demands on both hospitals and practices to ensure competency before granting or renewing surgical privileges for particular procedures as well as the deficit of real-life training opportunities in which clinicians in training can learn initial competency have made the availability of controlled, safe clinical scenarios increasingly important.

While minimally invasive surgery for the purpose of hysterectomy has garnered much attention over the past 15 years or so, ACOG continues to recommend total vaginal hysterectomy as "the preferred method among minimally invasive procedures". Moreover ACOG has strongly endorsed and recommended use of simulators for training, surgery skill

evaluation, and emergency drills. They provide simulation as a topic for use as a Part IV MOC fulfillment and make toolkits available that can be employed for simulated obstetric emergencies such as shoulder dystocia and obstetric hemorrhage.

One can find video instruction in the fashioning of makeshift simulators for training in hysterectomy, laparoscopic surgery, and other procedures, some placed on the internet by ACOG.

While these more rudimentary devices have some value, they do not provide high-quality, reproducible, and reliable means of transferring skills to actual patient-care settings, and these devices are never meant to substitute for real-life training or experience. This opens a demand for simulators that mimic patient care scenarios as closely as possible, ideally providing opportunities for training and skills assessment that might reflect actual practice. With that in mind several higher quality simulator have become commercially available in recent years.

Among these is the Miya Model(TM) pelvic training simulator. This device has several advantages: it is highly faithful to true pelvic anatomic relationships, has replaceable parts for multiple users, and can be used for the training and assessment of a variety of variety of gynecologic procedures.

In an attempt to extend the use of this simulator, the authors employ a validation process slightly modified from one devised by Messick and colleagues (in preferential use by the American Psychological Association, the American Educational Research Association, and others) to determine the integrity of inter-user reproducibility, consistency, and reliability of results among the 11 faculty and 39 trainees recruited for the study. The device itself, as described, has a high degree of fidelity to normal anatomic relationships and allows for significant interaction by the user by supplying disposable anatomic elements. Significantly it does not contain the ability to simulate blood loss and control and one assumes this will limit its utility at the highest level.

The unfamiliarity of this topic to many practicing clinicians along with technical jargon that can seem inscrutable to those who are not intimate with the topic an make this a difficult read. Moreover this reviewer does not have the statistical familiarity with the tools employed by the authors to validate this device, and such a review is essential before acceptance.

Nonetheless the opportunity exists for the curious reader to learn about this interesting and for many of us novel yet emerging topic in our specialty. While the authors appear to have validated the components under study for this device,

View Letter

2 of 7 6/29/2020, 2:32 PM

(4)

this highly technical definition of validity should not be interpreted to mean that clinical application of the skills learned from the simulator is validated. The authors make no such assertion. However, this is a step in that direction and as such represents an important potential milestone in the emergence of a reliable simulator in the training and evaluation of pelvic surgeons.

Reviewer #3: The authors report the results of a study to establish validity evidence for a vaginal hysterectomy (VH) simulation model. The strengths of the study include use of a modern framework for validity studies and safeguards to minimize bias in the evaluation of VH skills. I have several questions and suggestions for the authors.

1. The introduction is wordy and can be reduced in length without sacrificing the statement of the problem and gap in the literature addressed by the study.

2. Line 86 reports that a pilot study established the "acceptability and realism" of the model. However, the citation (5) refers to a study that aimed to "present pilot data on validity and reliability of the model" and did not report outcomes on acceptability and realism.

3. The sentence on lines 87-89 states, "Use of the model to assess vaginal hysterectomy skills coupled with previously validated global rating scales (including the Vaginal Surgical Skills Index (VSSI)(6) also resulted in moderate to high interrater reliability." Citation (6) refers to a study providing reliability and validity evidence for VSSI in real VH cases and not the use of the model. In what publication was moderate to high interrater reliability reported for the model? Are you referring to the pilot study (citation 5)?

4. The methods section includes a lengthy description of the Miya Model and uses a figure to illustrate it. This

information and the figure were published previously in reference 5. A shorter description, with or without republication of the figure, would suffice here if you refer readers to the prior published description of the model.

5. Line 139: please describe how the study participants were selected. Were they a convenience sample recruited over a specific time period, or was there a sampling strategy to assure a balance of novice and expert surgeons? Were the 10 novice and 10 expert gynecologists used in the pilot study (reference 5) included in this study also? Have any other studies been conducted evaluating the Miya VH model that did not contribute to this dataset?

6. Line 145: provide a citation regarding the procedural steps outlined by SGS and ACOG that you mention here.

7. Lines 152-153: provide additional detail on the verbal descriptors or anchors used on the 5-level VSSI subscales and the 7-level GS overall performance item.

8. Line 153: the pass-fail "criterion" or "metric" or "standard" is extremely important to detail because it is used as the gold standard to define "competent" performance in subsequent analyses. How was the pass-fail criterion on the model established and did it predict VH performance on actual patients? I did not see a citation providing this information from previous studies. A comment regarding the validity evidence for the pass-fail criterion should be include in the discussion.

9. Lines 168-172: outcomes are categorized into primary (VH skill assessments) and secondary (establishing a VSSI cut point for "competent" performance) followed by participants' post-simulation evaluation of the model. This same order should be used to report the results later the manuscript on lines 195-227.

10. Line 189: each participant performed only 1 VH on the Miya Model. How accurate is one case for appraising

performance? Have other studies evaluated the test-retest reliability (reproducibility) of the model? If not, this should be discussed in the limitations section of the discussion.

11. Line 195: as mentioned above, the presentation of results should follow the order of primary, secondary and additional outcomes, and not feature the least meaningful outcome first.

12. Line 209: it is surprising that the number of hysterectomies performed showed only a moderate correlation with VSSI and GS scores. The R-squared of these correlations shows that only 30-34% of the variance in performance was explained by experience.

13. Line 212: the wide range of intraclass correlation and inter-rater reliability is explained as "depending on the exact pair of evaluators that were being compared." The method section reports that each video-recorded case on the model was evaluated by two independent experts. How many total experts were used, and what was the overall reliability of the group? Did the group of expert evaluators overlap with the expert gynecologic surgeons who participated in the study?

14. Line 216: the ROC analysis using the pass-fail metric as a gold standard is only valid if the gold-standard is independent of the predictor variables (VSSI and GS). As mentioned above, further detail regarding the pass-fail metric will clarify this, and help determine whether ROC analysis was appropriate.

15. Line 217: the ROC cut-point of 27 is surprising in light of Figure 1 showing that this level was achieved by many of the novices with very little VH experience. This again begs the question of what the pass-fail gold standard is for these analyses, and whether that gold standard has any predictive relationship to VH performance with real patients.

16. Line 236: the first sentence in the discussion states that the current study "demonstrated the Miya Model to be an effective tool when coupled with either the VSSI or GS to assess vaginal hysterectomy skills." However, it is premature to report the model is effective without further understanding the pass-fail gold standard used in the analyses, particularly how independent it is of the VSSI and GS scoring systems and how it predicts actual VH performance.

17. Line 261: this paragraph emphasizes the limitations of other VH training models, but does not report any studies comparing the different models to support these criticisms.

18. Line 283: the limitations refers to not assessing the model as a teaching tool. However, that was not the study's purpose and it isn't as important to mention as the methodologic limitations that would threaten the validity of the study's conclusions. I've mentioned several in the comments above.

19. Line 296: the establishment of cut-scores for "competent" and "incompetent" may not be appropriate to report, depending on the nature and limitations of the pass-fail criterion.

20. Figures and Tables: (a) The results (and tables) should follow the same order as the primary, secondary and additional analysis. (b) The current Table 2 can be reported more briefly in the text of the article rather than a table. (c) The current Table 3 does not report correlations so its title should be changed; the columns with p-values should be

View Letter

7 6/29/2020, 2:32 PM

(5)

eliminated and replaced with a footnote reporting the p-value for linear trend by training level (ANOVA with contrast analysis instead of pairwise t-tests). (d) The current Table 4 has "interrater agreement" in the title but does not present data on interrater agreement; the p-values can be indicated by a footnote; the column heading should agree with the narrative on 225 in which you use "incompetent" rather than "novice. (e) Figure 1 has been published previously. (f) Figures 3 and 4 estimate correlation coefficients using nearly bimodal data due to the study's focus on novice and expert gynecologic surgeons. These curves may not accurately estimate correlations that include mid-range levels of experience.

(g) Figure 4 uses a circle to denote the cut-point, but the circle is too large to see where it intersects the curve; use an arrow instead.

STATISTICAL EDITOR COMMENTS:

The Statistical Editor makes the following points that need to be addressed:

1. lines 57, 212: Should round the ICC to nearest 0.01, not to 0.001 precision.

2. lines 214-215: Need to round the kappa to nearest 0.01.

3. Tables 1, 2 (and in text): Since the denominators for the two groups were n = 33 and n = 17, need to round all %s to nearest integer%, not to nearest 0.1% precision.

4. Table 3: The counts for most training levels is modest, so the power to discern whether the distributions of VSSI or GS scores were normal vs non-normal has limited power. Should default to using a non-parametric test and cite the scores as median(IQR or range), rather than as mean±SD. The numerical differences are large, so the inferences likely will be unchanged, although some p-values may be higher.

5. Figs 2, 3: Although these r values are statistically significant, it is made so by the clustering of scores at very low vs high numbers of vag hysts performed. Could be included as supplemental material, but does not seem crucial to the exposition of results.

6. Fig 4 and legend and lines 216-222: Need to include the AUC (with CIs) for the ROC. The optimal cutpoint (score = 27 in this case) has a confidence interval. Need to include the CIs. Also need to include CIs for the sens and spec at score = 27.

ASSOCIATE EDITOR - GYN

Thank you for this research article. We have been experiencing record numbers of submissions and as a result have strict page limit considerations - please keep this in mind as you work through the revisions above. Also, consider which figures might be okay to move an online-only Appendix. It is likely that some text will be cut from your revised submission and/or figures moved to the appendixes anyway due to our need to not go over our page limit restrictions for the year.

EDITOR COMMENTS:

1. The Editors of Obstetrics & Gynecology are seeking to increase transparency around its peer-review process, in line with efforts to do so in international biomedical peer review publishing. If your article is accepted, we will be posting this revision letter as supplemental digital content to the published article online. Additionally, unless you choose to opt out, we will also be including your point-by-point response to the revision letter. If you opt out of including your response, only the revision letter will be posted. Please reply to this letter with one of two responses:

A. OPT-IN: Yes, please publish my point-by-point response letter.

B. OPT-OUT: No, please do not publish my point-by-point response letter.

2. Obstetrics & Gynecology uses an "electronic Copyright Transfer Agreement" (eCTA). When you are ready to revise your manuscript, you will be prompted in Editorial Manager (EM) to click on "Revise Submission." Doing so will launch the resubmission process, and you will be walked through the various questions that comprise the eCTA. Each of your coauthors will receive an email from the system requesting that they review and electronically sign the eCTA.

Please check with your coauthors to confirm that the disclosures listed in their eCTA forms are correctly disclosed on the manuscript's title page.

3. Our journal requires that all evidence-based research submissions be accompanied by a transparency declaration statement from the manuscript's lead author. The statement is as follows: "The lead author* affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained."

View Letter

4 of 7 6/29/2020, 2:32 PM

(6)

*The manuscript's guarantor.

If you are the lead author, please include this statement in your cover letter. If the lead author is a different person, please ask him/her to submit the signed transparency declaration to you. This document may be uploaded with your submission in Editorial Manager.

4. Tables, figures, and supplemental digital content should be original. The use of borrowed material (eg, lengthy direct quotations, tables, figures, or videos) is discouraged. If the material is essential, written permission of the copyright holder must be obtained.

Both print and electronic (online) rights must be obtained from the holder of the copyright (often the publisher, not the author), and credit to the original source must be included in your manuscript. Many publishers now have online systems for submitting permissions request; please consult the publisher directly for more information. Permission is also required for material that has been adapted or modified from another source. Increasingly, publishers will not grant permission for modification of their material. Creative Commons licenses and open access have also made obtaining permissions more challenging. In order to avoid publication delays, we strongly encourage authors to link or reference to the material they want to highlight instead of trying to get permission to reprint it. For example, "see Table 1 in Smith et al" (and insert reference number). For articles that the journal invites, such as the Clinical Expert Series, the journal staff does not seek permission for modifications of material — the material will be reprinted in its original form.

When you submit your revised manuscript, please upload 1) the permissions license and 2) a copy of the original source from which the material was reprinted, adapted, or modified (eg, scan of book page(s), PDF of journal article, etc.).

If the figure or table you want to reprint can be easily found on the internet from a reputable source, we recommend providing a link to the source in your text instead of trying to reprint it in your manuscript.

5. Responsible reporting of research studies, which includes a complete, transparent, accurate and timely account of what was done and what was found during a research study, is an integral part of good research and publication practice and not an optional extra. Obstetrics & Gynecology supports initiatives aimed at improving the reporting of health research, and we ask authors to follow specific guidelines for reporting randomized controlled trials (ie, CONSORT), observational studies (ie, STROBE), observational studies using ICD-10 data (ie, RECORD), meta-analyses and systematic reviews of randomized controlled trials (ie, PRISMA), harms in systematic reviews (ie, PRISMA for harms), studies of diagnostic accuracy (ie, STARD), meta-analyses and systematic reviews of observational studies (ie, MOOSE), economic evaluations of health interventions (ie, CHEERS), quality improvement in health care studies (ie, SQUIRE 2.0), and studies reporting results of Internet e-surveys (CHERRIES). Include the appropriate checklist for your manuscript type upon submission.

Please write or insert the page numbers where each item appears in the margin of the checklist. Further information and links to the checklists are available at http://ong.editorialmanager.com. In your cover letter, be sure to indicate that you have followed the CONSORT, MOOSE, PRISMA, PRISMA for harms, STARD, STROBE, RECORD, CHEERS, SQUIRE 2.0, or CHERRIES guidelines, as appropriate.

6. Standard obstetric and gynecology data definitions have been developed through the reVITALize initiative, which was convened by the American College of Obstetricians and Gynecologists and the members of the Women's Health Registry Alliance. Obstetrics & Gynecology has adopted the use of the reVITALize definitions. Please access the obstetric and gynecology data definitions at https://www.acog.org/About-ACOG/ACOG-Departments/Patient-Safety-and-Quality- Improvement/reVITALize. If use of the reVITALize definitions is problematic, please discuss this in your point-by-point response to this letter.

7. Because of space limitations, it is important that your revised manuscript adhere to the following length restrictions by manuscript type: Original Research reports should not exceed 22 typed, double-spaced pages (5,500 words). Stated page limits include all numbered pages in a manuscript (i.e., title page, précis, abstract, text, references, tables, boxes, figure legends, and print appendixes) but exclude references.

8. Specific rules govern the use of acknowledgments in the journal. Please note the following guidelines:

* All financial support of the study must be acknowledged.

* Any and all manuscript preparation assistance, including but not limited to topic development, data collection, analysis, writing, or editorial assistance, must be disclosed in the acknowledgments. Such acknowledgments must identify the entities that provided and paid for this assistance, whether directly or indirectly.

* All persons who contributed to the work reported in the manuscript, but not sufficiently to be authors, must be

acknowledged. Written permission must be obtained from all individuals named in the acknowledgments, as readers may infer their endorsement of the data and conclusions. Please note that your response in the journal's electronic author form verifies that permission has been obtained from all named persons.

* If all or part of the paper was presented at the Annual Clinical and Scientific Meeting of the American College of Obstetricians and Gynecologists or at any other organizational meeting, that presentation should be noted (include the exact dates and location of the meeting).

View Letter

7 6/29/2020, 2:32 PM

(7)

9. The most common deficiency in revised manuscripts involves the abstract. Be sure there are no inconsistencies between the Abstract and the manuscript, and that the Abstract has a clear conclusion statement based on the results found in the paper. Make sure that the abstract does not contain information that does not appear in the body text. If you submit a revision, please check the abstract carefully.

In addition, the abstract length should follow journal guidelines. The word limit for Original Research articles is 300 words.

Please provide a word count.

10. Only standard abbreviations and acronyms are allowed. A selected list is available online at http://edmgr.ovid.com /ong/accounts/abbreviations.pdf. Abbreviations and acronyms cannot be used in the title or précis. Abbreviations and acronyms must be spelled out the first time they are used in the abstract and again in the body of the manuscript.

11. The commercial name "Miya Model" (with the generic name in parentheses) may be used once in the body of the manuscript. Use the generic name at each mention thereafter. Commercial names should not be used in the title, running title, précis, or abstract.

12. The journal does not use the virgule symbol (/) in sentences with words. Please rephrase your text to avoid using

"and/or," or similar constructions throughout the text. You may retain this symbol if you are using it to express data or a measurement.

13. In your Abstract, manuscript Results sections, and tables, the preferred citation should be in terms of an effect size, such as odds ratio or relative risk or the mean difference of a variable between two groups, expressed with appropriate confidence intervals. When such syntax is used, the P value has only secondary importance and often can be omitted or noted as footnotes in a Table format. Putting the results in the form of an effect size makes the result of the statistical test more clinically relevant and gives better context than citing P values alone.

If appropriate, please include number needed to treat for benefits (NNTb) or harm (NNTh). When comparing two procedures, please express the outcome of the comparison in U.S. dollar amounts.

Please standardize the presentation of your data throughout the manuscript submission. For P values, do not exceed three decimal places (for example, "P = .001"). For percentages, do not exceed one decimal place (for example, 11.1%").

14. Please review the journal's Table Checklist to make sure that your tables conform to journal style. The Table Checklist is available online here: http://edmgr.ovid.com/ong/accounts/table_checklist.pdf.

15. The American College of Obstetricians and Gynecologists' (ACOG) documents are frequently updated. These

documents may be withdrawn and replaced with newer, revised versions. If you cite ACOG documents in your manuscript, be sure the reference you are citing is still current and available. If the reference you are citing has been updated (ie, replaced by a newer version), please ensure that the new version supports whatever statement you are making in your manuscript and then update your reference list accordingly (exceptions could include manuscripts that address items of historical interest). If the reference you are citing has been withdrawn with no clear replacement, please contact the editorial office for assistance ([email protected]). In most cases, if an ACOG document has been withdrawn, it should not be referenced in your manuscript (exceptions could include manuscripts that address items of historical

interest). All ACOG documents (eg, Committee Opinions and Practice Bulletins) may be found at the Clinical Guidance page at https://www.acog.org/clinical (click on "Clinical Guidance" at the top).

16. When you submit your revision, art saved in a digital format should accompany it. If your figure was created in

Microsoft Word, Microsoft Excel, or Microsoft PowerPoint formats, please submit your original source file. Image files should not be copied and pasted into Microsoft Word or Microsoft PowerPoint.

When you submit your revision, art saved in a digital format should accompany it. Please upload each figure as a separate file to Editorial Manager (do not embed the figure in your manuscript file).

Art that is low resolution, digitized, adapted from slides, or downloaded from the Internet may not reproduce.

Figure 1: This appears to be a photo from the manufacturer. Please provide a letter of permission for print and online use.

Figure 2: Please move the numbers from the bottom right corner to the legend.

Figure 3: Please upload a higher resolution version of this figure and move the numbers from the bottom right corner to the legend.

17. Each supplemental file in your manuscript should be named an "Appendix," numbered, and ordered in the way they are first cited in the text. Do not order and number supplemental tables, figures, and text separately. References cited in appendixes should be added to a separate References list in the appendixes file.

View Letter

7 6/29/2020, 2:32 PM

(8)

18. Authors whose manuscripts have been accepted for publication have the option to pay an article processing charge and publish open access. With this choice, articles are made freely available online immediately upon publication. An

information sheet is available at http://links.lww.com/LWW-ES/A48. The cost for publishing an article as open access can be found at http://edmgr.ovid.com/acd/accounts/ifauth.htm.

Please note that if your article is accepted, you will receive an email from the editorial office asking you to choose a publication route (traditional or open access). Please keep an eye out for that future email and be sure to respond to it promptly.

***

If you choose to revise your manuscript, please submit your revision through Editorial Manager at

http://ong.editorialmanager.com. Your manuscript should be uploaded in a word processing format such as Microsoft Word.

Your revision's cover letter should include the following:

* A confirmation that you have read the Instructions for Authors (http://edmgr.ovid.com/ong/accounts/authors.pdf), and

* A point-by-point response to each of the received comments in this letter.

If you submit a revision, we will assume that it has been developed in consultation with your co-authors and that each author has given approval to the final form of the revision.

***Again, your paper will be maintained in active status for 30 days from the date of this letter. If we have not heard from you by Jun 25, 2020, we will assume you wish to withdraw the manuscript from further consideration.***.

Sincerely,

The Editors of Obstetrics & Gynecology 2018 IMPACT FACTOR: 4.965

2018 IMPACT FACTOR RANKING: 7th out of 83 ob/gyn journals __________________________________________________

In compliance with data protection regulations, you may request that we remove your personal registration details at any time. (Use the following URL: https://www.editorialmanager.com/ong/login.asp?a=r). Please contact the publication office if you have any questions.

View Letter

7 of 7 6/29/2020, 2:32 PM

(9)

12 June 2020 Dear Editors,

We respectfully re-submit our original manuscript entitled, “Establishing Validity for a Vaginal Hysterectomy Simulation-Based Assessment.” We have responded to the reviewers’ comments point by point attached to this letter. We have also made changes to the manuscript when appropriate. In addition, although the disclosure for Dr. Lockrow has not changed, we updated the cover page with disclosure language required by the Uniformed Services University of the Health Sciences. We also added additional acknowledgements: Cara Olsen MS DrPH for statistical support (for the revisions), data collection support by Noah Miyazaki BA and Jesse Ykimoff, MD, and data interpretation and manuscript readability as it pertains to the validation framework by Edgar LeClaire MD MS. We have obtained written permissions from all individuals mentioned in the acknowledgements (Cara Olsen Ms DrPH, Sorana Raiciulescu MSc, Eliza Burr BA, Noah Miyazaki BA, Jesse Ykimoff, MD, Edgar LeClaire MD MS).

All authors contributed substantially to the concept, design, analysis, and/ or interpretation of the data presented in the manuscript and have approved the final version of the manuscript for submission. Specifically, as the lead author, I spearheaded the project and was responsible for all stages of project completion including the IRB application, research design, implementation, analysis, and manuscript writing. In addition, as per the Obstetrics

& Gynecology transparency declaration, I affirm that “this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained”.

Other authors include Dr. Miyazaki who is the owner of Miyazaki Enterprises and Dr. Anderson who is a shareholder in Miyazaki Enterprises. Both Drs. Miyazaki and Anderson contributed substantially to the

conception and design of the model, and study design and implementation; however, they did not participate in data entry, analysis, or interpretation due to potential conflicts of interest. Drs. Lockrow, Destephano, Nihira, Kammire, Matthews, and Landrum contributed to the acquisition, analysis, and interpretation of the data as well as manuscript writing. Dr. Miyazaki is a speaker for Coloplast and Boston Scientific in addition to owning Miyazaki Enterprises, Dr. Matthews is a consultant for and has received grant funding from Boston Scientific, has received grant funding from Neomedic (Terrasa, Barcelona), and has been an expert witness for Johnson &

Johnson. Dr. Nihira is a legal expert for Ethicon, a consultant for Boston Scientific, Pacira, and Hologic, and has received honoraria from the Surgical Pain Consortium, ACOG, and ABOG, and Dr. Anderson is a

shareholder in Miyazaki Enterprises and principal and owner of Augmented Reality Systems, Inc. Please find the conflict of interest forms for all authors in our uploaded documents. We have also uploaded the Author Statement form signed for all authors. Thank you for considering our work.

Sincerely,

Chi Chiung Grace Chen, MD MHS

(10)

EDITOR COMMENTS:

1. The Editors of Obstetrics & Gynecology are seeking to increase transparency around its peer-review process, in line with efforts to do so in international biomedical peer review publishing. If your article is accepted, we will be posting this revision letter as supplemental digital content to the published article online. Additionally, unless you choose to opt out, we will also be including your point-by-point response to the revision letter. If you opt out of including your response, only the revision letter will be posted. Please reply to this letter with one of two responses:

A. OPT-IN: Yes, please publish my point-by-point response letter.

B. OPT-OUT: No, please do not publish my point-by-point response letter.

We are agreeable to proceeding with option A.

2. Obstetrics & Gynecology uses an "electronic Copyright Transfer Agreement" (eCTA). When you are ready to revise your manuscript, you will be prompted in Editorial Manager (EM) to click on "Revise Submission."

Doing so will launch the resubmission process, and you will be walked through the various questions that comprise the eCTA. Each of your coauthors will receive an email from the system requesting that they review and electronically sign the eCTA.

Please check with your coauthors to confirm that the disclosures listed in their eCTA forms are correctly disclosed on the manuscript's title page.

This is done.

3. Our journal requires that all evidence-based research submissions be accompanied by a transparency

declaration statement from the manuscript's lead author. The statement is as follows: "The lead author* affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no

important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained." *The manuscript's guarantor.

If you are the lead author, please include this statement in your cover letter. If the lead author is a different person, please ask him/her to submit the signed transparency declaration to you. This document may be uploaded with your submission in Editorial Manager.

This is done.

4. Tables, figures, and supplemental digital content should be original. The use of borrowed material (eg, lengthy direct quotations, tables, figures, or videos) is discouraged. If the material is essential, written permission of the copyright holder must be obtained.

This is done.

Both print and electronic (online) rights must be obtained from the holder of the copyright (often the publisher, not the author), and credit to the original source must be included in your manuscript. Many publishers now have online systems for submitting permissions request; please consult the publisher directly for more information. Permission is also required for material that has been adapted or modified from another

source. Increasingly, publishers will not grant permission for modification of their material. Creative Commons licenses and open access have also made obtaining permissions more challenging. In order to avoid publication delays, we strongly encourage authors to link or reference to the material they want to highlight instead of trying to get permission to reprint it. For example, "see Table 1 in Smith et al" (and insert reference number).

For articles that the journal invites, such as the Clinical Expert Series, the journal staff does not seek permission for modifications of material — the material will be reprinted in its original form.

When you submit your revised manuscript, please upload 1) the permissions license and 2) a copy of the

original source from which the material was reprinted, adapted, or modified (eg, scan of book page(s), PDF of

(11)

journal article, etc.).

If the figure or table you want to reprint can be easily found on the internet from a reputable source, we recommend providing a link to the source in your text instead of trying to reprint it in your manuscript.

This is done.

5. Responsible reporting of research studies, which includes a complete, transparent, accurate and timely account of what was done and what was found during a research study, is an integral part of good research and publication practice and not an optional extra. Obstetrics & Gynecology supports initiatives aimed at improving the reporting of health research, and we ask authors to follow specific guidelines for reporting randomized controlled trials (ie, CONSORT), observational studies (ie, STROBE), observational studies using ICD-10 data (ie, RECORD), meta-analyses and systematic reviews of randomized controlled trials (ie, PRISMA), harms in systematic reviews (ie, PRISMA for harms), studies of diagnostic accuracy (ie, STARD), meta-analyses and systematic reviews of observational studies (ie, MOOSE), economic evaluations of health interventions (ie, CHEERS), quality improvement in health care studies (ie, SQUIRE 2.0), and studies reporting results of Internet e-surveys (CHERRIES). Include the appropriate checklist for your manuscript type upon submission.

Please write or insert the page numbers where each item appears in the margin of the checklist. Further

information and links to the checklists are available at http://ong.editorialmanager.com. In your cover letter, be sure to indicate that you have followed the CONSORT, MOOSE, PRISMA, PRISMA for harms, STARD, STROBE, RECORD, CHEERS, SQUIRE 2.0, or CHERRIES guidelines, as appropriate.

While we agree with the editorial board regarding the importance of transparency and reporting research using a specific guideline, instead of modifying one of the aforementioned guidelines, we have used the Simulation-Based Research recommendations which are specific extensions to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines and include the following simulation-specific elements: participant orientation, simulator type, simulation environment, simulation event/ scenario, instructional design, feedback and debriefing (Cheng A, Kessler D, Mackinnon R, et al.

Reporting Guidelines for Health Care Simulation Research: Extensions to the CONSORT and STROBE Statements. Simul Healthc. 2016;11(4):238‐248) . In addition, we have also added an assessment of our study methodology by including the Median Medical Education Research Study Quality Instrument (MERSQI), which was developed to score the quality of medical education research (Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA . 2007;298(9):1002‐1009) . We have also made the following manuscript edits:

Method additions:

Line 399: This study is reported in accordance with the Simulation-Based Research recommendations, which are specific extensions to the STROBE guidelines, and includes the following simulation-specific elements: participant orientation, simulator type, simulation environment, simulation event and scenario, instructional design, feedback and debriefing.(13) Additionally, we also evaluated our study methodology using the Median Medical Education Research Study Quality Instrument (MERSQI), which was

developed to score the quality of medical education research using the following criteria: study design, sampling, type of data, validity of evaluation instrument, data analysis, outcomes. Each criteria is scored out of 3 with a potential range of 5 – 18 for the entire instrument.(14)

Results additions:

Line 523: Using the MERSQI domains (study design, sampling, type of data, validity of evaluation

instrument, data analysis, outcomes), our study scored a 14.5/18. We achieved the maximum score in all

domains with points lacking in “study design” as this is most accurately described as a cross-sectional

study and in “outcomes” as we did not gather data on changes in skills, behaviors or patient outcomes.

(12)

Discussion additions:

Line 638: Additionally, using the MERSQI, our study had a total score of 14.5 with a maximum score of 3 in the validity domain, which is higher than what the authors of the MERSQI found in their review of published medical education research studies as they determined a mean total MERSQI score of 9.96 (SD 2.34, range 5 – 16) with the lowest scores in the validity domain (mean 0.69).

Additionally, using the MERSQI, our study scored a 14.5, which is higher than what the authors of the MERSQI found for many published medical education research studies with a mean MERSQI score of 9.96 (SD 2.34, range 5 – 16) with the lowest scores found in the validity domain (mean 0.69).(14)

6. Standard obstetric and gynecology data definitions have been developed through the reVITALize initiative, which was convened by the American College of Obstetricians and Gynecologists and the members of the Women's Health Registry Alliance. Obstetrics & Gynecology has adopted the use of the reVITALize definitions. Please access the obstetric and gynecology data definitions at https://www.acog.org/About- ACOG/ACOG-Departments/Patient-Safety-and-Quality-Improvement/reVITALize. If use of the reVITALize definitions is problematic, please discuss this in your point-by-point response to this letter.

We have reviewed this document and confirmed that our manuscript is in accordance with the reVITALize definitions.

7. Because of space limitations, it is important that your revised manuscript adhere to the following length restrictions by manuscript type: Original Research reports should not exceed 22 typed, double-spaced pages (5,500 words). Stated page limits include all numbered pages in a manuscript (i.e., title page, précis, abstract, text, references, tables, boxes, figure legends, and print appendixes) but exclude references.

Our manuscript complies with these guidelines: 4603 words and 19.5 double-spaced pages excluding references.

8. Specific rules govern the use of acknowledgments in the journal. Please note the following guidelines:

* All financial support of the study must be acknowledged.

* Any and all manuscript preparation assistance, including but not limited to topic development, data collection, analysis, writing, or editorial assistance, must be disclosed in the acknowledgments. Such acknowledgments must identify the entities that provided and paid for this assistance, whether directly or indirectly.

* All persons who contributed to the work reported in the manuscript, but not sufficiently to be authors, must be acknowledged. Written permission must be obtained from all individuals named in the acknowledgments, as readers may infer their endorsement of the data and conclusions. Please note that your response in the journal's electronic author form verifies that permission has been obtained from all named persons.

* If all or part of the paper was presented at the Annual Clinical and Scientific Meeting of the American College of Obstetricians and Gynecologists or at any other organizational meeting, that presentation should be noted (include the exact dates and location of the meeting).

This is done.

9. The most common deficiency in revised manuscripts involves the abstract. Be sure there are no

inconsistencies between the Abstract and the manuscript, and that the Abstract has a clear conclusion statement based on the results found in the paper. Make sure that the abstract does not contain information that does not appear in the body text. If you submit a revision, please check the abstract carefully.

This is done.

In addition, the abstract length should follow journal guidelines. The word limit for Original Research articles is 300 words. Please provide a word count.

We have followed journal guidelines and the abstract is 265 words in length.

10. Only standard abbreviations and acronyms are allowed. A selected list is available online

(13)

at http://edmgr.ovid.com/ong/accounts/abbreviations.pdf. Abbreviations and acronyms cannot be used in the title or précis. Abbreviations and acronyms must be spelled out the first time they are used in the abstract and again in the body of the manuscript.

Our manuscript complies with this list and guideline.

11. The commercial name "Miya Model" (with the generic name in parentheses) may be used once in the body of the manuscript. Use the generic name at each mention thereafter. Commercial names should not be used in the title, running title, précis, or abstract.

We have made this change in all relevant portions of the manuscript; however, we did retain the name of the simulator in both the introduction and the methods. If the editorial board feels this is still too many mentions, please let us know and we will modify the introduction further to remove the name of the model.

12. The journal does not use the virgule symbol (/) in sentences with words. Please rephrase your text to avoid using "and/or," or similar constructions throughout the text. You may retain this symbol if you are using it to express data or a measurement.

We have removed all virgule symbols where indicating “and/or” but have retained them where they indicate necessary distinctions in data categories, for example “PGY2/3” and “PGY4/Fellow”.

13. In your Abstract, manuscript Results sections, and tables, the preferred citation should be in terms of an effect size, such as odds ratio or relative risk or the mean difference of a variable between two groups, expressed with appropriate confidence intervals. When such syntax is used, the P value has only secondary importance and often can be omitted or noted as footnotes in a Table format. Putting the results in the form of an effect size makes the result of the statistical test more clinically relevant and gives better context than citing P values alone.

This is done where appropriate.

If appropriate, please include number needed to treat for benefits (NNTb) or harm (NNTh). When comparing two procedures, please express the outcome of the comparison in U.S. dollar amounts.

This is not relevant for our study as we did not study the efficacy of a specific intervention.

Please standardize the presentation of your data throughout the manuscript submission. For P values, do not exceed three decimal places (for example, "P = .001"). For percentages, do not exceed one decimal place (for example, 11.1%").

This is done.

14. Please review the journal's Table Checklist to make sure that your tables conform to journal style. The Table Checklist is available online here: http://edmgr.ovid.com/ong/accounts/table_checklist.pdf.

This is done.

15. The American College of Obstetricians and Gynecologists' (ACOG) documents are frequently updated.

These documents may be withdrawn and replaced with newer, revised versions. If you cite ACOG documents in

your manuscript, be sure the reference you are citing is still current and available. If the reference you are citing

has been updated (ie, replaced by a newer version), please ensure that the new version supports whatever

statement you are making in your manuscript and then update your reference list accordingly (exceptions could

include manuscripts that address items of historical interest). If the reference you are citing has been withdrawn

with no clear replacement, please contact the editorial office for assistance ([email protected]). In most

cases, if an ACOG document has been withdrawn, it should not be referenced in your manuscript (exceptions

could include manuscripts that address items of historical interest). All ACOG documents (eg, Committee

Opinions

(14)

and Practice Bulletins) may be found at the Clinical Guidance page at https://www.acog.org/clinical (click on

"Clinical Guidance" at the top).

This is done.

16. When you submit your revision, art saved in a digital format should accompany it. If your figure was created in Microsoft Word, Microsoft Excel, or Microsoft PowerPoint formats, please submit your original source file. Image files should not be copied and pasted into Microsoft Word or Microsoft PowerPoint.

When you submit your revision, art saved in a digital format should accompany it. Please upload each figure as a separate file to Editorial Manager (do not embed the figure in your manuscript file).

This is done.

Art that is low resolution, digitized, adapted from slides, or downloaded from the Internet may not reproduce.

Figure 1: This appears to be a photo from the manufacturer. Please provide a letter of permission for print and online use.

Figure 2: Please move the numbers from the bottom right corner to the legend.

Figure 3: Please upload a higher resolution version of this figure and move the numbers from the bottom right corner to the legend.

This is done for Figure 1. We have removed Figures 2 and 3 per the recommendation by the Statistical Editor.

17. Each supplemental file in your manuscript should be named an "Appendix," numbered, and ordered in the way they are first cited in the text. Do not order and number supplemental tables, figures, and text separately.

References cited in appendixes should be added to a separate References list in the appendixes file.

We have included the following documents in Appendix:

1. Appendix 1: Miya Model Costs: this is a detailed list of the costs of the parts of this simulator.

2. Appendix 2: Vaginal Surgical Skills Index (VSSI): this is the non-modified (complete) version of this global rating scale

18. Authors whose manuscripts have been accepted for publication have the option to pay an article processing charge and publish open access. With this choice, articles are made freely available online immediately upon publication. An information sheet is available at http://links.lww.com/LWW-ES/A48. The cost for publishing an article as open access can be found at http://edmgr.ovid.com/acd/accounts/ifauth.htm.

We would like to make this article open access.

REVIEWER COMMENTS:

Reviewer #1: Review of Manuscript ONG-20-957 "Establishing validity for a vaginal hysterectomy simulation-based assessment"

Chen and colleagues have submitted results from a group of 50 surgeons that utilized a commercial vaginal hysterectomy trainer and then had "Experts" assess videos of the study participants in order to determine competency or lack thereof based on several previously evaluated, although modified in at least 1 case,

scales. The authors provide a reasonable commentary on the importance of surgical simulation and why fidelity is needed in this important area of medicine. As a minor note, there are several stray and remaining track changes present in the document. I have the following questions and comments.

We apologize for this issue and have rectified it.

Title - Consider noting in the title some about a surgical training model - don't have to mention the model by

name - and/or video review which more closely captures what you all did.

(15)

While we appreciate this comment, we did not address surgical training in this study. Regardless we have revised our title to better reflect that this is a simulation model: Establishing Validity for a Vaginal Hysterectomy Simulation Model for Surgical Skills Assessment

Précis - I think you could consider stating that it "… appears to be an effective tool…"

This is done.

Abstract - Line 43 - Really this is just for vaginal hysterectomy skills unless data demonstrates otherwise that it is for all vaginal surgical procedures.

This is done in all appropriate areas in the Abstract.

Line 49 - Do you identify later who the blinded reviewers were - authors/investigators on the study?

We clarified this point in the Methods:

Line 277: Each video recorded surgical performance was independently assessed by two or three expert vaginal surgeons (co-authors on this study except BA, DM who have financial ties to this simulation model), using a modified 10-item Vaginal Surgical Skills Index (VSSI) global rating scale (each item in the VSSI was scored on a 4-point anchored Likert-type scale (0–4), with higher scores indicating better performance), a 1-item global scale (GS) of overall operative performance (non-anchored Likert-type scale (1-7) with higher scores indicating better performance), and a pass-fail criterion, which was assessed as a separate item from the modified VSSI and GS.

Line 55 - Appears to be a typo as the same p value is represented 2 different ways.

Thank you for your detailed review and we have addressed this point.

Line 57 - Why not include information re: the AUC here for competency or not?

This is done.

Line 60-1 - Do you know if these skills translate to other vaginal surgical procedures? If not, should you state that this assessment applies to vaginal hysterectomy surgical skills?

We only assessed vaginal hysterectomy skills and have specified this in the abstract and other relevant sections of the paper.

Introduction - The authors have outlined issues related to why the study was performed and built on previous experiences.

Thank you for this comment.

Methods -

Line113-4 - Can you provide commentary on the fact that while each cadaver may be different, full anatomy, including lower limbs are present and while individuals are likely not performing a vaginal hysterectomy on a cadaver with the limbs in place, was thought given on how to at least mimic the presence of extremities?

We have incorporated the above suggestion:

Line 192: In contrast to biologic models such as cadavers, the features of this simulator are uniform, ensuring consistent training and assessment experiences for all learners; however, the simulator does not allow for the practice and evaluation of other aspects of the surgical process such as proper placement of the patient in lithotomy position.

Line 136 - Thank you for providing information on the approximate cost of the model. Since it appears that the inserts have to be replaced after each use, is there any cost and/or cost effectiveness information available?

While there is no cost-effectiveness analysis completed on this simulator, we have provided a more

detailed list and explanation of the costs of the various parts of the model in the supplemental file

(Appendix 1), if the editorial board feel that inclusion is appropriate.

(16)

Line 137 - Only the traditional more expensive insert was used for this study correct? If so should clarify that while another insert may be available it was not evaluated in this study.

This is done.

Line 202: This study utilized the original, more complete, disposable and nonreusable structures.

Line 141 - How were residents identified for participation? Did their program director know if they did or did not participate?

We have clarified this point:

Line 208: Advertisements for trainee and faculty volunteers were administered through the residency program office at each institution to obtain a convenience sample of study participants. Specific

instructions were given to participants regarding the anonymous nature of the video recordings and, for the trainees, assurance that performance on the simulator would have no impact on their residency evaluation.

Line 150 - Is data available regarding the applicability and/or appropriateness of using the modified versus the original VSSI global rating scale? How was it modified? What was utilized in the previous study you

referenced?

There is no data on the applicability/ appropriateness of using the modified VSSI as the VSSI has only been used either in its entirety or in the modified form used in this study and on the pilot study using this same simulator. We have clarified this point in appropriate sections of the manuscript and also moved up the specific modifications of the VSSI to the Methods section immediately after the first mention of VSSI:

Line 284: The original VSSI is a global rating scale with 13 items developed and validated to specifically evaluate vaginal surgical skills in live surgery (Appendix 2).(5) Certain metrics on the VSSI (initial inspection, electro-surgery, hemostasis) that were not able to be scored using this model were eliminated from use in this study. Specifically, “Initial inspection” was not evaluated because it was not often

possible to determine from the video recording how well an initial inspection was being performed. It was not possible to use electrocautery on this model. The “hemostasis” metric could also not be assessed as the material used to make vascular tubing for the vascular system in this version of the model was too stiff resulting in incomplete ligation of the vessels even with proper technique.

We also modified the Discussion:

Line 534: These results are consistent with the previous pilot study using this simulation model and the modified VSSI as well as other studies using both the VSSI and GS in live surgery.(5,6)

Line 153 - See comment re: line 150 above in terms of information about the validity of modifying this scale.

See above.

Line 155 - How do we know the "experts" that reviewed the videos are in fact experts? Did they perform the procedure on the model and have someone else assess them? What is their background? If the entire basis of the paper is that experts can successful adjudicate and assess videos to determine competence and lack thereof, more information on why they are experts is needed.

As the reviewer astutely noted, it is difficult to define surgical expertise and specifically expertise in

assessing surgery on a simulator. In order to address this complexity and avoid the expertise trap, we

assembled a group of reviewers from different subspecialties with varying levels of experience in the

different domains. Furthermore, although “board certification” is ubiquitously used as a surrogate for

technical expertise in the literature, there is precedent in the vaginal surgery literature to employ case

numbers as a surrogate for expertise that is probably more meaningful than obtaining certification

(Hardré PL, Nihira M, LeClaire E. Developing Expertise in Gynecologic Surgery: reflective perspectives

of international experts on learning environments and processes. Psychology Research and Behavior

Management. 2017:10: 17-30; Hardré PL, Nihira M, LeClaire E, Moen M. Defining Expertise in

(17)

Gynecologic Surgery: Perspectives of Expert Gynecologic Surgeons. Female pelvic medicine &

reconstructive surgery. 2016 22(6): 399-403). We have added the following in the Methods:

Line 311: Evaluating surgeons were all fellowship trained and or board certified in either female pelvic medicine and reconstructive surgery, minimally invasive gynecologic surgery, or gynecologic oncology. In addition to performing and teaching vaginal hysterectomy on this simulator on other occasions, all

evaluating surgeons had performed 120 – 900 lifetime vaginal hysterectomies, and have an average of 20 years of practice (range 9 – 30) with roles in resident and fellow education and expertise in simulation education and research.

Line 160 - Appears to be the modifications. Perhaps this should be moved up with the previous comments?

This is done.

Results - Line 205 - Since you modified the VSSI, should you also make it clear in the results that this was a modified VSSI?

This has been done in all relevant portions of the manuscript.

Line 206-7 - First portion seems to be more commentary/discussion, please remove and just note the data and differences observed.

This is done and this sentence has been modified:

Line 430: The number of hysterectomies performed moderately correlated with the composite modified VSSI scores and GS scores (Pearson r = 0.55, p<0.001, and Pearson r = 0.58, p<0.001, respectively).

Discussion - Line 236 - Sorry to harp on this again, but if the measure was modified you should note that it was modified throughout the manuscript.

This has been done in all relevant portions of the manuscript.

Line 270-1 - You note that the participants had no commercial ties to the product, but it appears that at least 2 of the authors (Drs. Anderson and Myazaki) have a commercial tie/interest in the product, which is a potential cause of bias. How was this managed for the study? Did either of these authors review the videos - I presume Dr. Anderson did not as he is not a Medical Doctor?

Both Drs. Miyazaki (DM) and Anderson (BA) did not review participants’ performances and this has been clarified in the Methods and the Discussion sections:

Line 277: Each video recorded surgical performance was independently assessed by two of three expert vaginal surgeons (co-authors on this study except BA, DM who have financial ties to this simulation model), using a modified 10-item Vaginal Surgical Skills Index (VSSI) global rating scale (each item in the VSSI was scored on a 4-point anchored Likert-type scale (0–4), with higher scores indicating better performance), a 1-item global scale (GS) of overall operative performance (non-anchored Likert-type scale (1-7) with higher scores indicating better performance), and a pass-fail criterion, which was assessed as a separate item from the modified VSSI and GS.

Line 627: The expert reviewers had no commercial ties to this model. The two co-authors on this manuscript who had commercial ties to this model (BA, DM) did not participate in the surgical performance evaluation, or the data entry, analysis or interpretation.

In the cover letter, we have also clarified the roles of the co-authors with commercial ties to this model:

Both Drs. Miyazaki and Anderson contributed substantially to the conception and design of the model, and study design and implementation; however, they did not participate in data entry, analysis, or interpretation due to potential conflicts of interest.

Tables - Table 1 - is the column entitled "Competent" only referring to individuals that were fully competent per

(18)

the assessments or more "experienced" surgeons? I think the heading may need to be "Novice Surgeons" and

"Experienced Surgeons" since you are trying to determine competence based on the assessment of their performance on the model.

This is done.

Table 2 - Can be supplementary in my mind as you note the key points in the manuscript already.

Considering this recommendation and the recommendation from Reviewer 3, we have removed Table 2.

While we have preserved the text summarizing the main findings of Table 2, we have moved it to later in the Results section per the recommendation of Reviewer #3 (Lines 500-507).

Table 3 - So I presume that the PGY1 and PGY 2/3 are the 33 novices from table 1. If this is the case, then perhaps for Table 1 it should be "Novice Surgeons (PGY1-3)" and likewise "Experienced Surgeons (PGY4+)"

This is done.

Table 4 - Please adjust column headings as previously noted.

We have clarified the column headings that this is a comparison between non-competent and competent surgeons using our modified VSSI cutoff score of 27.

Figures - Figure 1 - Important and fine

Figures 2 and 3 - Perhaps could be combined into a Figure 2A and 2B?

Due to the Statistical Editor’s comment, we have removed Figures 2 and 3.

Figure 4 - No comments.

Referensi

Dokumen terkait

The results showed that the teacher's question 1} in the preliminary activities, was used to check the attendance of students, students 'understanding of the previous lesson, and

Research Method The following methodology flowchart in this study : Analysis of the data used in this study is a direct survey to the location so that the writer can classifies the