Can I use ChatGPT to generate a fully AI-authored thesis?

(1)

Can I use ChatGPT to generate a fully AI-authored thesis?

Reflecting on prompting strategies and useful detection tools

Kirstin Krauss

WWIS Research Advisor Visiting Researcher – STADIO

Iontaofa Intelligence Ltd.

1

(2)

2023/06/29 © KEM Krauss 2

• Honours in Computing (Unisa)

• 1400+ proposal students (2021/2)

• 40+ mini-dissertation students (2021/2)

• Qualitative Case Study Research &

Thematic Analysis

(3)

2023/06/29 © KEM Krauss 3

(4)

4 phases of writing

Where ChatGPT was useful

• In the first phase your brainstorming ideas. In such case you can write

author or article driven. Writing as a form of thinking.

• In the second phase, you need to start constructing a conversation, an argument

• In the third phase you do editing and focus on presentation

• In phase four:

Engagement/Validation from the scientific community – reviewers, examiners, editors

4

2023/06/29 © KEM Krauss

4. Scientific peer-review

1. Generative

writing

(5)

Combination of detection tools is needed

References checking

AI writing detection Similarity

Plagiarism

2023/06/29 © KEM Krauss 5

Quality of sources

Questionable sources

Alignment

(6)

“Examiner” feedback

• While I was reading it felt a bit stilted (bursty) but I think that's because I'm used to reading a lot of dissertations. The sentence construction could have been more

smooth.

• I would have liked it if the theory was used to support the research questions. I don't know if you intentionally left out objectives or not. There should have been a conclusion.

• The digital divide can make a big difference in SA and perhaps the AI program could have highlighted that.

• Overall this is a very good essay and I believe few lecturers would have noticed the difference and I doubt Turnitin would have flagged similarity.

• The problem that arises now is that there are more people developing AI programs than there are developing anti-plagiarism programs. This can mean that false positives can occur for honest students. In your essay, which is very well written, it will not be picked up easily. No one is going to find out from which sources some passages were used and this makes things difficult for honest students.

2023/06/29 © KEM Krauss 6

(7)

Similarity / Plagiarism checking AI detection

2023/06/29 © KEM Krauss 7

(8)

2023/06/29 © KEM Krauss 8

(9)

2023/06/29 © KEM Krauss 9

(10)

2023/06/29 © KEM Krauss 10

(11)

2023/06/29 © KEM Krauss 11

(12)

2023/06/29 © KEM Krauss 12

(13)

2023/06/29 © KEM Krauss 13

(14)

GPTZero

2023/06/29 © KEM Krauss 14

(15)

2023/06/29 © KEM Krauss 15

(16)

Can I generate references retrospectively?

2023/06/29 © KEM Krauss 16

(17)

“Write a 4-page research paper that identifies factors that affect information system (IS) process innovation adoption decisions in three organizational environments. The analysis must be based on Rogers's (1995) theory of Diffusion of Innovations (DOI).

Highlight the factors that affect IS process innovation adoption, including need

recognition, availability of technological infrastructure, past experience, own trials, autonomous work, ease of use, learning by doing and standards. The methodology

should be a qualitative Case Study research.

Make sure to use proper references.”

2023/06/29 © KEM Krauss 17

(18)

2023/06/29 © KEM Krauss 18

(19)

2023/06/29 © KEM Krauss 19

(20)

Sourcing references Searching &

summarising sources

Chat with/summarise sources Generating topic statements AI detection

Plagiarism detection

Checking references

2023/06/29 © KEM Krauss 20

(21)

Checking references alignment

2023/06/29 © KEM Krauss 21

(22)

2023/06/29 © KEM Krauss 22

(23)

2023/06/29 © KEM Krauss 23

(24)

2023/06/29 © KEM Krauss 24

(25)

4 phases of writing

Where ChatGPT was useful

• In the first phase your brainstorming ideas. In such case you can write

author or article driven. Writing as a form of thinking.

• In the second phase, you need to start constructing a conversation, an argument

• In the third phase you do editing and focus on presentation

• In phase four:

Engagement/Validation from the scientific community – reviewers, examiners, editors

25

2023/06/29 © KEM Krauss

4. Scientific peer-review

1. Generative

writing

(26)

2023/06/29 © KEM Krauss 26

1. Generative

writing 2. Drafting 3. Editing

• Scientific argumentation

• Argument flow

• Paragraph flow

• Hedging techniques

• Weaving the golden research

• Particularising Methodology

• Theoretical elaboration

• Writefull

• Grammarly

• MSWord

• Language editing

• Literature searches

• Discovery tools

• Library databases

• Search strategies

• Organising strategies

• Plagiarism / Similarity

• Predatory publishing

• Retracted research

• Citation pollution

4. Scientific peer-

review

(27)

Elements of an argument (Toulmin et al., 1984)

Claim – testing of the destination

Grounds – required foundation if solid / reliable

Warrant – testing of grounds; judging solidity, reliability

Backing of warrant (authorised arguments)

• What exactly are we discussing?

• Where precisely are we to stand on this issue?

• And what position must we consider agreeing to as the outcome of the argument?

• What information (or facts) are you going on?

• What grounds is your claim based on?

• Where must we ourselves begin if we are to see whether we can take the steps you propose and so end by agreeing to your claim?

• Given the starting point, how do you justify the move from these grounds to that claim?

• What road do you take to get from this starting point to that destination?

• Is this really a safe move to make?

• Does this route take us to the required destination securely and reliably?

• And what other general information do you have to back up your trust in this particular warrant?

Rebuttals

Boundary conditions

Could this point to new contributions/knowledge

gaps?

2023/06/29

(28)

Structuring a section or paragraph

(Krauss 2017)

Focusing on the discussing of

themes/ideas/claims rather than on authors

28

(29)

29

Typical Contributions:

• New themes emerging from the data

• Themes from lit that don’t feature in the data

• Particularising of themes

“I believe that the

researcher’s best tool for analysis is his or her own mind, supplemented by the minds of others when work and ideas are

exposed to them”

(Walsham, G., 2006. Doing interpretive research. European journal of information systems, 15(3), pp. 320-330.)

Theoretical Elaboration

(30)

Teaching basic research

through prompting ChatGPT

A few examples

(31)

Teaching constructive alignment through prompting

(32)

Constructive alignment between literature themes and interview questions

(33)

The importance of constructive alignment in the findings chapter

(34)

Selecting research participants

(35)

The importance of using references throughout

(36)

What introduction should do

(37)

Thematic development of a literature review

(38)

Learning about consistency in tenses

(39)

The importance of argument roadmaps

(40)

Self-assessing research questions

(41)

The importance of explaining data analysis

(42)

Kirstin Krauss

43