3.12.1 The Research Instrument
There are several methods for collecting data for studies namely, examination of secondary sources, observation, semi-structured or unstructured interviews and surveys or questionnaires.
Questionnaires tend to provide the best method of data collection for explanatory research where causal relationships between variables need to be investigated (Saunders et al., 2016).
According to Sekaran and Bougie (2016), questionnaires can take the forms of personally administered questionnaires or self-administered questionnaires which can be distributed by mail, via online survey platforms or via delivery and collection mechanisms. Due to the nature of the study, a questionnaire appeared to be the most appropriate method for determining the relationship between the usage of AI solutions in law firms and the effect it could have on the propensity of law firms hiring less legal practitioners.
The data collection instrument for the study was chosen to be a self-administered structured questionnaire. The questionnaire originally adopted a hard copy/printed survey format which was hand delivered and collected with the intended the face to face natured relationship that the researcher had with customers and colleagues. This soon proved onerous due to unavailability of legal professionals, cancelled meetings and geographical dispersion. The hard copy server was then converted to fillable word document and e-mailed to prospective participants. This technique proved to be more amenable to legal professionals who value their time, given that it is a commodity upon which they make their living.
The questionnaire covered the following sections:
Demographic Information – 2 Questions – (1, 2)
o These question help to identify ages and gender to draw conclusions if there are any predisposition to adopting AI solutions
Company Type, Job Roles and Job Functions – 4 Questions – (3, 4, 5, 6)
o These questions help identify the type of companies the legal professionals work in, what their job roles are and what job function these company perform
Awareness of Artificial Intelligence in Law – 2 Questions – (7.1, 7.2)
o These questions were used to satisfy the first research objective, which was to ascertain if employees at law firms and legal publishers are aware of advancements of AI in the legal industry.
Utilisation of Artificial Intelligence Solutions – At Present – 2 Questions – (8.1, 8.2) o These questions are to uncover whether the respondent either uses AI technology
at present or is aware of AI solutions being used in their law firms
Utilisation of Artificial Intelligence Solutions – Capability – 1 Question – (9)
o These questions were used to satisfy the second research objective what was: To determine if there is a perception that law firms and legal publishers would use AI if it was accurate, quick and consistent to use.
Utilisation of Artificial Intelligence Solutions – Affordability – 1 Question – (10) o These questions were used to satisfy the third research objective what was: To
investigate if there is a perception that law firms and legal publishers would use AI if it was affordable.
Impact on Legal Professionals – 2 Questions - (11.1, 11.2)
o These questions were used to satisfy the fourth research objective what was: To determine the perception of the effect that AI solutions would have on the hiring of legal professionals at law firms and legal publishers.
The Influence the respondent has within the Organisation – 1 Question – (12)
o This question was to determine whether the respondent had any influence within the organisation to suggest usage of AI technology or hiring behaviour of staff.
A structured questionnaire with closed-ended questions was developed. However, limited open- ended questions were presented due respondents needing to provide specific details of their company types and job roles. The questionnaire was four pages long and took approximately 10 minutes to complete during the pilot testing phase. The survey instrument was hand delivered and e-mailed to the respondents at their places of work with the aid of research assistants. A cover letter explaining the purpose of the study and assuring the respondents anonymity and confidentiality of the information was attached to the questionnaire along with a 1 page infographic which contained information about the availability and current usage trends of AI products in Law – See Appendix A. Questions were answered by simply checking the box from a set of possible answers.
The advantage of using a self-administered hand delivered questionnaire was that it was quicker to get mass responses versus an interview style questionnaire. The fact that respondents were informed of the collection date facilitated a higher response rates versus the e-mail prompting of online questionnaires. The disadvantage with self-administered questionnaires was that it took longer to distribute it to respondents as well as printing costs being higher versus online questionnaires.
3.12.2 Measurement Scale
According to Creswell (2013) a measurement scale had to be chosen to quantify the behavioral responses by the respondents in relation to items which were attempted to be measured with in the instrument. These scales could take the form of continuous scales e.g. strongly agree to strongly disagree or could be discrete/ordinal scales e.g. numerical or yes/no. These scales are used to codify behavioral responses into quantitative measurements to ensure that consistent sentiment was measured and analysed by the respondents (Sekaran & Bougie, 2016).
A Likert-style rating scale format is an ordinal scale which is used to measure the respondents opinion by allowing them to indicate how strongly they agree or disagree with the statement (Saunders et al., 2016). A five-category Likert scale was selected for the purposes of codifying the responses of the respondents into quantifiable integers ranging from 1 to 5. A matrix style format was selected for the question style and responses options ranged from strongly disagree or strongly agree. An example of a question from the questionnaire can be seen in Table 3.1 below.
Table 3.1 Example of Likert Scale Matrix Format Question Strongly
Disagree
Disagree Neutral Agree Strongly Agree My company is aware of the
advancements of Artificial Intelligence in Law.
1 2 3 4 5
The questionnaire itself can be viewed in Appendix C.
3.12.3 Pilot Testing
While there are many types of pre-testing that can be performed, the method selected for this particular study was Pilot Testing. This makes sure that the questionnaire is reviewed to assess the suitability, usability and effectiveness, making certain that the questions are clear and concise, ensuring that the highest possible response rate is obtained with a minimal margin for error (Saunders et al., 2016). Pilot Testing, tested the instrument on a reduced set of individuals from the sample frame and was completed on approximately 5 respondents.
Responses were retrieved from the respondents and the questionnaire was revised based on this feedback. The first issue identified in the instrument was in Question 6, where ‘Legal Document Creation’ was duplicated and ‘Contract Management’ was excluded. The second revision to the questionnaire was in Question 11.2 which asked respondents to indicate the number of years it would take for AI Solutions to replace lawyers. The time selection options ranged from (< 3 years to > 10 years). Respondents asked for a time range of ‘Never’ to be included as they believed that AI Solutions would never replace lawyers. These participants were excluded from the final sample for the study (Creswell, 2013).