Developing an Audit Trail for PhD research: a case study

Introduction

The Research Excellence Framework (REF) includes in its definition of research:

‘…a process of investigation leading to new insights, effectively shared…scholarship, the invention and generation of ideas, images, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes…’ (UKQC, 2017)

It is under the REF that the Papé Audit Trail (PAT), in chart format, is proposed.

The chart (Table 2) was developed during the course of studying and writing-up the thesis for a PhD research project⁷. It is intended as an example for future doctoral students to understand the complexities of study and the recursive nature of steps needed to be undertaken in qualitative methodology, a feature of hermeneutic phenomenology (Butt, 2006). It is not offered as an instruction to be followed but more as a framework or scaffold in the Vygotskyian sense (Wells, 1998), which might assist the process.

The audit trail, which emanated from the research, comprised 3 discernible stages:

Table 1: Stages of Research

Each stage was completed before the subsequent stage commenced, which was confirmed by a Scrutiny Panel, which was the mechanism by which the awarding body⁸ accepted the level of study as being appropriate for the award.

Table 2: Papé Audit Trail (PAT) for PhD Research

Stage 1: Preparation for Research

In preparing to undertake the research, the first stage comprised five steps as shown in PAT (Table 2).

Step 1.1: Emplace supervision regime

The first step was to appoint a supervisor or supervisors with the appropriate skills and subject knowledge to support and encourage research (UKQC, 2016) from which the PAT is extracted. At least one supervisor will be employed by the awarding body, who will have some subject knowledge. There will be a requirement for a second supervisor with subject-specific knowledge. In emplacing the regime it is important to decide on the mode of supervision, that is whether the structure will allow for co-supervision; that it is being supervised by both supervisors concurrently; individual supervision, that is being supervised by one supervisor at a time; a mixture of the two modes. Whilst logistical disadvantages were recognised, the author preferred the first mode as it allowed for clarity of understanding without the fear of contradictory thoughts. However occasionally the student can also benefit from individual supervisory contact over particular enquiries. The number (10 per year) of supervision sessions was set by the awarding body and the duration was agreed at 1.5 hours per session. Permission was sought from supervisors to record supervision sessions which obviated the need for copious notes, which the author found preferable (McLeod, 2001). Once the parameters were set an agreement was reached between all parties on the focus of each session.

Step 1.2: Define focus of research

The author had an initial general idea of interest and thoughts about possible titles. The context was set according to the researcher’s experience of working in British universities. However only limited rationality was expressed with little understanding of what might be achievable.

At this stage supervisory intervention helped the researcher’s movement from original ideas of a wide and shallow focus to a narrow in-depth focus in three areas: association between students’ relationships with classroom instructors (tutors) (Nuthall & Alton-Lee, 1997; Attwood, 2010) and skills acquisition (Nelson-Jones, 2002) and professional development (Jones et al, 2008). There was also an agreement established that the research would report what was discovered, rather than attempt to prove a thesis. Applying these parameters, all parties agreed that the focus would be to ascertain from a limited number of student participants, what they considered were the main factors during study that helped in their learning. It was not until Stage 3 that the final title was developed and adopted.

Step 1.3: Apply for research degree approval

Application for approval was the first of many times the award degree institute makes a decision regarding the proposed research. Approval is needed to confirm the subject was deemed appropriate and support is in place to offer the researcher a reasonable chance of success. During this time, the researcher, in conjunction with supervisors, undertook a number of tasks, the first of which was to identity the proposed methodology for analysing data. Quantitative, qualitative and integrative methodology was considered and after considerable discussion the research team decided qualitative methodology was best suited to this small case-study research. The next task was to consider the research design. Firstly all agreed there must be a pilot study to test the tools for the research, that there would be sections on a selected literature review, methodology to include philosophy of the research and research design, findings, analysis and conclusions. Evidence would be collected by interview (Mishler, 1986) and ethical issues would be addressed. All would then be consolidated into an application to be presented to the research ethics committee.

Step 1.4: Ethics committee

The committee that ensures research ethics are complied with checks that all elements of the proposed research respect university protocols. The researcher has to establish that any framework or code of ethics under which research is engaged (BERA 2012; BACP 2013) is fully compliant with university requirements. The ethics committee certificates appropriate standards have been met and the draft proposal is then sent to the scrutiny panel.

Step 1.5: Scrutiny panel

The final check is made by the university scrutiny panel in order that the research can commence. The researcher is advised of the decision, which if positive will trigger commencement of the project.

Stage 2: Pilot Study

A pilot study is deemed necessary to test the selected tools and processes for the main research. It also offers an opportunity to practice interviewing technique and prepare the researcher for challenges of the main research.

Step 2.1: Obtain sample pool of interviewees

The first step is to decide from where raw data will be collected and create a group of potential interviewees.

Step 2.2: Randomised choice of 2 interviewees

Once the group is made randomising interviewee selection will assume importance and ethical considerations will play a part. For small scale case study research 2 or 3 subjects for the pilot study will offer comparative opportunities.

Step 2.3: Commence interviewing

Data collection by interview starts at this time; you will already have an idea of the broad areas you will research; you will decide if your questions will be semi-structured offering an opportunity for dialogic interventions. What is sought is emerging information from which analysis will produce a more focussed list of questions for the main study. Seeking interviewee consent to record interviews will enable concentration on dialogue.

Step 2.3a: Start Literature Review

During the above 3 steps a search for referenced literature is commenced. Think about the main subjects and theories that might inform the research. Review of literature is a recursive exercise and be prepared to keep the title or research aim in focus.

Step 2.4: Transcribe interviews

Once the researcher has collected data, interviews can be transcribed verbatim. This will enable a greater understanding of the subject matter plus expose any collateral nuanced remarks.

Step 2.5: Submit transcripts for member checks

This process enables the interviewee to check content accuracy as well intent of what was said. Allow editing opportunities by interviewees.

Step 2.6: Agree accuracy

This important stage is crucial to research and progression can only be made once written confirmation is received from the interviewee that the transcript is accurate and acceptable.

Step 2.7: Reflective interview

If the design allows arrange a second or reflective interview to allow for greater depth of interviewee response and repeat Steps 4-6 above

Step 2.8: Focussed Literature Review

Throughout these early steps, in collaboration with supervisors, the opportunity exists for researching additional texts and focussing the literature review.

Step 2.9: Analyse agreed texts

The tool for analysis has been agreed at the design stage (nVivo is one option) and can be tested at this time. Themes will emerge which will become foci for the main research.

Step 2.10: Researcher reflection

This step allows for stepping away and reflecting on the content and process of the research. Decisions will be made whether the selected tools are fit for purpose and whether the hypothesis stands up to further scrutiny. The design will be tested and relevant changes made. This is a recursive process and supervisory oversight and intervention assume high importance.

Step 2.11: Submit to Scrutiny Panel

Once agreement is reached that the pilot study fulfils bits purpose it is sent for scrutiny for feedback and recommendation.

Step 2.12: Researcher reflection

Further reflection on the results from feedback will suggest a greater understanding of the hypothesis and processes under which the research is undertaken.

Step 2.13: Incorporate re-commendations

This allows the research to be enhanced, be selective not defensive about incorporating others’ ideas. When completed the main research can be commenced.

Stage 3: The Research

The research outcome is to provide a written thesis of a specific word-count (usually about eighty thousand words) with a high standard of presentation evidencing the researcher as an expert in the chosen field.

Step 3.1: Obtain sample pool of interviewees

Careful thought and planning will have prepared for this step. Decisions about interviewees, randomised selection and interview structure have essentially been made earlier, with amendment as a result of pilot study experiences. The pool of interview candidates must be large enough for random selection to be meaningful with reserves built-in for the possibility of drop-out. All potential candidates must be aware they might not be selected and give consent to being involved in the selection process.

Step 3.2: Randomised choice of subjects for interview

Careful thought must be given to the choice of interviewees. From the sample pool a decision needs to be made on the number of participants (perhaps around 8) and the mechanics for ensuring random selection. It is important in case study research not to choose from known participants (ethnographic research differs) and to understand fully ethical implications.

Step 3.3: First interviews

Experience from the pilot study will help form a frame for the first interview. Decide on the duration and allow for tiredness and concentration spans. Ensure all main points are covered and be prepared for material to emerge unfettered; a dialogic approach will help in this regard. If a point has been missed it can be raised in the second reflective interview. If a point is ignored this might in itself provide material for analysis.

Step 3.4: Transcribe texts

The transcription will be a verbatim record of the interview from direct recordings in terms of content and process.

Step 3.5: Submit transcripts for member checks

Each interviewee will need to have the opportunity to check the transcript of the individual interview for both content and intent, that is what was said and what was meant to be said. This recursive process might go through a number of edits.

Step 3.6: Agreement of accuracy

It is essential the interviewee agrees the accuracy of the transcript in writing (a bespoke form can be used) as this is the raw material of the research. Be prepared for an iterative process of editing as the respondent might want to be absolutely sure of correctness of evidence.

Step 3.7: Reflective interview

Step 3.8: Transcribe recording

Step 3.9: Submit texts for member checks

Step 3.10: Agreement of accuracy

Steps 7-10 as above if the design allows for second or reflective interview

Step 3.11: Identify appropriate literature

The literature review will have been an ongoing process from commencement and at this stage the review will be nearing completion. The literature that informs the research will be drawn from current and historic journals and publications and focussed on the main areas under study.

Step 3.12: Critically review literature

Think clearly about the literature for inclusion, keep the research focus in mind and prepare against side-track and repetition. It is unlikely all literature published on the subject will be included, so consider inserting ‘selected’ into the review title. Use of primary texts will ensure authenticity, use of substantial texts or excerpts will not enhance the study unless absolutely necessary to the arguments.

Step 3.12: Write literature review

This will be an iterative process involving supervisors and researcher. Be prepared to debate the effectiveness and relevance of any inclusion. Consider the order of presentation and offer continuity. Sub headings might assist with increasing readability for moderators. Every word used will have significance, keep sentences simple and concise.

Step 3.13: Transcript and text analysis

Use of IT software will enable analysis, especially in the event of time pressures. NVivo is particularly useful for qualitative analysis as it enables emergence of themes through a structure of nodes, groups and families. Other software or manual analysis can be equally effective.

Step 3.14: Identify themes for discussion

Through text analysis themes and sub-themes will emerge. Present might be overarching themes which occur throughout the research. Other themes will be less significant. Decide through a recursive process the themes for inclusion and reflect on choice and exclusion.

Step 3.15: Prepare thematic analysis

This necessarily could be a lengthy process. Decide on structure, typically might be included introduction, analysis and conclusion. Use of interviewee’s own words can highlight points made. Back up any statement with clear referencing to literature. Consider order of presentation and sort into chapters. Write clearly and concisely, pay attention to grammar and presentation.

Step 3.16: Write methodology

State chosen methodology and explain reasons for selection. Describe methods for collecting data and include an ethics section.

Step 3.17: Prepare Discussion and Conclusion

This is the final chapter and will include sections on findings and conclusions as well as summarising the research.

Steps 12 to 17 will involve many rewrites and consultations with supervisors.

Step 3.18: Make decisions on Appendix content

Material that is connected to but not central to the research may be presented in appendices.

Step 3.19: Submission and attend Viva

The awarding body will agree a submission date for both electronic and hard copies versions. A Viva will be arranged in which you give a verbal account of your research in response to questions by the panel.

Conclusion

This audit trail is complete and might provide the reader with more questions than answers. Undoubtedly, any trepidation might be caused by the comprehensive nature of each stage and contained steps. However, if prioritised, the suggested format can be a tool to assist the research process and thus be a resource which could help the candidate to a successful conclusion.