“I consider myself lucky for getting a chance to go through ‘The Unofficial Guide to Medical Research, Audit and Teaching’. It’s a wonderful & down to earth book. It has dealt with the topic of medical research and publication in a most practical and step wise manner. I am sure it will act as a light house and illuminate the path for so many young medical professionals who have started their career, are full of motivation and ideas but are clueless.I will recommend this book to my undergraduate as well as postgraduate students.”
Alok Sinha
Assistant Professor, Manipal College of Medical Sciences, Pokhara, Nepal
“This will be a great introductory text for those who want to pursue a clinical academic career, and for everyone else will provide a range of concepts, tools and approaches which will be useful in any speciality.”
Michael Ross
Co-Editor-in-Chief, The Clinical Teacher
Senior Clinical Lecturer, Centre for Medical Education, The University of Edinburgh
Simon Patterson-Brown
Consultant General and Upper GI Surgeon, Edinburgh Royal Infirmar
Honorary Senior Lecturer, University of Edinburgh.
The Unofficial Guide to Medical Research, Audit and Teaching is our response to the need for medical students and junior doctors to be more than just competent clinical practitioners to get the job they deserve. Speciality training posts are very competitive, and the reality is that your CV needs research, audit, and teaching to make you stand out.
This book is a response to that need, and will give you that competitive edge to excel over your colleagues in the ruthless selection process.
And if you like the style, then there is an opportunity to join over 200 people like yourself working on such titles. You are the experts in your current learning needs after all. In 2012 when the MDU helped fund the start of the Unofficial Guide to Medicine’ project, it was one book written by a few friends. Now it is a global success, with over 300 people involved across the world. You could be one of them!
It is crucial that clinicians are able to interpret research findings and understand the implications for their practice. Is this statistically significant result also clinically important for my patients? Was this particular research done on patients comparable to the one that I am treating?
Clinicians must also be aware of the clinical exceptions when interpreting guidelines. Hence, a firm grasp of evidence-based medicine is beneficial, even for clinicians who are not actively carrying out research.
Key Features
Written in line with the other books in the series, stations contain:
Ceen-Ming Tang
Co-editor
Final Year Medical Student, University of Oxford, UK
Other benefits that come from doing research include the development of teamwork and leadership skills. You learn not only to work within a team, but to work towards deadlines, deliver presentations, and write scientifically. You also have the opportunity to think critically, and find answers for yourself.
The answer “because that’s what it says in the textbook” or “that’s how things are done” simply isn’t good enough. If the answer to your question is unsatisfactory, as it is for many areas of medicine, you can learn to think critically and at least try to answer the question empirically for yourself.
This book does not aim to make you an expert in research. Our first and foremost goal is to encourage you to think critically. To that end, we include simple introductory guides to different aspects of research and convenient checklists to help you appraise the literature.
Basic Science Research Studies
Your lab book should be a real-time record of what you do in the lab, which will serve as a reliable reference for writing up the results of your study (Box 1). Our aim is to provide advice on why and how you should keep a good lab notebook.
Your lab notebook is first and foremost for yourself. Experiments may take months, or even years to complete. Without good records, analysing data and writing up results will be almost impossible! A complete record of why experiments were performed and how they were performed is also needed to encourage critical thinking. By jotting down observations from each experiment, you can speculate as to how your results might fit the bigger picture and ask further questions.
Secondly, you will often be working within a team. People come and go, and for this reason, a complete record of your work from the key findings down to the brand of reagent used should be made accessible to others. As unlikely as it may seem now, there might be someone who is interested in continuing your work.
Very occasionally, significant sums of money can be involved in who did what and when. Lab notebooks are a legal record which preserves your rights to patent a discovery when considering commercialisation.
Many advances in medicine are underpinned by research in the basic sciences. Several important advances which come to mind include Banting and Best’s discovery of insulin and Fleming’s discovery of penicillin. The aim of this chapter is to introduce the scientific principles and techniques behind the work of basic scientists. This includes a broad overview of commonly used laboratory equipment and calculations, as well as more specific techniques in molecular biology, drug discovery, cell culture, and animal studies.
How to Keep Your Notebook
The exact format of lab notebooks varies greatly between researchers, and many universities will have official guidelines for you to follow. However, it is always important to get the right balance between neatness and speed. Spend too little time on your lab notebook, and you will not have sufficient information to refer to when writing up. Conversely, spend too much time on the notebook instead of experiments, and you will have nothing of use to say!
- Keep a table of contents
Use the first two pages of your notebook for a table of contents. This will help you find the relevant section of your notebook when writing up. - Date your entries
You must date each of your entries (DD/MM/ YYYY) and show clearly where each entry starts and ends. - Write legibly
Write in a manner which everyone is able to read, and avoid re-copying notes where possible. It is tempting to re-write entries to make them more legible and to organise thoughts more cohesively, but in doing so, you will introduce copying errors and omit information that you feel is irrelevant. This information could be crucial to you later. - Correct mistakes
If you make a mistake, draw a single line through it and ensure that the writing remains visible. Resist the temptation to erase it or use correction fluid. You may decide at a later date that your original entry was correct. Similarly, never remove a page. Missing pages from a notebook may be viewed with suspicion
by colleagues, and assumed to contain data unfavourable to your research objectives. - Paste loose notes into the notebook
All loose notes, including printouts of graphs or results tables should be pasted into the notebook. This ensures a complete chronological order of your work and a continuous thought process.
What Should Go Into Your Notebook?
All loose notes, including printouts of graphs or results tables should be pasted into the notebook. This ensures a complete chronological order of your work and a continuous thought process.
- Objectives and background
What is the aim of your experiment? What is your hypothesis? Justify the need for and logic behind your experiments using references to textbooks and/or published research. This should be written before performing the experiment. - Experimental set-up and protocols
You should include enough detail about your experimental set-up so that other researchers can replicate your experiment. This usually includes a step-by-step description of the procedure, as well as more specific details such as the brand of reagents used. You may choose to include diagrams if appropriate. This should be written whilst performing the protocol. - Observations
It is not uncommon for things to fail on the first attempt. Were any mistakes made during the experiment? You should jot down notes about your mistakes and any other problems with
the procedure so that you can explain any odd results from your experiment. Review these mistakes for learning points, and ask yourself how you can make improvements. Similarly, make a note of your successes so that you can try replicating them at a later date. All observations and conclusions should be written immediately after the experiment. - Positive and negative data
Both positive and negative data must be recorded so as not to introduce bias to your conclusions. Data from experiments should be recorded clearly in well-labelled graphs and tables. - Statistical analysis
In your data analysis, it is important to include annotated calculations, or refer to spreadsheets in your lab notebook. Remember to include sufficient detail, such as the classification of outliers, types of statistical tests used, and error calculations. - Conclusions and next steps
Summarise the findings of your experiment, and make a note of what can be improved in the future. Try reconciling your data with your hypothesis, and consider how this compares to dogma in the literature.
Lab Notebook Checklist |
|||||
| 1. | Is there a table of contents, and has each entry been dated (DD/MM/YYYY)? Are entries legible to both yourself and other members of your team? | N | |||
| 2. | Were mistakes crossed out with a single line? | N | |||
| 3. | Were any pages removed from the book? | N | |||
| 4. | Have you stated the objectives and hypotheses for your experiments? | N | |||
| 5. | Have you included sufficient detail so that another researcher can re-create your work using the instructions for the procedure written in your notebook? | N | |||
| 6. | Did you note down your observations during the experiment (e.g. mistakes, problematic steps)? | N | |||
| 7. | Did you include all results from experiments, including loose materials (e.g. images/graphs) and calculations? | N | |||
| 8. | Have you reviewed the results from your experiment, and made an attempt to reconcile it with current literature? | N | |||
Critical Appraisal: A Worked Example Using a Published Manuscript
The following is an example of a critical appraisal of a randomised controlled trial using the checklist.
It is based on the following article, which is freely available online. We use the simple RCT checklist to show how it might be used to assess this paper:
Bar-Or D, Salottolo KM, Loose H, Phillips MJ, McGrath B, Wei N, Borders JL,Ervin JE, Kivitz A, Hermann M, Shlotzhauer T, Churchill M, Slappey D, Clift V: A randomised clinical trial to evaluate two doses of an intra-articular injection of LMWF-5A in adults with pain due to osteoarthritis of the knee. PLoS One 2014, 9:e87910. PMID: 24498399
Question |
Yes |
No |
Explanation |
| Is the rationale for the study clear? Is there a clear study objective / question? | Y | Knee osteoarthritis is a common and clinically important condition, and there is a shortage of effective treatments for more severe symptoms. There are concerns about the safety of intra-articular steroids, and effective and safe new treatments would be of great interest. The study question in PICO format could be summarised as “Among patients attending specialist outpatient clinics for osteoarthritis of the knee (P), what is the effect of low or high dose intra-articular injections of LMWF-5A (I) compared with placebo (C) on pain scores (O)?” | |
| Is it clear how participants were recruited, and how the number of participants was calculated? | Y | Participants were recruited from nine US clinics and from advertising to the public. The trial excluded people who had previously received opioid analgesia, intra-articular, and various other treatments. The study was adequately powered (> 80%) to detect a clinically meaningful difference. | |
| Were patients appropriately randomised to different study groups? | Y | Patients were assigned to different study groups by separate block randomisation at each of nine study clinics. | |
| In the trial, were patients and researchers both “blind” to the treatment allocation? | Y | Treatment allocation was conducted by an independent statistician. Both patients and researchers were blind to the treatment allocation. No measures to check the success of blinding were reported, though this is rarely reported in trials. | |
| Were the control and treatment groups similar at the start of the trial? Consider factors such as age, gender, and social class. | Y | Baseline characteristics were similar in treatment and control groups. | |
| Were steps taken to minimise bias? | Y | Extra attention was not given to patients in the treatment groups. Data was analysed at the same time points in all groups. | |
| Did the majority of individuals allocated to each treatment arm complete the trial? Were they amenable to further follow-up? | Y | Almost all patients completed the trial and were followed up at 12 weeks. Reasons were given for those that dropped out of the trial, and a CONSORT flow diagram was included. | |
| Was the outcome measured in the same manner between all treatment groups? Were appropriate statistical tests used? | Y | Outcome was measured by a pain score in all treatment and control groups at the same time points. Detailed information on adverse outcomes was also reported. Appropriate statistical tests were used. | |
| Did the study use intention-to-treat analysis? This means that patients are analysed in the groups to which they were randomly allocated. | Y | There was an intention-to-treat analysis. A per-protocol analysis was mentioned in the statistical analysis plan provided as an appendix to the paper, but it was not reported in the paper. Reporting of the intention-to-treat analysis is more important here. | |
| Was the precision of the treatment effect clear? | Y | The difference in mean pain scores was 0.25 (95% confidence interval 0.08, 0.41, p = 0.004) on a 5-point scale, a difference which is statistically significant. | |
| In the context of the study design and results, were the authors’ conclusions appropriate? | Unclear |
There was a statistically significant difference in pain scores between treatment and control groups, with the treatment group having a larger reduction in pain score. The mean change in pain score in the treatment group was slightly larger than a previously suggested minimum clinically important improvement. However, the improvement in pain scores in the control group was also substantial. The key outcome measure is not the change in the treatment group score, but the difference between the treatment group and the control group scores. So, whilst there is a statistically significant difference between the groups, the clinical importance of the difference between treatment and placebo groups is not completely clear. The active treatment was compared with placebo rather than with the most effective alternative available, though the range of alternative options may be limited. The authors report a number of subgroup analyses. The paper provides a statistical analysis plan as an appendix, and this plan indicates that these subgroup analyses were pre-specified. This is a strength of this particular paper, but in general, subgroup analyses are at higher risk of reporting false positive findings (type 1 errors). You should consider whether the relatively modest and short-term benefits described here are enough to support the authors’ claim that they represent “a potential major breakthrough”. This is the first published RCT of this treatment, and further trials are needed to confirm the findings. The authors accept that a longer follow-up time is also needed. In addition, the paper does not give an indication of the cost of the treatment compared with other options. |
|
| Have the authors discussed all relevant published clinical trials, and referred to current evidence / guidelines? | Y | There is only one other unpublished trial available, which is discussed. Other intra-articular treatments such as Hylan G-F 20 are also discussed, but there was no comparison with the intraarticular use of corticosteroids. There was some discussion of current guidelines for osteoarthritis treatment. | |
| Have the authors identified what the next important research steps are arising from this case? | Y | The authors are conducting an additional trial of the treatment with a longer observational period. No other future steps were identified. | |
| Was ethical approval obtained and were any conflicts of interest, including funding sources, disclosed? | Y |
The paper reports that the study gained formal ethical approval and also mentions that informed consent was obtained. A number of important conflicts of interest were noted. Several of the authors, including the first author, were employees of the company that manufactures the treatment being studied. The same company funded the study. When interpreting the results of this RCT you should bear in mind that studies funded by pharmaceutical companies tend to be more likely to report favourable results for their products than studies which do not have such conflicts of interest. |
|
Overall Summary
This paper describes a methodologically well-executed RCT with an appropriate analysis. The trial addresses a clinically important question. Treatment with LMWF- 5A produced a statistically significant benefit compared with placebo, but the size of this benefit was modest and the period of follow-up was relatively short. The study was designed and analysed by employees of the company manufacturing the product. This is an initial study, and the findings need to be replicated in other studies with longer follow-up before recommendations for practice should be changed.
It can be argued that the most important aspect of critical appraisal is to decide whether the findings apply to your patient. A case study that illustrates this point is shown in Box 3.
Checklists for Critical Appraisal by Study Type
Once the study design in a given article has been identified, the following critical appraisal checklists may be used as a rough guide to determine the quality of the study. Note that this is also useful when writing up a study yourself.
Basic Science |
||
| Is the rationale for the study clear? | You should think about whether the null hypothesis/main objective is clearly stated and if any additional secondary objectives or hypotheses are reasonable. | N |
| Was an appropriate experimental design used? | You should consider whether the design is clearly described and whether it seems suitable for addressing the study question. | N |
| Were animal models used appropriately in the study? | You should think about whether the authors were sufficiently cautious about extrapolating results to human settings. | N |
| Were the sample type, sample size, and number of replicates/repeats appropriate? | Too small a sample risks false negative results, whereas too large a study wastes resources. | N |
| Were possible confounding variables kept constant or measured accurately in the experimental set-up? Were controls utilised? Were there any counterproof experiments performed? | Counter-proof experiments remove the condition of interest to demonstrate it is needed for the desired effect. For example, if a particular gene is thought to be responsible for a drug effect, the experiment is done in a “knock out” mouse without that gene. If the drug effect disappears, this supports the original hypothesis. | N |
| Were the results clearly displayed? | Appropriate use of tables and figures. | N |
| Were appropriate statistical tests used based on the data type and distribution? | Data may be nominal, ordinal, or continuous, and the distribution may be normal or skewed. | N |
| Was the precision of the results shown? | You should look for standard deviations or confidence intervals in tables or error bars in figures. | N |
| Do the authors’ overall conclusions seem reasonable? | You should think about the study methodology, the statistical power of the study, and results from positive and negative controls. You should ask yourself whether the study answered the initial study question. | N |
| Are clinical implications/potential addressed? | The study needs to be contextualised into possible clinical impact. | N |
| Are the limitations of the study clearly described? | These need to be identified to allow accurate interpretation of results. | N |
| Is the next important research step on the topic of interest identified? | Allows academics reading the manuscript to identify future research directions. | N |
| Was ethical approval obtained, and were any conflicts of interests, including funding sources, disclosed? | Funding from parties benefiting from either positive or negative results may weaken the credibility of any results in their favour. | N |
Case Reports |
||
| Does the case report contain sufficiently detailed information about the history and examination? | This allows the reader to understand the specific case. | N |
| Were the investigations appropriate, and described in detail? | If investigations such as ECGs or imaging are crucial to the diagnosis, these should ideally be shown. | N |
| Is the diagnosis reasonable? | You should consider the information provided about presentation and the results of investigations. | N |
| Have the differential diagnoses been considered and successfully refuted? | It is important that the diagnosis is conclusive. | N |
| Does the case report include the patient’s outcome and, if applicable, follow-up data? | This ensures that any long-term impact of a new disease/ drug effect is documented. | N |
| Have the authors put the case into context? | There should be a clear and concise background explaining why this case is significant. | N |
| Have the authors discussed all relevant published cases, and referred to current evidence/guidelines? | The results should be contextualised with published resources. | N |
| Have the authors identified learning points for readers? | This ensures that the learning value from the case is clear. | N |
| Have the authors identified any important research steps arising from the case? | Allows academics reading the manuscript to identify future research direction. | N |
Editorials |
||
| Does the editorial focus on a clear topic or idea? | Gives the article a clear purpose. | N |
| Does the background information provide a concise overview of the current field and its controversies? | Contextualises the editorial. | N |
| In editorials written in response to new original research, have the authors summarised key points? | The authors should avoid repeating the original article at length. | N |
| Do the authors give an opinion, offer new insight or alternate theories, make corrections, or clarify details from a previous publication? | Ensures the editorial is not mere assertion, but a considered opinion. | N |
| Is the opinion supported by citable evidence? | Ensures any opinion is supported by evidence. | N |
| Have the authors proposed further research which would support their alternate theory? | Allows academics reading the manuscript to identify future research directions. | N |
| Is the article well-written? | Authors should avoid derogatory comments or attacks on other researchers. | N |
| Were any conflicts of interests, including funding sources, disclosed? | Funding from parties benefiting from either positive or negative results may weaken the credibility of any results in their favour. | N |
An Introduction to Clinical Governance
Clinical governance is defined as the system through which healthcare organisations are accountable for continuously improving the quality of their services and safeguarding high standards of care by creating an environment in which excellence in clinical care will flourish. The main components of this system are defined by seven pillars of clinical governance (Figure 1).
These seven pillars of clinical governance are deemed to be crucial in maintaining high standards of clinical care and continuous improvement of service provision.
- Clinical audit
Comparison of current clinical practice against agreed best practice. - Clinical effectiveness
Generation of robust evidence through scientific research. - Staffing and staff management/governance
Continuous professional development, monitoring and supervision of staff as appropriate, provision of acceptable working conditions. - Education and teaching
Maintenance of high standards through review and planning of education. - Risk management
Identification and management of clinical risk using guidelines, protocols, and other strategies. - Patient and public involvement
Ensuring that services are provided to suit patient needs and inviting patient and public feedback to aid this process. - Information and IT
Safeguarding patient confidentiality, maintaining accurate patient data, and promoting appropriate use of this data.
Projects which support clinical governance include clinical audit and service evaluation. Such projects are valued because of the potential to improve patient care. Unlike research projects, clinical audits and service evaluation projects do not usually require approval from an ethics committee. This is because such projects result in no change to the treatment of patients involved in the audit (Table 1).
On an individual level, clinical governance projects can allow you to develop your own ideas into meaningful projects that can be completed in a clinical environment alongside day-to-day training. They also provide valuable experience of the difficulties and pitfalls of driving a project to completion, and enhance your leadership skills through motivating others to achieve a common goal. Most applicants for junior doctor posts, and almost all applicants for higher medical positions, are expected to have experience in clinical governance.
Research |
Audit |
Service Evaluation |
|
| Purpose | Designed to generate new knowledge | Designed to identify ways to improve current care using already established knowledge | Defines current care |
| Methodology | Tests a specific hypothesis using established methodology | Compares services to established standards | Measures a service without reference to any particular external standard |
| Interventions | May involve patient intervention such as a new drug or investigation | Doesn’t involve new interventions | Doesn’t involve new interventions |
| Transferability of results | Results may be applicable to other organisations and settings | Results specific to service but implemented changes leading to improvement may have wider relevance | Results usually relevant only to the service being assessed |
Contact Us
Are you a medical trainee, a student, a doctor or medical professional?
We’d love to hear from you for feedback or if you would like to contribute towards the Unoffical Guide to Medicine series.