Featured Article| Volume 32, P1-7, July 2019

Medication Administration Evaluation and Feedback Tool: Simulation Reliability Testing


      • The use of simulation was an effective method to test the intrarater and inter-rater reliability of the Medication Administration Evaluation and Feedback Tool (MAEFT).
      • Fleiss' Kappa coefficient for intra-rater and inter-rater reliability of multiple observers using the MAEFT measured good reliability.
      • Intrarater and inter-rater reliability and accuracy of the designed MAEFT shows that it is reliable and accurate in a simulated environment.



      Incorrect medication administration risks errors and patient harm. The aim of this study was to test the reliability of the newly developed medication administration evaluation and feedback tool.


      An observational, fully crossed design using recorded scenarios in a simulated environment was used to test reliability and agreement of an evaluation tool for nurses administering medications.


      Intrarater and inter-rater reliability observed agreement overall were 84% and 82% with Fleiss Kappa coefficient 0.72 and 0.68, respectively. Overall intrarater and inter-rater reliability of the tool rated as good.


      The medication administration evaluation and feedback tool is reliable in a simulated environment.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Clinical Simulation In Nursing
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Australian Commission on Safety and Quality in Health Care
        National Safety and Quality Health Service Standards.
        2nd ed. ACSQHC, Sydney2017
        • Berdot S.
        • Gillaizeau F.
        • Caruba T.
        • Prognon P.
        • Durieux P.
        • Sabatier B.
        Drug administration errors in hospital inpatients: a systematic review.
        PLoS One. 2013; 8: e68856
        • Butler D.L.
        • Winne P.H.
        Feedback and self-regulated learning: a theoretical synthesis.
        Review of Educational Research. 1995; 65: 245
        • Calleja P.
        • Harvey T.
        • Fox A.
        • Carmichael M.
        Feedback and clinical practice improvement: a tool to assist workplace supervisors and students.
        Nurse Education in Practice. 2016; 17: 167-173
        • College of Nurses Ontario
        Medication Practice Standard.
        2017 (Retrieved from)
        • Colquhoun H.L.
        • Brehaut J.C.
        • Sales A.
        • Ivers N.
        • Grimshaw J.
        • Michie S.
        • Carroll K.
        • Chalifoux M.
        • Eva K.W.
        A systematic review of the use of theory in randomized controlled trials of audit and feedback.
        Implementation Science: IS. 2013; 8: 66
        • Davies K.
        • Coombes I.
        • Keogh S.
        • Whitfield K.
        Medication administration evaluation tool design: an expert panel review.
        Collegian. 2019; 26: 118-124
        • Davies M.
        • Fleiss J.L.
        Measuring agreement for multinomial data.
        Biometrics. 1982; 38: 1047-1051
        • Davies K.
        • Mitchell C.
        • Coombes I.
        The role of observation and feedback in enhancing performance with medication administration.
        Journal of Law and Medicine. 2015; 23: 316
        • Davies K.
        • Norris L.
        • O’Brien E.
        • Buksh F.
        • Rogers K.
        • Kubler P.
        • Donovan P.
        • Coombes I.
        Internal Medicine Services (IMS) medication administration observation & feedback.
        in: Paper Presented at the Royal Brisbane and Women's Hospital Symposium, Brisbane. 2015
        • Fleiss J.L.
        • Levin B.
        • Paik M.C.
        Statistical Methods for Rates and Proportions.
        3rd ed. Wiley, Hoboken2004
        • Gamer M.
        • Lemon J.
        • Fellows I.
        • Singh P.
        Irr: various coefficients of interrater reliability and agreement. R package version 0.84.
        (Retrieved from)
        Date: 2012
        Date accessed: April 26, 2018
        • Goodstone L.
        • Goodstone M.S.
        Use of simulation to develop a medication administration safety assessment tool.
        Clinical Simulation in Nursing. 2013; 9: e609-e615
        • Hallgren K.A.
        Computing inter-rater reliability for observational data: An overview and tutorial.
        Tutorials in Quantitative Methods for Psychology. 2012; 8: 23
        • Härkänen M.
        • Vehviläinen-Julkunen K.
        • Murrells T.
        • Rafferty A.M.
        • Franklin B.D.
        Medication administration errors and mortality: Incidents reported in England and Wales between 2007 ̶ 2016.
        Research in Social and Administrative Pharmacy. 2018;
        • Harris P.A.
        • Taylor R.
        • Thielke R.
        • Payne J.
        • Gonzalez N.
        • Conde J.G.
        Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support.
        Journal of Biomedical Informatics. 2009; 42: 377-381
        • Hattie J.
        • Timperley H.
        The power of feedback.
        Review of Educational Research. 2007; 77: 81-112
        • Hemingway S.
        • Baxter H.
        • Smith G.
        • Burgess-Dawson R.
        • Dewhirst K.
        Collaboratively planning for medicines administration competency: A survey evaluation.
        Journal of Nursing Management. 2011; 19: 366-376
        • INACSL Standards Committee
        INACSL standards of best practice: SimulationSM simulation glossary.
        Clinical Simulation in Nursing. 2016; 12: S39-S47
        • Jeffries R.P.
        • Rodgers R.B.
        • Adamson R.K.
        NLN Jeffries simulation theory: brief narrative description.
        Nursing Education Perspective. 2015; 36: 292-293
        • Johnson C.
        • Keating J.
        • Boud D.
        • Dalton M.
        • Kiegaldie D.
        • McKenzie W.
        • Nestel D.
        • Palermo C.
        • Molloy E.
        Identifying educator behaviours for high quality verbal feedback in health professions education: literature review and expert refinement.
        BMC Medical Education. 2016; 16: 96
        • Kaufman D.M.
        Applying educational theory in practice.
        BMJ. 2003; 326: 213-216
        • Knowles M.S.
        Informal Adult Education: A Guide for Administrators, Leaders, and Teachers.
        Association Press, New York1950
        • Merrifield N.
        RCN concerns over proposed changes to mentor training and medicines management.
        Nursing Times. 2017; (Retrieved from)
        • Nursing and Midwifery Council
        Standards for Medicines Management.
        2010 (UK. (Accessed 5 April 2018))
        • Nursing and Midwifery Council
        New NMC standards shape the future of nursing for next generation [Press release].
        (Retrieved from)
        • O'Brien E.
        Medication resource package. RBWH intranet: royal brisbane and women's hospital (RBWH).
        (Retrieved from)
        • Polit D.F.
        • Beck C.T.
        The content validity index: are you sure you know what's being reported? Critique and recommendations.
        Research in Nursing & Health. 2006; 29: 489-497
        • Sitzmann T.
        • Ely K.
        A meta-analysis of self-regulated learning in work-related training and educational attainment: what we know and where we need to go.
        Psychological Bulletin. 2011; 137: 421-442
        • Walshe N.
        • O’Brien S.
        • Hartigan I.
        • Murphy S.
        • Graham R.
        Simulation performance evaluation: Inter-rater reliability of the DARE2-patient safety rubric.
        Clinical Simulation in Nursing. 2014; 10: 446-454
        • Wang E.E.
        Simulation and adult learning.
        Disease-a-Month. 2011; 57: 664-678
        • World Health Organisation
        Education and Training: Technical Series on Safer Primary Care. (Licence: CC BY-NC-SA 3.0 IGO.).
        Geneva World Health Organization, 2016 (Retrieved from)
        • World Health Organisation
        Medication without harm: WHO's third global patient safety challenge.
        (Retrieved from)
        Date: 2017
        Date accessed: April 15, 2017
        • Yanes A.F.
        • McElroy L.M.
        • Abecassis Z.A.
        • Holl J.
        • Woods D.
        • Ladner D.P.
        Observation for assessment of clinician performance: a narrative review.
        BMJ Quality & Safety. 2015; 25: 46-55