Interrater Reliability Checklist Osce Kateterisasi Urin Di Program Studi Ilmu Keperawatan Fakultas Kedokteran Universitas Gadjah Mada

https://doi.org/10.22146/jkkk.44278

Hershinta Retno Martani(1*), Intansari Nurjannah(2)

(1) Program Studi Ilmu Keperawatan, Fakultas Kedokteran, Kesehatan Masyarakat, dan Keperawatan (FK-KMK), Universitas Gadjah Mada
(2) Departemen Keperawatan Jiwa Program Studi Ilmu Keperawatan Fakultas Kedokteran, Kesehatan Masyarakat dan Keperawatan Universitas Gadjah Mada
(*) Corresponding Author

Abstract


Background: Objective Structured Clinical Examination (OSCE) is one of summative test method for performance-based assessment. One of component that make up an OSCE is assessment instrumen. Whereas checklist is one of OSCE’s component that affect OSCE’s reliability. As long as this checklist was implemented in Nursing Science Program, Faculty of Medicine, UGM, the reliability of urinary catheterization checklist hasn’t been tested
Objective: This study aims to assess interrater reliability of OSCE checklist instruments for urinary catheterization in Nursing Science Program, Faculty of Medicine, Universitas Gadjah Mada.
Methods: This study is a psychometric testing study. Two rater consisted of a fourth-year student and a lecturer who performed measurement on 93 second-year students who was taking the OSCE examination. The measurement result were analyzed using kappa test and percent agreement (PA). Whereas the item’s reliability were analyzed using weighted kappa dan some items which is paradox can be count with Prevalence and Bias Adjusted Kappa-Ordinal Scale (PABAK-OS) to separate the bias and prevalence effect.
Results: The results of measurement of the reliability was 0,57, which indicated that the checklist was in the moderate category, and the PA was 78,49%. According to Osborne (2008) and Stemler and Tsai (2008), this checklist reliability considered as acceptable. Meanwhile, the result of measurement of each item indicated various reliabilities. Reliability value on this checklist’s item was around 0,24-0,96. Meanwhile, some factors that affect OSCE’s rating categorized as item and rater.
Conclusion: The checklist of urinary catheterization has moderate reliability value and can be used as an instrument for the OSCE assessment. However, there were 9 items that weren’t reliable and must be improved.

 

ABSTRAK

Latar belakang: Objective Structured Clinical Examination (OSCE) merupakan salah satu metode penilaian sumatif dalam penilaian berbasis performa. Salah satu komponen yang menyusun OSCE adalah instrumen penilaian. Instrumen checklist merupakan komponen OSCE yang mempengaruhi reliabilitas penilaian tersebut. Selama diterapkan di PSIK FK UGM, reliabilitas checklist kateterisasi urin belum pernah diuji.
Tujuan: Penelitian ini bertujuan untuk mengetahui interrater reliability checklist OSCE kateterisasi urin di Program Studi Ilmu Keperawatan Fakultas Kedokteran Universitas Gadjah Mada.
Metode: Penelitian ini merupakan penelitian psikometri. Dua rater yang terdiri dari mahasiswa tingkat  4 dan seorang dosen menilai performa 93 mahasiswa tahun kedua dalam stase kateterisasi urin saat OSCE. Hasil pengukuran akan dihitung dan diuji menggunakan uji kappa dan Percent Agreement (PA). Sedangkan reliabilitas tiap item kateterisasi urin akan dihitung dengan menggunakan weighted kappa, dan beberapa item yang mengalami paradoks akan dihitung menggunakan Prevalence And Bias Adjusted Kappa-Ordinal Scale (PABAK-OS) untuk menghilangkan efek bias dan prevalensi.
Hasil: Hasil dari penghitungan nilai kappa menunjukkan bahwa checklist kateterisasi urin memiliki nilai kappa sebesar 0,57, dan PA sebesar 78,49%. Sedangkan pengukuran item menunjukkan hasil yang bervariasi. Nilai kappa item berada pada kisaran 0,24-0,96. Adapun faktor yang mempengaruhi penilaian OSCE dapat dilihat dari sudut pandang item maupun rater.
Kesimpulan: Checklist kateterisasi urin merupakan checklist dengan kategori reliabilitas sedang dan merupakan ceklis yang reliabel. Namun, terdapat 9 item pada checklist tersebut yang memerlukan perbaikan karena reliabilitasnya tidak dapat diterima.


Keywords


OSCE; checklist; interrater reliability

Full Text:

PDF


References

McWilliam, P., Botwinski, C. Developing a Successfull Nursing Objective Structured Clinical Examination. Journal of Nursing Education. 2010; 49(1): 36-41.

Brannick, M.T., Erol-Korkmaz, H.T., Prewett, M. 2011. A systematic review of the reliability objective structured clinical examination scores. Medical Education 2011 (45):1181-1189.

Harden, R. McG., Stevenson, M., Downie, W.W., Wilson, G.M. Assessment of Clinical Competence using Objective Structured Examination. British Medical Journal. 1975 (1): 447-451

Newble, D. Techniques for Measuring Clinical Competence: Objective Structured Clinical Examination. Medical Education 2004. (38):199-203.

Medical Council of Canada. Guidelines for the Development of Objective Structured Clinical Examination (OSCE) Cases. 2013:1-13.

Khan, K.Z., Ramachandran, S., Gaunt, K., Pushkar, P. The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part I: An historical and theoretical perspective. Medical Teacher. 2013 (35): 1437-1446

McWilliam, P., Botwinski, C. Developing a Successfull Nursing Objective Structured Clinical Examination. Journal of Nursing Education. 2010; 49(1): 36-41

Silva, C.C.B.M., Lunardi, A.C., Mendes, F.A.R., Souza, F.F.P., Carvalho, C.R.F. Objective Structured Clinical Evaluation as an Assessment Method for Undergraduate Chest Physical Therapy Students: A Cross-Sectional Study. Rev Bras Fisioter. 2011; 15 (6): 481-486

Erfanian, F. dan Khadivzadeh, T. Evaluation of Midifery Students’ Competency in Providing Intrauterine Services Using Objective Structured Clinical Examination. Journal of Nurse Midwifery Research 2011; 16 (3): 191-196

Smith, S.A. Nurse Competence: A Concept Analysis. International Journal of Nursing Knowledge. 2011; 23(3):242-247

Sakurai, H., Sugiura, Y., Tomita, M., Tanabe, S. Standardization of Clinical Skill Evaluation in Physical/Occupational Thereapist Education –Effect of Introduction of an Education System Using OSCE-. Journal of Physical Therapy Science. 2013 (25): 1071-1077

Beckham, N.D. Objective Structured Clinical Evaluation Efectiveness in Clinical Evaluation for Family Nurse Practitioner Students. Clinical Simulation in Nursing 2013 (9):453-459 13.

Mitchell, M.L., Henderson, A., Groves, M., Dalton, M. Nuity, D. The Objective Structured Clinical Examination (OSCE): Optimising its value in the undergraduate nursing curriculum. Nursing Education Today. 2009 (29): 398-404

Moattari, M., Zargar, A.S., Mousavinasab, M., Zare, N., Marvdast, B. Reliability and Validity of OSCE in Evaluating Clinical Skills of Nursing Students. Journal of Medical Education 2009; 13 (3): 79-85

Mukwato P.K., Mwape, L., Makukula, M.K., Mweemba, P., Maimbolwa, M.C. Implementation of Objective Structured Clinical Examination for Assessing Nursing Students’ Clinical Competencies: Lessons and Implications. Creative Education 2013; 4(10A): 48-53

Rushfort, H.E. Objective Structured Clinical Examination (OSCE): Review of Literature and Implications for Nursing Education. Nurse Education Today. 2007 (27):481-490

Selim, A.A., Ramadan, F.H., El-Gueneidy, M.M., Gaafer, M.M. 2012. Using Objective Structured Clinical Examination (OSCE) in Undergraduate Psychiatric Nursing Education: Is it Reliable and Valid?. Nurse Education Today 2012 (30):283-288

Tudiver, F., Rose, D., Banks, B., Pfortmiller, D. Reliability and Validity Testing of an Evidence-based Medicine OSCE Station. Family Medicine. 2009; 41(2):89-91

Miller, M.D., Linn, R.L., Gronlund, N.E. 2009. Measurement and Assessment in Teaching. United States of America. Pearson

Feinstein, A.R. dan Ciccheti, D.V. 1990. High Agreement but low kappa: I. The Problems of Two Paradox.

Warrens, M.J. 2013. Weighted Kappa for 3x3 Tables. Journal of Probability and Statistics 2013 (13):1-10

Graham, M., Milanowski, A., Miller, J., Westat. Measuring and Promoting Inter-rater Agreement of Teacher and Principal Performance Rating. Wisconsin: Center for Educator Compensation Reform 2012: 1-33

Osborne, J. Best Practice in Quantitative Methods. California: SAGE Publishing. 2008: 32-48

Stemler, S.E. dan Tsai, J. 3 Best Practice in Interrater Reliability: Three Common Approaches. Arizona: SAGE Publication. 2008: 1-45

McCray, G. Assessing inter-rater agreement for nominal judgement variables. Paper presented at the Language Testing Forum. Nottingham, November 15-17. 2013: 1-23

Streiner, D.L., Norman, G.R., dan Cairney, J. Health Measurement Scales: A practical guide to their development and use 5th ed.Oxford: Oxford University Press 2015:177

Sim, J. dan Wright, C.C. The Kappa Statistic in Reliability Studies: Use, Interpretation, and Sample Size Requiremets. Phys Ther 2005 (85):257-268

Kottner, J. dan Dassen, T. An interrater Reliability Study of the Braden Scale in Two Nursing Homes. International Journal of Nursing Studies 2008 (45): 1501-1511

Besar, M.N.A., Siraj, H.H., Manap, R.A., Mahdy, Z.A., Yaman, M.N., Kamarudin, M.A., Mohamad, N. 2012. Should a single clinician examiner be used in objective structured clinical examination?. Procedia-Social and Behavioral Science 2012 (60):443-449

Moineau, G., Power, B. Pion, A.M.J., Wood, T.J., Murto, S.H. 2010. Comparison of Student Examiner to Faculty Examiner Scoring and Feedback in an OSCE. Medical Education 2010 (45):183-191



DOI: https://doi.org/10.22146/jkkk.44278

Article Metrics

Abstract views : 2025 | views : 1859

Refbacks

  • There are currently no refbacks.


Copyright (c) 2017 Hershinta Retno, Intansari Nurjannah



Jurnal Keperawatan Klinis dan Komunitas (Clinical and Community Nursing Journal) 
collaborates with DPW PPNI DIY

Lisensi Creative Commons  

Jurnal Keperawatan Klinis dan Komunitas (Clinical and Community Nursing Journal) is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.