Aspek-Aspek yang Perlu Diperhatikan Rater dalam Verifikasi Isi Instrumen Pengukuran
Abstract
Rater memiliki peran penting dalam memvalidasi skor hasil pengukuran, sehingga rater diharapkan memiliki keahlian, salah satunya pemahaman tentang psikometrika dan ilmu pengukuran. Akan tetapi, pada kenyataannya terdapat rater yang tidak memiliki kompetensi tersebut sehingga terdapat beberapa kesalahan dalam memverifikasi isi instrumen pengukuran. Oleh karena itu, artikel ini bertujuan untuk menyoroti pentingnya kompetensi rater, bagaimana rater harus membangun kompetensi tersebut, dan bagaimana seharusnya peneliti memilih rater. Metode yang digunakan meliputi literature review dan pengalaman reflektif yang menghasilkan bahwa terdapat beberapa prinsip dan kaidah dalam memvalidasi instrumen pengukuran yang harus dikuasai dan dipahami oleh rater. Secara etika, ketika seseorang kurang memahami kaidah tersebut, maka sebaiknya tidak bersedia untuk memverifikasi isi instrumen pengukuran. Selain itu, para akademisi juga perlu untuk membangun kompetensinya di bidang pengukuran untuk mengisi kelangkaan jumlah seseorang yang pantas menjadi rater. Di sisi lain, para peneliti juga hendaknya menelusuri secara mendalam tentang kompetensi dan rekam jejak ilmiah seseorang yang akan diminta menjadi rater.
Keywords
DOI: 10.22146/buletinpsikologi.95539
References
AERA, APA, & NCME. (2014). Standards for educational and psychological testing. Washington, D.C., United States: American Educational Research Association.
Aiken, L. R. (1985). Three Coefficients For Analyzing The Reliability And Validity Of Ratings. Educational And Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012
Ambuehl, B., & Inauen, J. (2022). Contextualized Measurement Scale Adaptation: A 4-Step Tutorial for Health Psychology Research. International Journal of Environmental Research and Public Health, 19(12775), 1–24. https://doi.org/10.3390/ ijerph191912775
Aros, M., Narvaez, G., & Aros, N. H. (2009). The semantic differential for the discipline of design: a tool for the product evaluation. Product Engineering, 422–433.
Azwar, S. (2016). Reliabilitas dan Validitas (4th Ed). Yogyakarta: Pustaka Pelajar.
Azwar, S. (2021). Penyusunan Skala Psikologi (3rd Ed). Yogyakarta: Pustaka Pelajar.
Bandura, A. (1977). Self-Efficacy: Toward A Unifying Theory Of Behavioral Change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191
Bandura, A. (2006). Guide For Constructing Self-Efficacy Scales. In F. Pajares & T. Urdan (Eds.), Self-Efficacy Beliefs Of Adolescents (pp. 307–337). Greenwich, CT: Information Age Publishing. https://doi.org/10.1017/CBO9781107415324.004
Berk, R. A. (1990). Importance of Expert Judgment in Content-Related Validity Evidence. Western Journal of Nursing Research, 12(5), 659–671. https://doi.org/10.1177/019394599001200507
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 110(2), 203–219. https://doi.org/10.1037/0033-295X.110.2.203
Cizek, G. J. (2015). Validating test score meaning and defending test score use: different aims, different methods. Assessment in Education: Principles, Policy & Practice, 23(2), 212–225. https://doi.org/10.1080/0969594X.2015.1063479
Cohen, R. J., Schneider, W. J., Tobin, R., Swerdlik, M., & Sturman, E. (2022). Psychological Testing and Assessment: An Introduction to Tests and Measurement. New York, New York, United States: McGraw Hill.
Davis, D. E., Rice, K., Hook, J. N., Van Tongeren, D. R., DeBlaere, C., Choe, E., & Worthington, E. L. (2015). Development of the sources of spirituality scale. Journal of Counseling Psychology, 62(3), 503–513. https://doi.org/10.1037/cou0000082
Drost, E. A. (2011). Validity and Reliability In Social Science Research. Educational Research And Perspectives, 38(1), 105–123.
Eichenbrenner, L.-E., & Helmes, E. (2016). Social Desirability and Affect: Linking Domains of Content. Advances in Social Sciences Research Journal, 3(11), 119–125. https://doi.org/10.14738/assrj.311.2277
Embretson, S. E. (2007). Construct Validity: A Universal Validity System or Just Another Test Evaluation Procedure? Educational Researcher, 36(8), 449–455. https://doi.org/10.3102/0013189x07311600
Finn, A., & Kayande, U. (2004). Scale modification: Alternative approaches and their consequences. Journal of Retailing, 80(1), 37–52. https://doi.org/10.1016/j.jretai.2004.01.003
Fishman, J., Yang, C., & Mandell, D. (2021). Attitude theory and measurement in implementation science: a secondary review of empirical studies and opportunities for advancement. Implementation Science, 16(87), 1–10. https://doi.org/10.1186/s13012-022-01204-9
Gafni, N. (2016). Comments on implementing validity theory. Assessment in Education: Principles, Policy & Practice, 23(6), 284–286. https://doi.org/10.1080/0969594X.2015.1111195
Gorin, J. S. (2007). Reconsidering Issues in Validity Theory. Educational Researcher, 36(8), 456–462. https://doi.org/10.3102/0013189X07311607
Gross, J. J. (2002). Emotion Regulation: Affective, Cognitive, And Social Consequences. Psychophysiology, 39, 281–291. https://doi.org/10.1017.S0048577201393198
Haddock, G., & Maio, G. R. (2008). Attitudes: Content, Structure and Functions. In M. Hewstone, W. Stroebe, & K. Jonas (Eds.), Introduction to social psychology: a European perspective (4th Ed, pp. 112–133). Oxford, United Kingdom: Blackwell Publishing.
Hambleton, R. K. (1980). Test score validity and standard-setting methods. In R A Berk (Ed.), Criterion-referenced measurement: The state of the art. Baltimore, Md: John Hopkins University Press.
Henriques, G., & Michalski, J. (2020). Defining Behavior and its Relationship to the Science of Psychology. Integrative Psychological and Behavioral Science, 54(2). https://doi.org/10.1007/s12124-019-09504-4
Holt, G. D. (2014). Asking questions, analysing answers: relative importance revisited. Construction Innovation, 14(1), 2–16. https://doi.org/10.1108/CI-06-2012-0035
ITC, I. T. C. (2017). The ITC guidelines for translating and adapting tests (2nd Ed). https://doi.org/005
Kane, M. T. (2015). Explicating validity. Assessment in Education: Principles, Policy & Practice, 23(2), 198–211. https://doi.org/10.1080/0969594X.2015.1060192
Kaplan, R. M., & Saccuzzo, D. P. (2017). Psychological Testing: Principles, Applications, And Issues (9th Ed). Boston, Massachusetts, United States: Cengage Learning.
Knekta, E., Runyon, C., & Eddy, S. (2019). One size doesn’t fit all: Using factor analysis to gather validity evidence when using surveys in your research. CBE Life Sciences Education, 18(1). https://doi.org/10.1187/cbe.18-04-0064
Kwon, M., Kim, D. J., Cho, H., & Yang, S. (2013). The smartphone addiction scale: Development and validation of a short version for adolescents. PLoS ONE, 8(12), e83558. https://doi.org/10.1371/journal.pone.0083558
Kyllonen, P. C., & Zu, J. (2016). Use of response time for measuring cognitive ability. Journal of Intelligence, 4(4), 1–29. https://doi.org/10.3390/jintelligence4040014
Lane, S. (2014). Validity evidence based on testing consequences. Psycothema, 26(1), 127–135. https://doi.org/10.7334/psicothema2013.258
Larson, R. B. (2018). Controlling social desirability bias. International Journal of Market Research, 61(5), 534–547. https://doi.org/10.1177/1470785318805305
Lawshe, C. H. (1975). A Quantitative Approach To Content Validity. Personel Psychology, 28, 563–575.
Likert, R. (1932). A Tehcnique for the Measurement of Attitudes. Archives of Psychology, 22(140), 5–55.
Lissitz, R. W., & Samuelsen, K. (2007). A Suggested Change in Terminology and Emphasis Regarding Validity and Education. Educational Researcher, 36(8), 437–448. https://doi.org/10.3102/0013189X07311286
Lozano, L. M., García-Cueto, E., & Muñiz, J. (2008). Effect of the Number of Response Categories on the Reliability and Validity of Rating Scales. Methodology, 4(2), 73–79. https://doi.org/10.1027/1614-2241.4.2.73
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement. Washington, DC: Collier Macmillan Publishers.
Moss, P. A. (2007). Reconstructing Validity. Educational Researcher, 36(8), 470–476. https://doi.org/10.3102/0013189X07311608
Padilla, J. L., & Benítez, I. (2014). Validity evidence based on response processes. Psicothema, 26(1), 136–144. https://doi.org/10.7334/psicothema2013.259
Pillet, J.-C., Carillo, K. D., Vitari, C., & Pigni, F. (2023). Improving scale adaptation practices in information systems research: Development and validation of a cognitive validity assessment method. Focus On Research Methods, 33(4), 842–889. https://doi.org/10.1111/isj.12428
Reivich, K., & Shatté, A. (2002). The Resilience Factor: 7 Essential Skills for Overcoming Life’s Inevitable Obstacles. New York, New York, United States: Broadway Books.
Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psycothema, 26(1), 108–116. https://doi.org/10.7334/psicothema2013.260
Rosenberg, B. D., & Navarro, M. A. (2018). Semantic Differential Scaling. In B. B. Frey (Ed.), The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. Thousand Oaks, California, United States: SAGE Publications, Inc. https://doi.org/10.4135/9781506326139.n624
Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Special Work Research, 27(2), 94–104. https://doi.org/10.1093/swr/27.2.94
Saifuddin, A. (2020). Penyusunan Skala Psikologi (Pertama). Jakarta: KENCANA (Divisi dari Prenadamedia Group).
Saifuddin, A. (2021). Validitas Dan Reliabilitas Alat Ukur Psikologi. Depok: RajaGrafindo Persada.
Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43–53. https://doi.org/10.1016/j.newideapsych.2011.02.007
Seligman, M. E. P. (2006). Learned Optimism: How to Change Your Mind and Your Life. New York, New York, United States: Vintage Books.
Stark, R., & Glock, C. Y. (1968). American Piety: The Nature of Religious Commitment. Berkeley: University of California Press.
Steinberg, L., & Rogers, A. (2022). Changing the Scale: The Effect of Modifying Response Scale Labels on the Measurement of Personality and Affect. Multivariate Behavioral Research, 57(1), 79–93. https://doi.org/10.1080/00273171.2020.1807305
Syaiful, I. A., & Roebianto, A. (2020). Adapting and Examining the Factor Structure of the Self-Compassion Scale in Indonesian Version. Jurnal Psikologi, 47(3), 175–205. https://doi.org/10.22146/jpsi.57608
van Heerden, D. B. J., & Mellenbergh, G. J. (2003). Validity And Truth. In H. Yanai, A. Okada, K. Shigemasu, Y. Kano, & J. J. Meulman (Eds.), New Developments In Psychometrics (pp. 1–8). Tokyo, Japan: Springer. https://doi.org/10.1007/978-4-431-66886-8_36
Widhiarso, W. (2016). Peranan Butir Unfavorabel Dalam Menghasilkan Dimensi Baru Dalam Pengukuran Psikologi. Jurnal Psikologi Perseptual, 1(1), 40–52. https://doi.org/10.24176/perseptual.v1i1.1078
Willis, G. B. (1999). Cognitive Interviewing: A “How To” Guide, Reducing Survey Error through Research on the Cognitive and Decision Processes in Surveys. In Research Triangle Park, NC: Research Triangle Institute.
Willis, G. B., Royston, P., & Bercini, D. (1991). The use of verbal report methods in the development and testing of survey questionnaires. Applied Cognitive Psychology, 5(3), 251–26. https://doi.org/10.1002/acp.2350050307
Wilson, B. F., & Peterson, L. S. (1999). Using the NCHS cognitive lab to help design cycle VI of the national survey of family growth. Proceedings of the Survey Research Methods Section, American Statistical Association, 997–1002.
Refbacks
- There are currently no refbacks.
Copyright (c) 2024 Buletin Psikologi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
* Corresponding Author
