The Development of Performance Validity Tests (PVTs) for Indonesia: A Literature Review and Recommendations

Widhi Adhiatma; Marc P. H. Hendriks; Magdalena S. Halim; Roy P. C. Kessels

doi:10.22146/buletinpsikologi.73390

The Development of Performance Validity Tests (PVTs) for Indonesia: A Literature Review and Recommendations

Widhi Adhiatma, Marc P. H. Hendriks, Magdalena S. Halim, Roy P. C. Kessels

(Submitted 4 March 2022)

(Published 27 June 2023)

Abstract

Neuropsychological tests are proliferating in Indonesia. So far, unfortunately, it is not accompanied by the development of a performance validity test (PVT). According to international neuropsychological standards, using PVTs is essential to determine the validity of the neuropsychological test results. To date, there is no single standardized PVT available in Indonesia. In this article, we describe (1) the concept of performance validity testing, (2) the detection strategy, and (3) the procedure of PVT. Furthermore, several factors which affect PVT failure and its implication on the interpretation of a neuropsychological assessment are discussed. Considering the importance of PVTs incorporation and that it has become a standard by some neuropsychological associations, it is crucial to develop PVTs in Indonesia. Finally, the utilization of PVT should be performed cautiously, as many factors might influence the PVT result.

Keywords

performance validity test (PVT); malingering; underperformance; suboptimal effort; neuropsychology

Full Text: PDF

DOI: 10.22146/buletinpsikologi.73390

References

Adhiatma, W., & Halim, M. S. (2019). Exploratory factor analysis of the Indonesian version of MMPI-2-RF Restructured Clinical (RC) Scales. HUMANITAS: Indonesian Psychological Journal, 16(1), 66–73. https://doi.org/10.26555/humanitas.v16i1.9420

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorder: DSM-5. American Psychiatric Publishing, Inc.

Ardila, A. (2005). Cultural values underlying psychometric cognitive testing. Neuropsychology Review, 15(4), 185–195. https://doi.org/10.1007/s11065-005-9180-y

Babikian, T., Boone, K. B., Lu, P., & Arnold, G. (2006). Sensitivity and specificity of various digit span scores in the detection of suspect effort. The Clinical Neuropsychologist, 20(1), 145–159. https://doi.org/10.1080/13854040590947362

Barker, M. D., Horner, M. D., & Bachman, D. L. (2010). Embedded indices of effort in the repeatable battery for the assessment of neuropsychological status (Rbans) in a geriatric sample. The Clinical Neuropsychologist, 24(6), 1064–1077. https://doi.org/10.1080/13854046.2010.486009

Bigler, E. D. (2012). Symptom validity testing, effort, and neuropsychological assessment. Journal of the International Neuropsychological Society, 18, 632–642. https://doi.org/10.1017/S1355617712000252

Binder, L. M. (2003). The Portland Digit Recognition Test: A Review of Validation Data and Clinical Use. Journal of Forensic Neuropsychology, 2(3–4), 27–41. https://doi.org/10.1300/J151v02n03_02

Binder, L. M., Larrabee, G. J., & Millis, S. R. (2014). Intent to fail: Significance testing of forced choice test results. The Clinical Neuropsychologist, 28(8), 1366–1375. https://doi.org/10.1080/13854046.2014.978383

Binder, L. M., & Willis, S. C. (1991). Assessment of Motivation After Financially Compensable Minor Head Trauma. Psychological Assessment, 3(2), 175–181. https://doi.org/10.1037/1040-3590.3.2.175

Bodner, T., Merten, T., & Benke, T. (2019). Performance validity measures in clinical patients with aphasia. Journal of Clinical and Experimental Neuropsychology, 41(5), 476–483. https://doi.org/10.1080/13803395.2019.1579783

Bush, S. S., Ruff, R. M., Tröster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., Reynolds, C. R., & Silver, C. H. (2005). Symptom validity assessment: Practice issues and medical necessity: NAN Policy & Planning Committee. Archives of Clinical Neuropsychology, 20(4), 419–426. https://doi.org/10.1016/j.acn.2005.02.002

Butcher, J. N. (2005). A beginner’s guide to the MMPI-2 (2nd ed.). American Psychological Association.

Carone, D. A., Iverson, G. L., & Bush, S. S. (2010). A model to approaching and providing feedback to patients regarding invalid test performance in clinical neuropsychological evaluations. The Clinical Neuropsychologist, 24(5), 759–778. https://doi.org/10.1080/13854041003712951

Critchfield, E., Soble, J. R., Marceaux, J. C., Bain, K. M., Bailey, K. C., Webber, T. A., Alverson, W. A., Messerly, J., González, D. A., & O’Rourke, J. J. F. (2019). Cognitive impairment does not cause invalid performance: Analyzing performance patterns among cognitively unimpaired, impaired, and noncredible participants across six performance validity tests. The Clinical Neuropsychologist, 33(6), 1083–1101. https://doi.org/10.1080/13854046.2018.1508615

Dandachi-FitzGerald, B., & Martin, P. K. (2022). Clinical judgment and clinically applied statistics: Description, benefits, and potential dangers when relying on either one individually in clinical practice. In R. W. Schroeder & P. K. Martin (Eds.), Validity assessment in clinical neuropsychological practice: Evaluating and managing noncredible performance (pp. 107–125). The Guilford Press.

Dandachi-FitzGerald, B., Merckelbach, H., & Ponds, R. W. H. M. (2017). Neuropsychologists’ ability to predict distorted symptom presentation. Journal of Clinical and Experimental Neuropsychology, 39(3), 257–264. https://doi.org/10.1080/13803395.2016.1223278

Davis, J. J., & Millis, S. R. (2014). Examination of performance validity test failure in relation to number of tests administered. The Clinical Neuropsychologist, 28(2), 199–214. https://doi.org/10.1080/13854046.2014.884633

Dewiyana, Satiti, S., & Gofir, A. (2019). Penggunaan HIV dementia scale dan international HIV dementia scale pada skrining demensia. Berkala Neurosains, 18(2), 84–89.

Erdodi, L. A., & Lichtenstein, J. D. (2017). Invalid before impaired: an emerging paradox of embedded validity indicators. The Clinical Neuropsychologist, 31(6–7), 1029–1046. https://doi.org/10.1080/13854046.2017.1323119

Foussias, G., Siddiqui, I., Fervaha, G., Mann, S., McDonald, K., Agid, O., Zakzanis, K. K., & Remington, G. (2015). Motivated to do well: An examination of the relationships between motivation, effort, and cognitive performance in schizophrenia. Schizophrenia Research, 166(1–3), 276–282. https://doi.org/10.1016/j.schres.2015.05.019

Galioto, R., Dhima, K., Berenholz, O., & Busch, R. (2020). Performance Validity Testing in Multiple Sclerosis. Journal of the International Neuropsychological Society, 26(10), 1028–1035. https://doi.org/10.1017/S1355617720000466

Gorissen, M., Sanz, J. C., & Schmand, B. (2005). Effort and cognition in schizophrenia patients. Schizophrenia Research, 78(2–3), 199–208. https://doi.org/10.1016/j.schres.2005.02.016

Green, P. (2008). Green’s Non-Verbal Medical Symptom Validity Test (NV-MSVT): User’s Manual for Microsoft Windows. Green’s Publishing Inc.

Greher, M. R., & Wodushek, T. R. (2017). Performance validity testing in neuropsychology: Scientific basis and clinical application - A brief review. Journal of Psychiatric Practice, 23(2), 134–140. https://doi.org/10.1097/PRA.0000000000000218

Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of Malingered Amnesia Measures With a Large Clinical Sample. Psychological Assessment, 6(3), 218–224. https://doi.org/10.1037//1040-3590.6.3.218

Guilmette, T. J. (2012). The role of clinical judgment in symptom validity assessment. In D. A. Carone & S. S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment and malingering (pp. 31–43). Springer Publishing Company.

Guilmette, T. J., Sweet, J. J., Hebben, N., Koltai, D., Mahone, E. M., Spiegler, B. J., Stucky, K., & Westerveld, M. (2020). American Academy of Clinical Neuropsychology consensus conference statement on uniform labeling of performance test scores. Clinical Neuropsychologist, 34(3), 437–453. https://doi.org/10.1080/13854046.2020.1722244

Hathaway, S. R., & McKinley, J. C. (1940). A multiphasic personality schedule (Minnesota): I. Construction of the schedule. Journal of Personality, 10, 249–254.

Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., & Millis, S. R. (2009). American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129. https://doi.org/10.1080/13854040903155063

Husein, N., Lumempouw, S. F., Ramli, Y., & Herqutanto. (2010). Uji validitas dan reliabilitas Montreal Cognitive Assessment versi Indonesia (MoCA-Ina) untuk skrining gangguan kognitif. Neurona, 27(4).

Indrajaya, A. W., Lumempouw, S. F., Ramli, Y., & Prihartono, J. (2013). Nilai normal pemeriksaan neuropsikologi CERAD di Jakarta. Neurona, 30(3).

Jones, A., Ingram, M. V., & Ben-Porath, Y. S. (2012). Scores on the MMPI-2-RF scales as a function of increasing levels of failure on cognitive symptom validity tests in a military sample. The Clinical Neuropsychologist, 26(5), 790–815. https://doi.org/10.1080/13854046.2012.693202

Kessels, R. P. C., & Hendriks, M. P. H. (2022). Neuropsychological Assessment. In H. S. Friedman (Ed.), Encyclopedia of Mental Health (Vol. 3). Academic Press. https://doi.org/https://doi.org/10.1016/B978-0-323-91497-0.00017-5

Khairunnisa, G., Putri, P., Cheerson, F., Junita, F., Suwartono, C., & Halim, M. S. (2014). Uji validitas konstruk The Modified Mini Mental State-Test (3MS). Jurnal Pengukuran Psikologi Dan Pendidikan Indonesia, 3(4), 329–359.

Kiewel, N. A., Wisdom, N. M., Bradshaw, M. R., Pastorek, N. J., & Strutt, A. M. (2012). A retrospective review of Digit span-related effort indicators in probable Alzheimer’s disease patients. The Clinical Neuropsychologist, 26(6), 965–974. https://doi.org/10.1080/13854046.2012.694478

Kolb, B., & Whishaw, I. Q. (2015). Fundamentals of human neuropsychology (7th ed.). Worth Publishers.

Larrabee, G. J. (2012). Performance validity and symptom validity in neuropsychological assessment. Journal of the International Neuropsychological Society, 18(4), 625–631. https://doi.org/10.1017/S1355617712000240

Larrabee, G. J. (2015). The multiple validities of neuropsychological assessment. American Psychologist, 70(8), 779–788. https://doi.org/10.1037/a0039835

Larrabee, G. J., Millis, S. R., & Meyers, J. E. (2009). 40 plus or minus 10, a new magical number: Reply to Russell. The Clinical Neuropsychologist, 23(5), 841–849. https://doi.org/10.1080/13854040902796735

Larrabee, G. J., Rohling, M. L., & Meyers, J. E. (2019). Use of multiple performance and symptom validity measures: Determining the optimal per test cutoff for determination of invalidity, analysis of skew, and inter-test correlations in valid and invalid performance groups. The Clinical Neuropsychologist, 33(8), 1354–1372. https://doi.org/10.1080/13854046.2019.1614227

Laurent, R. L., Whiteside, D. M., & Basso, M. R. (2019). Performance Validity Testing in an Older Adult Population: Considerations for Clinical Practice. In L. D. Ravdin & H. L. Katzen (Eds.), Handbook on the Neuropsychology of Aging and Dementia (2nd ed., pp. 77–88). Springer Nature Switzerland AG. https://doi.org/10.1007/978-1-4614-3106-0

Loring, D. W., & Goldstein, F. C. (2019). If Invalid PVT Scores Are Obtained, Can Valid Neuropsychological Profiles Be Believed? Archives of Clinical Neuropsychology, 34(7), 1192–1202. https://doi.org/10.1093/arclin/acz028

Martin, P. K., & Schroeder, R. W. (2020). Base rates of invalid test performance across clinical non-forensic contexts and settings. Archives of Clinical Neuropsychology, 35(6), 717–725. https://doi.org/10.1093/arclin/acaa017

Martin, P. K., & Schroeder, R. W. (2022). A framework for providing clinical feedback when patients invalidate testing. In R. W. Schroeder & P. K. Martin (Eds.), Validity assessment in clinical neuropsychological practice: Evaluating and managing noncredible performance (pp. 47–69). The Guilford Press.

Martin, P. K., Schroeder, R. W., Olsen, D. H., Maloy, H., Boettcher, A., Ernst, N., & Okut, H. (2020). A systematic review and meta-analysis of the Test of Memory Malingering in adults: Two decades of deception detection. The Clinical Neuropsychologist, 34(1), 88–119. https://doi.org/10.1080/13854046.2019.1637027

McGuire, C., Crawford, S., & Evans, J. J. (2019). Effort Testing in Dementia Assessment: A Systematic Review. Archives of Clinical Neuropsychology, 34, 114–131. https://doi.org/10.1093/arclin/acy012

McWhirter, L., Ritchie, C. W., Stone, J., & Carson, A. (2020). Performance validity test failure in clinical populations - A systematic review. Journal of Neurology, Neurosurgery and Psychiatry, 91, 945–952. https://doi.org/10.1136/jnnp-2020-323776

Merckelbach, H., Dandachi-FitzGerald, B., van Helvoort, D., Jelicic, M., & Otgaar, H. (2019). When Patients Overreport Symptoms: More Than Just Malingering. Current Directions in Psychological Science, 28(3), 321–326. https://doi.org/10.1177/0963721419837681

Morra, L. F., Gold, J. M., Sullivan, S. K., & Strauss, G. P. (2015). Predictors of Neuropsychological Effort Test Performance in Schizophrenia. Schizophrenia Research, 162, 205–210. https://doi.org/10.1016/j.schres.2014.12.033

Nauta, I. M., Bertens, D., van Dam, M., Huiskamp, M., Driessen, S., Geurts, J. J. G., Uitdehaag, B. M. J., Fasotti, L., Hulst, H. E., de Jong, B. A., & Klein, M. (2022). Performance validity in outpatients with multiple sclerosis and cognitive complaints. Multiple Sclerosis Journal, 28(4), 642–653. https://doi.org/10.1177/13524585211025780

Nijdam-Jones, A., Rivera, D., Rosenfeld, B., & Arango-Lasprilla, J. C. (2019). The effect of literacy and culture on cognitive effort test performance: An examination of the Test of Memory Malingering in Colombia. Journal of Clinical and Experimental Neuropsychology, 41(10), 1015–1023. https://doi.org/10.1080/13803395.2019.1644294

Patrick, R. E., & Horner, M. D. (2014). Psychological characteristics of individuals who put forth inadequate cognitive effort in a secondary gain context. Archives of Clinical Neuropsychology, 29(8), 754–766. https://doi.org/10.1093/arclin/acu054

Pearson. (n.d.). Validity Indicator Profile (VIP). Retrieved July 13, 2021, from https://www.pearsonassessments.com/store/usassessments/en/Store/Professional-Assessments/Cognition-%26-Neuro/Validity-Indicator-Profile/p/100000284.html?tab=faqs

Rambe, A. S. (2008). Neuropsychological evaluation against geriatric background. Majalah Kedokteran Nusantara, 41(4).

Richey, L. N., & Doninger, N. A. (2020). A Comparison of Performance Validity Measures in Predicting MMPI-2 Lie Scale Results. The Clinical Neuropsychologist, 34(2), 353–367. https://doi.org/10.1080/13854046.2019.1643922

Rogers, R. (2018a). An introduction to response styles. In R. Rogers & S. D. Bender (Eds.), Clinical Assessment of Malingering and Deception (pp. 3–17). The Guilford Press.

Rogers, R. (2018b). Detection Strategies for Malingering and Defensiveness. In R. Rogers & S. D. Bender (Eds.), Clinical Assessment of Malingering and Deception (4th ed., pp. 18–41). The Guilford Press.

Rogers, R., & Correa, A. A. (2008). Determinations of malingering: Evolution from case-based methods to detection strategies. Psychiatry, Psychology and Law, 15(2), 213–223. https://doi.org/10.1080/13218710802014501

Ruiz, I., Raugh, I. M., Bartolomeo, L. A., & Strauss, G. P. (2020). A Meta-Analysis of Neuropsychological Effort Test Performance in Psychotic Disorders. Neuropsychology Review, 30(3), 407–424. https://doi.org/10.1007/s11065-020-09448-2

Ryan, J. J., Yamaguchi, T., & Kreiner, D. S. (2019). Preliminary Validation of the Rey 15-Item Test and Reliable Digit Span in Native Japanese Samples. Psychological Reports, 122(5), 1925–1945. https://doi.org/10.1177/0033294118792697

Schagen, S., Schmand, B., De Sterke, S., & Lindeboom, J. (1997). Amsterdam short-term memory test: A new procedure for the detection of feigned memory deficits. Journal of Clinical and Experimental Neuropsychology, 19(1), 43–51. https://doi.org/10.1080/01688639708403835

Schroeder, R. W., & Martin, P. K. (2022). Incorporation of validity assessment and validity findings when writing clinical reports. In R. W. Schroeder & P. K. Martin (Eds.), Validity assessment in clinical neuropsychological practice: Evaluating and managing noncredible performance (pp. 70–85). The Guilford Press.

Sherman, E. M. S., Slick, D. J., & Iverson, G. L. (2020). Multidimensional malingering criteria for neuropsychological assessment: A 20-year update of the malingered neuropsychological dysfunction criteria. Archives of Clinical Neuropsychology, 35(6), 735–764. https://doi.org/10.1093/arclin/acaa019

Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary (3rd ed.). Oxford University Press.

Strutt, A. M., & Stinson, J. M. (2022). Performance validity testing with culturally diverse individuals and non-native English speakers: The need for a cultural perspective in neuropsychological practice. In R. W. Schroeder & P. K. Martin (Eds.), Validity assessment in clinical neuropsychological practice: Evaluating and managing noncredible performance (pp. 211–232). The Guilford Press.

Sulastri, A., Utami, M. S. S., Jongsma, M., Hendriks, M., & Luijtelaar, G. van. (2019). The Indonesian Boston Naming Test: Normative Data among Healthy Adults and Effects of Age and Education on Naming Ability. International Journal of Science and Research, 8(11), 134–139. https://doi.org/10.21275/ART20202307

Suwartono, C., Halim, M. S., Hidajat, L. L., Hendriks, M. P. H., & Kessels, R. P. C. (2014). Development and Reliability of the Indonesian Wechsler Adult Intelligence Scale—Fourth Edition (WAIS-IV). Psychology, 05(14), 1611–1619. https://doi.org/10.4236/psych.2014.514171

Syahroni, Lumempouw, S. F., Lastri, D. N., & Herqutanto. (2011). Validasi transkultural dan uji reliabilitas Screening Test for Luria Nebraska Neuropsychological Battery (ST-LNNB) untuk penapisan gangguan kognitif. Neurona, 28(2).

Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM). Multi-Health System, Inc.

Utami, M. S. S., Sulastri, A., & Guritno, H. (2019). Rey Auditory Verbal Learning Test of University Students. Psikodimensia, 18(1), 37–48. https://doi.org/10.24167/psidim.v18i1.1740

van de Vijver, F., & Tanzer, N. K. (2004). Bias and equivalence in cross-cultural assessment: An overview. Revue Europeenne de Psychologie Appliquee, 54(2), 119–135. https://doi.org/10.1016/j.erap.2003.12.004

Votruba, K. L., Rykulski, N., Dumitrescu, C., & Abeare, C. A. (2020). Handedness and performance validity test performance. Psychology and Neuroscience, 13(2), 196–205. https://doi.org/10.1037/pne0000188

Wahyuningrum, S. E., van Luijtelaar, G., & Sulastri, A. (2021). An online platform and a dynamic database for neuropsychological assessment in Indonesia. Applied Neuropsychology:Adult, 1–10. https://doi.org/10.1080/23279095.2021.1943397

Western Psychological Services. (2021). The b Test. https://www.wpspublish.com/the-b-test

Widyarini, N., Lumempouw, S. F., Ramli, Y., & Herqutanto. (2013). Nilai normal pemeriksaan Status Mental Neurologi Strub dan Black versi Indonesia. Neurona, 30(3).

Zenisek, R., Millis, S. R., Banks, S. J., & Miller, J. B. (2016). Prevalence of below-criterion Reliable Digit Span scores in a clinical sample of older adults. Archives of Clinical Neuropsychology, 31(5), 426–433. https://doi.org/10.1093/arclin/acw025

Refbacks

There are currently no refbacks.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.