Teori kritis terhadap analisis sitasi untuk kajian kuantitatif sains dan evaluasi kinerja riset

  • Yaniasih Yaniasih Indonesian Institute of Sciences
Keywords: citation analysis, critical theory, machine learning


Introduction. Citation is the main indicator in research performance evaluation using the quantitative approach. There have been many criticisms of citations since they were used half a century ago, but they have not yet succeeded in bringing new concepts and methods. This paper aims to criticize and propose a new approach to citation analysis.

Data Collection Method. A contemporary critical theory methodology was adopted as a framework to collect and analyze the data. Scientific publications related to citation analysis was collected from several databases such as Google scholar, Microsoft academic search, dan Garuda Ristekdikti.

Analysis Data. Publications data were critically reviewed and analyzed narratively by using open coding.

Results and Discussions. The results mapped the lack of citation analysis form various aspects: (1) criticism of the positivist paradigm which did not succeed in achieving its objectives, (2) criticism of methods that produce invalid results, and (3) criticism of ethical issues of the researcher and  bias in implementation. The proposed solution and recommendation is to change the citation analysis method from a simple measurement of bibliographic data to text and context analysis based on a computer science approach (machine learning techniques).

Conclusion. This new method has the potential to be developed within the framework of quantitative in Science and Technology studies to overcome existing criticisms. Subsequent multidisciplinary studies are needed to lay a strong philosophical and technical foundation particularly in applying the in-text citation analysis method for evaluating research performance in accordance with the Indonesian context.


Abu-jbara, A., Ezra, J., & Radev, D. (2013). Purpose and polarity of citation : Towards NLP-based bibliometrics. Proceedings OfNAACL-HLT, (June), 596–606. Atlanta, Georgia: Association for Computational Linguistics.

Andersen, J. P. (2017). An empirical and theoretical critique of the Euclidean index. Journal of Informetrics, 11(2), 455–465. https://doi.org/10.1016/j.joi.2017.02.009

Athar, A., & Teufel, S. (2012). Context-enhanced citation sentiment detection. Conference Ofthe North American Chapter Ofthe Association for Computational Linguistics: Human Language Technologies, 597–601. Montr´eal, Canada.

Belter, C. W. (2015). Bibliometric indicators: Opportunities and limits. Journal of the Medical Library Association, 103(4), 219–221. https://doi.org/10.3163/1536-5050.103.4.014

Bernstein, R. (1991). Habermas and modernity (R. Bernstein, ed.). Cambridge, Massachusetts: The MIT Press.

Buranyi, S., & McKenzie, D. (2017). Is the staggeringly profitable business of scientific publishing bad for science? https://doi.org/10.1371/journal.pmed.0050201

Cobo, M. J., Lopez-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). Science mapping software tools : review , analysis , and cooperative study among tools. Journal of American Society for Information Science and Technology, 62(7), 1382–1402. https://doi.org/10.1002/asi

De Bellis, N. (2010). Bibliometrics and citation analysis: From the science citation index to cybermetrics (Vol. 23). https://doi.org/10.1087/20100312

Elabdi, M., Smine, B., & Ben Yahia, S. (2018). DFBICA: A new distributed approach for sentiment analysis of bibliographic citations. Proceedings - International Conference on Research Challenges in Information Science, 2018-May, 1–6. https://doi.org/10.1109/RCIS.2018.8406672

Finardi, U. (2013). Correlation between journal impact factor and citation Performance: An experimental study. Journal of Informetrics, 7(2), 357–370. https://doi.org/10.1016/j.joi.2012.12.004

Garfield, E. (1983). Citation indexing, Its theory and application in science, technology, and humanities. Philadelphia: ISI Press.

Geisler, E. (2005). The measurement of scientific activity : Research directions in linking philosophy of science and metrics of sciences and technology outputs. Scientometrics, 62(2), 269–284.

Hood, W., & Wilson, C. (2003). Hood, Wilson - 2003 - Informetric studies using databases Opportunities and challenges.pdf. Scientometrics, 58(3), 587–608.

Hussain, S. J., Maqsood, S., Jhanjhi, N., Khan, A., Supramaniam, M., & Ahmed, U. (2019). A Comprehensive evaluation of cue-words based features and in-text citations based features for citation classification. International Journal of Advanced Computer Science and Applications, 10(7), 209–218. https://doi.org/10.14569/ijacsa.2019.0100730

Ikram, M. T., & Afzal, M. T. (2019). Aspect based citation sentiment analysis using linguistic patterns for better comprehension of scientific knowledge. Scientometrics, 119(1), 73–95. https://doi.org/10.1007/s11192-019-03028-9

Iwan, I. (2014). Menelaah teori kritis Jurgen Habermas. Jurnal Edoeksos, III(2), 145–165.

Jappe, A., Pithan, D., & Heinze, T. (2018). Does bibliometric research confer legitimacy to research assessment practice? A sociological study of reputational control, 1972-2016. PLoS ONE, 13(6), 1–28. https://doi.org/10.1371/journal.pone.0199031

Kilicoglu, H., Peng, Z., Tafreshi, S., Tran, T., Rosemblat, G., & Schneider, J. (2019). Confirm or refute?: A comparative study on citation sentiment classification in clinical research publications. Journal of Biomedical Informatics, 91(January), 103123. https://doi.org/10.1016/j.jbi.2019.103123

Kinouchi, R. R. (2014). Scientometrics : the project for a science of science transformed into an industry of measurements. Scientiae Studia, 12, 147–159.

Kodir, A. (2016). Corporate social responsibility (CSR), Ideologi dan keberpihakan di Indonesia : Telaah teori kritis madzhab frankfurt corporate social responsibility, ideology and aligment in Indonesia : Study of Frankfurt School Critical Theory. Jurnal Sosiologi Pendidikan Humanis, 1(2), 149–160.

Leydesdorff, L., & Milojevic, S. (2015). Scientometrics. In M. Lynch (Ed.), International encyclopedia of social and behavioral sciences (2nd Editio, pp. 322–327). Oxford: Elsevier.

MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5), 342–349. https://doi.org/10.1002/(SICI)1097-4571(198909)40:5<342::AID-ASI7>3.0.CO;2-U

Magnis-Suseno, F. (1992). Filsafat sebagai ilmu kritis. Yogyakarta: Kanisius.

Maricic, S., Spaventi, J., Pavicic, L., & Pifat-mrzljak, G. (1998). Citation context versus the frequency counts of citation history. Journal of American Society for Information Science, 49(6), 530–540. https://doi.org/https://doi.org/10.1002/(SICI)1097-4571(19980501)49:6<530::AID-ASI5>3.0.CO;2-8

Merton, R. (1942). Science and technology in a democratic order. Journal of Legal and Political Sociology, (1), 115–126.

Molas-Gallart, J., & Ràfols, I. (2018). Why bibliometric indicators break down: Unstable parameters, incorrect models and irrelevant properties. BiD, 40(40). https://doi.org/10.1344/BiD2018.40.23

Ronda, D. (2019). Kartelisasi sitasi ditinjau dari perspektif Kristen. Integritas: Journal Teologi, 1(1), 29–37.

Sengupta, I. (1992). Bibliometrics, informetrics, scientometrics and librametrics: An overview. Libri: International Library Review, 42(2), 75–98. https://doi.org/https://doi.org/10.1515/libr.1992.42.2.75

Shahid, A., Afzal, M. T., & Qadir, M. A. (2015). Lessons learned: The complexity of accurate identification of in-text citations. International Arab Journal of Information Technology, 12(5), 481–488.

Sinay, L., Sinay, C., Carter, B., & Martins, A. (2019). Reflections about Garfield’s Algorithm. RAUSP Management Journa, 2531–0488. https://doi.org/10.1108/RAUSP-05-2019-0079

Strydom, P. (2011). Contemporary critical theory and methodology. New York: Routledge Taylor & Francis Group.

Tennant, J. (2018). Elsevier are corrupting open science in Europe. Retrieved November 12, 2019, from The Guardian website: https://www.theguardian.com/science/political-science/2018/jun/29/elsevier-are-corrupting-open-science-in-europe

Vosloo, J. (2011). Chapter 5: Research design and methodology.

Wallin, J. A. (2005). Bibliometric methods: Pitfalls and possibilities. Basic and Clinical Pharmacology and Toxicology, 97(5), 261–275. https://doi.org/10.1111/j.1742-7843.2005.pto_139.x

White, H. D. (2004). Citation analysis and discourse analysis revisited. Applied Linguistics, 25(1), 89-116+132. https://doi.org/10.1093/applin/25.1.89

Wilkinson, J. (2018). Bibliometrics and research evaluation: Uses and abuses,. Journal of Web Librarianship, 12(1), 80–80. https://doi.org/10.1080/19322909.2018.1408448

Wisri, W., & Mughni, A. (2016). Paradigma dasar fenomenologis, hermeneutika dan teori kritis. Jurnal Lisan Al-Hal, 10(1), 5–21.

Wuryanta, E. (2018). Teori kritis dan varian paradigmatis dalam Ilmu Komunikasi. https://doi.org/https://doi.org/10.31227/osf.io/wk4p9

Wyatt, S., Milojević, S., Park, H. W., & Leydesdorff, L. (2015). Quantitative and qualitative STS : The intellectual and practical contributions of scientometrics. Retrieved from SSRN website: http://ssrn.com/abstract=2588336

Yousif, A., Niu, Z., Chambua, J., & Khan, Z. Y. (2019). Multi-task learning model based on recurrent convolutional neural networks for citation sentiment and purpose classification. Neurocomputing, 335, 195–205. https://doi.org/10.1016/j.neucom.2019.01.021

Yousif, A., Niu, Z., Tarus, J. K., & Ahmad, A. (2017). A survey on sentiment analysis of scientific citations. Artificial Intelligence Review, 1–34. https://doi.org/10.1007/s10462-017-9597-8

Žukauskas, P., Vveinhardt, J., & Andriukaitienė, R. (2018). Philosophy and paradigm of scientific research. In Management Culture and Corporate Social Responsibility (pp. 111–139). https://doi.org/http://dx.doi.org/10.5772/intechopen.70628

How to Cite
Yaniasih, Y. (2020). Teori kritis terhadap analisis sitasi untuk kajian kuantitatif sains dan evaluasi kinerja riset. Berkala Ilmu Perpustakaan Dan Informasi, 16(1), 127-141. https://doi.org/10.22146/bip.v16i1.72