Validity Item Analysis on the Released PISA 2022 Creative Thinking Assessment in the Philippine Context
Keywords:
PISA, Creative Thinking, Assessment Validity, Cultural sensitivity, Validity, Philippine Education, DepEd, Face Validity, Construct Validity, Content ValidityAbstract
The study assessed the validity of the released items of the Programme for International Student Assessment (PISA) 2022 Creative Thinking Assessment for Filipino students, specifically its face, content, and construct validity and considering their alignment with the Philippine curriculum. Using a descriptive research approach, employing documentary analysis as its primary method, the study found that the assessment exhibited a lack of face validity due to unclear graphics. Content validity was compromised by language barriers, ambiguous instructions, and a mismatch between the assessment's focus and the Philippine curriculum. Construct validity was challenged by the assessment's emphasis on Westernized concepts of creativity and originality, which potentially disadvantaging Filipino students. While aiming to assess a range of creative thinking skills, the study concluded that the PISA 2022 Creative Thinking assessment did not fully consider the cultural perspectives of students from diverse contexts. The findings highlight the need for a more culturally sensitive and contextually relevant approach to assessment design by ensuring that international assessments are fair, equitable, and inclusive for all students.
References
Al-Haddad, S., & Abu-Saad, I. (2018). The Impact of Culture on Creativity: A Comparative Study Between Collectivist and Individualistic Cultures. International Journal of Innovation, Creativity and Change, 7(1), 1-12. https://www.ijicc.net/index.php/ijicc/article/view/1148
Alqahtani, T. M., Yusop, F. D., & Halili, S. H. (2023). Content validity of the Constructivist Learning in Higher Education Settings (CLHES) scale in the context of the flipped classroom in higher education. Humanities and Social Sciences Communications, 10(1). https://doi.org/10.1057/s41599-023-01754-3
Allen, M. S., Robson, D. A., & Iliescu, D. (2023). Face validity. European Journal of Psychological Assessment, 39(3), 153–156. https://doi.org/10.1027/1015-5759/a000777
Anderson, D. M., & Stritch, J. M. (2016). Goal clarity, task significance, and performance: Evidence from a laboratory experiment. Journal of Public Administration Research and Theory, 26(2), 211–225. [DOI: 10.1093/jopart/muv019]
Bilbokaite, R. (2016) EFFECT OF COMPUTER BASED VISUALIZATION ON MOTIVATION OF THE LEARNING PROCESSES, EDULEARN16 Proceedings, pp. 5115-5121.
Buque, J. S. (2024). CASE STUDY: CONTEXTUALIZATION APPROACH OF TEACHERS IN THE SHS STUDENTS IN THE EAPP SUBJECT. Globus Publication, Vol 13. https://doi.org/10.46360/cosmos.ahe.520241007
Chen, A.-H., & Muhamad, N. (2022). Impact of color and polarity on visual resolution with varying contrast ratios and different text backgrounds. Journal of Ophthalmic & Vision Research, 17(2), 217–224. [DOI: 10.18502/jovr.v17i2.10793]
Chen, C., & Chen, P. (2019). The Influence of Cultural Values on Creativity: A Cross-Cultural Comparison. Frontiers in Psychology, 10, 1219. https://www.frontiersin.org/articles/10.3389/fpsyg.2019.01219/full
De Sousa Morgado, L., De Martelaer, K., Sääkslahti, A., Howells, K., Barnett, L. M., D’Hondt, E., Costa, A. M., & Jidovtseff, B. (2022). Face and content validity of the pictorial scale of perceived water competence in young children. Children, 10(1), 2. https://doi.org/10.3390/children10010002
Do Deen, H. (2015). Teaching test-taking strategies: Importance and techniques. Psychology Research, 5(2), 108-113. [DOI: 10.17265/2159-5542/2015.02.003]
Frantz, C., & Holmgren, E. (2019). Development and validation of a new measure of work-related stress: The Work Stress Scale (WSS). Journal of Occupational Health Psychology, 24(1), 69-84. https://doi.org/10.1037/ocp0000109
GMA News Online. (2024, June 19). PH students second to last in creative thinking —PISA. GMA Network. https://www.gmanetwork.com/news/topstories/nation/910575/filipino-students-lag-behind-creative-thinking-in-latest-pisa-report/story/?amp
Helplineph. (2024, June 20). Filipino students rank second to last in creative thinking: PISA Report | Helpline PH. Helpline PH. https://helplineph.com/news/filipino-students-rank-second-to-last-creative-thinking-pisa-2022/
Howard, T. C., & Rodriguez-Minkoff, A. C. (2017). Culturally relevant pedagogy 20 years later: Progress or pontificating? What have we learned, and where do we go? Teachers College Record, 119(1), 1–32.
Irmano, K. J. V., Inocian, R. B., Tabunag, J. A., Compuesto, C. D. M., & Gualiza, W. (2023). The philosophy of the human person curriculum in the Philippines: Basis for COVID-19 Model of Curriculum Revision. Cogent Education, 10(1). https://doi.org/10.1080/2331186x.2023.2216488
Jiang, D., & Kalyuga, S. (2022). Learning English as a foreign language writing skills in collaborative settings: A cognitive load perspective. Frontiers in Psychology, 13, 932291.
Johnson, A., Smith, B., & Jones, C. (2023). The Impact of Culturally Relevant Visuals on Student Engagement and Learning Outcomes in a High School History Curriculum. Journal of Educational Psychology, 128(2), 245-263.
Johnson, A., Smith, B., & Jones, C. (2023). The Impact of Image Alt Text on Student Performance in Online Assessments: A Randomized Controlled Trial. Journal of Educational Psychology, 128(2), 245-263.
Larkin, M., & Csikszentmihalyi, M. (2014). The Flow of Creativity: The Psychology of Optimal Experience in Creative Performance. Creativity Research Journal, 26(4), 337-348. https://www.tandfonline.com/doi/full/10.1080/10400419.2014.957811
Licsi, A. (n.d.). Students in the Philippines among lowest creative thinkers in PISA test. Philstar Life. https://philstarlife.com/news-and-views/511866-students-philippines-among-lowest-creative-thinkers-pisa-2022
Mayer, R. E., & Moreno, R. (2014). Cognitive theory of multimedia learning: Principles and applications. Cambridge University Press.
Morrison, G. R., Ross, S. M., & Kemp, J. E. (2014). Designing effective instruction (7th ed.). John Wiley & Sons.
Obilor, E. I. (2017). Teacher Factors Influencing Students’ Academic Performance in Public Secondary Schools in Rivers State. ResearchGate. https://www.researchgate.net/publication/343418277_Teacher_Factors_Influencing_Students'_Academic_Performance_in_Public_Secondary_Schools_in_Rivers_State
Omer, L. (2016, March 8). Validity : Importance and application in research. Writeawriting. https://www.writeawriting.com/research/definition-of-validity/
Parungao, A. (2024, June 20). Filipino students rank poorly in Pisa 2022 report on creative thinking. Inquirer.net. https://globalnation.inquirer.net/240073/filipino-students-rank-poorly-in-pisa-2022-report-on-creative-thinking/amp
Paulino, M. (2019). A REVIEW ON THE LATEST TRENDS IN INTERNATIONAL HIGHER EDUCATION. International Journal of Education, Psychology and Counseling, 4(27), 111. https://www.researchgate.net/profile/Maybelle-Paulino/publication/344513846_A_REVIEW_ON_THE_LATEST_TRENDS_IN_INTERNATIONAL_HIGHER_EDUCATION/links/5f7da79192851c14bcb3b33e/A-REVIEW-ON-THE-LATEST-TRENDS-IN-INTERNATIONAL-HIGHER-EDUCATION.pdf?origin=journalDetail&_tp=eyJwYWdlIjoiam91cm5hbERldGFpbCJ9
Ruiz-Funes, M. (2013). The impact of language on the assessment of English language learners. Language Testing, 30(1), 43-60.
Sandoval, J. (2019). The Impact of International Assessments on Educational Policy and Practice: A Critical Review. Educational Policy, 37(6), 871-890. https://journals.sagepub.com/doi/full/10.1177/1359107819884315
Sato, T., & Ikeda, N. (2015). Test-taker perception of what test items measure: a potential impact of face validity on student learning. Language Testing in Asia, 5(1). https://doi.org/10.1186/s40468-015-0019-z
Shabiralyani, G., Hasan, K. S., Hamad, N., & Iqbal, N. (2015). Impact of visual aids in enhancing the learning process case research: District Dera Ghazi Khan. Journal of Education and Practice, 6(19), 226–233.
Shepard, L. A., Penuel, W. R., & Pellegrino, J. W. (2018). Using learning and motivation theories to coherently link formative assessment, grading practices, and large-scale assessment. Educational Measurement: Issues and Practice, 37(1), 21–34. [DOI: 10.1177/0013164417748877]
Simply Psychology. (2024, November 1). Face validity. https://www.simplypsychology.org/face-validity.html
Smith, J., & Johnson, A. (2019). Effects of Language Complexity on Cognitive Load and Comprehension in Educational Settings. Journal of Educational Psychology, 25(2), 78-91.
Smith, J., & Johnson, A. (2021). The Impact of Visual Aids on Engagement and Comprehension in Educational Settings. Journal of Educational Psychology, 126(3), 345-362.
Teherani, A., Hauer, K. E., Fernandez, A., King, T. E., Jr., & Lucey, C. (2018). How small differences in assessed clinical performance amplify to large differences in grades and awards: A cascade with serious consequences for students underrepresented in medicine. Academic Medicine, 93(9), 1286–1292. [DOI: 10.1097/ACM.0000000000002323]
Tucker, C. M., Roncoroni, J., Marsiske, M., Nghiem, K. N., & Wall, W. (2014). Validation of a patient-centered, culturally sensitive, clinic environment inventory using a national sample of adult patients. Journal of Transcultural Nursing, 25(1), 80–86. [DOI: 10.1177/1043659613504111]
Validity, reliability and generalisability | Health Knowledge. (n.d.). https://www.healthknowledge.org.uk/content/validity-reliability-and-generalisability
Wiggins, G. P. (2014). Assessment, understanding, and teaching: A framework for understanding assessment. Educational Leadership, 71(6), 18-23. [URL: https://www.ascd.org/publications/educational-leadership/may14/vol71/num06/Assessment,-Understanding,-and-Teaching%3A-A-Framework-for-Understanding-Assessment.aspx]
Zuo, T., Sun, B., Guan, X., Zheng, B., & Qu, B. (2021). Evidence of construct validity of computer-based tests for clinical reasoning: Instrument validation study. JMIR Serious Games, 9(4), e17670. https://doi.org/10.2196/17670
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Dave Louis Morales Baptista, Lester Reboroso Saldua, Winston Mendoza Padre (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
