The validity of the assessment methods in an undergraduate psychiatry module examination among a sample of fourth year Malaysian medical students

Lai Fong Chan, Hatta Sidi, Zaleha A. Mahady, Nabishah Mohamad, Suriati Mohamed Saini, Muhamad Saiful Bahri Yusoff

Research output: Contribution to journalArticle

Abstract

Background: Assessment of medical students' competencies is fundamental in the education of future doctors. An ideal assessment should play an integral role to provide insight into actual competencies as well as to improve overall performance of students. Objective: This study aims to assess the convergent, discriminant and predictive validity of the clinical, theory and continuous assessment components of an undergraduate Psychiatry Module examination on the overall examination performance of fourth year medical students in the Psychiatry Module rotation. Method: A retrospective study design on archival records of a medical undergraduate assessment program was done. The target population was fourth year undergraduate medical students in Universiti Kebangsaan Malaysia Medical Centre (n = 127) that took the end of semester Psychiatry Module examination in February 2011 (academic year of 2010/2011) after completing an 8-week Psychiatry Module. The Pearson correlation test was used to determine the discriminant and convergent validity of the assessment tools. The linear regression test was used to evaluate the predictive validity of each assessment tool used to assess medical student performance. Results: Pearson's correlation test showed an acceptable level of convergent and discriminant validity for the theory and continuous assessment components. The clinical component was found to be lacking in convergent validity. There were eight test formats that predicted the overall examination performance of the medical students: OBA, EMQ, KFQ, case write-up, log-book, OSCE 1, OSCE 2 and OSCE 10. The KFQ (·= 0.44, p < 0.001) was the most dominant test format assessment tool predicted the overall examination performance followed by EMQ (·= 0.21, p < 0.001), OSCE 1 (·= 0.16, p < 0.01), case write-up (·= 0.15, p < 0.01), log-book (·= 0.15, p < 0.05), OSCE (·= 0.14, p < 0.01), OBA (·= 0.13, p < 0.01) and OSCE 10 (·= 0.13, p < 0.01). The cumulative total OSCE mark (·= 0.50, p < 0.001) was the most influential assessment component. Conclusion: The assessment methods used in the Psychiatry Module examination had an acceptable level of convergent, discriminant and predictive validity. Nevertheless, the clinical test formats should be relooked and reviewed in line with the assessment principles and the utility of assessment methods to improve its validity in future.

Original languageEnglish
Pages (from-to)347-351
Number of pages5
JournalInternational Medical Journal
Volume19
Issue number4
Publication statusPublished - Dec 2012

Fingerprint

Medical Students
Psychiatry
Health Services Needs and Demand
Malaysia
Medical Records
Linear Models
Retrospective Studies
Students
Education

Keywords

  • Assessment
  • Education
  • Malaysia
  • Medical
  • OSCE
  • Teaching/method
  • Undergraduate
  • Validity

ASJC Scopus subject areas

  • Medicine(all)

Cite this

@article{cc90e599aaa640f3ac4075268f5a182b,
title = "The validity of the assessment methods in an undergraduate psychiatry module examination among a sample of fourth year Malaysian medical students",
abstract = "Background: Assessment of medical students' competencies is fundamental in the education of future doctors. An ideal assessment should play an integral role to provide insight into actual competencies as well as to improve overall performance of students. Objective: This study aims to assess the convergent, discriminant and predictive validity of the clinical, theory and continuous assessment components of an undergraduate Psychiatry Module examination on the overall examination performance of fourth year medical students in the Psychiatry Module rotation. Method: A retrospective study design on archival records of a medical undergraduate assessment program was done. The target population was fourth year undergraduate medical students in Universiti Kebangsaan Malaysia Medical Centre (n = 127) that took the end of semester Psychiatry Module examination in February 2011 (academic year of 2010/2011) after completing an 8-week Psychiatry Module. The Pearson correlation test was used to determine the discriminant and convergent validity of the assessment tools. The linear regression test was used to evaluate the predictive validity of each assessment tool used to assess medical student performance. Results: Pearson's correlation test showed an acceptable level of convergent and discriminant validity for the theory and continuous assessment components. The clinical component was found to be lacking in convergent validity. There were eight test formats that predicted the overall examination performance of the medical students: OBA, EMQ, KFQ, case write-up, log-book, OSCE 1, OSCE 2 and OSCE 10. The KFQ (·= 0.44, p < 0.001) was the most dominant test format assessment tool predicted the overall examination performance followed by EMQ (·= 0.21, p < 0.001), OSCE 1 (·= 0.16, p < 0.01), case write-up (·= 0.15, p < 0.01), log-book (·= 0.15, p < 0.05), OSCE (·= 0.14, p < 0.01), OBA (·= 0.13, p < 0.01) and OSCE 10 (·= 0.13, p < 0.01). The cumulative total OSCE mark (·= 0.50, p < 0.001) was the most influential assessment component. Conclusion: The assessment methods used in the Psychiatry Module examination had an acceptable level of convergent, discriminant and predictive validity. Nevertheless, the clinical test formats should be relooked and reviewed in line with the assessment principles and the utility of assessment methods to improve its validity in future.",
keywords = "Assessment, Education, Malaysia, Medical, OSCE, Teaching/method, Undergraduate, Validity",
author = "Chan, {Lai Fong} and Hatta Sidi and Mahady, {Zaleha A.} and Nabishah Mohamad and {Mohamed Saini}, Suriati and Yusoff, {Muhamad Saiful Bahri}",
year = "2012",
month = "12",
language = "English",
volume = "19",
pages = "347--351",
journal = "International Medical Journal",
issn = "1341-2051",
publisher = "Japan International Cultural Exchange Foundation",
number = "4",

}

TY - JOUR

T1 - The validity of the assessment methods in an undergraduate psychiatry module examination among a sample of fourth year Malaysian medical students

AU - Chan, Lai Fong

AU - Sidi, Hatta

AU - Mahady, Zaleha A.

AU - Mohamad, Nabishah

AU - Mohamed Saini, Suriati

AU - Yusoff, Muhamad Saiful Bahri

PY - 2012/12

Y1 - 2012/12

N2 - Background: Assessment of medical students' competencies is fundamental in the education of future doctors. An ideal assessment should play an integral role to provide insight into actual competencies as well as to improve overall performance of students. Objective: This study aims to assess the convergent, discriminant and predictive validity of the clinical, theory and continuous assessment components of an undergraduate Psychiatry Module examination on the overall examination performance of fourth year medical students in the Psychiatry Module rotation. Method: A retrospective study design on archival records of a medical undergraduate assessment program was done. The target population was fourth year undergraduate medical students in Universiti Kebangsaan Malaysia Medical Centre (n = 127) that took the end of semester Psychiatry Module examination in February 2011 (academic year of 2010/2011) after completing an 8-week Psychiatry Module. The Pearson correlation test was used to determine the discriminant and convergent validity of the assessment tools. The linear regression test was used to evaluate the predictive validity of each assessment tool used to assess medical student performance. Results: Pearson's correlation test showed an acceptable level of convergent and discriminant validity for the theory and continuous assessment components. The clinical component was found to be lacking in convergent validity. There were eight test formats that predicted the overall examination performance of the medical students: OBA, EMQ, KFQ, case write-up, log-book, OSCE 1, OSCE 2 and OSCE 10. The KFQ (·= 0.44, p < 0.001) was the most dominant test format assessment tool predicted the overall examination performance followed by EMQ (·= 0.21, p < 0.001), OSCE 1 (·= 0.16, p < 0.01), case write-up (·= 0.15, p < 0.01), log-book (·= 0.15, p < 0.05), OSCE (·= 0.14, p < 0.01), OBA (·= 0.13, p < 0.01) and OSCE 10 (·= 0.13, p < 0.01). The cumulative total OSCE mark (·= 0.50, p < 0.001) was the most influential assessment component. Conclusion: The assessment methods used in the Psychiatry Module examination had an acceptable level of convergent, discriminant and predictive validity. Nevertheless, the clinical test formats should be relooked and reviewed in line with the assessment principles and the utility of assessment methods to improve its validity in future.

AB - Background: Assessment of medical students' competencies is fundamental in the education of future doctors. An ideal assessment should play an integral role to provide insight into actual competencies as well as to improve overall performance of students. Objective: This study aims to assess the convergent, discriminant and predictive validity of the clinical, theory and continuous assessment components of an undergraduate Psychiatry Module examination on the overall examination performance of fourth year medical students in the Psychiatry Module rotation. Method: A retrospective study design on archival records of a medical undergraduate assessment program was done. The target population was fourth year undergraduate medical students in Universiti Kebangsaan Malaysia Medical Centre (n = 127) that took the end of semester Psychiatry Module examination in February 2011 (academic year of 2010/2011) after completing an 8-week Psychiatry Module. The Pearson correlation test was used to determine the discriminant and convergent validity of the assessment tools. The linear regression test was used to evaluate the predictive validity of each assessment tool used to assess medical student performance. Results: Pearson's correlation test showed an acceptable level of convergent and discriminant validity for the theory and continuous assessment components. The clinical component was found to be lacking in convergent validity. There were eight test formats that predicted the overall examination performance of the medical students: OBA, EMQ, KFQ, case write-up, log-book, OSCE 1, OSCE 2 and OSCE 10. The KFQ (·= 0.44, p < 0.001) was the most dominant test format assessment tool predicted the overall examination performance followed by EMQ (·= 0.21, p < 0.001), OSCE 1 (·= 0.16, p < 0.01), case write-up (·= 0.15, p < 0.01), log-book (·= 0.15, p < 0.05), OSCE (·= 0.14, p < 0.01), OBA (·= 0.13, p < 0.01) and OSCE 10 (·= 0.13, p < 0.01). The cumulative total OSCE mark (·= 0.50, p < 0.001) was the most influential assessment component. Conclusion: The assessment methods used in the Psychiatry Module examination had an acceptable level of convergent, discriminant and predictive validity. Nevertheless, the clinical test formats should be relooked and reviewed in line with the assessment principles and the utility of assessment methods to improve its validity in future.

KW - Assessment

KW - Education

KW - Malaysia

KW - Medical

KW - OSCE

KW - Teaching/method

KW - Undergraduate

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=84870260043&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84870260043&partnerID=8YFLogxK

M3 - Article

VL - 19

SP - 347

EP - 351

JO - International Medical Journal

JF - International Medical Journal

SN - 1341-2051

IS - 4

ER -