Combining course- and program-level outcomes assessments through embedded performance assessments at key courses: A proposal based on the experience from a Japanese dental education program

Kayo Matsushita, Kazuhiro Ono, Yugo Saito

Abstract


This paper addresses how to combine the course- and program-level assessments and presents a new method illustrated by a case of dental education program in Japan. Performance assessments are considered effective for evaluating knowledge integration and higher-order skills, while placing a burden on faculty, hence their feasibility as the program-level assessment is regarded lower than standardized tests or questionnaire surveys. We have developed several performance assessments at the course level, such as Modified Triple Jump for the PBL course. Based on this experience, we propose Pivotal Embedded Performance Assessment (PEPA) as a method for combining assessment at the course and program levels. The method limits the range of performance assessment to key courses directly linked to program goals and placed at the critical juncture points of curriculum, while entrusting the assessment of other courses to expert judgment of individual teachers. PEPA consists of the following procedures: systematization of curriculum and selection of key courses; design and implementation of performance assessments by a faculty team; setting passing criteria with incorporating the function of formative assessment; certifying the completion of the degree program. PEPA thus enables maintaining assessment feasibility and compatibility with a credit system, while ensuring assessment validity and reliability.

Received: 27 September 2018
Accepted: 13 November 2018
Published online: 29 November 2018


Keywords


performance assessment; embedded assessment; program-level assessment; curriculum; PBL (problem-based learning); dental education

Full Text:

[PDF] [HTML]

References


Akiba, Nami, Masako Nagasawa, Kazuhiro Ono, Takeyasu Maeda, and Katsumi Uoshima. “An Introduction to the Undergraduate Comprehensive Model Practice Course at the Faculty of Dentistry, Niigata University.” The Journal of Japanese Dental Education Association 33 (2017): 106-14. [In Japanese.]

Alverno College Faculty. Student Assessment-as-Learning at Alverno College. Milwaukee: Alverno College Institute, 1994.

Banta, Trudy W., and Catherine A. Palomba. Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. 2nd ed. San Francisco: Jossey-Bass, 2015.

Barrows, Howard S. “The Essentials of Problem-Based Learning.” Journal of Dental Education 62 (1998): 630-33.

Biggs, John, and Catherine Tang. Teaching for Quality Learning at University. 4th ed. Berkshire: The Society for Research into Higher Education & Open University Press, 2011. Kindle.

Blake, Jennifer M., Geoffrey R. Norman, and E. Kinsey M. Smith. “Report Card from McMaster: Student Evaluation at a Problem-Based Medical School.” The Lancet 345 (1995): 899-902.

Cummings, Rhoda, Cleborne D. Maddux, and Aaron Richmond. “Curriculum- Embedded Performance Assessment in Higher Education: Maximum Efficiency and Minimum Disruption.” Assessment & Evaluation in Higher Education 33, no. 6 (2008): 599-605.

Earl, Lorna M. Assessment as Learning: Using Classroom Assessment to Maximize Student Learning. Thousand Oaks: Corwin Press, 2003.

Ewell, Peter, Karen Paulson, and Jillian Kinzie. Down and In: Assessment Practices at the Program Level. Champaign: National Institute for Learning Outcomes Assessment, 2014. http://www.learningoutcomeassessment.org/documents/NILOAsurveyreport2011%20-%20Down%20and%20In%2010-20.pdf.

Fujii, Noritaka, Syoji Takenaka, Koichi Tabeta, Naoko Sato, Nami Akiba, Yohei Oda, Yuji Katsumi, Kazuhiro Ono, and Takeyasu Maeda. “Competency Assessments for Undergraduate Students in Clinical Clerkships at the Faculty of Dentistry, Niigata University.” The Journal of Japanese Dental Education Association 33 (2017): 4-11. [In Japanese.]

Kuh, George D., Natasha Jankowski, Stanley O. Ikenberry, and Jillian Kinzie. Knowing What Students Know and Can Do: The Current State of Student Learning Outcomes Assessment in U.S. Colleges and Universities. Champaign: National Institute for Learning Outcomes Assessment, 2014. http://www.learningoutcomeassessment.org/documents/2013%20Abridged%20Survey%20Report%20Final.pdf.

Lane, Suzanne. “Performance Assessment: The State of the Art.” In Beyond the Bubble Test: How Performance Assessments Support 21st Century Learning, edited by Linda Darling-Hammond and Frank Adamson, chap. 5. San Francisco: Jossey-Bass, 2014. Kindle.

Matsushita, Kayo. “Making Learning Outcomes Visible.” Japanese Journal of Higher Education Research 20 (2017): 93-112. [In Japanese.]

Matsushita, Kayo, Kazuhiro Ono, and Yusuke Takahashi. “Development of a Rubric for Writing Assessment and Examination of Its Reliability.” Journal of the Liberal and General Education Society of Japan 35, no. 1 (2013): 107-15. [In Japanese.]

Middle States Commission on Higher Education. Student Learning Assessment: Options and Resources. 2nd ed. 2007. https://www.msche.org/publications/SLA_Book_0808080728085320.pdf.

Miller, George E. “The Assessment of Clinical Skills/Competence/Performance.” Academic Medicine 65, no. 9 (September Supplement 1990): S63-S67.

Ministry of Education, Culture, Sports, Science and Technology. “Japanese University Reforms Including Those in Educational Contents.” November, 2017. http://www.mext.go.jp/a_menu/koutou/daigaku/04052801/__icsFiles/afieldfile/2017/12/13/1398426_1.pdf. [In Japanese.]

Model Core Curriculum Revision Coordination Committee and Model Core Curriculum Revision Specialist Research Committee. Model Core Curriculum for Dental Education: AY 2016 Revision. 2017. [In Japanese.] http://www.mext.go.jp/component/b_menu/shingi/toushin/__icsFiles/afieldfile/2017/12/26/1383961_02_3.pdf.

Mtshali, Ntombifikile G., and Lyn Middleton. “The Triple Jump Assessment: Aligning Learning and Assessment.” In New Approaches to Problem-Based Learning: Revitalising Your Practice in Higher Education, edited by Terry Barrett and Sarah Moore, 187-200. New York: Routledge, 2011.

Newman, Mark J. “Problem Based Learning: An Introduction and Overview of the Key Features of the Approach.” Journal of Veterinary Medical Education 32 (2005): 12-20.

Oda, Yohei, Kazuhiro Ono, Noritaka Fujii, Tadaharu Kobayashi, and Takeyasu Maeda. “Development and Use of a Web-Based E-Portfolio for Dental Clinical Training.” The Journal of Japanese Dental Education Association 33 (2017): 65-73. [In Japanese.]

Ono, Kazuhiro, Kayo Matsushita, and Yugo Saito. “Prospects for Direct Assessment of Problem Solving Competence: Development of Modified Triple Jump in Problem-Based Learning.” Journal of the Liberal and General Education Society of Japan 36, no. 1 (2014): 123-32. [In Japanese.]

Ono, Kazuhiro, and Kayo Matsushita. “Assessment of Writing in First-Year Education.” In Assessment of Active Learning, edited by Kayo Matsushita and Terumasa Ishii, 26-43. Tokyo: Toshindo, 2016. [In Japanese.]

_____. “PBL Tutorial Linking Classroom to Practice: Focusing on Assessment as Learning.” In Deep Active Learning: Toward Greater Depth in University Education, edited by Kayo Matsushita, 183-206. Singapore: Springer, 2017.

Pike, Gary R. “Limitations of Using Students’ Self-Reports of Academic Development as Proxies for Traditional Achievement Measures.” Research in Higher Education 37, no. 1 (1996): 89-114.

Rhodes, Terrel. Assessing Outcomes and Improving Achievement: Tips and Tools for Using the Rubrics. Washington D.C.: Association of American Colleges and Universities, 2009.

Rhodes, Terrel, and Ashley Finley. Using the VALUE Rubrics for Improvement of Learning and Authentic Assessment. Washington D.C.: Association of American Colleges and Universities, 2013.

Richman, W. Allen, and Laura Ariovich. All-in-One: Combining Grading, Course, Program, and General Education Outcomes Assessment. Champaign: National Institute for Learning Outcomes Assessment, 2013. http://learningoutcomesassessment.org/documents/Occasional%20Paper%2019%20FINAL.pdf.

Rohlin, Madeleine, Kerstin Petersson, and Gunnel Svensäter. “The Malmö Model: A Problem-Based Learning Curriculum in Undergraduate Dental Education.” European Journal of Dental Education 2 (1998): 103-14.

Saito, Yugo, Kazuhiro Ono, and Kayo Matsushita. “Correlations of Direct Measures Based on Performance Assessment and Indirect Measures Based on Student Self-report.” Japan Journal of Educational Technology 40 (Suppl.) (2016): 157-60. [In Japanese.]

Suskie, Linda. Assessing Student Learning: A Common Sense Guide. 2nd ed. San Francisco: Jossey-Bass, 2009. Kindle.

Winning, Tracy, Elaine Lim, and Grant Townsend. “Student Experiences of Assessment in Two Problem-Based Dental Curricula: Adelaide and Dublin.” Assessment & Evaluation in Higher Education 30, no. 5 (2005): 489-505.




DOI: http://dx.doi.org/10.18543/tjhe-6(1)-2018pp111-142

Refbacks

  • There are currently no refbacks.