19-4 “Success by Accountability and Assessment” Resource Guide

May 15th, 2008 | By | Category: 19-4: Success by Accountability and Assessment, Online resource guides, Resource Guides, Web Exclusive
By Anne Marie Karlberg, Ph.D.

Introduction

Everyone involved in higher education – whether at tribal colleges or at other colleges and universities – must deal with assessment in one form or another. Assessment is a systematic process of gathering, interpreting, and using information regarding student performance and perceptions for the purposes of improving student learning and accomplishing the college’s mission. The unique mission of tribal colleges — with their focus on the self-determination of Native peoples — provides them with substantial motivation to succeed in their assessment efforts.

With the new emphasis on learning outcomes, tribal colleges have an opportunity to re-define their own measures of success and, therefore, their own curricular and pedagogical values and approaches in more culturally appropriate ways. By deriving from the mission statement their own standards to measure success, tribal colleges can view assessment programs as a means of pursuing their mission, building local capacity, and advancing processes of self-empowerment, self-determination, and decolonization among Native peoples.

The following list of assessment resources is intended to assist those who wish to further explore this important aspect of education. There are also several articles about assessment in this issue of Tribal College Journal (Vol. 19, No. 4).

Dissertation

Karlberg, A. M. (2007). Assessment in a tribal college context: A case study of Northwest Indian College. Unpublished doctoral dissertation, University of British Columbia, Vancouver. http://ww2.nwic.edu/faculty/assessment/assessment.htm (posted under the link “Survey Results and Reports”).

This study was designed to advance theoretical and applied knowledge in the field of assessment within tribal colleges. Karlberg uses a case study methodology to examine a specific assessment program that is being developed at Northwest Indian College (NWIC).

In this study, Karlberg provides responses to three research questions: (1) What criteria are best used to evaluate an assessment program in a tribal college context? (2) Which elements of the NWIC assessment program are most and least successful according to the evaluative criteria established in Research Question 1? and (3) What preconditions and other contextual factors contribute to the relative success or failure of different elements of the NWIC assessment program?

Karlberg reviews the history of Native Americans in higher education, provides an overview and critique of the emerging assessment movement, and discusses the state of assessment within the tribal college system. This study demonstrates that assessment can be done in a tribal college context in a culturally respectful and meaningful way and provides insights into how this can be approached.

Books

Angelo, T., & Cross, K. P. (1993). Classroom assessment techniques (2nd ed.). San Francisco: Jossey-Bass.

Classroom assessment techniques are direct methods for assessing student learning that are not graded. They are short and frequent, and they provide immediate in-class feedback from students about which concepts are clear and which are not. These formative assessment techniques provide instructors with the opportunity to adjust courses mid-stream (or even mid-class) and to build trust and relationships with their students. Scholars are convinced that employing these strategies improves student retention and engagement and provides instructors with feedback as to what students are or are not learning.

This handbook was written for college teachers regardless of their prior training in pedagogy, assessment, or education. This book provides an introduction to classroom assessment, introduces the basic tools of classroom assessment, describes how faculty can plan and carry out classroom assessment projects, provides examples of classroom assessment case studies, and presents 50 Classroom Assessment Techniques.

Banta, T. (Ed.). (2004). Community college assessment. San Francisco: Jossey-Bass.

Abstract (from www.josseybass.com/WileyCDA/WileyTitle/productCd-0787972878.html): Good measures of student learning are scarce, and all institutions are struggling with the challenge of effectively and reliably assessing student learning. Nowhere is this challenge felt more strongly, however, than at the community college, where students enter with diverse education goals and are more likely to transfer, stop out, or even drop out.

This makes the need more critical than ever for assessment methods of demonstrated value in the community college setting. This booklet is designed to address this need, presenting some of the strongest illustrations of good practice that have appeared in the pages of the newsletter, Assessment Update.

Articles address issues of concern to community college faculty and administrators such as evaluating transfer success, students’ perceptions of student engagement, assessing learning communities, assessing employer needs, and the role of corporate partnerships in assessment. Drawing on both faculty-created and standard measures (such as the Community College Student Experiences Questionnaire and the Community College Survey of Student Engagement), the authors explore the effectiveness of various approaches and how they can be used to make curricular changes that can lead to improved student-learning outcomes.

Boyer, P. (2003). Building tribal communities: Defining the mission and measuring the outcomes of tribal colleges. In M. Benham & W. Stein (Eds.), The renaissance of American Indian higher education: Capturing the dream (pp. 136-147). Mahwah, NJ: Lawrence Erlbaum Associates.

The section on Recommendations argues for the importance of assessment in tribal colleges and proposes that assessment can help colleges fulfill their mission statements and become models of excellence in higher education. Ultimately, assessment makes tribal colleges more accountable to students, the tribe, and their cultural heritage. However, it is necessary to take this discussion to the next step by systematically investigating how tribal colleges are responding to the assessment movement. Turtle Mountain Community College and United Tribes Technical College are cited here. Their work deserves more careful scrutiny, and creative approaches at other tribal colleges deserve to be spotlighted. Most critically, tribal colleges need to develop and share alternative approaches to assessment, both within and outside the classroom.

Maki, P. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus Publishing.

Abstract (from www.styluspub.com/Books/BookDetail.aspx?productID=85566): This book offers colleges and universities a framework and tools to design an effective and collaborative assessment process appropriate for their culture and institution. Maki starts with a definition of assessment as a process that enables us to determine the fit between what we expect our students to understand and be able to do and what they actually demonstrate at points along their educational careers. She then presents a framework that will assist all stakeholders in the institution to develop and sustain assessment of student learning as an integral and systematic core institutional process. The framework is accompanied by extensive examples of processes, strategies, and illustrative campus practices, as well as key resources, guides, worksheets, and exercises.

Mentkowski, M., Rogers, G., Doherty, A., & Loacker, G. (2000). Learning that lasts: Integrating learning, development, and performance in college and beyond. San Francisco: Jossey-Bass.

Abstract (from www.alverno.edu/for_educators/publications.html): This book explores how learners transform themselves and how educators foster skills for learning, leading, teamwork, and adapting with integrity in college and beyond. The authors’ theory includes practical strategies for enabling a wide range of students to cultivate integrative and expansive capabilities across a lifetime. The theory draws on two decades of longitudinal studies of student learning in the Alverno College (Milwaukee) curriculum, leading educational theories, and experience in institutional consortia. The authors illustrate how faculty and academic staff forge effective curricula, design innovative programs, implement key institutional goals, and renegotiate the college culture.

Nichols, J. (2002). A practitioner’s handbook for institutional effectiveness and student outcomes assessment implementation (3rd ed.). New York: Agathon Press.

Abstract (from www.csmd.edu/library/outcomes.html): This handbook is for educators who are setting up programs for institutional effectiveness and outcomes assessment. Its focus is on the how of institutional effectiveness or outcomes assessment rather than the why. The ideas outlined in this book, with contributions from many different professionals in the field, lead to what they call a “long term implementation and not a ‘quick fix.’”

Palomba, C., & Banta, T. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass.

Abstract (from www.josseybass.com/WileyCDA/WileyTitle/productCd-0787941808.html): This step-by-step guide provides the practices for developing assessment programs on college and university campuses. Assessment Essentials outlines the assessment process from the first to the last step and is filled with illustrative examples of assessment on today’s academic campuses. It is especially useful for faculty members and others who may be new to the assessment process. Palomba and Banta describe effective assessment programs and offer a thorough review of the most up-to-date practices in the field. Each chapter of the book addresses a specific aspect of assessment and is designed to walk users through various steps of the assessment process.

Walvoord, B., & Anderson, V. (1998). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.

Abstract (from www.josseybass.com/WileyCDA/WileyTitle/productCd-0787940305.html): Effective Grading is written for the faculty member who believes the grading process is a valuable measure of student learning. This hands-on guide for evaluating student work offers an in-depth examination of the linkage between teaching and grading. It uses grades not as isolated artifacts but as part of a process that, when integrated with course objectives, provides rich information about student learning. The authors reveal how the grading process can also be used for broader assessment objectives, such as curriculum and institutional assessment.

As practical as it is informative, Effective Grading contains a wealth of special materials, including the American Association for Higher Education’s Principles of Good Practice for Assessing Student Learning, types of assignments and tests, and a plan for a faculty workshop on grading and assessment. In addition, the book provides background to the principles of the grading process as well as a wealth of illustrative examples, offering faculty both a sound basis in assessment theory and the practical tools they need to put it to work.

Walvoord, B. (2004). Assessment clear and simple: A practical guide for institutions, departments, and general education. San Francisco: Jossey-Bass.

Abstract (from www.josseybass.com/WileyCDA/WileyTitle/productCd-0787973114.html): Assessment Clear and Simple is Assessment 101 in a book – a concise and step-by-step guide written for everyone who participates in the assessment process. This practical book helps to make assessment simple, cost-efficient, and useful to the institution, while at the same time meeting the requirements of accreditation agencies, legislatures, review boards, and others. It explores a variety of topics. It shows how to build on an existing assessment process, use classroom work and grading process, get faculty and department on board, assess hard to define goals such as moral and civic development, develop workable learning goals, tailor assessment to its purposes, select sensible assessment measures, make criteria explicit, use assessment to improve learning, establish effective oversight without an assessment bureaucracy, write an assessment report, and interpret the institution’s culture to external audiences.

Journal Articles

Kirkness, V., & Barnhardt, R. (1991). First Nations and higher education: The four Rs – Respect, relevance, reciprocity, responsibility. Journal of American Indian Education, 30(3), 1-15. Retrieved February 19, 2008, from http://jaie.asu.edu/v30/V30S3fir.htm

Abstract: American Indian/First Nations/Native people have been historically under-represented in the ranks of college and university graduates in Canada and the United States. From an institutional perspective, the problem typically has been defined in terms of low achievement, high attrition, poor retention, weak persistence, etc., thus placing the onus for adjustment on the student. From the perspective of the Indian student, however, the higher educational system is the problem. It should respect them for who they are, become relevant to their view of the world, offer reciprocity in their relationships with others, and help them exercise responsibility over their own lives. This paper examines the implications of these differences in perspective and identifies initiatives that are transforming the landscape of higher education for First Nations/American Indian people in both Canada and the United States.

Nichols, R., & LaFrance, J. (2006). Indigenous evaluation: Respecting and empowering Indigenous knowledge. Tribal College Journal, 18(2), 32-35. Retrieved February 19, 2008, from http://tribalcollegejournal.org/themag/backissues/winter2006/winter2006rg.htm

This resource guide provides a list of essential books, articles, web sites, and other resources about Indigenous evaluation.

Ortiz, A. M., & Boyer, P. (2003). Student assessment in tribal colleges. New Directions for Institutional Research, 118(Summer 2003), 41-49. Retrieved February 21, 2008 at www3.interscience.wiley.com/cgi-bin/abstract/104541802/ABSTRACT?CRETRY=1&SRETRY=0.

Abstract: The unique culture and traditions of Native American communities affect student outcomes and attendance patterns. The interpretation of assessment and accountability mechanisms needs to account for these contextual issues.

Organizations / Online Resources

Alaska Native Knowledge Network (www.ankn.uaf.edu/)

This website provides resources for compiling and exchanging information related to Alaska Native knowledge systems and ways of knowing. The curriculum focus should be shifted from teaching/learning about cultural heritage as another subject to teaching/learning through the local culture as a foundation for all education. All forms of knowledge, ways of knowing, and world views should be recognized as equally valid, adaptable and complementary to one another in mutually beneficial ways. The Assembly of Alaska Native Educators articulated a set of culturally responsive standards for students, educators, schools, curriculum, and community, which may serve as a model for Indigenous colleges. The document “Alaska standards for culturally responsive schools: Cultural standards for students, educators, schools, curriculum, and community” was published in 1998 and can be accessed at www.ankn.uaf.edu/publications/culturalstandards.pdf. Other cultural standards and guidelines are available at www.ankn.uaf.edu/publications/#standards.

Alverno College: Student Assessment-as-Learning (http://depts.alverno.edu/saal/index.html)

In the United States, a few colleges have taken the lead in engaging students in the assessment process itself — as a means of promoting learning. Alverno College, a small Catholic liberal arts college for women in Milwaukee, WI, is one of these. Alverno College has been a leader in assessment for the last 30 years and has pioneered the concept of assessment as learning. Faculty and staff at Alverno College believe that students should learn something through every assessment process and, if assessment is done well, the result should be learning that lasts.

Thus, assessment at Alverno College is part of the learning process, not just the evaluation process. The success of this approach has been impressive. Serving a predominantly low income, minority population, Alverno College (both its faculty and students) has received numerous awards and widespread recognition for its achievements. This website has links to information about inspiring assessment workshops that Alverno offers, publications, its diagnostic digital portfolio, assessment essentials, self-assessment, student responses, and other resources.

Astin, A, Banta, T., El-Khawas, E., Ewell, P., Hutchings, P., et al. (1992). Nine principles of good practice for assessing student learning. Washington, DC: American Association for Higher Education. Retrieved on February 19, 2008 at www.higher-ed.org/resources/assessment-AAHE.htm.

This document elaborates on these nine principles.

Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. American Association for Higher Education Bulletin, 39(7), 14. Retrieved February 19, 2008 at http://websvr.smith.edu/deanoffaculty/Chickering%20and%20Gamson.pdf

In this short article, Chickering and Gamson elaborate on the seven principles.

Community College Survey of Student Engagement (http://www.ccsse.org/)

One innovative approach to assessment in two-year colleges is the use of student engagement surveys. The Community College Survey of Student Engagement is a standardized national survey that attempts to assess student motivation and involvement in their learning process. This survey solicits students’ perceptions of their level of engagement in various aspects of college life, such as collaborative learning, student effort, academic challenge, student-faculty interaction, and support for learners.

This information is valuable because research indicates that students who are more engaged in educationally purposeful activities like these are more likely to succeed. Colleges can use the results from this respected national survey to initiate meaningful conversations about increasing retention rates and improving student learning, as well as improving the quality of the overall college experience.

Higher Education (http://www.higher-ed.org/resources/Assessment.htm), NC State University (www2.acs.ncsu.edu/UPA/assmt/resource.htm)

These websites offer many assessment-related resources from various other websites, including assessment web resources, college and university assessment web sites, assessment publications, and assessment electronic discussion lists.

Northwest Indian College (NWIC) Assessment (www.nwic.edu/faculty/assessment/assessment.htm)

This website provides access to various teaching, learning, and assessment materials, reports, and data. The NWIC assessment website is a working website for NWIC faculty who are developing course, program, and college outcomes. It provides links to NWIC’s course evaluation website, graduate survey, and instructor peer review forms as well as to many reports, blank surveys, and survey results. Finally, it has assessment educational presentations posted on it.

Systemic Research. (2006). AIMS: American Indian Measures of Success. Alexandria, VA: American Indian Higher Education Consortium. (www.systemic.com/aims/)

The American Indian Higher Education Consortium (AIHEC) created the American Indian Measures for Success (AIMS) project with two goals: to define relevant quantitative and qualitative indicator data of American Indian student success (as determined by the tribal colleges) and to develop and implement a strategy for collecting, analyzing, and presenting annually the success indicator data using electronic information management tools. Through capacity building in data collection and analysis at tribal colleges, this project is expected to be the foundation for systemic reform that significantly increases — and for the first time, accurately measures — American Indian success in higher education. The first fact book (October 2006) can be found at the AIHEC website, www.aihec.org/documents/PDFS/AIMS/AIHEC_AIMS_2005FactBook.pdf

World Indigenous Higher Education Consortium (www.win-hec.org)

The World Indigenous Nations Higher Education Consortium (WINHEC) formed in 2002. It is “committed to building partnerships that restore and retain Indigenous spirituality, cultures and languages, homelands, social systems, economic systems and self determination.” It is committed to self-determination through control of higher education. WINHEC provides an international forum and support for Indigenous peoples to pursue common goals through higher education. The WINHEC accreditation handbook (2nd edition) can be accessed at http://www.ankn.uaf.edu/IEW/WINHEC/WinhecHandDec2004.pdf.

Anne Marie Karlberg served as the director of assessment at Northwest Indian College from 2002 through 2008. She has a Ph.D. in higher educational studies from the University of British Columbia, with a specialization in assessment at tribal colleges.

Leave a Comment

You must be logged in to post a comment.