Volume 18 Winter 2006 Issue No. 2

In This Issue:
Traditional Wisdom Our Strength

subscribe to Tribal College Journal

RESOURCES

Indigenous Evaluation: Respecting and Empowering Indigenous Knowledge

by Richard Nichols (Santa Clara Pueblo), B.S., and Joan LaFrance (Turtle Mt. Chippewa), Ed.D., MPA

BASKET WEAVING DEMONSTRATION Photo courtesy of Ed LeBlanc, Archivist, Institute of American Indian Arts.

INTRODUCTION

Though based on Western research models, it has been said that evaluation is a practical craft; evaluators engage in the craft to contribute to program quality. Given this nod toward practicality, evaluators are free to explore cultural ways of knowing different from those traditionally taught in the Western epistemological tradition.

This is especially true when such exploration contributes to the usefulness and validity of evaluation within the program operations context. Just as action research models have evolved, evaluation practice has become more collaborative and responsive to evaluation stakeholders, including American Indian tribes, schools, and communities. Indeed, evaluation capacity building has become an embedded principle in such theoretical models as empowerment evaluation.

The American Indian Higher Education Consortium (AIHEC)’s Indigenous Evaluation Framing project is described elsewhere in this issue. Funded by the National Science Foundation, the project informs and creates evaluation designs that ensure validity and reliability based on indigenous ways of knowing and core values common to most, if not all, Indian communities. The following evaluation resources provide an overview of indigenous evaluation as an emergent field.

BOOKS AND ARTICLES

LaFrance, J. (Summer 2004). Culturally competent evaluation in Indian Country. New Directions for Evaluation, No. 102, 39-50. Special issue: In search of cultural competence in evaluation: Toward principles and practices. Jossey-Bass and the American Evaluation Association.

Conducting culturally competent evaluation in Indian Country requires an understanding of the rich diversity of tribal peoples and recognition of Indian self-determination and tribal sovereignty, according to Dr. Joan LaFrance’s chapter in this special issue. If an evaluation can be embedded within an indigenous framework, it is more responsive to tribal ethics and values. She says appropriate tribal protocols should be utilized for evaluation and argues that an indigenous orientation to evaluation has specific implications for the use of appropriate methodological approaches, for partnerships between the evaluator and the program, and for reciprocity.

LaFrance, J. & Nichols, R. (In Press). Summary report: Stories from the focus groups on an indigenous framing for evaluation. Alexandria, VA: American Indian Higher Education Consortium (AIHEC).

Developed with support from the National Science Foundation (NSF), this document highlights regional focus groups conducted as phase one of the Indigenous Evaluation Framing project. Three focus groups were conducted in the Southwest, Northwest, and Central Plains region inviting American Indian cultural traditionalists, educators, and evaluators to consider how to assess program merit or worth from a traditionalist perspective.

Based on a concept of evaluation as a joint journey between the evaluator and evaluation stakeholders to “create knowledge about a program,” the focus group participants’ information is being used to develop a training curriculum on indigenous evaluation. Information was collected about tribal experiences with evaluation and cultural values regarding knowledge, judgment, and assessment of what is valued in Indian education. The stories also provide advice on practice and methodological implications for evaluation in American Indian communities. The Summary Report will be available at the AIHEC website (www.aihec.org) in Winter 2007.

The National Science Foundation, Directorate for Education and Human Resources, Division of Research, Evaluation, and Communications. The Cultural Context of Educational Evaluation: The Role of Minority Evaluation Professionals. Workshop Proceedings, June 1-2, 2000.

Since the 1990s, the National Science Foundation (NSF) has been concerned with increasing the capacity of evaluation practitioners in order to provide high quality evaluation services more responsive to minority concerns and community needs. This 2000 workshop was held to discuss issues related to increasing the supply of minority evaluators for mathematics and science programs and projects. Since then, the senior staff of NSF’s Division of Research, Evaluation, and Communications has worked with the American Evaluation Association (AEA) to promote more minority participation in AEA’s annual meetings and in its publications. NSF is in the process of developing theoretical models for training and capacity building in evaluation -- models that will incorporate contextual factors and their influence on the process of evaluation.

The National Science Foundation, Directorate for Education and Human Resources, Division of Research, Evaluation, and Communications. The Cultural Context of Educational Evaluation: A Native American Perspective. Workshop Proceedings, April 25-26, 2002.

The National Science Foundation sponsored this 2-day workshop to discuss issues of culturally responsive educational evaluation pertinent to Native Americans. The goal of this workshop was to offer direction for future planning of NSF evaluations and research activities and to focus on capacity building within the field of educational evaluation. Participants included evaluation and education experts from a variety of tribes and experience in national organizations, federal agencies, and schools and higher education institutions across the nation. This workshop was structured around three major themes:

• evaluation issues relating to the academic achievement of Native American students;
• education/training opportunities for Native American evaluators; and
• developing, maintaining, and expanding a network of Native American evaluators.

The discussions highlighted the history of research exploitation in Indian Country, which raises issues for evaluation. Evaluation is different from research in that it should respond to programs and not the Western notion of empiricism, which is the goal of research.

Participants noted that evaluation in Indian Country should be attentive to community ownership and participation, as well as traditional protocols and codes of ethics. It was noted that creating a cadre of Native American evaluators was only in its beginning stages and that there was a need for continued support from federal agencies such as NSF to keep the momentum going.

Evaluation should not be an “underfunded afterthought,” doing only summative evaluation IS passé. Evaluation should be formative, positive, developmental, and ongoing. Furthermore, evaluations should involve not only administrators but also “frontline educators” and community members in the interpretation of data and their implications.

This resource, based on the input of many recognized experts in Indian education, provides an important introduction to the evolution of evaluation. New theoretical models (such as participatory, collaborative, and empowerment approaches) are being developed and implemented in Indian Country. The document can be downloaded at www.nsf.gov/pubs/2003/nsf03032/start.htm.

Nelson-Barber, S., LaFrance, J., Trumbull, E., & Aburto, S. Promoting culturally reliable and valid evaluation practice. In S. Hood, R. Hopson & H. Frierson (Eds.) (2005), The Role of Culture and Cultural Context In Evaluation. Greenwich, CT: Information Age Publishing.

This chapter describes the authors’ work with diverse communities, especially their work in Indian communities. The authors describe how contemporary approaches to evaluation (participatory practices, empowerment evaluation, etc.) alone cannot ensure cultural responsiveness. They outline standards for extemporary evaluation practice that considers cultural context and audiences. They argue that these standards suggest a new paradigm and roles for evaluators.

Senese, G. The PENAL Project: Program evaluation and Native American liability. In S. Hood, R. Hopson, & H. Frierson (Eds.) (2005), The Role of Culture and Cultural Context In Evaluation. Greenwich, CT: Information Age Publishing.

In this provocative chapter, Senese argues that cultural competency and responsiveness is important in evaluation. However, it fails to call attention to the ways in which the culture of capitalism feeds the culture of racism and the structural inequalities of oppressed people. He evaluated a community wellness program connected to a Navajo school.

Smith, L. T. (1999). Decolonizing methodologies: Research and indigenous peoples. London: Zed Books, Ltd.

This seminal text challenges Western research practices and advocates for the development and use of indigenous research methodologies that are more inclusive of indigenous “cultural protocols, values and behaviors” (p.15). Linda Smith, a Maori academician, notes that “research is not an innocent or distant academic exercise.” It occurs within “a set of political and social conditions” (p. 5). Chapter 8 of Smith’s text, “Twenty-Five Indigenous Projects,” provides generalized descriptions of research projects, i.e., research activities, which can resonate within an evaluation context in other Native communities.

White, C. & Hermes, M. Learning to Play Scholarly Jazz: A Culturally Responsive Evaluation of the Hopi Teachers for Hopi Schools Project, in S. Hood, R. Hopson & H. Frierson (Eds.) (2005),The Role of Culture and Cultural Context In Evaluation. Greenwich, CT: Information Age Publishing.

The authors describe the tensions involved in planning a culturally responsive evaluation study of the Hopi Teachers for Hopi Schools Project (HTHS). They were concerned with the problem of using “colonial” research tools and the English language in a program developed for and by tribal people. Using the metaphor of jazz, which they describe as a music form that offers the possibility of hearing the world differently, the authors explain how they approached program design and evaluation to be responsive and culturally appropriate. Using pragmatic, strategic, and self-reflective methodological approaches, they list methods for their evaluation that are comprehensive as well as responsive. They describe how they will explore new forms of presenting final reports that are culturally valid and open new ways of knowing.

OTHER RESOURCES

American Evaluation Association. Guiding Principles for Evaluators. www.eval.org.

Evaluation practitioners interested in how other professional evaluators view their craft might find the American Evaluation Association (AEA’s) principles of interest. AEA strives to promote ethical practice in the evaluation of programs, products, personnel, and policy. The principles are: (a) systematic inquiry, (b) competence, (c) integrity/honesty, (d) respect for people, and (e) responsibilities for general and public welfare. AEA has an Indigenous Peoples Topical Interest Group comprised of Native educators and evaluators who are American Indian, Hawaiian, and Maori, as well as non-Native educators and evaluators practicing in Native communities. www.eval.org/aboutus/organizations/tigs.asp.

Aroturuki me te Arotakenga. (Ministry of Maori Development),Te Puni Kökiri (Monitoring and Evaluation Branch), 1999. Evaluation for Maori: Guidelines for Government Agencies. www.tpk.govt.nz/publications/subject/default.asp.

The Te Puni Kökiri is required to monitor and act as a liaison with each New Zealand government department and agency that provides — or has responsibility to provide — services to or for Maori people. The purpose of this monitoring function is to ensure the adequacy of services, based on the governmental treaty responsibility and priorities for serving the Maori population (which is approximately 20% of the population). The branch also advises appropriate evaluation systems for Maori. These guidelines were developed to ensure the collection of appropriate and quality information by agencies undertaking evaluations. The guidelines are a set of minimum critical success factors that should be considered by agencies when evaluating their programs. The guidelines include ethical issues for consideration, ensuring that Maori are involved in evaluation planning and that their participation is built into the evaluation design, as well as the analysis of data and reporting of evaluation findings. They include many practical considerations that may be applicable to evaluators conducting evaluations in Indian communities. The documents are all freely downloadable.

Kamehameha Schools. Evaluation Hui. www.ksbe.edu/pase/pdf/EvaluationHui/03_04_17.pdf

This Native Hawaiian hui — meeting or group — was borne out of the question of whether current approaches and methodologies for program evaluation and research were appropriate for the Native Hawaiian population. The hui is comprised of evaluators, program administrators, researchers, and educators that serve the Native Hawaiian population. In collaboration with New Zealand Maori colleagues, the hui serves as a forum for the exchange of knowledge, ideas, and experiences about culturally responsive methods and protocols for the evaluation of programs serving these two indigenous cultures. The Evaluation Hui’s vision is to create and initiate evaluation methods and outcomes that benefit the Kanaka Maoli and Maori. Their website contains a digital library of publications that document the conceptions and progress of the Evaluation Hui. The documents are all freely downloadable.

Richard Nichols (Santa Clara Pueblo) is president of Colyer Nichols, Inc., a research and evaluation company (P.O. Box 5132, Fairview, NM 87533). He can be reached via email at colyrnickl@cybermesa.com. Joan LaFrance (Turtle Mountain Chippewa) is the owner of Mekinak Consulting, a company specializing in program evaluation (612 NW 80th St, Seattle, WA 98117). She may be reached via email at joanlafrance1@msn.com.