This third Kaupapa Māori evaluation blog post describes the methodology, methods and lessons learned from a Kaupapa Māori evaluation conducted in Aotearoa New Zealand in the late 1990s. BackgroundHe Oranga Poutama was part of a broader effort called Omangia Te Oma Roa (Run the Long Journey/Participate for Life), which aimed to facilitate Māori aspirations for health and wellbeing by coordinating policy developments and service delivery across various government agencies. The philosophy of Omangia Te Oma Roa was that physical leisure (sport and recreation) was an effective medium for reaching Māori and promoting health and wellness. He Oranga Poutama was initially developed by the Hillary Commission and subsequently established as a key strategy for promoting Māori health through physical activity. The initiative involved the recruitment and training of Māori health and wellbeing coordinators (kaiwhakahaere) who worked to increase awareness of the benefits of physical activity and injury prevention within Māori communities. The focus of the second-year evaluation of He Oranga Poutama was on changes in Māori community behaviours and attitudes towards sports and recreation as a result of the initiative. The evaluation covered events organized by kaiwhakahaere (coordinators) at four project sites: Tauranga, Whakatane, Auckland Central, and Rotorua. Kaupapa Māori MethodologyThe evaluation of He Oranga Poutama was conducted using a Kaupapa Māori (Māori approach) framework, which emphasizes Māori values, cultural practices, and community involvement. Kaupapa Māori research is concerned with methodology rather than method, focusing on culturally appropriate and community-centred approaches. This involved several key principles, including:
MethodsThe methods used in the evaluation of He Oranga Poutama were designed to be culturally responsive, respectful, and empowering. By aligning with the principles of Kaupapa Māori, the evaluation process honoured Māori values and perspectives, ensuring that the findings were meaningful and relevant to the community. The combination of survey questionnaires, in-depth interviews, focus groups, participant observation, and reflective journals provided a rich and comprehensive understanding of the events’ impact on Māori health and well-being. This approach exemplifies the potential of culturally grounded methodologies in conducting effective and meaningful evaluations within Indigenous communities. 1. Survey Questionnaires
Strengths: The use of survey questionnaires allowed for the collection of standardized data, making it possible to compare results across different sites and events. The involvement of local interviewers enhanced the cultural appropriateness of the method. Challenges: Ensuring high response rates and accurate data collection in a dynamic event environment can be challenging. Training and supporting interviewers was essential to address these challenges. 2. In-Depth InterviewsDescription: In-depth interviews were conducted with key stakeholders, including event organizers (kaiwhakahaere), community leaders, and participants. These interviews aimed to gather detailed qualitative data on their experiences, motivations, and perceptions of the events. Application: Interviewers used semi-structured interview guides to ensure that key topics were covered while allowing for flexibility and in-depth exploration of issues. Interviews were conducted in a respectful and culturally appropriate manner, often incorporating elements of tikanga Māori (Māori customs and traditions). Strengths: In-depth interviews provided rich, detailed insights into the experiences and perspectives of stakeholders. This method allowed for a deeper understanding of the cultural context and the impact of the events on the community. Challenges: Conducting in-depth interviews requires skilled interviewers who can build rapport with participants and navigate sensitive topics. Ensuring the confidentiality and comfort of participants was crucial. 3. Focus GroupsDescription: Focus groups were organized to facilitate discussions among participants and stakeholders. These group discussions provided an opportunity to explore collective views, experiences, and suggestions for improvement. Application: Focus groups were conducted in a relaxed and supportive environment, encouraging open and honest discussion. Facilitators guided the conversations, ensuring that all participants had an opportunity to contribute. The focus groups often started with a whakawhanaungatanga (relationship-building) activity to establish trust and connection. Strengths: Focus groups allowed for the exploration of shared experiences and collective insights. This method facilitated the identification of common themes and issues, enriching the overall understanding of the events’ impact. Challenges: Managing group dynamics and ensuring that all voices were heard required skilled facilitation. Creating a safe and respectful environment was essential for meaningful discussion. 4. Participant ObservationDescription: Participant observation involved evaluators immersing themselves in the events, observing interactions, activities, and the overall atmosphere. This method aimed to provide a holistic understanding of the events and their impact on the community. Application: Evaluators participated in the events as observers, taking detailed field notes on their observations. They focused on aspects such as participant engagement, interactions between whānau (extended family) members, and the integration of health promotion messages. Strengths: Participant observation provided context and depth to the evaluation findings. It allowed evaluators to capture the nuances of the events and the cultural dynamics at play. Challenges: Maintaining an objective perspective while being immersed in the event can be challenging. Balancing participation and observation required careful consideration and reflexivity. 5. Reflective Journals
Strengths: Reflective journals supported evaluators in maintaining a reflexive approach, allowing them to critically examine their own biases, assumptions, and positionality. This method contributed to the integrity and authenticity of the evaluation. Challenges: Maintaining consistent and detailed journal entries required discipline and commitment. Ensuring that reflections were honest and constructive was essential for the effectiveness of this method. Key Evaluation FindingsDemographics: The majority of participants were Māori, with a significant portion aged between 16 and 40 years. There was a relatively even gender distribution. Attendance and Motivation: Participants primarily attended to support whānau, play sports, or enjoy the event. Whānau networks were a crucial factor in recruiting participants and spectators. Health Behaviours: Many participants reported reducing smoking and alcohol consumption during the events. A significant number set personal goals related to increasing fitness and participating in more sports. Event Success: Most participants and stakeholders felt the events were successful, highlighting the importance of whanaungatanga (kinship) and community involvement. Key Learnings from the Use of Kaupapa Māori Evaluation MethodologyThese key points highlight the valuable lessons learned from employing a Kaupapa Māori evaluation methodology in the He Oranga Poutama initiative. The culturally responsive and community-centred approach not only enhanced the evaluation process but also contributed to the empowerment and well-being of the Māori communities involved. Cultural relevance enhances engagement
Building trust through local interviewers
Community ownership and empowerment
Holistic understanding through mixed methods
Importance of reflexivity
Continuous improvement through feedback
Cultural integrity in evaluation
ConclusionHe Oranga Poutama successfully promoted sports and healthy lifestyles among Māori communities. The events fostered a sense of whanaungatanga (relationships) and provided platforms for participants to set and achieve personal health goals. The initiative's culturally relevant approach and strong community engagement were key to its success. Ongoing support, innovative health promotion strategies, and continuous improvement based on community feedback will further enhance the program's impact. The initiative stands as a model for culturally grounded health promotion, demonstrating the power of Indigenous knowledge and practices in fostering well-being. The use of Kaupapa Māori evaluation methodology in the evaluation of He Oranga Poutama was instrumental in creating a culturally responsive, respectful, and empowering evaluation process. This methodology not only enhanced the engagement and participation of Māori communities but also ensured that the evaluation findings were meaningful, relevant and useable. By honouring Māori values and perspectives, this Kaupapa Māori evaluation provided a robust framework for understanding and promoting health and well-being within Māori communities. Glossary
Other blog posts in this series:
|