Phase 1: Implementation of the Student Equity in Higher Education Evaluation Framework: Sector Consultation Report

Executive Summary

2022 HEPPP Reporting Template

The sector generally viewed the latest HEPPP reporting template as an improvement, finding it more user-friendly and streamlined. However, there were concerns that it might omit useful data due to its simplification, particularly regarding the effectiveness of programs and their outcomes.

General SEHEEF Preparedness and Familiarity with Tools

• Participants expressed a need for greater clarity regarding timeframes for SEHEEF implementation and expectations for reporting and evaluation.
• The perceived lack of clarity was impacting preparedness and planning.
• Additionally, there were concerns about yearly funding cycles and the availability of student data for reporting on HEPPP programs accurately.
• Implementation preparedness for SEHEEF varied across the sector, with institutional leadership support influencing investment in data infrastructure and resources.
• Lack of staff and evaluation expertise were identified as factors affecting implementation readiness.

CQI Preparedness

• Some participants described embedding evaluation frameworks into their planning, though not always based on SEHEEF.
• Most CQI practices and tools were applied to both existing and new programs, with the main difference being the application of program logic models, more apparent in new programs.
• Stakeholders expressed a desire for further training on CQI.

Impact Evaluation Preparedness

• Most participants had not undertaken Impact Evaluation, with minimal planning for Quantitative Impact Evaluation (QIE) or Theory-Based Impact Evaluation (TBIE).
• The majority perceived their institutions as not prepared for QIE or TBIE, citing lack of systems, data collection methods, staffing, and evaluation expertise.
• Challenges included data linkage across different datasets in universities’ information systems.

Training and Support Needs

• There was a need for training materials ranging from basic to advanced aspects of evaluation, with a focus on simplicity and practicality.
• They also sought clarity on SEHEEF rollout timeframes and additional time/personnel for evaluation.
• The Communities of Practice (CoP) concept was supported, with participants enthusiastic about sharing best practices and learning from each other.