As of January 1, 2018, stewardship of Hive NYC will transition from Mozilla to Partnership for After School Education (PASE). You can read more about this here. Please visit the link to learn more about Partnership for After School Education.

Jul 06 2017

Digital Media Learning Evaluation Cohort – Making Meaning of Evaluations

Community, Connected Learning, Hive Learning Networks, Hive NYC, Members, Research, Teaching Resources

On June 7th, the Hive NYC’s Youth Digital Media Learning Evaluation cohort met to reflect on the information gathered from program evaluations that measured social emotional learning outcomes in digital learning programs.  Cohort members explored how to make observations of young people’s collective growths, identifying effectiveness of youth development practices, and how to translate results to action plans for their work.  Using a sample Youth Development Impact Learning System (YDiLS) post report, volunteered by one of our Hive members, Algorhythm supported the cohort investigate key findings in student’s learning. The sample report explored provided a focused assessment of gains in six SEL capacities:

  • Self-Management: A young person’s ability to make choices, take positive risks and persist through life’s challenges.
  • Contribution: A young person’s capacity to give energy and time to help their family, peers, community, and society.
  • Academic Self-Efficacy: A young person’s motivation and confidence in their academic performance.
  • Social Skills: A young person’s ability to take others’ perspectives into consideration, as well as express caring and empathy.
  • Positive Identity: A young person’s internal sense of positive self-worth and self-efficacy as they explore who they are.
  • Social Capital: A young person’s positive bonds with people who can provide advice, counsel and access to what they need to succeed.

The above capacities were drawn out via field-research Algorhythm conducted with partnering youth serving organizations.  These SEL outcomes aid young people in building the critical thinking skills, attitudes, and behaviors to effectively deal with challenges in order to reach success.

The cohort examined the evaluation data to determine key indicators from this report of most impactful youth development practices.  Making meaning of these results is invaluable for youth workers as it can fuel the necessary steps to improve programs and communications strategies.  The cohort followed up this conversation by working in small breakout groups to translate data to actionable steps.

Using Evaluation Data for External Communications

Making meaning of evaluation data provides the opportunity to support cohort members in communication strategies meant for a variety of audiences (i.e. youth, education staff, funders, supporters etc).  The measurement of SEL outcomes could help to inform reports and discussions with funders in a way that’s in line with members’ values – not only highlighting numbers but showcasing the impact of best practices and pedagogy through youth’s qualitative reflections and feedback.  They experimented with a Communications MadLib tool  (designed by Algorhythm) to draft a report to illustrating their evaluation process and how they are using their findings to improve  digital media learning programs.  This techniques also supported the visioning of ways to showcase on different platforms (i.e. website, presentations, reports etc.) in order to share reports and connect with supporters of the work they are doing.  The cohort also discussed methods of how and when to share evaluation data with youth in their programs, as it could present an opportunity to build more autonomy over their learning experiences and potential collaboration with young people on program design.

Using Evaluation Data for Program Improvement

Member’s post evaluation reports can be used the reinforcement and adaptation of effective PYD practices in programs to best fit youth’s interests and needs.  For example, if evaluation data shows a higher percentage in social capital gains, assessed through youth’s positive experience with peer-to-peer learning rather than learning on their own, it’s significant information for program staff to invest more in co-learning education models.  Using Algorhythm’s post report action plan worksheet, members discussed a structure that would allow to move evaluation data to tasks and projects in order to adapt programs.  Apart from program improvement, members visioned that reports could also prove useful in providing context to establish refined language that help organziations articulate work and intended project outcomes to education staff.  This can be essential in the professional development and orientation of new staff – providing perspective to organizations theory of change, pedagogical practice, and program planning.

In the coming weeks, Algorhythm will be supporting the cohort in analyzing findings and identifying trends that connect across members’ programs and best practices.

For cohort members, upholding and advocating the importance of social emotional learning in youth’s holistic growth is pivotal to their shared values of positive youth development. The intentional process of assessing young people’s experiences in programs not only provides a framework for collecting data but also reinforces the investment we have on our student’s drives, passions, and futures. The cohort hopes to engage further in this conversation with more youth workers and adult allies in the coming Hive NYC Learning Network Meetup in July. A report back on this discussion will be available on our Blog.

See the list below for resources and materials the cohort used during the Youth Digital Media Learning Evaluation cohort meeting.

Leave a Comment