Mixed-methods remote program evaluation

Knowledge graph centered on Mixed-methods remote program evaluation with 8 nodes and 13 connections. Top connected: Aedes communis, “How Do We Do This at a Distance?!” A Descriptive , Rosemary J. Smith, Rex D. Cole, S. A. Lopez.

Method synopsis

Combined structured program documentation, focus groups, and surveys to evaluate remote undergraduate research program implementation and student experiences during COVID-19.

Synthesized from method descriptions across 1 paper using this protocol.

Procedure as described in the canonical source

Steps below were extracted from the paper that introduces this protocol — “How Do We Do This at a Distance?!” A Descriptive Study of Remote Undergraduate Research Programs during COVID-19 (2022), CBE Life Sciences Education. Implementations in other papers (listed below) may differ.

1. Recruit participating programs
   Invited 25 programs that involved students in remote undergraduate research in 2020. Twenty-three programs chose to participate, hosted by 24 organizations across 18 states and one U.S. territory.
   Quantities: 25 programs invited, 23 programs participated, 24 host organizations, 18 states plus 1 U.S. territoryDuration: Summer 2020Conditions: During COVID-19 pandemic when programs shifted to remote delivery
2. Collect written program descriptions
   Used structured templates to document when, where, and how each program was implemented from perspective of Principal Investigators. Templates covered expectations, introductory and culminating events, and weekly activities to ensure PIs could describe implementation with accuracy.
   Quantities: 23 program descriptions collectedDuration: After programs were completedConditions: Structured template format for consistency
   Equipment: structured templates
3. Conduct student focus groups
   Held focus groups at midpoint and end of program timepoints with average 81% and 67% student participation respectively. Students chose focus groups that suited their schedules. All focus groups were recorded to ensure feedback was captured accurately.
   Quantities: Focus groups with 81% participation at midpoint, 67% at end of programDuration: Midpoint and end-of-program timepointsConditions: Students selected focus groups based on their availability
   Equipment: recording equipment for focus groups
4. Administer student surveys
   Surveyed students at end of programs regarding extent to which they experienced programs synchronously versus asynchronously, quality of relationships with research mentors, and level of connectedness in their programs.
   Quantities: Surveys administered to program participantsDuration: End of programConditions: Complemented focus group data collection
   Equipment: survey instruments
5. Analyze qualitative data
   Used inductive qualitative content analysis of focus group reports. Evaluation team independently read student strengths and suggestions, created brief descriptive summaries with illustrative quotes. Team met as group to discuss and refine themes, developed definitions, and ensured coherent representation.
   Quantities: Analysis of all focus group transcriptsDuration: Post-data collection analysis periodConditions: Iterative team-based analysis approach

Typical Equipment

• structured templates
• recording equipment for focus groups
• survey instruments

Output Measurements

• program implementation descriptions
• student satisfaction ratings
• qualitative feedback themes

Papers Using This Protocol (1)

Species Studied With This Protocol

Places Where Used