To collect employer insights on the effectiveness of the EPP’s advanced programs in preparing graduates for their professional roles, specifically with regard to professional knowledge and student impact.
Research Questions
Data Collection ProceduresTwo participant groups were invited to provide employer satisfaction feedback via virtual focus groups. As suggested by Krueger and Casey (2015), the focus groups included participants who were generally homogenous and have common characteristics that are relevant to the topic under consideration. Participants were recruited using a combination of purposive sampling, “in which participants are deliberately selected because they are most likely to provide insight into the phenomenon being investigated due to their position, experience, and/or identity markers” (Saldana & Omasta, 2018, p. 96), and convenience sampling, in which participants are selected because of ease of access and availability (Tracy, 2020). Participants in each group were invited for three key reasons: 1. their roles in the hiring and supervision of completers from our advanced programs, 2. their location in a district where a higher concentration of completers are hired, and 3. the ease of access due to willingness to participate and connection to the advanced programs.
Two focus groups were conducted to enable questions specific to the professional area of concentration, but overall results were analyzed across programs for trends regarding strengths and weaknesses of completers in the areas of professional knowledge and student impact as well as recommendations for program improvement. The first group included Superintendents, Assistant Superintendents, Human Resources Staff, and other Central Office personnel who were asked to speak to completers of the advanced programs focused on educational leadership. The second participant group included Central Office personnel, school principals, and district-level administrators who were asked to speak to completers of the advanced programs focused on curriculum and instruction. Each focus group had one facilitator, and 4-6 participants, ensuring engaging conversations with adequate interaction without becoming unwieldy (Griffith et al., 2020; Lewis & Muzzy, 2020). Virtual focus groups require additional considerations, such as using smaller groups, limiting the amount of time, and additional training for facilitators (Griffth et al., 2020), which were accounted for by facilitators discussing those additional concerns and calibrating their approaches in advance, intentionally limiting participation to six for each group, and careful construction of protocols for guiding the groups. Facilitators specifically focused on key best practices like ensuring all participants are heard, asking follow-up questions or including prompts, developing rapport, setting the tone, establishing the purpose of the group, and keeping the discussion focused and on track (Brown, 2022; Krueger & Casey, 2015) Participants were invited to focus groups via email (see Appendix A), and groups were scheduled in the evening to accommodate participant schedules. Semi-structured focus group protocols were developed by in alignment with EPP advanced program goals and CAEP Standard 4. Once protocols were drafted, they were reviewed by multiple faculty and revised according to feedback. Finalized focus group protocols are included as appendices to this report (see Appendix B and C). Focus groups were held virtually via Zoom to accommodate both the wide geographical spread of participants and for scheduling convenience (Brown, 2022; Griffith et al., 2020). Focus groups were recorded using the internal recording tool in Zoom and transcribed using Otter.ai, which powers the closed captioning feature in Zoom.
Data AnalysisThe two facilitators of the focus groups analyzed transcripts using a process of holistic coding for content analysis (Saldana, 2021). Holistic coding provides a “broad brushstroke representation” of the data that is appropriate for determining broad patterns and key concepts in content analysis (Saldana, 2021, p. 33). That analysis resulted in key areas of strength and weakness that employers identified as well as strengths and weaknesses of the advanced programs. Additionally, the analysis yielded recommendations for program improvement.
Focus group transcripts, analysis documents, and notes were all de-identified to ensure participant confidentiality. Only audio recordings of the focus groups were retained rather than videos containing personally identifying information. Data materials are stored in a password-protected file with two-factor authentication to which only the focus group facilitators have direct access. All findings and interpretations are reported in summary form.
FindingsFindings from focus groups are divided by program area: curriculum and instruction and educational leadership and organized according to completer strengths, completer weaknesses, and program feedback.
C&I Focus GroupOverall, participants found program completers from Gardner-Webb MAIT, MTLCI (now MCLIC) and EDCI programs to be more well prepared than completers of other programs, noting their ability to advance their careers, recognize the importance of research, and make sound decisions.
Strengths. In particular, the participants noted three key strengths of Gardner-Webb completers:
These areas reflected candidates’ strengths in skills and dispositions related to program outcome goals as well as to their ability to apply their professional knowledge in the context of their current positions. Candidates specifically demonstrated strengths relative their abilities to apply theoretical and content knowledge especially with regard to assessment and instructional practice. Additionally, completers show particular strengths in their use of data and technology in leadership roles.
Weaknesses. There were also areas identified by participants as completer weaknesses.
One area noted was the knowledge of relevant legislation and policies that impact curriculum and that instructional leaders should be conversant in. Another area where completers don’t have the same depth of knowledge is social emotional learning and behavior issues.
Program feedback. In addition to the areas for growth related to completer professional knowledge and skills, employers provided specific feedback about the program. Strengths of the program participants noted was the use of feedback and continuous improvement as well as the cohort model, which facilitates building and leading teams and effective communication. More than anything, participants highlighted the program’s effectiveness at producing reflective practitioners who Participants emphasized the need for the program to constantly update and include relevant texts, which they acknowledged is already happening, in order to keep up with the fast-paced and ever-changing educational environment. They also noted that additional emphasis on possible career paths or opportunities would be useful so that completers are aware of advanced teacher roles in addition to administrative or other leadership roles.
LS Focus GroupOverall, participants found program completers from Gardner-Webb MSA and EDLS programs to be as prepared or more prepared than program completers from other EPPs.
These areas reflected candidates’ strengths in skills and dispositions related to program outcome goals as well as to their ability to apply their professional knowledge in the context of leadership positions. Specifically, completers showed strength when encountering new situations and were able to apply their theoretical knowledge to problem solve. Additionally, completers demonstrated particular capacity for strengths based, collaborative, and shared leadership in which they were able to build effective teams to address issues and improve schools (“move a school”).
Weaknesses. There were also some areas that the participants noted were weaker for Gardner-Webb completers:
In particular, relative to other areas, curriculum and instruction was seen as a weaker area of preparation for completers. Notably, employers emphasized that Gardner-Webb graduates quickly adapted and understood those challenges when put in positions to lead curriculum. Participants also commented that sometimes the micropolitics that come within specific community contexts are more difficult to navigate, though they mentioned that this particular challenge was not unique to Gardner-Webb completers. They noted the importance of having an understanding of professionalism that extended beyond specific roles and responsibilities and into how to navigate relationships with those they lead to avoid potential pitfalls. Program Feedback. In addition to the areas for growth related to completer professional knowledge and skills, employers provided specific feedback about the program. Program strengths included the cohort model, which supported completers’ strong communication skills and networking, and the depth of content and theoretical knowledge completers gain. Program areas for growth included opportunities for more applied internship experiences. Participants did not think more hours or greater requirements were necessary but rather than a reimagining of the structure of clinical hours might facilitate hands-on experiences.
Implications and RecommendationsAs we examined the trends in employer satisfaction focus group feedback, we noticed key areas of strength that cut across all advanced programs as well as areas that were specific to the general concentrations of curriculum and instruction and educational leadership. Areas of strength noted by employers were the effectiveness of Gardner-Webb completers as strengths-based leaders who were skilled at constructing teams that capitalized on the strengths of each member and collaborative solving problems. That effectiveness was attributed to systems thinking that is fostered through the programs. Participants also noted that the cohort model in the programs fostered meaningful collaboration, the ability to build and lead a team, and networking. A key area for improvement across participant feedback is career discernment, whether it’s understanding how to determine fit for a first leadership position or to understand the possible careers enabled by the advanced degree, participants felt completers could be more well-versed in the landscape of career decision making.
Some participants were able to speak from a program perspective as well because they had either previously completed the program or had been involved with the program in some capacity, such as serving as an adjunct instructor, supporting clinical experiences, or serving on doctoral committees. Their feedback took into account multiple aspects of program completer competence and their satisfaction as employers, so they were able to highlight the communication and support structures that exist across the programs between various program stakeholders and how that infrastructure benefits completers. Participants also noted that completers of the Gardner-Webb masters level programs were more well-prepared for doctoral-level work than completers from other EPPs and specifically noted the MCLIC program as preparing students for EDCI.
Participant also made recommendations to the EPP for strengthening the program, particularly in light of contemporary needs and trends in public education.
References
Brown, R. (2022). Best practices for conducting virtual focus groups. Publications and Scholarship, 6. https://source.sheridancollege.ca/ctl_publ/6
Griffith, I., Parekh, J., & Charles, C. (2020, September 16). Conducting successful virtual focus groups. Child Trends. https://www.childtrends.org/publications/conducting-successful-virtual-focus-groups
Krueger, R. A. & Casey, M.A. (2015). Focus groups: A practical guide for applied research (4th ed.). Sage.
Lewis, F., & Muzzy, S. (2020, April). Conducting virtual focus groups. MDRC. https://www.mdrc.org/work/publications/conducting-virtual-focus-groups
Saldaña, J. (2021). The coding manual for qualitative researchers (4th ed.). Sage.
Saldaña, J., & Omasta, M. (2018). Qualitative research: Analyzing life. Sage.
