Lesson 7

CQI: questions for each phase with sample answers

The following Q+A is designed to provide 100% Community course participants and action team members with a sample of how to answer the key continuous quality improvement questions (CQI) related to developing a local innovation. In this example, an action team has reviewed the 100% Community survey data and seen that many parents report having a lack of access to behavioral health care in general, along with their children not having access to it at school.

Q+A: Assessment

Question: What is the challenge you identified?
Example: Our students at our middle school face emotional challenges due in part to adverse childhood experiences. Students at our school use alcohol and recreational drugs at early ages, as well as consider suicide.

Question: What data did you use to identify the challenge?
Example: In addition to the countywide 100% Community survey results where 46% of parents reported challenges to accessing behavioral health care in the community or at school, the Youth Risk and Behavior Survey (YRBS), specifically the data around initiation of alcohol use by age 12 and rates of suicidal thoughts by both middle and high school students. Data from the 2009 Public Health Survey of Adults that asked questions about ACEs. Data from the article, “The Prevalence of Confirmed Maltreatment Among US Children, 2004 to 2011” in JAMA (2014) by Wildman, Emanuel and Leventhal, concluded that maltreatment will be confirmed for 1 in 8 US children by 18 years of age. The authors reported that this was far greater than the 1 in 100 children whose maltreatment has been confirmed annually by child welfare systems across the nation. For black children, the cumulative prevalence is 1 in 5; for Native American children, 1 in 7.

Question: What do data tell you about the size of the challenge?
Example: According to the most recent Youth Risk Behavior Survey (YRBS), 1 out of 5 middle school students report suicidal thoughts. Slightly less for HS students in the county. Many students at our middle school did not take the YRBS, therefore the need may be greater than we thought.

Question: What do data tell you about which populations are being impacted by the challenge?
Example: Data from the article, “The Prevalence of Confirmed Maltreatment Among US Children, 2004 to 2011” in JAMA 2014 by Wildman, Emanuel, and Leventhal that concluded that maltreatment will be confirmed for 1 in 5 black children, and 1 in 7 Native American children. 1 in 5 children live in households where the income is below the federal poverty income. Our middle school is located in a zip code that has a mix of incomes and many students with different ethnicities, with a significant proportion being Hispanic.

Question: What do data tell you about the current capacity of your agency, community or county to address the challenge?
Example: The rates of underage substance use and suicide ideation have not changed significantly for many years indicating a lack of community capacity to address the challenge. Our school does not have staff to address substance use with our middle schoolers nor their parents. There is limited funding for our school to increase behavioral health services for our students.

Question: Which data can confirm that your challenge is a priority?

Binge drinking rates, from the YRBS, among MS and HS students indicate a challenge.

Rates of suicidal thoughts are present in middle school and continue to high school (from the Youth Risk Behavior Survey). Depression among adults and ACEs scores from the latest Behavioral Risk Factor Surveillance System (BRFSS) public health survey of adults.

Question: What data and/or research illustrates the root causes of the challenge?
Example: We are exploring root causes of suicidal thoughts by looking at academic literature and books on suicidal thoughts. There can be a host of reasons — including Adverse Childhood Experiences. Past research on ACEs shows that there is a link between adult behavioral health outcomes and early childhood experiences. In our state 60% of adults report having at least one ACEs according our state Public Health Adult Survey. In small samples, as many as three-quarters of high school students report having 3 or more ACEs. Some with 7 to 10 ACEs.

Q+A: Planning Phase

Question: Which component of the challenge will you address with a local project?
Example: There were many issues we identified (substance use, depression, ACEs and suicidal thoughts). We wish to focus on addressing depression in students, as well as adverse childhood experiences. We want to explore evidence-informed prevention processes.

Question: Describe your review of research focused on your challenge?
Example: More ACE research to come:

CDC’s Adverse Childhood Experiences (ACEs) Information: https://aae.how/219
Category Archives: Adverse childhood experiences: https://aae.how/220
Robert Wood Johnson Foundation on Adverse Childhood Experiences: https://aae.how/221
Youth Risk Behavior Surveillance System (Depression in Youth): https://aae.how/222

Question: What potential evidence-informed solutions exist in the research?
Example: Because ACEs represent a number of challenges (child abuse, neglect, growing up in households with violence or substance abuse, or with adults with mental health challenges, etc.), we want to first focus on identifying the challenges/ACEs endured by our middle school students. We understand that our students and their parents may have untreated trauma due to ACEs. In some households ACEs continue. We seek, at first, to be able to offer to students and their family members behavioral health care that is easily accessible and culturally appropriate.

We reviewed research on current evidence-informed solutions for ACEs. There are many challenges and approaches to addressing child abuse, neglect, growing up in households with violence or substance abuse, etc. We understand that families have many needs such as stable housing, secure food and access to medical care and transportation. We have been reviewing the literature on community schools which have funding for school-based behavioral health care among many other services. We are prioritizing school-based behavioral health care.

Since we mainly have influence at our middle school, we want to focus on first identifying/ACEs endured by our middle school students and helping our students and their families access behavioral health care. One approach we found is called Screening, Brief Intervention and Referral to Treatment (https://aae.how/223). We are still reviewing many forms of support that could be made available if our school had a behavioral health care center.

Question: What is the hypothesis that illustrates how your actions might solve the challenge? (As in, “if you do A, then B will happen.”)
Example: If we can increase our capacity to provide behavioral health care at our school, we can implement ACEs screening. With a team of school-based behavioral health care providers, we can help students and their families to get quality, culturally appropriate and easily accessible behavioral health. Behavioral health treatment can lead to a reduction in early substance misuse, depression and suicidal thoughts. We can help parents to address their problematic behaviors so that ACEs end in their household.

Question: What are your key steps, timelines, roles and responsibilities related to your project?
Example: Our course project requires creating awareness of the epidemic levels of ACEs and the relationship between ACEs and early substance misuse, depression and suicidal thoughts. This awareness is a short-term outcome.

Intermediate-term, we are looking into creating funding for more behavioral health care staff at the school.

Long-term, we want to fund the creation of a school-based health/wellness center with the capacity to offer behavioral health care to both students and their family members. This is a long-term project requiring, increased funding, and buy in from stakeholders on the school, district, city and county levels. We may also need support from state government and health care institutions.

Question: What unintended consequences might you encounter?
Example: Other school-based health centers we have looked at have had different reactions. Some have been very welcomed by the entire school community. Others have had backlash from parents who do not approve of behavioral health care or school-based health centers for fear that “personal family issues” may be revealed or that health centers may provide unwelcome health advice to youth.

We also may find that there are not enough appropriate referrals in our community to help families access vital services once we do identify problems related to ACEs.

Some behavioral health care agencies are limited to which services they can provide.

Q+A: Action Phase

Question: How will you secure buy in for the course project (innovation or change initiative) in your workplace and/or community?
Example: We will start with qualitative data. We hope to do informational interviews with local school management, school board members, parents, students, city hall, county government and the local hospital and behavioral health care providers. We will use their input to get buy-in. We need to build support among potential funders. We can then seek to gather data on the magnitude of ACEs, along with substance misuse and suicidal ideation, within our student and parent populations.

Question: Which, of all your proposed activities, are the most vital?
Example: Buy-in from school principal and school board for phase one: creating awareness of the challenges and potential solutions. We also need ongoing awareness of ACEs from our school staff and parents. For our long-term goals, we will need establishing funding to be one of our most vital activities. For all of our efforts, we will need data (qualitative and quantitative) as a vital component to educate and ensure buy-in along the way.

Question: How will you begin implementation of the course project?
Example: We will start by contacting the state Department of Health (DOH), Office of School and Adolescent Health and the state coalition for school-based health to learn the latest policies on school-based health centers. We will also explore how some school-based health centers were funded. We have identified fully-resourced community schools in the state and schools with fully-funded behavioral health care centers. From there we begin our process of information gathering through interviews with stakeholders.

Question: How will you record and monitor activities as your course project unfolds?
Example: All information interviews with stakeholders will be recorded, documented and filed on Google docs. We will also do surveys — via Google forms and record findings.

Question: What will your process be for making adjustments to your course project?
Example: We will meet monthly to assess progress. We will consider adjustments using the data we are collecting along the way as part of our process.

Question: How will you ensure data is collected along the way as your course project unfolds?
Example: We will have a monthly report at our meeting on all findings (qualitative and quantitative data, research articles on our topic areas) — and create a quarterly update sent out to all participants and stakeholders.

Q+A: Evaluation Phase

Question: How will you analyze and share all relevant data with those working on the experiment?
Example: Analyses will require our team working with a data specialist. We have identified one at our local college. We will create a database that all our members can access and a project management system called Freedcamp, to share all documents.

Question: How will you ensure that your data and evaluation are presented in an easy to understand manner?
Example: We will have a data expert on our team and run all data by a data committee. We will ask for support from the Office of School Health-DOH — in reviewing data. We will use data visualization (for example Tableau software) to help increase awareness.

Question: How will you determine the strengths and weaknesses of the course project?
Example: Strengths will be assessed by the aspects of the project that are going well and moving forward with support from stakeholders. Weaknesses will be identified by areas that have limited ability to make movement forward on our project.

Question: How will you determine the impact of the course project on your challenge?
Example: The overall impact for the project is long-term. Our initial phase is awareness which will be determined by follow-up surveys. Our intermediate phase will be determined by assessing the amount of funding we have been able to secure and number of providers we have been able to hire. Long-term outcomes can be monitored over the years. We can use measures from the YRBS on substance use, depression and suicidal thoughts, the number of ACEs screens we are able to complete at our middle school per year, and the number of students and families that are receiving services due to school ACEs screening.

Question: How will you identify the unintended consequences?
Example: We need to set up a process to track the parent’s reactions to school-based behavioral health. Surveys and informational interviews can be valuable tools to identify the unintended consequences.

Question: How will you measure if people (employees and/or residents impacted by the course project) are better off?
Example: We will use surveys and existing data to assess:

Short term: The amount of constructive dialogue between school staff, parents, students and district personal on the topic of ACEs and the need to address it

The percentage of awareness of school staff, parents and students of the local rates of ACEs, depression and substance misuse and potential evidence-based solutions

Usage of local behavioral health agencies by parents and youth

Usage of the counseling staff by students and family members

Amount of funding for school and community support for school-based behavioral health including financial support

Long term: significant changes in access to behavioral health care for our students and their family members, we should see a decrease in problems associated with youth depression, substance use and suicidal thoughts — as indicated by Youth Risk and Behavior Survey

Question: How will you ensure that you communicate with all stakeholders on the results of the experiment?
Example: We will use the internet and social networking tools to share our progress and successes — as well as challenges. We will make sure our monthly meetings include time to plan communication with stakeholders who are not able to attend meetings.

Finding answers

As you can see from this example of CQI question and answers, local community stakeholders can create a very informative and thoughtful strategic plan to take on a project. Please let the 100% Community course instructors or coaches know if you need assistance answering any question as you develop your project plan. We look forward to supporting the development of your course project/innovation. Make sure to review the course website’s links to research articles and other resources.