Lesson 4

March 9 2020

Course – Lesson 4: VI. Nest Steps and Assignments

NEXT STEPS AND ASSIGNMENTS

LESSON 4:VI – What are assignments and next steps?

The Steps You Take

Your steps may be different depending on your situation. Ideally, you are taking this course as part of a cohort: a group made up of local 100% New Mexico initiative participants. You may be the co-community organizer, action team leader or member. This is ideal, as you can discuss the course and all the information presented with people you’ll be collaborating with. However, you might be taking this course because you wish to start a 100% New Mexico initiative and today, you are a solo champion seeking to recruit stakeholders. Either way, the following steps enrich your learning experience and prepare you for Lesson 5: Evaluate.

Visit the Virtual Innovation Center: We provide you with links to innovations in all ten key family-serving sectors. These innovations include a menu of various projects, from across the state, nation and globe to consider implementing. The projects identified are designed to strengthen family health, safety and learning systems, and support your local 100% Community innovation. At this point, you should be very familiar with your sector’s innovations and projects.

Reviewing your course textbooks: Ideally, you have a good supply of sticky notes and sharpie pens to make comments and questions throughout all books. This makes group discussion more lively and focused as you get answers to questions and challenge assumptions. We strongly encourage you to dive into Adaptive Leadership, because you do not want to dive into action without knowing the difference between technical and adaptive challenges.

Reading Assignment: Course participants should finish reading “Chapter 30: Loss and change: understanding the difference between technical and adaptive challenges,” and “Chapter 31: Sharing the vision to achieve collective impact” in 100% Community before starting Lesson: 5. Evaluate.

Meet with an initiative team member or work colleague to discuss the following questions:

For your sector, are the biggest challenges adaptive or technical?
In your sector, what kind of pushback might you expect when you begin implementing change?
In your county, how is your action team set up to meet the 5 components of collective impact?

Convene with fellow initiative participants to discuss this lesson and the implementation process of projects: Set aside time with your fellow action team members to review all the projects that are being considered for implementation.

RETURN

Lesson 4:V

CQI QUESTIONS

Lesson 5:I

EVALUATE

March 9 2020

Course – Lesson 4: V. CQI QUESTIONS

Institute StaffLesson 4

CQI QUESTIONS

QUESTIONS FOR ACTION

LESSON 4:V – HOW DO CQI QUESTIONS GUIDE OUR PROJECT DEVELOPMENT?

What are the CQI questions for the action phase?

These are the (unskippable) questions you and your action team members will need to answer as you identify innovations and projects.

Question 1: How will you secure buy in for the innovation in your workplace and/or community?

Question 2: Which, of all your proposed activities, are the most vital?

Question 3: How will you begin implementation of the innovation?

Question 4: How will you record and monitor activities as your innovation unfolds?

Question 5: What will your process be for making adjustments to your innovation?

Question 6: How will you ensure data is collected along the way as your innovation unfolds?

RETURN

Lesson 4:IV

TO ACTION

Lesson 4:VI

NEXT STEPS

March 9 2020

Course – Lesson 4: IV. Turning Data Into Action

Institute StaffLesson 4

TO ACTION

TURNING DATA INTO ACTION

LESSON 4:IV – Q+A WITH AUTHORS KATHERINE ORTEGA COURTNEY, PHD AND DOMINIC CAPPELLO

How can data inform your entire innovation?

Dominic Cappello responds:

Data can help identify problems within agency workflow or community systems, and show where progress is being made.

How does it benefit you to use both qualitative data and quantitative data to inform your innovation?

Katherine Ortega Courtney, PhD responds:

Quantitative data are information collected from numbers. For example, the percentage of parents who report not being able to access medical care. The benefit is to describe large portions of a community at one time and get an idea of the overall functioning of a community

Qualitative data are information gained from phrases, sentences, or words that paint a picture. An example would be the description of a parent’s experience with housing support services. Both types of data together paint a larger, more complete story. The benefit is gaining more detailed information. Another benefit of qualitative data is that it provides stories, which are very helpful when presenting data to various audiences.

In the action phase of CQI, what challenges may occur?

Katherine Ortega Courtney, PhD responds:

As an experimental innovation unfolds there may be unintended consequences, or costs may be higher than projected, or buy-in from staff may diminish, or the manager of the experiment may feel unsupported. We recommend reviewing Adaptive Leadership for a great overview of why change is difficult even when it seems to benefit the community.

What is the importance of asking the right questions to identify the data-informed solution?

Dominic Cappello responds:

Using data is an iterative process. This means the process is cyclical and repetitive. Data can be very complex. It can be easy to head down the wrong path or get sidetracked with a question unrelated to the solution. It is important to be familiar with the data being used and understand the categories into which it can be grouped. When analyzing and interpreting data, we become translators of the data to the world outside. We need to interpret it correctly in order not to mislead others about what’s actually happening in practice.

How can data be visualized?

Katherine Ortega Courtney, PhD responds:

There are many ways to visualize data including pie charts, line graphs, bar graphs and tables. Data are also made more understandable by infographics. Information graphics or info-graphics are graphic visual representations of information, data or knowledge intended to present complex information quickly and clearly. Graphic design and infographics can play an important role in educating the public about challenges.

What are trends in data and what can they reveal?

Dominic Cappello responds:

Trends can more easily display rises and falls in activity. It can also reveal seasonality of certain activity. For example, one may see that over three years, more children are aging out of the child welfare system every year. Or one may see more students graduating from high school.

It important to note that trends can only be accurately assessed if the sample is large enough. Significance of changes is influenced by sample size. Changes can appear to be very dramatic in a small sample size, but that doesn’t mean the changes are meaningful.

How can you assess the quality and accuracy of data collected for your innovation?

Katherine Ortega Courtney, PhD responds:

To check the accuracy of the data you are collecting, you always need to ask a few questions of the data and check it:

Are data being recorded in the same way by everyone?

Are all data that should be recorded actually being recorded?

Are the data that is recorded reviewed for accuracy on a consistent basis?

Why is data a vital resource for the entities providing health and safety services?

Dominic Cappello responds:

Data do so many things. First, it tells us which community members are at risk for health and safety challenges. Data can tell us where to allocate resources, where problems occur and where programs are achieving goals or not.

Data, in the form of surveys or informational interviews, can also reveal points within social interaction between people that need attention.

Does collecting data tell us how to solve a problem?

Katherine Ortega Courtney, PhD responds:

Data cannot tell us how exactly to fix a problem or what to do, but it can guide a problem-solving process from identifying problems to monitoring progress toward solving them.

When beginning the action phase with a new project, what are the benefits of alerting all local initiative action teams (even though they may focus on different social sectors?)

Especially in a small city or town, new innovations attract attention. Some residents may cheer them on while other view with worry or suspicion. It’s best to let all action team members know when any new innovation is launching to gather as much support as possible. Additionally, the challenges in sectors may be interrelated. For example the mentorship team may discover that one of the biggest barriers to mentorship is transportation. Regular communication between the action teams can help identify and solve challenges.

RETURN

Lesson 4:III

TESTING

Lesson 4:V

CQI QUESTIONS

March 9 2020

Course – Lesson 4: III. Testing a Hypothesis

sssuperadminLesson 4

TESTING

TESTING A HYPOTHESIS

LESSON 4:III – Q+A WITH AUTHORS KATHERINE ORTEGA COURTNEY, PHD AND DOMINIC CAPPELLO

What is "fidelity" in terms of our team's innovation, and why is it important?

Katherine Ortega Courtney, PhD responds:

In the planning phase we designed the innovation or initiative. In the action phase we wish to follow that plan in all its details. For example, let’s say a school district has a new training for all school staff to help them identify students who might be experiencing trauma associated with ACEs (Adverse Childhood Experiences.) The training is four days with half the time spent on coaching. The evaluation of the training showed high marks from participants in terms of increased knowledge. A second school district decides to do the training but reduces it to two days and eliminates the coaching aspect. They have changed the training significantly and fidelity is not being respected. The second school district may get very different outcomes (knowledge increase among participants may be lower) as a result of changing the intervention).

What types and means of data collection are useful to increase fidelity?

Dominic Cappello responds:

The first step towards increasing fidelity with an experiment is to ensure that those responsible for its implementation are on the same page. This might be accomplished through a training or team meeting, and followed up with a short survey to identify any potential inconsistencies in how the specific elements of the experiment are understood.

Second, it is helpful to track what experimental activities are occurring at the individual or case levels, depending on the experiment. This might be done with the use of a spreadsheet (such as in Microsoft Excel or Google Sheets); a paper tracking form; or even an online survey tool like Survey Monkey.

How do we establish a baseline prior to implementing our innovation?

Katherine Ortega Courtney, PhD responds:

Baseline data is necessary to gauge what, if any, changes occur as a result of your experiment. Baseline data related to your experiment area may be readily available to you prior to implementing your activities; however, some experiments may address challenges for which no applicable baseline data exists, and that data must be collected. A good array of baseline data is likely to consist of both.

The Resilient Community Experience Survey will serve as great baseline data. If this survey is implemented at the beginning of the 100% New Mexico initiative, before any innovations have taken place, that can serve as a baseline. If it is implemented again after action teams have begun implementing innovations, it could also serve as a great way to measure change.

How would the concept of a "control group" apply to our local experiment?

Dominic Cappello responds:

In the world of science, experiments are considered “controlled” when there is, in addition to a population receiving the treatment or intervention, a similar group who does not receive it — the control group. This way, the magnitude of any changes noted in the “experimental group” can be accounted for through comparison with the control group.

Carefully designed experiments will usually assign similar clients or cases at random to either the experimental group or the control group (thus making the experiment not only controlled, but “randomized”). However, in the field (the “real world”) this isn’t always possible or practical. In some situations, clients will self-select to be part of one or the other.

How does my team anticipate what data we will need for evaluation?

Katherine Ortega Courtney, PhD responds:

Data for evaluation will need to include: data that are aligned with any baseline data collected (For example, results of the initial Resilient Community Experience Survey). Another example would be pre- and post-survey responses to a particular intervention such as a training for law enforcement on trauma sensitive practices); data that track fidelity (For example, data related to possible alternative explanations (things that could have an effect on outcomes but are not related to the actual experiment or intervention in question).

What are other effects of implementing a new practice that needs to be evaluated?

Dominic Cappello responds:

Our 100% New Mexico initiative will be testing many new innovative ways of increasing service quality and accessibility. The implementation and evaluation of any new program, practice or policy should also take into account the circumstances and secondary effects of implementing a new practice. When evaluating an experiment, it is important to consider not only whether the activities themselves are effective towards their goal, but: whether they are sustainable; what effects they have on staff morale; whether they require additional resources; and if they are applicable to address other challenges.

RETURN

Lesson 4:II

CONCEPTS

Lesson 4:IV

DATA TO ACTION

March 9 2020

Course – Lesson 4: II. Terms to Explore

Institute StaffLesson 4

CONCEPTS

TERMS TO EXPLORE

LESSON 4:II – WHAT ARE KEY CONCEPTS AND TERMS TO EXPLORE?

INTRODUCTION TO DATA INFORMING ACTION

In this session we focus on how data assist us in our efforts to implement the action phase of the CQI process. A data-informed CQI process provides us with an avenue to examine our program practice with an eye to improving outcomes for the children and families we serve.

As discussed in previous lessons, the process of using data often starts with asking the right questions. These questions help us develop data-informed solutions to problems. As we begin action, our questions continue to be of vital importance.

We will work to refine our hypothesis that directs our innovation and test all our assumptions about the area we are focusing on. Data also help us facilitate our innovation and make decisions as challenges arise during the action phase as new insights and problems may reveal themselves.

Data help a local action team move through the action step of the CQI process, helping us see where modifications to the innovation may be needed. Further, data can identify where problems exist in an agency, organization or community process. Data allow us to assess the magnitude of a problem and what resources or procedures might exist to solve the problem. Data inform course corrections that might be needed during the innovation implementation phase. The length of the action phase can vary depending on the size and complexity of the problem.

RETURN

Lesson 4:I

ACT

Lesson 4:III

TESTING

March 9 2020

Course – Lesson 4: I. Act with Partnerships

Institute StaffLesson 4

ACT

ACT WITH PARTNERSHIPS

LESSON 4:I – WHAT ARE THE LEARNING OBJECTIVES OF THIS LESSON?

WHAT ARE WE LEARNING?

Welcome to Lesson 4: Action. By now you’ve grounded yourself in assessment and planning and are ready to explore what it means to take on data-driven action. When a project begins, data help a local office move through the action step of the CQI process, helping you see where you might need course corrections.

We always face unintended consequences, so being informed and flexible help tremendously during the implementation phase of any project. Depending on what type of course project you choose to take on, the length of the action phase can vary depending on the size and complexity of the problem and solution.

In this session, you will be introduced to the action phase of CQI and how data will help you monitor success in the implementation phase. This section will also focus on testing a hypothesis. Participants also can assess their capacity to be change agents.

Participants completing the classroom, coaching and online lesson will be able to:

Document innovation design, sources of data, and the people responsible for data collection and analysis
Demonstrate the ability to create a logic model with hypothesis, activities and outcomes
Demonstrate ability to test a hypothesis
Demonstrate action phase for local project
Use data to assess progress with the activities that are part of an innovation
Describe how data has been used for the assessment, planning and action phases of their local project
Communicate with local leaders and stakeholders about change and being change agents
Communicate with local office staff the goals of the third phase of the local project/innovation: action