Menu

Lesson 3

baby-parents12x6

Course – Lesson 3: VII. Next Steps

NEXT
NEXT ASSIGNMENTS

LESSON 3:VII – What are the lesson assignments and next steps?

The Steps You Take

Your steps may be different depending on your situation. Ideally, you are taking this course as part of a cohort — a group made up of local 100% New Mexico initiative participants. You may be the co-community organizer, action team leader or member. This is ideal, as you can discuss the course and all the information presented with people you’ll be collaborating with. However, you might be taking this course because you wish to start a 100% New Mexico initiative and today, you are a solo champion seeking to recruit stakeholders. Either way, the following steps enrich your learning experience and prepare you for Lesson 4: Act.

Visit the Virtual Innovation Center. We provide you with links to innovations in all ten key family-serving sectors. These innovations include a menu of various projects, from across the state, nation and globe to consider implementing. The projects identified are designed to strengthen family health, safety and learning systems, and support your local 100% Community innovation.

Reviewing your course textbooks. Ideally, you have a good supply of sticky notes and sharpie pens to make comments and questions throughout Anna, Age Eight, 100% Community, and Adaptive Leadership. This makes group discussion more lively and focused as you get answers to questions and challenge assumptions.

Reading Assignment: Course participants should finish reading the chapter on their particular sector from “Part 4: Workbook for Action Teams” in 100% Community before starting Lesson: 4. Act.

Meet with an initiative member (preferably from your sector) or work colleague to discuss the following questions focusing on your sector:

  1. In your sector, how well is the “supply” of services understood and easy to access online in your county (e.g. is there a directory of services)?
  2. In your sector, how well is the “supply” meeting the “demand” in your county (e.g. is there an unmet need)? How do you know?
  3. In the chapter that focuses on your sector, which of the innovations seem most promising to meet the specific needs of your county? What data do you currently have support this?

Convene with fellow initiative participants to discuss this lesson and the planning process. Meet with your fellow action team members to review and develop all potential projects and the logic models guiding them.

RETURN

Lesson 3:VI

CQI QUESTIONS

NEXT

Lesson 4:I

ACT

baby-headband-asleep12x6

Course – Lesson 3: VI. CQI Questions

CQI QUESTIONS
QUESTIONS FOR PLANNING

LESSON 3:VI – HOW DO CQI QUESTIONS GUIDE OUR PROJECT DEVELOPMENT?

What are the CQI questions for the planning phase?

These are the (unskippable) questions you and your action team members will need to answer as you identify innovations and projects.

Question 1: Which component of the challenge will you address with a local innovation?

Question 2: Describe your review of research focused on your challenge?

Question 3: What potential evidence-based solutions exist in the research?

Question 4: What is the hypothesis that illustrates how your actions might solve the challenge? (As in: “If you do A, then B will happen.”)

Question 5: What are your key steps, timelines, roles and responsibilities related to your innovation?

Question 6: What unintended consequences might you encounter?

RETURN

Lesson 3:V

HYPOTHESIS

NEXT

Lesson 3:VII

NEXT STEPS

laugharmsE12x6

Course – Lesson 3: V. A Hypothesis Guides US

HYPOTHESIS
A HYPOTHESIS GUIDES US

LESSON 3:V – Q+A WITH AUTHORS KATHERINE ORTEGA COURTNEY, PHD AND DOMINIC CAPPELLO

What do the hypothesis and assumption mean?

What do the hypothesis and assumption mean?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

A hypothesis is a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation. Assumptions are accepted cause and effect relationships, or estimates of the existence of a fact from the known existence of other fact(s). Although useful in providing basis for action and in creating “what if” scenarios to simulate different realities or possible situations, assumptions are dangerous when accepted as reality without thorough examination.

What is an experiment?

What is an experiment?

Dominic Cappello responds:

Dominic Cappello responds:

An experiment is a scientific procedure a scientific procedure undertaken to make a discovery, test a hypothesis or demonstrate a known fact.

What is the term theory?

What is the term theory?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Theory is a well-substantiated explanation of some aspect of the natural world that is acquired through the scientific method, and repeatedly confirmed through observation and experimentation. As with most forms of scientific knowledge, scientific theories are inductive; that is, they seek to supply strong evidence for but not absolute proof of the truth of the conclusion and they aim for predictive and explanatory force.

If a substantial amount of evidence is gathered that consistently suggests the validity of a hypothesis, the hypothesis can be converted into a theory.

How do data inform innovations?

How do data inform innovations?

Dominic Cappello responds:

Dominic Cappello responds:

Data are the information needed to disprove or prove hypotheses; show progress or regress toward measurement goals; and display results.

What are local experiments and innovations and what can they achieve?

What are local experiments and innovations and what can they achieve?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Local innovations and projects (often these terms are interchangeable) are designed to address a health or safety challenge informed by the assessment phase. They are researched, created and developed by local 100% New Mexico initiative action teams. The projects are implemented and tracked on the agency, city, community and/or county level.

How does one structure an innovation?

How does one structure an innovation?

Dominic Cappello responds:

Dominic Cappello responds:

To make sure all participants have a shared understanding of an experiment, it is best to create a plan with details about the objective, goals, and activities involved in the experiment. All should understand what changes are involved, the timeline, who plays what role, what data are to be collected, who is to collect the data, how often the data is to be reviewed and what the measurements are that will show progress or regress toward the goals.

How does one get buy-in for an experiment or innovation?

How does one get buy-in for an experiment or innovation?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Buy-in for an experiment, like any team project, requires communication. The need for an innovation should be based on the results of the assessment phase. Ideally, the action team will agree on a needed innovation based on the assessment, as well as the planning phase. Time constraints are always a concern so a description of how the experiment will be managed needs to be addressed, along with a timeline, how the experiment will be implemented and evaluated.

How can technology help manage an innovation?

How can technology help manage an innovation?

Dominic Cappello responds:

Dominic Cappello responds:

Technology affords us the means to research current experiments and evidence-based practices through the Internet. It also affords us the means to collect and store data to be analyzed for an experiment.

How does one create and test a hypothesis?

How does one create and test a hypothesis?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Before creating one’s own hypothesis, it is important to do research to find out if anyone else has tested the same hypothesis and, if so, what the results were.

Creating a hypothesis involves developing a clear statement of what the results will look like if the assumptions about the changes are implemented as described. For example, I predict that I will gain 10 pounds of weight after 3 months if I stop exercising and eat lots of fat. Then, I have to put that statement to the test and find out if I am correct once the 3 months have passed.

How does one evaluate an innovation?

How does one evaluate an innovation?

Dominic Cappello responds:

Dominic Cappello responds:

Evaluation of the experiment outlines details about where the experiment stands at a particular time regarding activities and what the measurements look like at this time. It gives us some ideas about how the experiment is going. In order to evaluate my experiment above, I will need to track my activities and diet over the three months as well as my weight. At the end of the three months, the data will show whether or not I have gained the 10 pounds as expected. I could also describe challenges that occurred during the three months.

RETURN

Lesson 3:IV

EVIDENCE

NEXT

Lesson 3:VI

CQI QUESTIONS

boy-camera12x6

Course – Lesson 3: IV. Evidence-informed Strategies

EVIDENCE
EVIDENCE-INFORMED STRATEGIES

LESSON 3:IV – Q+A WITH AUTHORS KATHERINE ORTEGA COURTNEY, PHD AND DOMINIC CAPPELLO

What are evidence-based and evidence-informed practices?

What are evidence-based and evidence-informed practices?

Dominic Cappello responds:

Dominic Cappello responds:

Evidence-based practices are approaches to prevention or treatment that are validated by some form of documented scientific evidence. This could be findings established through scientific research, such as controlled clinical studies, and other methods of establishing evidence that are validated. The study and findings have been published in a well-known peer journal.

Evidence-informed strategies use practice knowledge to guide program design and implementation. It might not be based on research and not always validated by science. This informed practice allows for innovation and incorporates the lessons learned from the existing practice.

What are best practice and promising practice and the differences between them?

What are best practice and promising practice and the differences between them?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Best practices have documentation behind them that validate a strategy, ideally more than one study that supports the efficacy of the practice.

Promising practices have a least one study that finds the practice to be better than a comparison practice or nothing.

How do data inform our understanding of effective practice and policy?

How do data inform our understanding of effective practice and policy?

Dominic Cappello responds:

Dominic Cappello responds:

Best practices will have data to back up the theory behind the practice.

Best practices can lead to policy that is clear and effective.

How can action teams find research, best practices and innovation within their sectors?

How can action teams find research, best practices and innovation within their sectors?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

There are many places to look for research. Our virtual innovation center provides all ten action teams with 10–20 projects that have the potential to increase access to services and quality of services. We recommend starting there and using the links provided to explore further innovations. These links to potential innovations are also provided in the book 100% Community in the workbook section (part 4).

How can one understand if a strategy successfully used in another state or jurisdiction can work locally?

How can one understand if a strategy successfully used in another state or jurisdiction can work locally?

Dominic Cappello responds:

Dominic Cappello responds:

One needs to compare all the assumptions of the hypothesis being considered for the innovation with the assumptions used in the other state to see how similar they are. This may determine whether the practice could be replicated in a local community (what works in Pittsburgh may not work in Roswell). Even though an idea is promising and initial data suggests it is appropriate, an innovation might not meet the cultural needs of a community.

How can data and research change policy?

How can data and research change policy?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Data and research can reveal policy and practice that do not work or could be improved. Through experiments and documentation of data related to practice, the workforce can improve practice and change policy.

How is policy made on the state level?

How is policy made on the state level?

Dominic Cappello responds:

Dominic Cappello responds:

It is important to have a good understanding of how public policy is created, changes, and evolves concerning services to individuals and families. It is what gives framework to procedures and practice by the workforce.
Public policy making can be characterized as a dynamic, complex and interactive system through which public problems are identified and countered by creating new public policy or by reforming existing public policy. Public problems can originate in endless ways and require different policy responses (such as regulations, subsidies, quotas, and laws) on the local, state, national or international level.

Public policy making is a continuous process that has many feedback loops. Verification and evaluation are essential to the functioning of this system.

In public policy making, numerous individuals and interest groups compete and collaborate to influence policymakers to act in a particular way. The large set of actors in the public policy process, such as politicians, civil servants, lobbyists, domain experts, and industry representatives, use a variety of tactics and tools to advance their aims, including advocating their positions publicly, attempting to educate supporters and opponents, and mobilizing allies on a particular issues. Since societies have changed in the past decades the public policy making system changed, too.

Today public policy making is increasingly goal-oriented, aiming for measurable results and goals, and decision-centric, focusing on decisions that must be taken immediately. Furthermore, mass communications and technological changes have caused the public policy system to become more complex and interconnected.
These changes pose new challenges to the current public policy systems and pressure them to evolve in order to remain effective and efficient. This dynamic process makes it important to collect, analyze and report data effectively.

RETURN

Lesson 3:III

LOGIC

NEXT

Lesson 3:V

HYPOTHESIS

kids-chalk12x6

Course – Lesson 3: III. Logic Guides Us

LOGIC
LOGIC GUIDES US

LESSON 3:III – Q+A WITH AUTHORS KATHERINE ORTEGA COURTNEY, PHD AND DOMINIC CAPPELLO

What is a logic model, and how is it used to guide the development and implementation of an experiment or initiative?

What is a logic model, and how is it used to guide the development and implementation of an experiment or initiative?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

A logic model is a visual representation of your innovation that combines graphic elements and text to show the logical flow of your experiment, from problem identification and planning through implementation and evaluation. It helps organize the steps and participants in the process so that each component has a purpose, and needs are anticipated for each stage of the process.

What are the first steps in creating a logic model?

What are the first steps in creating a logic model?

Dominic Cappello responds:

Dominic Cappello responds:

The life of a logic model begins with identifying the purpose and goals of the project. These are often statements identifying a need and should relate back to your assessment phase. A purpose is related to the broad problem you wish to address: “Improve access to behavioral health in my county.”

Goals are a little bit different. The goals stated in your logic model should be directly related to your purpose but also more specific. Using the example above, one might identify one goal as “Reduce staff vacancy rate in existing providers to 10%, sustained for a 12-month period.”

Logic model goals do not have to be so specific and measurable, however. They can also be items thought to contribute to the problem you are addressing in your purpose statement, such as “Increase employee morale” or “Create more office accountability and better communication.” It’s okay if you don’t yet know just how you’re going to do all of the above; that’s where the logic model comes in.

What are logic model "inputs" and how are they identified?

What are logic model "inputs" and how are they identified?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Put very simply, inputs are all the things (and people) that you currently have that are related to both the problem and the (potential) solution. Even when they are ostensibly linked to the challenge you’re addressing, inputs are still resources. For example, if you wanted to consider the development of a school based behavioral health care center, the following stakeholders would be considered “inputs.” School board, superintendent, school communities, behavioral health care providers and funders who might be city or county government.

What are logic model "activities" and how are they identified?

What are logic model "activities" and how are they identified?

Dominic Cappello responds:

Dominic Cappello responds:

Activities are the specific steps/actions that move your experiment forward and should be performed by, about, with, or “to” the inputs. Each of the inputs in your logic model should be linked in some way to at least one activity.

An activity for the staff retention challenge discussed above may be an employee survey to discover areas of concern related to work conditions and office climate/culture. This may be created by research and development within the entity (Input A); administered by management within the entity (Input B) to the rest of the entity staff (Input C); and shared with CQI training coaches and colleagues (Input D). The team conducting the experiment may discuss the survey results within the context of an academic research article about similar issues (Input E).

What are logic model "outputs" and how are they identified?

What are logic model "outputs" and how are they identified?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Outputs are the specific things that result from Inputs and Activities combining together to create change. In examples shown in this course, the survey results are an output.

For experiments with the goal of implementing a new kind of practice in the office, that new practice will likely be the output of the activities. The outcomes that we hope to get from the new practice are a different component of the logic model.

It is always a good idea to look ahead and anticipate outputs so that you can be sure that the data you need for evaluation is somehow included. For example, if you will need data from a tracking tool to show how your experiment was applied, this data should be included as an output stemming from the activity of developing and using a tracking system (and by which inputs).

What are the different levels of outcomes in a logic model?

What are the different levels of outcomes in a logic model?

Dominic Cappello responds:

Dominic Cappello responds:

In a logic model, outcomes — short-term, intermediate, and long-term — are the sequential changes (and, we hope, improvements) that you expect will happen as a result of your activities and outputs.

Short-term outcomes generally should occur within 2-6 months (or less or more, depending on your project) and may be described as “individual-level” outcomes. These types of changes might involve staff, stakeholders, and/or clients, and probably won’t go beyond changes in knowledge, attitudes, beliefs, behaviors; or small changes in workflow, practice, or protocols. An example of a short-term, individual-level outcome (again using the staff retention example) might be: school board members agreeing to allow meetings discussing the importance of behavioral health care in schools.

Intermediate outcomes will likely take between 6 months and 2 years to appear and are sometimes called “agency-level” outcomes. With ongoing application of your experiment or initiative, you may start to see improvements in the “target numbers” related to your overall challenge at this point. Intermediate outcomes may also include consistent use of a new practice (an all-new “business as usual”); stronger relationships with stakeholders; and secondary gains (or unintended consequences) related to your innovation, such as improved collaboration or need for more resources. Using the example of a school based behavioral health care center being developed, it might be that a school principal has agreed to be a pilot site for such a center, and funding is being identified.

Long-term outcomes should relate most closely with the stated goal(s) of your logic model. These outcomes may take 2 years or more to materialize but should be well-established and consistent. At this point, if your experiment has continued apace, you should hope to see positive changes in your specific challenge area and possibly improvements in related areas. For the development of a school based behavioral health center a long term outcome would be that the doors are open in the center and students and families are receiving behavioral health care.

How are outputs different from outcomes?

How are outputs different from outcomes?

Katherine Ortega Courtney, PhD responds:

Katherine Ortega Courtney, PhD responds:

Making the distinction between outputs and outcomes is easy in theory, but as you start to build your logic model it may feel murky at times. A good way to know for sure whether something is an output vs. an outcome is to ask this question: “Can I prove that this happened due to an activity we did, and not because of something else?” If the answer is yes (such as, you are holding survey results in your hand that came from the survey you administered), it is an output. If the answer is no (such as, your coworkers reported better job morale in a follow-up survey)… it is an outcome.

RETURN

Lesson 3:II

CONCEPTS

NEXT

Lesson 3:IV

EVIDENCE

blackandwhite-boy-locks12x6

Course – Lesson 3: II. Terms to Explore

CONCEPTS
TERMS TO EXPLORE

LESSON 3:II – WHAT ARE KEY CONCEPTS AND TERMS TO EXPLORE?

INTRODUCTION TO PLANNING WITH CQI

In this session we will continue to learn about CQI and explore how data assist us in our efforts to plan a new innovation, initiative, experiment or program. A data-informed CQI process provides us with an avenue to examine our program practice and community work. The focus is to improve outcomes for the children and families we seek to serve.

CQI helps a local office plan the entire innovation process — moving through assessment of a problem to planning a solution.

The more carefully thought out a plan, the better chance for conducting a successful local innovation.

In this session we will learn about evidence-based practice, policies, and programs in behavioral health, child welfare, public health, youth development, child development and education. We focus on research-based strategies. There are many sources for evidence-informed practice available through research on the Internet.

Policy analysis (the critical review of policy, procedures and programs) is a vital skill that helps us understand why certain strategies are called evidence-based while others may have little quality research to support them.

By using data and measurement, we are able to demonstrate how our practice, including the protocols we follow and the policies we work with, impacts outcomes for our children and families. The research conducted during this planning phase of CQI will play a vital role in informing the development of a local innovation.

In the planning phase, we also explore using logic models as a planning tool for our innovations and initiatives.

Our logic models, graphic representations of a project plan, create a shared vision for moving forward. Logic model components include:

Hypothesis: A statement that follows science-based thinking, “If we do A, then B will happen.”

Goal + Purpose: Identifying what you hope to achieve and why.

Inputs: What partners, resources and processes you need in place to achieve goals.

Activities: What data-driven activities you will implement to create change.

Outputs: What occurs or is created as a result of activities.

Outcomes: Short, intermediate and long-term measurable results. The planning phase prepares you to implement a science-based process with measurable and meaningful outcomes. The logical model guides the process.

RETURN

Lesson 3:I

PLAN

NEXT

Lesson 3:III

LOGIC

girl-nature-1200x600

Course – Lesson 3: I. Plan With Research

PLAN
PLAN WITH RESEARCH

LESSON 3:I – WHAT ARE THE LEARNING OBJECTIVES OF THIS LESSON?

planning-gallery-image1
planning-gallery-image1
planning-gallery-image2
planning-gallery-image3
planning-gallery-image4
planning-gallery-image5
planning-gallery-image6
planning-gallery-image7

What are we learning?

Welcome to Lesson 3: Plan. We know you’re quite skilled in planning and understand that the more thoughtfully a plan is designed, the better chance for a successful project. In this lesson we explore a variety of topics: evidence-based practice, research and identifying programs and policies that hold promise for your agency and city.

As you move through the lesson, we wish to support you in connecting the dots between your project’s hypothesis, goals, measurable activities and outcomes. Enjoy the learning.

In this lesson, you will be introduced to the planning phase of continuous quality improvement (CQI). You will explore programs and policies on evidence-informed strategies. You’ll engage in analysis of research to strengthen your ability to identify and analyze practice, protocols and policies. This section will also focus on creating a hypothesis and logic model.

Participants completing the classroom, coaching and online lesson will be able to:

  • Describe the second component of CQI: Plan
  • Demonstrate ability to plan
  • Articulate how CQI relates to strategic planning
  • List ways that CQI can affect local office practice
  • Understand the role data plays in planning
  • Create a hypothesis (if we do “A” then “B” will happen) based on research
  • Demonstrate the ability to describe a logic model with hypothesis, activities and outcomes
  • Demonstrate the planning phase for a local project
  • Articulate the difference between evidence-based and evidence-informed practices
  • Articulate difference between best practice and promising practices
  • Identify benefits of evidence-informed practice
  • Identify trusted and well-known sources for behavioral health, child welfare, public health, youth development and education research
  • Identify research to inform solutions
  • Describe how data may inform policy
  • Create a hypothesis
  • Conduct the planning phase for local innovation
  • Communicate the course process with local staff the goals of the second phase of the local project/innovation: planning

RETURN

Lesson 2

ASSESS

NEXT

Lesson 3:II

CONCEPTS

0