Indicator of effectiveness

After performing the literature review, WYSAC researchers found that multiple studies evaluating the same strategy often had different research designs, and/or evaluated different substances, age groups/populations, and outcomes. Some studies evaluated strategies individually, whereas others evaluated strategies as part of multi-component interventions that included several additional strategies. Therefore, they had to determine a way to assess the effectiveness of the evidence that could account for variations in how the strategies were evaluated.

To address the complexity of evaluating the effectiveness of the evidence, researchers developed a color-coded indicator. The color of the indicator represents the effectiveness of the strategy based on the available literature. The following provides a description of each color-indicator:

Effective

A green indicator implies that for the specific substance, population, and/or outcome evaluated in the literature, the strategy was found to have a statistically significant (p<.05) effect in the desired direction.

Varied evidence of effectiveness

A yellow indicator implies that the evidence base has varied results regarding the effectiveness of the strategy. For example, research may support the effectiveness of the strategy when used for one substance, but not for another—or for one population, but not another. Additionally, research may support the effectiveness of the strategy when evaluated for one outcome, but not for another. A yellow indicator may also signal that the strategy was found to be effective as part of a multi-component intervention, but not as a stand-alone approach, or vice versa. A strategy with a yellow indicator implies further investigation is necessary to determine if the strategy is effective for the desired substance, outcome, and/or population.

Not effective

A red indicator implies that the strategy was not found to have a significant effect in the desired direction for any of the substances, outcomes, and/or populations reviewed in the evidence base.

Although the indicator is intended to reflect the effectiveness of a given strategy based on the most-prominent, current literature available, it should not be viewed as a static, unchanging symbol indicating the usefulness of all environmental prevention strategies. When selecting a prevention strategy, a prevention professional must consider the level of readiness of their community, the cost-effectiveness of each strategy, and the current prevention programming available in their community. WYSAC researchers hope this tool serves as a first step toward identifying prevention strategies that may work within a specific community for a specific goal.