Module 4 – Assess, Scale, and Sustain

Module 4 will teach you:

• To assess change (outcome evaluation)
• How to report on your wider impact
• To think like a marketer and how you can do it better the next time
• About program sustainability

We have arrived at Module 4. This step is one of the most important in a social marketing program design process, yet it is often incompletely done, or even forgotten altogether. Step 4 of the social marketing program design process generates and provides evidence of the effectiveness of the designed social marketing program. It links directly back to Module 1, when we set out to understand whether we have achieved our aims. Now, the original program objectives and actual achievements come together. So, remind yourself: what were your objectives, and what other set Key Performance Indicators (KPIs) need to be ¾externally and internally¾reported?

 

What, when, where and how to assess change?

Assessing change in the target audience is known as outcome evaluation—it is the measurement of the results of your efforts. Equally important is how well the program was delivered—this is called process evaluation. Assessing both allows you to determine whether a poor outcome within a particular group (e.g. no behaviour change) was because the program itself was not effective, or it was not implemented correctly. If we return to the Road Crew example, perhaps your outcome measures indicated that drink driving offences and crashes have reduced significantly in and around 20 towns, but only marginally in five other towns. Your process evaluation indicates that brochures were not delivered on time in three out of those five towns, or due to mechanical failure and driver unavailability, limousines were not available in two towns for one month during your program. This suggests that in those five towns, program elements were not available at all times; sometimes individuals were not aware of the program; and these factors affected program uptake and subsequent efficacy. For further readings, we recommend Moore et al. (2015), who offer a framework which emphasises the relations between implementation, mechanisms, and context. For example, implementation of a new intervention will be affected by its existing context, but the intervention may also, in turn, change aspects of the delivery context.

 

Assessing effectiveness includes measuring the knowledge, attitudes, and most importantly, the behaviour of target audiences. In order to measure changes over time, it is necessary to question your target audience about these on at least two separate occasions¾before and after the program. Ideally, following and understanding short- to long-term behavioural changes by asking participants the same questions at, for example, three, six, and even 12 month intervals, you will better understand short- and longer-term effects of the social marketing program.

 

Figure 22 – What, when, where and how, who

In addition to measuring changes in our target audiences’ behaviour during the social marketing program, it is essential to evaluate the experiences of front-line staff as well as the feedback from our target audience regarding the program itself. Useful questions will go beyond what people enjoyed (satisfaction measures), or what worked well, to include the thoughts of all stakeholder groups involved.

 

I was told that I need a control group?

Scientific research design will also feature a control group, i.e. a group that has not taken part in the program but whose behaviour is assessed in the same way as the ‘intervention’ group. Control (or comparison) groups are important to attribute program success to the intervention itself, and not to other environmental effects extraneous to the social marketing program. Talk to us if you need some help on setting up an empirical evaluation for your program.

 

Why do we need to evaluate our social marketing program?

Evaluation of the social marketing program allows us to measure the effect of the intervention, and determine where and how we can improve our social marketing program for its future use in other groups. Measuring the effect of the program allows us to demonstrate a return delivered to funding bodies, and provides the rationale for seeking further or new funding to implement longer-term and self-sustaining social marketing programs. Funding bodies will demand a return on their investment even when that return is social good. This is true for universities, government organisations, non-government organisations, corporate social responsibility initiatives, and charities. Successful social marketers work to a high standard of evaluation to provide an understanding of what worked, and didn’t work, well. These insights then help us to understand how we can deliver better social marketing programs in the future. So what should we be reporting on?

 

• Where your money has been invested, or what you have produced, i.e. your social marketing program, promotional items, equipment and resources.
• The calculated impacts¾tangible and intangible¾on the social issue that you have been working on. For example, health care savings from a smoking cessation program (monetary) or implementation of a program into the national school curriculum (non-monetary).

Excurse: Calculating the Return on Investment

In business, we frequently hear the term “Return on Investment” (ROI). In simple terms, the ROI is a performance measure to evaluate the efficiency of an investment. The ability to provide an ROI is a selling point that gives funding agencies and boards of management substantive evidence of return on their invested money. To calculate ROI, only the following information is needed (adapted from Lee & Kotler, 2015):

• Money spent to implement the program (all costs)
• The level of behaviour change achieved (how many people)
• Benefit of one changed behaviour

Lee and Kotler (2015) provide a compelling example of a three year quit-tobacco program demonstrating indisputable ROI:

• All costs: US$3 Million
• 30,000 smokers called the quitline over thr34ee years; 3,900 (13%) quit smoking during the program
• Public health care costs of one smoker per year (US$1,800) multiplied by three years = US$5,400

Interesting fact: We can also calculate the costs of influencing one tobacco user to quit for at least three years by dividing 3,900 “quitters” by the US$3M invested, leaving us with a US$770 “dividend”, far outweighing the public health care costs for one smoker!

 

In summary, we can state that while the three million dollar investment in the program might seem a big outlay, the return on investment in both health and financial terms significantly outweighs this cost. This is why the ROI calculation is a measure of success that is far more convincing than simple outcome assessments. Talk to us if you need help to calculate your ROI footprint.

Writing up your results

Lastly, when social marketers make their learnings available and share them with the wider community other change agents gain leverage from their findings. Evaluation findings should be documented and reported. What should this report look like? The following key headings indicate the minimum that should be featured and expanded upon:

 

• Background
• Aims and objectives
• Methodology (design, participants, measures, analysis)
• Evaluation (assessment)
• Recommendations

 

Assess, scale and sustain

Change happens over time, and solutions need to be implemented sustainably. How likely is it that your program (or at least components of it) will continue beyond the funding running out? Can the program be developed into a sustainable business model which is self-funding (e.g. social enterprise)? Will someone manage the program, or are there earmarked follow-up funds or resources (e.g. grants, sponsorship, volunteers) to give the program longevity? Perhaps among your stakeholders are those who could manage certain components of your program¾think particularly about those whose objectives are similar or shared most closely.

 

Another approach to ensuring the sustainability of your social marketing programs is to design them as small, self-sufficient business units. While this is not easy¾diligence, planning and proper execution is necessary¾ it is a powerful way of cutting through in the social change business:

 

Watch this space for our new social entrepreneurship module coming your way soon!

 

Last words

Once you have managed your first delivery of a social marketing program, it is important to take your insights and all that you have learned to do it again, and better. The learning curve is steep, and in the game of long-term behaviour change you need to stay aware of trends, continuously monitor and tweak your program, and your social marketing strategies and tactics. Change is constant, not only in the environment in terms of technology, the economy and political systems¾our target audience is always changing. We want you to succeed, so please do not hesitate to get in touch with us and see how, together, we can achieve change for good.

 

Thanks for taking part in Social Marketing @ Griffith’s Masterclass. Please stay tuned for more modules, workshops and case studies.