Establishing a Theoretical Model with MOST and Understanding Optimization
ESTABLISHING A THEORETICAL MODEL
A good theoretical model should specify
- the determinants of and influences on the health behavior or outcome to be impacted by the behavioral intervention;
- all important mediators and moderators;
- what psychosocial and/or biological theories inform which parts of the model--in most cases there will be several theories that inform a theoretical model;
- which intervention components are aimed at which mediators; and
- which intervention components, if any, are expected to interact with each other.
Sometimes the theoretical model is not a model of a health behavior per se, but a model of maintaining treatment fidelity, promoting adherence or compliance, or the like. For an example, see Collins, Kugler, and Gwadz (2015).
It is a lot of work to develop a solid theoretical model, but thoughtful development of a comprehensive theoretical model is absolutely essential for MOST. The theoretical model
- provides a conceptual foundation for the intervention;
- is based on prior scientific literature and empirical findings;
- identifies gaps in prior scientific literature;
- informs the choice of intervention components and shows clearly what role each component is to play in the intervention process;
- informs the choice of the design for the component selection experiments;
- in situations where the outcome of interest occurs far in the future, provides a rationale for selection of more proximal outcome measures for use in component selection;
- guides the specification of a priori hypotheses; and
- provides a framework for interpretation of unexpected results; sometimes such results will suggest alterations to the theoretical model (see "What is the continuous optimization principle?").
Well-thought-out, comprehensive theoretical models are often publishable.
What is meant by “optimized”?
According to the online Concise Oxford Dictionary of Mathematics, the optimized solution is “the best possible solution… subject to given constraints.” Note that optimized does not mean best in an absolute or ideal sense. The constraints are always part of the definition.
What is an optimization criterion?
It is necessary to identify an optimization criterion in order to optimize a behavioral intervention. The optimization criterion is an operational definition of what is meant by “best possible subject to constraints.”
Here are a few examples of possible optimization criteria. For all of these, one set of constraints is the set of intervention components under consideration.
- An intervention with no inactive components
- Most efficacious/effective intervention that can be delivered without exceeding some upper limit on cost. For example, you might know that the intervention you are delivering will be practical only if it costs less than $100/person to implement. Using MOST it is possible to select a set of intervention components that will give you the largest effect subject to the constraint of this upper limit on cost.
- Most efficacious/effective intervention that can be delivered without exceeding some upper limit on time. For example, you might know that the intervention you are delivering will be practical only if it takes less than 15 minutes of clinic time per person to implement. Using MOST it is possible to select a set of intervention components that will give you the largest effect subject to the constraint of this upper limit on time.
I’ve already evaluated my intervention, and it has a statistically significant effect. What is the difference between this and optimization?
Evaluation and optimization are different. Both are important. Evaluation is done to determine whether an intervention has a statistically significant effect. Optimization is done to build an intervention that meets a prespecified optimization criterion (see "What is an optimization criterion," above).
Many interventions today have a statistically significant effect but have not been optimized. It is possible that they could be more effective or use fewer resources if they were optimized. It is also possible for an intervention to be optimized and not have a statistically significant effect. When an optimized intervention does not have a statistically significant effect, the optimization criterion, the approach to intervention, and/or the theoretical model need to be rethought.
For more about the difference between evaluation and optimization, see Collins et al. (2011). NOTE: this paper uses a previous formulation of MOST that has different phases.
Collins, L. M., Baker, T. B., Mermelstein, R. J., Piper, M. E., Jorenby, D. E., Smith, S. S., Schlam, T. R., Cook, J. W., & Fiore, M. C. (2011). The multiphase optimization strategy for engineering effective tobacco use interventions. Annals of Behavioral Medicine, 41, 208-226. PMCID: PMC3053423
So isn’t optimization just deciding what is “good enough”? How is this so different from the treatment package approach? Every clinical trial uses this “good enough” approach.
Simply saying “well, this is good enough” is not the same as optimization. Optimization involves
- identifying the ideal,
- identifying the constraints that are imposed in a given situation, and
- working in a systematic and principled manner to come as close to the ideal as you can while operating within the constraints.
In fact, optimized is not the same as “good enough;” the optimized product may not be good enough!
Let’s take a straightforward hypothetical example. Suppose you were charged with building and optimizing a GPS. The IDEAL solution would be to pinpoint your location. The way to come closest to this would be to spare no expense in building the GPS. However, suppose your charge is to build a GPS that costs $100 or less to manufacture. This constraint means that you cannot choose the most precise and, therefore, most expensive components for your GPS. In this example, optimization means finding the combination of components that comes closest to the ideal without exceeding the $100 limit. Maybe that means going with all moderately-priced components; maybe it means that one key component has to be on the expensive side and the others can be cheap.
Suppose the best GPS you could manufacture for $100—that is, the optimized GPS—can identify locations only within a 400 foot range. This might not be “good enough” even though it is optimized. Then the company might decide to change the constraints, for example, allow $150 in manufacturing costs.