Overview of MOST | The Methodology Center

Overview of MOST

Black BoxWhat is the multiphase optimization strategy (MOST)?

MOST is a comprehensive, principled, engineering-inspired framework for optimizing and evaluating multicomponent behavioral interventions. MOST includes a randomized controlled trial (RCT) for intervention evaluation, but unlike the standard approach to intervention development, also includes other phases of research before the RCT. These phases of research are aimed at intervention optimization using criteria selected by the scientist. The goal may be to develop a cost-effective intervention, an intervention that achieves a specified level of effectiveness, the briefest intervention that achieves a minimum level of effectiveness, or any other reasonable and explicitly operationalized goal. The MOST framework relies heavily on resource management by strategic use of highly efficient experimental designs. MOST is designed to be practical, and holds out the possibility of achieving more rapid long-run improvement of interventions without requiring a dramatic increase in intervention research resources.

 

 

Where can I read more about MOST?

View the MOST recommended reading list.

 

 

What qualifies as a “multicomponent behavioral intervention”?

A multicomponent behavioral intervention is any multicomponent intervention in which at least one of the components is behavioral. In other words, an intervention that combines behavioral and pharmaceutical components would be considered behavioral according to this definition.

 

 

Do biological, pharmaceutical, or medical interventions qualify as behavioral interventions? For example, what about pre-exposure prophylaxis (PrEP) for prevention of HIV?

There are many important behavioral factors related to biological, pharmaceutical, and medical interventions. These include compliance, adherence, and management of side effects. In our view, as soon as a biological intervention includes even a single component aimed at addressing one of these behavioral factors, it becomes a multicomponent behavioral intervention.

 

 

What is an intervention component?

In MOST we define an intervention component as any part of an intervention that can reasonably be separated out for study. This definition is meant to be both broad and practical, because what constitutes a component can be very different in different situations. A component can be a part of the content of the intervention, such as topics in a drug abuse prevention program; a feature of the intervention that promotes compliance or adherence, such as reminder phone calls; features aimed at improving fidelity, such as a phone number for program delivery staff to call with questions; or any other aspect of a behavioral intervention.

 

An intervention scientist conducts MOST because he or she needs to make decisions about what to include in an intervention. If you are asking yourself “Should I include X?” then X can probably be considered a component of the intervention.

 

An intervention component can impact efficacy, effectiveness, and/or cost-effectiveness. To contrast some very different takes on intervention components, read the articles in the Implementations of MOST section of the recommended reading list. 

 

  

Why is some of the terminology used on this web site different from the terminology used in the articles in the recommended reading list?

The MOST framework has been refined to make it more consistent with the goals and objectives of intervention science, as opposed to engineering. Beginning in about 2014, articles reflect this change.

 

 

What are the phases of MOST?

Figure 1 below is a flow chart depicting the three phases of MOST: preparation, optimization, and evaluation. MOST is a framework, not an off-the-shelf procedure, so the details can vary somewhat from study to study. This means that a variation on this flow chart may do a better job of describing any particular application of MOST.

 

The material below is adapted from Collins, Nahum-Shani, and Almirall (2014).

 

The flow chart begins with an information input/output box; the list inside reads: “Theory.  Scientific literature.  Clinical experience.  Data analysis results.  Other.”   Then proceed to an action box with Step 1 “Establishment of theoretical model.”   Next is an action box: Step 2 “Identification of set of intervention components to be examined.”    Next is an action box: Step 3A “Experimentation to examine individual intervention components.  Step 3A leads to Step 3B.  An alternate path leads to the information input/output box “Centralized database.”    The information input/output box “Centralized database” is outside of the main steps of the flow chart.  The information input/output box “Centralized database” in turn leads to an action box also outside of the main flow chart steps.  The action box reads “Exploratory data analyses.”  The arrow from this action box returns to the very beginning of the chart with the information input/output box: “Theory.  Scientific literature.  Clinical experience.  Data analysis results.  Other.”  From Step 3A, proceed to Step 3B, the action box “Refinement via experimentations and other methods.”  This action is optional.  Step 3B is followed by Step 4.  Step 3B also has an alternate path to the information input/output box “Centralized database.”  Step 4 is an action box “Assembly of beta intervention.”  Step 4 leads to a decision point: “Is beta intervention expected to be effective?”.  If the answer is no, then return to Step 1 “Establishment of theoretical model.”  If the answer is yes, then proceed to Step 5.  Step 5 is an action box “Confirmation of effectiveness of beta intervention via RCT [randomized controlled trial]”.  Step 5 has two arrows.  One leads to the information input/output box “Centralized database.”  The other arrow from Step 5 leads to a decision point: “Is beta intervention effective?”.    If the answer to this question is no, then return to Step 1 “Establishment of theoretical model.”  If the answer is yes, proceed to Step 6.    Step 6 is a product “Release of new intervention.”  This is the end of the flow chart.

Download this image. You may use it in presentations, online or in publications free of charge.

 

The preparation phase 

The purpose of the preparation phase is to lay the groundwork for optimization of the intervention. Information from sources such as behavioral theory, scientific literature, and secondary analyses of existing data is used to form the basis of a theoretical model. This model is critical for guiding decisions in MOST, in particular, the selection of which intervention components to examine. Any pilot testing of intervention components (highly recommended) is done in this phase.   

 

An essential part of the preparation phase is identifying and operationalizing a clear optimization criterion. This is a definition of the end product to be achieved by optimizing the intervention. One straightforward optimization criterion is simply “no inactive components.” Constraints such as limits on cost, time, and participant logistical or cognitive burden can be incorporated into the optimization criterion, but they must be explicitly identified and operationalized. For example, a project funded by the National Institute of Diabetes and Digestive and Kidney Diseases (B. Spring and L. Collins, co-PIs) is using MOST to develop the most effective weight reduction intervention that can be implemented for $500 per person or less.

 

The optimization phase

As the name implies, in this phase the investigator optimizes the intervention. This involves selecting the components and component levels that make up the intervention that meets the optimization criterion. It is necessary to gather empirical information to do this. The approach depends on what information is required. Very often estimates of the individual and combined effectiveness of a set of intervention components are required for optimization. This information is typically gathered by means of an efficient randomized experiment, such as a factorial experiment, fractional factorial experiment, or sequential, multiple assignment, randomized trial. However, we emphasize that any experimental design can be used, provided it has been selected based on the resource management principle. The results from this experiment, possibly augmented by secondary analyses on the experimental data, form the basis for making decisions about component selection and formation of the optimized intervention. 

 

At the end of the optimization phase, the investigator has selected the intervention components and component levels that make up the optimized intervention. Figure 1, above, shows a diamond immediately after the optimization phase, indicating that a decision is required. At this point the investigator has a rough sense of the likely overall effectiveness of the optimized intervention. If, based on the effect size estimates obtained in the component selection experiment, it appears that the optimized intervention will have a sufficiently large effect to justify evaluating it with an RCT, it would make sense to proceed to the evaluation phase. On the other hand, the results of the component selection experiment may indicate that the optimized intervention is likely to have a very small effect. For example, the optimization criterion may call for an intervention comprised of only active components, and perhaps the experiment indicates that there is only one such component. In this case, it may not make sense to proceed to the evaluation phase. As Figure 1 indicates, it may be advisable to return to the preparation phase and reconsider the theoretical model, pilot test some new intervention components, and so on. 

 

Note: If the intervention under consideration is a time-varying adaptive intervention, it is known that all the components to be included are effective, and intensive longitudinal data are being collected, it may be possible to optimize using ideas from engineering and computer science. For more about this, see the Other approaches to optimization section of recommended reading, Daniel Rivera’s web site, and Susan Murphy’s web site.

 

The evaluation phase

The evaluation phase consists of a standard RCT comparing the optimized intervention to a suitable control or comparison condition. As Figure 1 shows, there is another decision point immediately after the evaluation phase. If the RCT indicates that the optimized intervention is not effective, then it is necessary to return to the preparation phase and reconsider the theoretical model or the approach to intervention. If the RCT indicates that the optimized intervention is effective, it can be released to the public.  

 

Figure 1 shows arrows running from the optimization and evaluation phases to a rhombus on the left labeled “Data for future secondary analyses.” This indicates that the data gathered in these phases contain much information that can be used to inform the preparation phase of future cycles of MOST.  

 

 

What is the resource management principle?

The resource management principle is one of two fundamental principles underlying MOST (the other is the continuous optimization principle). According to this principle, research resources—money, experimental subjects, time, equipment, personnel, etc.—must be managed strategically to move intervention science forward fastest. Sometimes this will mean thinking creatively about the following:

 

  • using experimental designs other than the standard RCT and its relatives, when it is more efficient to do so;
  • taking calculated risks to move science forward faster;
  • conducting experimentation in a sequential way so that the results of one experiment inform the planning and execution of the next one; and
  • doing what will move intervention science forward fastest and best in the long run, even if in the short run it means somewhat slower progress, or the appearance of slower progress.

 

 

What is the continuous optimization principle?

The continuous optimization principle is one of two fundamental principles underlying MOST (the other is the resource management principle). According to this principle, optimization is an ongoing process. Every behavioral intervention, including the theoretical model that provides its conceptual foundation, is a perennial work in progress. Once an intervention has been optimized, a cycle of MOST should be started again to effect further improvements. An intervention can always be made more potent or more efficient.

 

Return to MOST home page.

 

Read about establishing a theoretical model.

Follow Us