Practical Questions About Implementing MOST

I’ve already collected a bunch of data. Can I use MOST to analyze this existing data set?

Black BoxNo. MOST is not a data analysis technique. It requires carrying out experiments.

 

However, secondary analyses of data, for example data from previously conducted RCTs, can be very informative in planning how you will use MOST to optimize and evaluate a behavioral intervention. For example, such analyses can provide important leads for intervention components to be tested.

 

 

MOST seems like a great idea in theory, but I do not see how it can be conducted without massive resources. It just does not seem practical.

By definition, MOST works using the resources at hand, so it is meant to be very practical. This is not to say that MOST is suitable for every situation. But before concluding that it is not suitable for your situation, it would be good to take careful stock of the resources you have at your disposal for intervention development and evaluation, and think creatively about this question: How can these resources be leveraged to move intervention science forward fastest and best? See "What is the resource management principle?".

 

 

How can MOST be conducted within the five-year limit on NIH grants, and it is necessary to do so?

Whether you can conduct a cycle of MOST within five years is hugely dependent on the area in which you are working. There are several factors that influence this:

  • How far along the field is. In some cases, there is a well-accepted theoretical model in place, and the intervention components have already been selected and pilot tested when a grant proposal is written. This means that the project can start with Step 3A in MOST. In other cases, the theoretical model must be developed before anything else can be done.
  • The lag time between when the intervention is implemented and when it makes sense to assess the primary outcome. In areas like smoking cessation, the outcome can be assessed soon after the intervention is over. In areas like weight loss, it may be necessary to wait longer. (In some areas there can be a lag of years and a somewhat different approach may be required; see "What should I use as an outcome variable for the component selection experiment?")
  • The rate at which subjects can be obtained. When a universal intervention is being developed for first-year college students, it is often possible to work with a cohort of all first-year students at a university. By contrast, in clinic-based interventions it is usually necessary to wait for subjects to “trickle in.”

 

In the study described in Baker et al. (2011) and Collins et al. (2001) we successfully proposed conducting MOST up to and including Step 4, with the idea that we would propose the RCT in a subsequent application. So it may not be necessary to finish an entire MOST cycle in one five-year grant.

 

It may be desirable not to propose the RCT in the same funding cycle anyway. In my experience NIH reviewers tend to get nervous if there are any uncertainties in the application (e.g., “If A happens we will do X, but if A does not happen we will do Y”). There is no guarantee that when you reach Step 4 of MOST you will have a viable set of intervention components ready for an RCT (just as in the treatment package approach, there is no guarantee that the RCT will demonstrate a significant treatment effect). This uncertainty might be more than reviewers can tolerate. You can propose to go up to, say, forming the beta version of the intervention, and say you will propose an RCT in a subsequent application.

 

References

Baker, T. B., Mermelstein, R. J., Collins, L. M., Piper, M. E., Jorenby, D. E., Smith, S. S., Schlam, T. R. Cook, J. W., & Fiore, M. C. (2011). New methods for tobacco dependence treatment research. Annals of Behavioral Medicine, 41, 192-207. PMCID: PMC3073306  View abstract

Collins, L. M., Baker, T. B., Mermelstein, R. J., Piper, M. E., Jorenby, D. E., Smith, S. S., Schlam, T. R., Cook, J. W., & Fiore, M. C. (2011). The multiphase optimization strategy for engineering effective tobacco use interventions. Annals of Behavioral Medicine, 41, 208-226. PMCID: PMC3053423  View abstract

 

You may be interested in Resources for planning a grant involving MOST.

Follow Us