Analysis of treatment response data from eligibility designs

In this paper, we develop and compare two alternative approaches for calculating the effect of the actual intake when treatments are randomized, but compliance with the assignment in the treatment arm is less than perfect for reasons that are correlated with the outcome. The approaches are based on different identification assumptions about these unobserved confounders. In the first approach, which stems from [Sommer, A., Zeger, S., 1991. On estimating efficacy in clinical trials. Statistics in Medicine 10, 45–52], the unobserved confounders are modeled by a discrete indicator variable that represents subject-type, defined in terms of the potential intake in the face of each possible assignment. In the second approach, confounding is modeled without reference to subject-type in the spirit of the Roy model. Because the two models are non-nested, and model comparison and assessment of the approaches in a real data setting is one of our central goals, we formulate the discussion from a Bayesian perspective, comparing the two models in terms of marginal likelihoods and Bayes factors, and in terms of inferences about the treatment effects. The latter we calculate from a predictive perspective in a way that is different from that in the literature, where typically only a point summary of that effect is calculated. Our real data analysis focuses on the JOBS II eligibility trial that was implemented to test the effectiveness of a job search seminar in decreasing the negative mental health effects commonly associated with job loss. We provide a comparative analysis of the data from the two approaches with prior distributions that are both reasonable in the context of the data and comparable across the model specifications. We show that the approaches can lead to different evaluations of the treatment.