Statistical Assessment of the Replicability of Scientific Results
Presenter: Marina Bogomolov
When: Tuesday, July 12, 2016 Time: 9:00 AM - 10:30 AM
Room: Salon A Carson Hall (Level 2)
Session Synopsis:
Assessing replicability of findings across two studies of multiple features
Replicability analysis aims to identify the findings that replicated across independent studies that examine the same features. These features can be single-nucleotide polymorphisms (SNPs) examined for associations with disease, genes examined for differential expression, protein pairs examined for protein-protein interactions, etc. We provide powerful novel replicability analysis procedures for two studies with a guaranteed control over false replicability claims. We consider both the case where one is interested to find the results that replicated from a primary to a follow-up study, and the case where one is interested to assess replicability of findings across two studies with no division into primary and follow-up. Each of the proposed procedures computes for each of the examined features a number, the r-value, which quantifies the strength of replication.
Statistical Assessment of the Replicability of Scientific Results
Presenter: Qunhua Li
When: Tuesday, July 12, 2016 Time: 9:00 AM - 10:30 AM
Room: Salon A Carson Hall (Level 2)
Session Synopsis:
A REGRESSION FRAMEWORK FOR ASSESSING COVARIATE EFFECTS ON THE REPRODUCIBILITY OF HIGH-THROUGHPUT EXPERIMENTS
The outcome of high-throughput biological experiments is affected by many operational factors in the experimental and data analytical procedures. Understanding how these factors affect the reproducibility of the outcome is critical for establishing workflows that produce replicable discoveries. In this work, we propose a regression framework, based on a novel cumulative link model, to assess the covariate effects of operational factors on the reproducibility of findings from high-throughput experiments. In contrast to existing graphical approaches, our method allows one to succinctly characterize the simultaneous and independent effects of covariates on reproducibility and to compare reproducibility while controlling for potential confounding variables. By establishing a connection between our regression model and Archimedean copula models, we develop a procedure to choose functional forms of the regression model and provide an interpretation of the regression model in the context of multivariate dependence models. Using simulations, we show that our method produces calibrated type I error and is more powerful in detecting the difference in the reproducibility than existing measures of agreement. We illustrate the usefulness of our methods using ChIP-seq and microarray studies.
Statistical Assessment of the Replicability of Scientific Results
Presenter: Daniel Yekutieli
When: Tuesday, July 12, 2016 Time: 9:00 AM - 10:30 AM
Room: Salon A Carson Hall (Level 2)
Session Synopsis:
Inferring replicability from the Cochrane Collaboration reviews
The Cochrane Collaboration is an independent (non-profit and non-governmental) organization that conducts extensive reviews of health-care interventions that are meant to help health-care providers, patients, and policy makers to make informed evidence-based medical decisions. I will present a statistical testing framework for using the Cochrane Collaboration review, for a given outcome, to provide confidence statement regarding the distribution of the treatment effect of the same outcome in a new treatment group. Specifically, I will show how to construct confidence interval for the CDF and for quantiles of the distribution of the odds ratio in the new treatment group. I will also show how this framework can be extended to provide confidence statement regarding the conditional distribution of the new group treatment effect in the case where we also have data from a small pilot study assessing the treatment efficacy in the new treatment group.
Statistical Assessment of the Replicability of Scientific Results
Presenter: Yoav Benjamini
When: Tuesday, July 12, 2016 Time: 9:00 AM - 10:30 AM
Room: Salon A Carson Hall (Level 2)
Session Synopsis:
What can be learned from replication studies about ways to increase replicability?
Some replication studies in phenotyping mice, in medical research and in experimental psychology will be discussed. Different ways to assess the replicability of their results will be shown. Recommendations will be drawn from these assessments about ways to increase the replicability of an original (stand-alone) study.
