To Replicate or Not to Replicate
Dr. Aslibekyan and colleagues propose that an alternative to replication of genetic association studies be considered in an article titled “To Replicate or Not to Replicate: The Case of Pharmacogenetic Studies” in Circulation: Cardiovascular Genetics(2013). It is understandable why replicating the positive results of a genetic association study is, as the authors state, “the gold standard” of validation. Replication is a useful tool to confirm the likelihood of an association from a previous genetic association study.However, the results of genetic association studies, including genome wide association studies (GWAS), are often difficult to reproduce as evidenced in pharmacogenomics. The authors provide multiple reasons why false positives and false negatives in genetic association studies prevent results from being reproduced. Much of the data in pharmacogenomic studies come from groups of patients who may have differing drug regimens and intervention strategies. Other factors include low minor allele frequencies, small effect size of the variants, limited sample sizes, and differences in phenotype definitions. The authors conclude that as an alternative to replication a combination of methods should be used to validate results. They’ve termed this multi-method validation of genetic association studies “triangulation”. They propose to validate results through a combination of functional validation in vitro and in animal models, joint analyses of several populations, and simulation-based methods, and cite a precedent for methodological triangulation in the social sciences. Social science research involves highly complex systems, and the inherent ethical limitations in human subjects research render reproducibility an impractical method of validation. In a rebuttal, Dr. John Ioannidis argues that what pharmacogenetic association studies actually require are “better, more rigorous methods, and even more stringent replication, and clinical validation”. He proposes several strategies to yield better results from pharmacogenetic association studies including mining data from biobanks and electronic medical records, more stringent criteria for replication, improved methods of detecting and validating rare variants, focus on polygenic markers rather than on single genes, and validation with large, randomized clinical trials. In a response, Dr. Aslibekyan and colleagues agreed with much of Dr. Ioannidis’ argument. They did, however, reiterate the concern that lowering P-value thresholds for single nucleotide variants could cause many single gene variants with true associations to be overlooked due to factors such as gene-environment interactions and epistasis. Read the articles here:
To Replicate or Not to Replicate: the Case of Pharmacogenetic Studies: Establishing Validity of Pharmacogenomic Findings: From Replication to Triangulation. Circulation: Cardiovascular Genetics (2013)
Aslibekyan S, Claas SA, Arnett DK