Fig. 4

ROC summary plots in Yeast and MCF7. The Yeast case study produced reference standards that predicted between 55 and 70% of the genes in the genome as DEGs, while the MCF7 breast cancer cell lines predicted ~ 15% DEGs. In a, the intersection of all reference standard is used to produce what we would consider an “overtly-conservative” reference. The reference standard was constructed by taking intersection of the DEG lists from cohort analysis of the dataset with DESeq2, DEGSeq, edgeR, NOISeq-BIO (3118 genes as DE). Conversely, in b), the reference standard was constructed taking the union of all techniques (6425 genes as DE), resulting in an “anti-conservative” approach. The anti-conservative scenario facilitates the prediction task as a larger number of genes are called DEGs, which is advantageous to recall. In this case, methods like DEGseq stand out as they can maintain recall while not sacrificing precision since it will tend to call more genes as DEGs on average compared to its counterparts. DEGseq also operates invariantly at FDRs of 5–20%, making it highly suitable for precision medicine since an FDR of 5% is a default standard in clinical decision-making. In the overly conservative scenario with smaller number of DEGs in the gold standard, a more selective approach will perform better, highlighted in the precision parameter and illustrating the trade-offs available across all the tested techniques. An ensemble provides the analyst a robust trade-off alternative as it can build upon the strengths of all methods, and not suffer the issue of “performing well” in one dataset but not in another. In each panel, methods are ordered according to performance