Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Inter-assay variability of next-generation sequencing-based gene panels

BMC Medical Genomics volume 15, Article number: 86 (2022) Cite this article

Abstract

Background

Tumor heterogeneity has been known to cause inter-assay discordance among next-generation sequencing (NGS) results. However, whether preclinical factors such as sample type, sample quality and analytical features of gene panel can affect the concordance between two different assays remains largely unexplored.

Methods

Replicate sets of DNA samples extracted from formalin-fixed paraffin-embedded tissues (FFPE) (n = 20) and fresh frozen (FF) tissues (n = 10) were herein analyzed using a tumor-only (TO) and paired tumor–normal (TN) gene panel in laboratories certified by the Clinical Laboratory Improvement Amendment. Reported variants from the TO and TN panels were then compared. Furthermore, additional FFPE samples were sequentially sliced from the same FFPE block and submitted to another TN panel assay.

Results

Substantial discordance (71.8%) was observed between the results of the two panels despite using identical DNA samples, with the discordance rate being significantly higher for FFPE samples (p < 0.05). Among the 99 variants reported only in the TO panel, 32.3% were consistent with germline variants, which were excluded in the TN panel, while 30.3% had an allele frequency of less than 5%, some of which were highly likely to be artificial calls. The comparison of two independent TN panel assay results from the same FFPE block also showed substantial discordance rate (55.3%).

Conclusions

In the context of clinical settings, our comparative analysis revealed that inter-NGS assay discordance commonly occurred due to sample types and the different analytical features of each panel.

Peer Review reports

Background

Next-generation sequencing (NGS) gene panels can analyze hundreds of cancer-related genes for somatic alterations in cancer tissue. This novel technology has great potential for providing detailed profiling of variants, which could help improve diagnostic accuracy and treatment selection in patients with cancer [1,2,3].

Although a variety of gene panels have been introduced into daily clinical practice, the reproducibility of NGS results and the factors that cause discordance between different NGS panels have remained unclear [4,5,6,7]. After performing an inter-laboratory comparison to assess the proficiency of gene panel assays in identifying and reporting variants using formalin-fixed, paraffin-embedded (FFPE) tissues from the same FFPE block, Spence et al. reported high concordance (100%) in the identification of exonic variants with an allele frequency (AF) > 15% among all five clinical laboratories despite using different NGS panels [8]. With the increased interest in non-invasive testing, several studies have been conducted to examine concordance between tumor and plasma mutational profiling. Kuderer et al., who compared reported actionable variants between tumor and plasma panels, found a concordance rate of only 22% [9], while Jovelet et al. reported that 55% of patients enrolled in the Molecular Screening for Cancer Treatment Optimization trial were concordant for mutations after comparing tumor and plasma panels [10]. These studies raised many possibilities regarding the source of tumor–plasma discordance, such as tumor heterogeneity, differences in the timing of testing, assay limit of detection, and analytical processes. In a recent study utilizing identical plasma samples and orthogonal comparison of four plasma NGS panels, Stetson et al. suggested that technical factors might be a major source of assay discordance [11]. Such differences might affect clinical decision-making regarding patient treatment.

Nonetheless, inter-assay variability of tissue panels has remained largely unexplored. To address this issue, results of two different NGS gene panels [tumor-only (TO) and paired tumor–normal (TN) panels] using identical DNA samples analyzed in different laboratories accredited by the Clinical Laboratory Improvement Amendment (CLIA) were compared in 30 patients. Furthermore, the NGS results were compared using a pair of FFPE samples from the same FFPE block in 20 patients.

Methods

NGS-based gene panel assays

The TO panel (OncoPrime™) is a hybrid capture-based NGS gene panel assay with a total capture size of 1.33 Mb. This gene panel covers the entire coding region of 215 genes and can detect the rearrangement of 17 selected genes as previously described [12]. Illumina HiSeq 2500 was used for NGS sequencing in a CLIA-certified laboratory (EA Genomics; Morrisville, NC, United States). The minimum input DNA quantity required for creating libraries was 150 ng per sample, while the median depth of coverage was more than 3000. Variant calling software VarPROWL (r20278) was used for variant calling [13].

The paired TN panel (NCC Oncopanel v4, RUO version) is a hybrid capture-based NGS assay with a total capture size of 1.38 Mb. This gene panel covers the entire coding region of 114 genes and can detect the rearrangement of 12 selected genes as previously described [14]. With paired normal samples (DNA extracted from blood), this assay can determine germline variants. NGS was performed in a CLIA-certified and College of American Pathologist-accredited laboratory (Riken Genesis Co., Ltd; Kanagawa, Japan). Variant calling was done using variant calling software (cisCall and GATK) in Mitsui Knowledge Industry (Tokyo, Japan). Thresholds for mean read depth in the TN panel were set according to tumor cellularity, as defined by pathological examination. The threshold was set at 200 for samples with > 50% cellularity, 250 for samples with 20%-50% cellularity, and 500 for samples with < 20% cellularity [14]. Analytical features of the TO and TN panels are compared in Additional file 1: Table S1.

Patients and samples

Between April 2015 and March 2017, a total of 143 patients with histopathologically confirmed solid tumors underwent TO panel assay at Kyoto University Hospital (Kyoto, Japan). Among the 50 patients with actionable mutations, 30 who were available for archived tumor-derived DNA samples were selected for this study. Patients were classified into three groups according to the characteristics of tissue samples: Group FF (n = 10), fresh frozen (FF) tissue samples; Group FFPE-H (n = 10), FFPE samples that yielded DNA library concentrations ≥ 5 nM; and Group FFPE-L (n = 10), FFPE samples that yielded DNA library concentrations < 5 nM.

DNA extracted from FF tumor tissues (Group FF) or FFPE samples with tumor content ≥ 20% (Group FFPE-H and FFPE-L) were utilized for TO panel assay. Residual DNA samples were stored at -80℃ in the Cancer BioBank of Kyoto University Hospital until TN panel assay analysis.

To investigate the discordance due to different regions of the same tumor, additional FFPE samples were sequentially sliced from the same FFPE block (Group FFPE-H and FFPE-L, n = 20) and submitted to another TN panel assay (Fig. 1).

Fig. 1
figure 1

Study design. A total of 30 samples were classified into three groups according to sample type and DNA library concentration: Group FF (n = 10, yellow), fresh frozen (FF) tissue samples; Group FFPE-H (n = 10, green), FFPE samples with DNA library concentrations ≥ 5 nM; and Group FFPE-L (n = 10, purple), FFPE samples with DNA library concentrations < 5 nM. Identical DNA samples were analyzed using tumor-only (TO) and tumor–normal (TN) panel assays. Additional FFPE samples were sliced from the same FFPE block (Group FFPE-H and FFPE-L, n = 20) and submitted for TN panel assay

Full size image

Assessment of tumor DNA library concentration

The tumor DNA library was quantified using KAPA Library Quantification Kits (PN KK4824) following the manufacturer’s instructions. In brief, real-time polymerase chain reaction (PCR) was performed to determine the concentration of the 1:1600 dilution of the library in relation to the concentrations of DNA standards. Next, the difference in size between the average fragment length of the library and the DNA standard was estimated using the Agilent TapeStation system (Santa Clara, CA, United States). Finally, the concentration of the undiluted DNA library was calculated by considering the relevant dilution factor [13].

Q-value assessment

Extracted DNA was quantified using a Qubit dsDNA BR Assay Kit (Thermo Fisher Scientific, Waltham, MA, USA) and Qubit 3.0 Fluorometer (Thermo Fisher Scientific). Quantitative PCR analysis of the RPPH1 (RNase P) locus was performed as control, while the ratio of PCR-amplifiable DNA to total double-stranded DNA was used to calculate Q-value.

Definition of target variants for analysis

We verified the detection status of two panels with base substitutions up to a maximum length of 5 bases and InDel were defined as short variants.

Definition of actionability

In this study, we defined a genomic alteration as actionable if the identified alterations met any of several criteria as previously reported [12]:

  1. 1.

    It can be directly targeted by a US Food and Drug Administration (FDA)-approved drug.

  2. 2.

    It is a signaling pathway component that can be targeted by an FDA-approved drug.

  3. 3.

    It predicts treatment response to an FDA-approved drug.

  4. 4.

    It can be targeted directly or indirectly by an investigational agent that is available in clinical trials.

  5. 5.

    It is a biomarker for which only preclinical data is available.

Comparative tumor variant analyses

A discordant variant or alteration was defined as that detected only in either of two panels. The concordance rate was calculated by dividing the number of variants found in both panels by the number of variants found in both panels plus all discordant variants. Discordance rate was 100% minus concordance rate.

Statistical analysis

Difference in concordance rates between the three groups, FF, FFPE-H, and FFPE-L was examined using Fisher’s exact test. Differences in AF between variants detected only in the TO panel and those detected in both panels were evaluated using the Wilcoxon signed-rank test. Correlations were examined using the Spearman correlation method. All reported p values used herein were two-sided, with a p value < 0.05 indicating statistical significance. All data were analyzed using R (version 3.6.3) and Rstudio (version 1.3.959).

Results

Patient characteristics

Clinical and preanalytical characteristics of all 30 patients are summarized in Table 1. The median age was 61.5 years (range, 45–82 years), while 14 patients (46.7%) were males. The most common tumor types tested were pancreatic cancer (n = 7, 23.3%), followed by biliary tract cancers (n = 4, 13.3%), colorectal cancer (n = 4, 13.3%), and cancer of unknown origin (n = 4, 13.3%). Most of the samples were archived within 2 years before the date of sequencing (n = 29; 96.7%).

Table 1 Clinical and preanalytical characteristics of 30 patients
Full size table

Comparative analyses of tumor variants

The TO panel reported 216 short variants, 8 fusions, and no amplification (TO panel was not designed to report amplifications), while the TN panel reported 84 short variants, 8 fusions, and 4 amplifications. The TO panel reported 75 actionable short variants (34.3%), while the TN panel reported 58 actionable short variants (69.0%) (Additional file 2: Table S2). Although all reported fusions were discordant between the two panels, four fusions in the TN panel were actionable, whereas all of the detected fusions in the TO panel were variants of unknown significance (VUS) (Table 2 and Additional file 3: Table S3).

Table 2 Number of reported alterations in the tumor-only and tumor–normal panels
Full size table

Since the two panels target different set of genes, we focused our comparative analysis on short variants in the 92 genes covered in both panels (Additional file 4: Table S4 and Additional file 10: Fig. S1A and B). Among 130 discordant short variants, 99 were detected only in the TO panel, while 31 were found only in the TN panel (Fig. 2A). The number of actionable short variants detected only in the TO panel, only in the TN panel, and in both panels was 17, 14, and 44, respectively. Thus, the concordance rate in identifying actionable variants and VUS between the two panels was 56.7% and 6.6% (Fig. 2B, C).

Fig. 2
figure 2

Number of variants reported by the two panels. The number of variants reported in both or each of the tumor-only (TO) and tumor–normal (TN) panels after focusing on 92 genes covered in both panels. Green circles represent variants reported in the TO panel, while blue circles represent variants reported in the TN panel. A The number of all short variants reported in each or both of the TO and TN panels. B The number of actionable short variants reported in each or both of the TO and TN panels. C The number of VUS reported in each or both of the TO and TN panels

Full size image

Effects of sample types on concordance between NGS results

To identify the source of discordance, the effects of sample types on NGS results were analyzed. As shown in Fig. 3A and Additional file 5: Table S5, the number of reported short variants in TO and TN panels increased markedly in Group FFPE-L, with this trend being more prominent in the TO panel. The concordance rate differed significantly between all three groups (Fisher’s exact test, p < 0.005), with Group FF (54.1%) showing a higher concordance rate compared to Group FFPE-H (20.4%) and Group FFPE-L (22.1%) (Fig. 3B and Additional file 5: Table S5).

Fig. 3
figure 3

Sample types and concordance rate between the two panels. A The number of variants reported in both or each of the tumor-only (TO) and tumor–normal (TN) panels were categorized due to sample type and DNA library concentration (Group FF, FFPE-H, and FFPE-L). The vertical axis shows the number of counted variants. B The percentage of reported variants in Group FF, FFPE-H, and FFPE-L were expressed as columns. The vertical axis shows the percentage of variants reported in both or each of the TO and TN panels. The green column represents variants reported only in the TO panel, the blue column represents variants reported only in the TN panel, and the pastel pink column represents variants were reported in both panels

Full size image

The concordance rate between the two panels was correlated with neither DNA library concentration (r = 0.36, p = 0.06) nor Q-value (r = 0.18, p = 0.36) (Additional file 11: Fig. S2A and 2B).

AF analysis of discordant variants

Next, the association between AF and discordant variants were analyzed. As plotted in Fig. 4A, variants detected in the TO panel alone had a lower AF compared to common variants detected in both panels (0.08 and 0.22; Wilcoxon signed-rank test, p < 0.005). Further analysis revealed that most of the discordant variants in Group FF and FFPE-H had an AF of approximately 50%, suggesting the possibility of germline variants (Fig. 4B). To support this hypothesis, 32 of the 99 variants (32.3%) reported only in the TO panel were confirmed in normal control bam data of the TN panel (Additional file 6: Table S6).

Fig. 4
figure 4

Allele frequency and concordance rate between the two panels. A The distribution of allele frequency of variants detected only in the tumor-only (TO) panel and those detected in both panels. B The distribution of mean allele frequency of variants reported by the TO panel and/or tumor–normal (TN) panels in Group FF (pastel pink dots), FFPE-H (blue dots), and FFPE-L (green dots). The vertical axis shows the mean allele frequency in logarithmic scale. The horizontal red line and blue line indicate a frequency of 5% and 15%, respectively. C The distribution of allele frequency of variants detected only in the TN panel and those detected in both panels

Full size image

On the other hand, approximately 30% of the 99 variants had an AF of less than 5%, with such variants being more commonly observed in Group FFPE-L, suggesting the possibility of unreliable calls (Fig. 4B and Additional file 12: S3). Indeed, discordant variants detected with low AF in the TO panel were not detected through the variant calling process of the TN panel (Additional file 12: Fig. S3 and Additional file 6: Table S6).

The median AF of the variants detected only in the TN panel was also lower than that of the concordant variants (0.07 and 0.24; Wilcoxon signed-rank test, p < 0.005) (Fig. 4C). Furthermore, 29.5% of these variants had an AF of < 5%, and 66.7% of them belonged to the FFPE-L samples. Approximately 50% of these variants had an AF of < 0.5% in the supportive reads and were derived from the FFPE-L samples.

Comparison of TN panel assays using different FFPE samples from the same FFPE block

To compare the results of independent TN panel assays using different FFPE samples from the same FFPE block, additional FFPE samples were sliced from the same FFPE block (Group FFPE-H and FFPE-L, n = 20) and submitted for another TN panel assay (Fig. 1). Although the TN panel reported a similar number of variants between the first and second assays (n = 62 and 61, respectively), the concordance rate remained as low as 44.7%. As shown in Fig. 5A, B, although the Group FFPE-L had a more reported variants than the Group FFPE-H (n = 56 and 29, respectively), no difference in concordance rate was observed between both groups (44.6% vs. 44.8%; Fisher’s exact test, p = 0.58).

Fig. 5
figure 5

Reported mutations in two different DNA samples. A The number of variants reported in both samples or only in one sample were categorized according to Group FFPE-H and FFPE-L. The vertical axis shows the number of counted variants. B The percentage of reported variants in Group FFPE-H and FFPE-L were expressed as columns. The vertical axis shows the percentage of variants reported in both samples or in only one sample. The green column represents variants were reported only in initial samples (sample 1), the blue column represents variants reported only in additional samples (sample 2), and the pastel pink column represents variants reported in both samples

Full size image

Discussion

To the best of our knowledge, this has been the first study to utilize identical DNA from the same FFPE/FF samples to investigate inter-assay variability between TO and paired TN gene panels, both of which were sequenced and analyzed in CLIA-certified laboratories. Because these panels are commercially available and > 10 institutions have been using them in daily clinical practice in Japan, our comparative analysis is important for understanding the scale and mechanism of discordance between NGS gene panels.

Surprisingly, substantial discordance (71.8%) was observed between the final output from both gene panels despite utilizing identical DNA samples. Considering that FF samples had a significantly higher concordance rate (50.4%) than FFPE samples (21.4%) (Fig. 3B), discordance seems to be partly attributable to DNA quality. Supporting this interpretation, previous studies have reported that DNA degradation is more evident in FFPE compared to FF samples due to formalin-related DNA fragmentation and chemical modification, which potentially increases the incidence of false-positive mutation calls [15,16,17,18,19]. Furthermore, it has been known that FFPE samples do not yield a uniform read-depth coverage across the targeted genes, which can be a probable source of inter-assay discordance [20].

To further investigate the effect of formalin fixation, we analyzed the incidence of C>T/G>A mutations in all samples. As a result, the number of C>T mutations were higher in the samples belonging to Group FFPE-L (Additional file 13: Fig. S4), suggesting the artifact effects of formalin fixation in these samples. This finding might also explain the increased number of short variants in FFPE-L (Fig. 3A). The number of C>T alterations was also higher in the discordant variants, suggesting a relationship between the chemical modification of DNA and discordance between the two NGS panels (Additional file 7: Table S7).

AF analysis of discordant variants showed biased distribution. Indeed, around 30% of discordant variants had an AF of less than 5% (Fig. 4A), most of which were observed in FFPE samples with low DNA library concentration (Group FFPE-L, Fig. 4B and Additional file 12: Fig. S3). Previous studies have reported that variants with low AF may represent subclonal passenger events or non-cancer-derived clones and should therefore be interpreted with caution [21,22,23]. However, our analysis suggested that the majority of discordant variants with low AF were unreliable calls given that most of them were not detected through the variant calling process of the TN panel and belonged to Group FFPE-L (Additional file 6: Table S6). Further analysis also revealed that the median read depth of the discordant variants reported in the TN panel was significantly lower than that of the concordant variants (189 and 555; Wilcoxon signed-rank test, p < 0.005). These findings are consistent with the fact that artificial calls were more common in FFPE samples with low DNA quality as mentioned earlier [15,16,17,18,19].

The difference in analytical pipelines of each NGS gene panel may also contribute to the observed discordance. As shown in Additional file 6: Table S6, among short variants that were found only in the TO panel, all discordant variants observed in FF samples was germline variants which were detected in control bam files of TN panel’s. On the other hand, 96.5% of discordant variants in FFPE-L samples was due to the variant calling process. In FFPE-H group, 55.5% of discordant variants could be attributed to germline variants while 40.7% was not detected by variant caller. Concerning short variants that were found only in the TN panel, discordance observed in FFPE samples was mainly due to variant calling process (66.7% in FFPE-H and 83.3% in FFPE-L).

Although most of the discordant variants were of unknown significance, three clinically relevant variants were reported in the TO panel [BRCA2 S871* (AF = 0.045, read depth = 2607), BRCA2 splice site 7977-2A>T (AF = 0.044, read depth = 1684), and ATM R1618*(AF = 0.854, read depth = 2884)]. We reviewed the bam files of the TN panel but found no signals related to these BRCA2 variants. On the other hand, the germline variants of ATM were not reported in the TN panel because it was also detected from control samples and was automatically classified as normal variants based on its algorithm.

Another specific distribution was observed in groups FF and FFPE-H such that most of the discordant variants had an AF of approximately 50% (Fig. 4B). Among the 99 genetic variants reported only in the TO panel, 32 (32%) were confirmed in the normal control bam file of the TN panel, suggesting that these variants were of germline origin and were accurately determined by a TN paired panel (Additional file 6: Table S6). In our study, Group FFPE-L had significantly fewer discordances due to heterozygous polymorphisms than Group FFPE-H or FF, which might be due to the fact that, in the TO panel, the maximum number of reportable variants was limited to 14 in their algorithm. Therefore, if the total number of detected variants exceeded 14 (as observed in Group FFPE-L), only the top 14 variants were selected based on their actionability, while other variants were not reported. We consider this the main reason why the discordance rate due to heterozygous polymorphisms was lower in Group FFPE-L. Because matched TN assay is known to increase the accuracy of somatic calls, certain discordant variants might be explained by the difference between the TO and matched TN assays [24,25,26].

A comparison between independent TN panel assays using different FFPE samples from the same FFPE block also revealed substantial discordance (Fig. 5A, B). Since the tumor cellularity was similar between the samples used in the first and second assays (Additional file 8: Table S8), the discordance appeared to derive from the heterogeneity of the subclones inside each sample or from the low quality of several samples. Several previous studies support the former explanation. The existence of subclones that varied between samples obtained from different regions of the same primary tumor has been demonstrated in breast cancer, renal cell carcinoma, and glioblastoma [27,28,29,30,31,32,33]. Moreover, one study showed that intratumor heterogeneity can pose challenges to the tumor genomic profiling of a single tiny tumor sample [6]. In addition to tumor heterogeneity, samples with low DNA quality appeared to affect the high discordant rate. Among the 24 short variants that were reported only in the first assay, 14 were derived from 5 FFPE-L samples, in which 1 sample had 9 discordant short variants. Likewise, among the 23 short variants that were reported only in the second assay, 17 were derived from 8 FFPE-L samples, in which 4 samples had > 3 discordant short variants. Moreover, the incidence of C>T mutations was higher in the samples belonging to Group FFPE-L, suggesting the artifact effects of formalin fixation in these samples (Additional file 9: Table S9).

In our study, all reported fusions were discordant between the two panels. However, it is difficult to speculate the cause of discordance since detection of fusion genes largely depends on target or partner primers of each NGS panel which are not fully disclosed due to trade secrecy.

The current study has several limitations worth noting. First, our sample size (n = 30) was relatively small. Second, there is significant variability in the cancer types, the concentration of library DNA, and the Q-values across the three groups of samples. Third, we could only speculate that discordant short variants with low AF were likely attributed to artificial calls given our inability to confirm it by second orthogonal methods, such as droplet digital PCR. Fourth, we could not extensively control the storage period of samples that might affect the degree of DNA degradation in different samples. Fifth, the current study design could not evaluate the contribution of pre-analytical factors such as library preparation, fragment size selection and nuance parameters related to the NGS assay pipeline.

Finally, considering that our study compared specific TO and TN panels with unique analytical features (Additional file 1: Table S1), our current observations might not be applicable to other panels with different analytical features.

To overcome the limitations of our present study, especially to minimize the confounding factors from clinical settings, future studies involving the enrolment of patients with similar baseline characteristics and application of stricter quality control for the collection, preparation and storage of samples should be considered. In addition, the validation of discordant variants using orthogonal methods, such as Sanger sequencing, will be helpful. In contrast, it is difficult for treating physicians to control the confounding factors emanating from laboratory settings, such as variant callers, minimum AF threshold, and maximum numbers of variants to report, since these parameters are set by independent NGS companies and are not fully disclosed due to trade secrecy.

Conclusions

In the context of clinical settings, our comparative analysis using identical tumor DNA samples revealed the existence of substantial discordance between the final output of two different gene panels analyzed by CLIA-certified laboratories. The degree of discordance was affected by sample types, DNA quality, and differences in the analytical pipelines of each NGS gene panel. Physicians engaged in daily clinical practice should therefore be aware that discordance is common between independent NGS assays and should pay more attention to the clinical interpretation of variants, especially those with low AF.

Availability of data and materials

The datasets generated during the current study are not publicly available because it is possible that individual privacy could be compromised, but are available from the corresponding author on reasonable request.

Abbreviations

AF:

Allele frequency

CLIA:

Clinical Laboratory Improvement Amendment

FF:

Fresh frozen

FFPE:

Formalin-fixed paraffin-embedded

NGS:

Next-generation sequencing

PCR:

Polymerase chain reaction

TO:

Tumor-only

TN:

Tumor–normal

References

  1. Bieg-Bourne CC, Millis SZ, Piccioni DE, Fanta PT, Goldberg ME, Chmielecki J, et al. Next-generation sequencing in the clinical setting clarifies patient characteristics and potential actionability. Cancer Res. 2017;77(22):6313–20.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Goodwin S, McPherson JD, McCombie WR. Coming of age: ten years of next-generation sequencing technologies. Nat Rev Genet. 2016;17(6):333–51.

    Article  CAS  PubMed  Google Scholar 

  3. Vogelstein B, Papadopoulos N, Velculescu VE, Zhou S, Diaz LA, Kinzler KW. Cancer genome landscapes. Science. 2013;339(6127):1546–58.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Richards S, Aziz N, Bale S, Bick D, Das S, Gastier-Foster J, et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med. 2015;17(5):405–24.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Bean LJH, Funke B, Carlston CM, Gannon JL, Kantarci S, Krock BL, et al. Diagnostic gene sequencing panels: from design to report-a technical standard of the American College of Medical Genetics and Genomics (ACMG). Genet Med. 2020;22(3):453–61.

    Article  PubMed  Google Scholar 

  6. Riggs ER, Andersen EF, Cherry AM, Kantarci S, Kearney H, Patel A, et al. Technical standards for the interpretation and reporting of constitutional copy-number variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics (ACMG) and the Clinical Genome Resource (ClinGen). Genet Med. 2020;22(2):245–57.

    Article  PubMed  Google Scholar 

  7. Endrullat C, Glökler J, Franke P, Frohme M. Standardization and quality management in next-generation sequencing. Appl Transl Genom. 2016;10:2–9.

    PubMed  PubMed Central  Google Scholar 

  8. Spence T, Stickle N, Yu C, Chow H, Feilotter H, Lo B, et al. Inter-laboratory proficiency testing scheme for tumour next-generation sequencing in Ontario: a pilot study. Curr Oncol. 2019;26(6):e717–32.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Kuderer NM, Burton KA, Blau S, Rose AL, Parker S, Lyman GH, et al. Comparison of 2 commercially available next-generation sequencing platforms in oncology. JAMA Oncol. 2017;3(7):996–8.

    Article  PubMed  Google Scholar 

  10. Jovelet C, Ileana E, Le Deley MC, Motte N, Rosellini S, Romero A, et al. Circulating cell-free tumor dna analysis of 50 genes by next-generation sequencing in the prospective MOSCATO trial. Clin Cancer Res. 2016;22(12):2960–8.

    Article  CAS  PubMed  Google Scholar 

  11. Stetson D, Ahmed A, Xu X, Barrett RBN, Lubinski TJ, Johnson JH, et al. Orthogonal comparison of four plasma NGS tests with tumor suggests technical factors are a major source of assay discordance. JCO Precis Oncol. 2019. https://doi.org/10.1200/PO.18.00191.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Kou T, Kanai M, Yamamoto Y, Kamada M, Nakatsui M, Sakuma T, et al. Clinical sequencing using a next-generation sequencing-based multiplex gene assay in patients with advanced solid tumors. Cancer Sci. 2017;108(7):1440–6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Quy PN, Kanai M, Fukuyama K, Kou T, Kondo T, Yamamoto Y, et al. Association between preanalytical factors and tumor mutational burden estimated by next-generation sequencing-based multiplex gene panel assay. Oncologist. 2019;24(12):e1401–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Sunami K, Ichikawa H, Kubo T, Kato M, Fujiwara Y, Shimomura A, et al. Feasibility and utility of a panel testing for 114 cancer-associated genes in a clinical setting: a hospital-based study. Cancer Sci. 2019;110(4):1480–90.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Srinivasan M, Sedmak D, Jewell S. Effect of fixatives and tissue processing on the content and integrity of nucleic acids. Am J Pathol. 2002;161(6):1961–71.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Quach N, Goodman MF, Shibata D. In vitro mutation artifacts after formalin fixation and error prone translesion synthesis during PCR. BMC Clin Pathol. 2004;4(1):1.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Hedegaard J, Thorsen K, Lund MK, Hein AMK, Hamilton-Dutoit SJ, Vang S, et al. Next-generation sequencing of RNA and DNA isolated from paired fresh-frozen and formalin-fixed paraffin-embedded samples of human cancer and normal tissue. PLoS ONE. 2014;9(5):e98187.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Jennings LJ, Arcila ME, Corless C, Kamel-Reid S, Lubin IM, Pfeifer J, et al. Guidelines for validation of next-generation sequencing-based oncology panels. J Mol Diagn. 2017;19(3):341–65.

    Article  PubMed  Google Scholar 

  19. Do H, Dobrovic A. Sequence artifacts in DNA from formalin-fixed tissues: causes and strategies for minimization. Clin Chem. 2015;61(1):64–71.

    Article  CAS  PubMed  Google Scholar 

  20. Robbe P, Popitsch N, Knight SJL, Antoniou P, Becq J, He M, et al. Clinical whole-genome sequencing from routine formalin-fixed, paraffin-embedded specimens: pilot study for the 100,000 Genomes Project. Genet Med. 2018;20(10):1196–205.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Gupta R, Othman T, Chen C, Sandhu J, Ouyang C, Fakih M. Guardant360 circulating tumor DNA assay is concordant with foundationone next-generation sequencing in detecting actionable driver mutations in anti-EGFR naive metastatic colorectal cancer. Oncologist. 2020;25(3):235–43.

    Article  CAS  PubMed  Google Scholar 

  22. Jahangiri L, Hurst T. Assessing the concordance of genomic alterations between circulating-free DNA and tumour tissue in cancer patients. Cancers (Basel). 2019;11(12):1938.

    Article  CAS  PubMed Central  Google Scholar 

  23. Taavitsainen S, Annala M, Ledet E, Beja K, Miller PJ, Moses M, et al. Evaluation of commercial circulating tumor DNA test in metastatic prostate cancer. JCO Precis Oncol. 2019. https://doi.org/10.1200/PO.19.00014.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Teer JK, Zhang Y, Chen L, Welsh EA, Cress WD, Eschrich SA, et al. Evaluating somatic tumor mutation detection without matched normal samples. Hum Genomics. 2017;11(1):22.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Rabizadeh S, Garner C, Sanborn JZ, Benz SC, Reddy S, Soon-Shiong P. Comprehensive genomic transcriptomic tumor-normal gene panel analysis for enhanced precision in patients with lung cancer. Oncotarget. 2018;9(27):19223–32.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Jones S, Anagnostou V, Lytle K, Parpart-Li S, Nesselbush M, Riley DR, et al. Personalized genomic analyses for cancer mutation discovery and interpretation. Sci Transl Med. 2015;7(283):283ra53.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Torres L, Ribeiro FR, Pandis N, Andersen JA, Heim S, Teixeira MR. Intratumor genomic heterogeneity in breast cancer with clonal divergence between primary carcinomas and lymph node metastases. Breast Cancer Res Treat. 2007;102(2):143–55.

    Article  PubMed  Google Scholar 

  28. Shah SP, Morin RD, Khattra J, Prentice L, Pugh T, Burleigh A, et al. Mutational evolution in a lobular breast tumour profiled at single nucleotide resolution. Nature. 2009;461(7265):809–13.

    Article  CAS  PubMed  Google Scholar 

  29. Navin N, Krasnitz A, Rodgers L, Cook K, Meth J, Kendall J, et al. Inferring tumor progression from genomic heterogeneity. Genome Res. 2010;20(1):68–80.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Gerlinger M, Rowan AJ, Horswell S, Math M, Larkin J, Endesfelder D, et al. Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N Engl J Med. 2012;366(10):883–92.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Snuderl M, Fazlollahi L, Le LP, Nitta M, Zhelyazkova BH, Davidson CJ, et al. Mosaic amplification of multiple receptor tyrosine kinase genes in glioblastoma. Cancer Cell. 2011;20(6):810–7.

    Article  CAS  PubMed  Google Scholar 

  32. Nickel GC, Barnholtz-Sloan J, Gould MP, McMahon S, Cohen A, Adams MD, et al. Characterizing mutational heterogeneity in a glioblastoma patient with double recurrence. PLoS ONE. 2012;7(4):e35262.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Szerlip NJ, Pedraza A, Chakravarty D, Azim M, McGuire J, Fang Y, et al. Intratumoral heterogeneity of receptor tyrosine kinases EGFR and PDGFRA amplification in glioblastoma defines subpopulations with distinct growth factor response. Proc Natl Acad Sci USA. 2012;109(8):3041–6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The abstract of this study has been published recently on Annals of Oncology, Volume 32, Supplement 4, 2021. https://doi.org/10.1016/j.annonc.2021.05.554. The authors would like to thank K. Ashida and H. Mochizuki for their excellent technical assistance and secretarial help. We also deeply thank all patients, clinicians, and medical staff who participated in this research.

Funding

The funders listed below played no role in the design of the study, analysis and interpretation of data, and in writing the manuscript. This study was supported by the Practical Research for Innovative Cancer Control and Program for Integrated Database of Clinical and Genomic Information from Japan Agency for Medical Research and Development, AMED (No. 17kk0205006h0002).

Author information

Author notes

  1. Pham Nguyen Quy and Keita Fukuyama have contributed equally to this work and share first authorship

Authors and Affiliations

  1. Department of Therapeutic Oncology, Graduate School of Medicine, Kyoto University, Kyoto, Japan

    Pham Nguyen Quy, Keita Fukuyama, Masashi Kanai, Tadayuki Kou, Tomohiro Kondo, Masahiro Yoshioka, Junichi Matsubara, Shigemi Matsumoto & Manabu Muto

  2. Department of Real World Data Research and Development, Graduate School of Medicine, Kyoto University, Kyoto, Japan

    Keita Fukuyama & Shigemi Matsumoto

  3. Biomedical Department, Mitsui Knowledge Industry Co., Ltd., Tokyo, Japan

    Tomohiro Sakuma

  4. Department of Diagnostic Pathology, Graduate School of Medicine, Kyoto University, Kyoto, Japan

    Sachiko Minamiguchi

Authors
  1. Pham Nguyen Quy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keita Fukuyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masashi Kanai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tadayuki Kou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tomohiro Kondo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Masahiro Yoshioka
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Junichi Matsubara
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tomohiro Sakuma
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Sachiko Minamiguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Shigemi Matsumoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Manabu Muto
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MM and MK supervised the study. MK, TK1, SM, MM provided the samples. TK1, PNQ, MK collected and/or assemble data. PNQ, MK, KF, TK1, TK2, MY, JM, TS, SM2, MM analyzed and interpretated data. SM1 performed the pathological review. PNQ, KF, MK wrote the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Masashi Kanai.

Ethics declarations

Ethics approval and consent to participate

The present study was approved by the Ethics Committee of the Kyoto University Graduate School of Medicine (Kyoto, Japan; G617 and G692). All patients provided written informed consent for the use of genomic and clinical data for research purposes.

Consent for publication

All participants granted consent in written form for the publication of the de-identified results.

Competing interests

TS is employees at Mitsui Knowledge Industry. MK and SM2 received honoraria from Chugai Pharmaceutical. MM received honoraria from Mistui Knowledge Inc., Sysmex, Riken Genesis, Taiho, SCREEN and Chugai Pharmaceutical. All remaining authors declare no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

Analytical features of the two panels.

Additional file 2: Table S2

. List of all short variants identified by the two panels with all pertinent information.

Additional file 3: Table S3

. List of all fusions identified by the two panels.

Additional file 4: Table S4.

List of the 92 genes covered by the two panels.

Additional file 5: Table S5.

Number of reported short variants in the two panels according to the sample type.

Additional file 6: Table S6.

Analysis of discordant variants that were detected only in one panel.

Additional file 7: Table S7

. Number of point mutations in the variants reported in the tumor-only and tumor–normal panels.

Additional file 8: Table S8.

Tumor cellularity in samples used for the first and second tumor–normal panel assays.

Additional file 9: Table S9.

Number of point mutations in the concordant and discordant variants reported in the two different tumor–normal panel assays.

Additional file 10: Figure S1.

The working flow of comparative analysis of tumor variants. (A) The overlapping and different genes between the tumor-only (TO) panel and tumor–normal (TN) panel. (B) Type and number of alterations that were reported and filtered during the comparative analysis.

Additional file 11: Figure S2.

DNA quality and concordance rate between the two panels. (A) Distribution of reported variants in terms of concordance rate between the two panels (vertical axis) and DNA library concentration (horizontal axis). (B) Distribution of reported variants in terms of concordance rate between the two panels (vertical axis) and Q-value (horizontal axis).

Additional file 12: Figure S3.

Analysis of discordant variants that were reported only in the tumor-only panel. A total of 99 variants detected only in the tumor-only (TO) panel were classified into five categories according to possible causes of discordance and mapped with allele frequency in Group FF, FFPE-H, and FFPE-L.

Additional file 13: Figure S4.

Incidence of point mutations in short variants reported only in the tumor-only panel. The incidence of C>T/G>A mutations in samples belonging to Group FF, FFPE-H and FFPE-L.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Quy, P.N., Fukuyama, K., Kanai, M. et al. Inter-assay variability of next-generation sequencing-based gene panels. BMC Med Genomics 15, 86 (2022). https://doi.org/10.1186/s12920-022-01230-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12920-022-01230-y

Keywords

BMC Medical Genomics

ISSN: 1755-8794

Contact us