Although the PBS-IIc SELDI-TOF MS apparatus has been extensively used in the search for better biomarkers, issues have been raised concerning the semi-quantitative nature of the technique and its reproducibility. To overcome these limitations, a new SELDI-TOF MS instrument has been introduced: the PCS 4000 series. In the current study, we compared the performances of the old PBS-IIc and new PCS 4000 series generation SELDI-TOF MS apparatus, by analysis of two sample sets.
Peak detection
For the CRC sample set, most peaks were detected with the new PCS 4000 series using the Ciphergen Express™ software, indicating a better sensitivity and less detector saturation of this apparatus. The latter allows for the application of increased laser intensities, after which proteins will desorb more comprehensively, resulting in detection of more peaks. However, for the BC sample set, most peaks were detected with the PBS-IIc instrument using the ProteinChip software, indicating the opposite. Interestingly, in both sample sets, fewer peaks were detected by Ciphergen Express™ than by the ProteinChip software in the spectra generated with the PBS-IIc, despite the fact that both software packages use the same algorithm with similar settings to generate peak clusters. Apparently, the spectrum processing algorithms underlying the visible settings are different for both software packages.
In the BC set, all peaks detected in the PBS-IIc generated spectra by the ProteinChip software, but missed by Ciphergen Express™ were < 4 in intensity. As peaks are detected by means of their signal-to-noise ratio, detection of these low intensity peaks becomes critical when either the noise increases or the signal decreases due to over-estimation of the baseline. Conceivably, the algorithm for noise and/or baseline estimation between both software packages has been changed. Due to the detector attenuation of the PCS 4000 instrument, matrix blanking has improved compared to the PBS-IIc. Hence, less chemical noise is expected when measuring with the PCS 4000 instrument, to which the algorithm applied in noise calculation might have been adapted. As such, for spectra generated with the PBS-IIc (in which relatively more chemical noise is present), the Ciphergen Express™ software will estimate the noise too high or the signal too low, the latter being the consequence of the baseline being estimated too high. Either way results in fewer detected peaks.
The difference between peaks detected by either software package in the PBS-IIc generated spectra was more pronounced in the BC set than in the CRC set. These two datasets differed in their deflector/detector attenuation settings (CRC: 2000 Da, BC: 1000 Da), but in both sets, the noise was calculated between 2 and 200 kDa. However, as matrix peaks are generally observed up to 2000 Da, their contribution to the noise will most likely increase with decreasing deflector settings. Hence, the difference in deflector settings could have caused higher noise estimation in the BC set compared to the CRC set. Combined with the probable noise overestimation by Ciphergen Express™ in PBS-IIc generated spectra, and the fact that relative to the CRC data sets, the BC data sets contained more low intensity peaks (30 and 70%, respectively), which were mainly present in the <10 kDa range, this might explain the more pronounced difference in number of peaks detected in the PBS-IIc generated BC dataset by both software packages.
The difference in deflector/detector attenuation settings might also explain why, contrary to the CRC set, in the BC set more peaks were detected by the ProteinChip software in the PBS-IIc spectra than by Ciphergen Express™ in the PCS 4000 spectra. Compared to the ProteinChip software, the noise calculation algorithm in Ciphergen Express™ apparently is more sensitive to the noise in the low molecular weight range. Due to the difference in detector attenuation settings, this low molecular weight range will contain a higher signal in the BC spectra than in the CRC spectra. Consequently, the noise is estimated higher and less peaks are detected. This hypothesis is supported by the observation that all peaks detected in the PBS-IIc spectra, but not in the PCS 4000 spectra were < 3 in intensity.
One of the alleged improvements of the PCS 4000 compared to its PBS-IIc predecessor is its special configuration for sensitivity in the high mass range that allows detection of proteins above 100 kDa. Indeed, in the BC set, four > 100 kDa peaks were detected exclusively in the PCS 4000 generated spectra, compared to two peaks in the PBS-IIc generated spectra. Moreover, all peaks that were detected exclusively in the PCS 4000 spectra by Ciphergen Express™ were above 10 kDa. However, in none of the CRC data sets any proteins > 100 kDa were detected, indicating no better sensitivity for proteins in the higher mass range for the PCS 4000 series. Most peaks detected only in data set 2 were in the 2–10 kDa range. The differences in detection of high molecular weight peaks could, however, be caused by the different array types used for the analyses of both sample sets.
Classification
As the ultimate gain of the improved performance of the PCS 4000 instrument would be detection of more and better biomarker candidates, we also assessed the classification potential of the data sets generated by both machines. For the CRC set, the improved performance of the new instrument was indeed reflected in the classifiers constructed, as the best classification was obtained with the data set generated by the PCS 4000 instrument, using the total number of peaks detected. When using the subset of peaks detected in all three datasets, the performance of the classifier build on dataset 1b and 2 was similar. For the BC data set, results were less unambiguous. While for data set 1a and 1b only one classifier was applied in the different optimum decision trees constructed, best performance was achieved in data set 1b. Apparently, the different spectrum processing algorithms underlying both software packages also contribute to the alleged improved performance of the PCS 4000 instrument. However, application of both the PCS 4000 and Ciphergen Express™ yielded no better classifiers. Hence, for the BC set, the superior performance of the PCS 4000 instrument in providing better biomarker candidates could not be confirmed. It can, however, not be precluded that our data sets do not contain any real biomarkers.
Reproducibility
For the CRC set, the reproducibility of peak intensities was largely similar across data sets, although a non-significant trend could be seen to a lower CV for data set 2 compared to 1a and 1b. Thus, the spot scanning in a raster and the less detector saturation with the PCS 4000 series does not seem to result in a significant better reproducibility. The fact that significant differences in CV were seen when all peaks were considered indicates that the surplus of peaks detected in data set 2 consists of more robust peaks than the ones also detected in the other data sets, causing the median CV to drop. Reproducibility of the PCS 4000 instrument as measured by the CV has been stated to be < 20% using an external standard [13]. It is not known to us in which m/z range this reproducibility was obtained and whether this was with manual or robotic sample handling. However, our observed median CV is well in concordance with this value, especially taking the manual sample handling into account.
Reproducibility in the BC data sets was assessed by calculation of Spearman's rank correlation coefficient on duplicate intensities of the 10 to 100% peaks with lowest intensity. When all peaks detected were included in this calculation, usage of the PCS 4000 and Ciphergen Express™ software package led to a better performance, as statistically significantly (p < 0.05) good correlations (R > 0.8) were already achieved upon inclusion of only 20% of lowest peaks, compared to the 80% of lowest peaks necessary to achieve comparable results in the PBS-IIc generated data set. However, when correcting for the excess of low intensity peaks detected in data set 1a relative to data set 2 by considering only the peaks detected across all three data sets, results obtained were highly similar for the three data sets. Thus, the improved features of the PCS 4000 instrument relative to the PBS-IIc apparatus do not lead to an improved reproducibility, as already observed in the CRC data sets.
Serum fractionation
Analysis of the PBS-IIc generated spectra by Ciphergen Express™ generally yielded the lowest number of peaks detected. Hence, the performance of the PCS 4000 in serum fractionation is indeed superior compared to the PBS-IIc instrument, reflecting the improved spot coverage and increased detector sensitivity. These observations are highly similar to the results obtained following peak detection in the three CRC data sets.
Although deflector/detector attenuation settings were different for the fractionation spectra on IMAC and CM10 chips, peak clustering results were highly similar for the two array types used, contrary to the results obtained in the CRC and BC sample sets. This could be due to the fact that these spectra have a higher noise level than spectra from crude serum (data not shown), limiting the influence of the different noise estimation between both software packages. Moreover, the number of peaks < 10 kDa is similar in the fractionation spectra from the IMAC and CM10 chips, contrary to the spectra from the CRC and BC set, which could also cause less influence of the noise estimation on peak detection.