Comparing the old and new generation SELDI-TOF MS: implications for serum protein profiling

Chemicals

All used chemicals were obtained from Sigma, St. Louis, MO, USA, unless stated otherwise.

Patient samples

The performances of both apparatus were assessed with two distinct sample sets. A first set of 45 sera from colorectal cancer (CRC) patients and 43 matched controls (CON) was prospectively collected between July 2003 and October 2005 (referred to as the CRC set). The second set consisted of 45 sera from breast cancer (BC) patients and 46 matched normal women (CON), collected between January 2003 and July 2005 (referred to as the BC set). Both sets were obtained at the Netherlands Cancer Institute, in Amsterdam, The Netherlands. Sample collection was performed with individuals' informed consent after approval by the institutional review boards.

Serum fractionation

Serum samples from three normal women were fractionated in duplicate on QhyperD beads with a strong anion exchange moiety (Bio-Rad Labs), according to manufacturer's protocol. Sample fractionation was performed with a Biomek 3000 Laboratory Automation Workstation (Beckman Coulter Inc.). First, sera were denatured with 9 M urea/2% 3 [(3-cholamidepropyl)-dimethylammonio]-propane sulfonate (CHAPS). After binding of denatured serum to the beads, the flow-through was collected and bound proteins were subsequently eluted with buffers with pH from 9 to 3. Remaining proteins were finally eluted with an organic buffer.

Protein profiling

For profiling of whole serum each sample was analysed according to previously developed protocols [4]. CRC samples and their matched controls were first denatured with 9 M urea/2% CHAPS/1% dithiotreitol. Then, each sample was applied in triplicate on CM10 arrays (weak cation exchange chromatography) with 20 mM sodium phosphate pH 5/0.1% TritonX-100 as a binding buffer and 20 mM sodium phosphate pH 5 as a wash buffer. BC samples and their matched controls were denatured in 9 M urea/2% CHAPS, after which each sample was applied in duplicate on IMAC30 arrays (immobilised metal affinity capture chromatography). Prior to sample application, IMAC30 arrays were charged twice with 50 μL 100 mM nickel sulphate (Braun, Emmenbrücke, Germany), followed by three rinses with deionised water. Phosphate Buffered Saline (PBS; 0.01 M) pH 7.4/0.5 M sodium chloride/0.1% TritonX-100 was applied as a binding buffer and PBS pH 7.4/0.5 M sodium chloride as a wash buffer. For both sample sets, a 50% sinapinic acid (SPA; Bio-Rad Labs) solution in 50% acetonitrile (ACN)/0.5% trifluoroacetic acid (TFA) was used as energy absorbing matrix.

Profiling of fractionated serum was performed on both CM10 chips and IMAC30 arrays. Binding and wash buffers were 100 mM sodium acetate pH 4 and 50 mM HEPES buffer for CM10. For IMAC30, 100 mM copper sulphate was used as charging solution, 100 mM sodium acetate as neutralizing buffer and 100 mM sodium phosphate pH 7/0.5 M sodium chloride as binding and wash buffer (all: Bio-Rad Labs). A solution of 50% SPA in 50% ACN/0.5% TFA was used as matrix.

During all profiling experiments arrays were assembled in 96-well format bioprocessors (Bio-Rad Labs), which were placed on a platform shaker at 350 rpm. Arrays were equilibrated twice with 200 μL of binding buffer, incubated with denatured sample or QHyperD serum fraction for 30 min and, after binding, washed twice with binding buffer, followed by two washes with wash buffer. Lastly, arrays were rinsed with deionised water. After air-drying, two times 1 μL of matrix was applied to the array spots.

Statistics and bioinformatics

To account for possible differences in data processing by the different software packages, data from the PBS-IIc were analysed with the ProteinChip Software, version 3.1 (Bio-Rad Labs) as well as with Ciphergen Express™ version 3.0.6. (Bio-Rad Labs). PCS 4000 data were only processed with the latter package. The PBS-IIc-generated spectra analysed by the ProteinChip Software and Ciphergen Express™ respectively will further be referred to as "data set 1a" and "data set 1b". The PCS 4000-generated spectra, analysed by Ciphergen Express™ will be referred to as "data set 2".

Spectra from the CRC and BC sets were analysed separately. Acquired spectra from each set were compiled and analysed as a whole. Both the ProteinChip and Ciphergen Express™ software spectra were baseline subtracted with the following settings: smooth before fitting baseline: 25 points, fitting width: 10 times expected peak width. Filtering was "on" using an average width of 0.2 times the expected peak width. The noise was calculated from 2000 or 1000 to 200,000 Da for the CRC and BC set respectively. Spectra were normalised to the total ion current in the same m/z range. For peak clustering with the ProteinChip Software, the Biomarker Wizard (BMW; Bio-Rad Labs) application was used. For clustering with the Ciphergen Express™ software (Bio-Rad Labs), identical clustering conditions were defined (see Table 1). In each set, peaks were auto-detected starting from 2000 Da.

For the CRC and BC set, peak intensities from the triplicate (CRC) and duplicate (BC) analyses were averaged and mean peak intensities between groups compared by the non-parametric Mann-Whitney U (MWU) test (p < 0.01 considered statistically significant). For the CRC set, the median CV in each sample set was calculated from the CV's of the triplicate analyses for all clustered peaks in each data set as well as for all common peaks present in each of the three data sets.

For the BC set, Spearman's rank correlation coefficient was calculated on peak intensities in each duplicate analysis for all three data sets. The majority of peaks (> 50%) detected spectrum wide in the three data sets are of relatively low average intensity (< 5), increasing the chance of finding potential biomarkers in the low intensity range. Hence, the reproducibility of peaks in the low intensity range is of special interest. However, correlation analyses are influenced by outliers (e.g. the few high intensity peaks detected), even when using non-parametric statistics. We therefore chose to assess the reproducibility in subsets of peaks, starting with inclusion of the 10%, 20%, 30%, etc. of peaks with lowest intensity, and ending with inclusion of all peaks detected. Spearman's rank correlation coefficient and corresponding p-values were subsequently plotted per subset of peaks.

Classification performance of the data sets obtained with both apparatus was assessed by building classification trees with the Biomarker Patterns Software (BPS; Bio-Rad Labs). Trees were generated with the gini method and the minimal cost tree was chosen in both the CRC and BC sample set. A ten-fold cross validation was used to estimate the sensitivity and specificity for each tree.

For the CM10 and IMAC30 serum fractionation sets, baseline correction and noise calculation was performed as described for the CRC and BC set. For each duplicate fraction, peaks were auto-detected by the ProteinChip software or Ciphergen Express™ by the settings described in Table 1. The number of peaks in each fraction was assessed, as well as the number of unique peaks across all fractions.

Protein profiling CRC set

Six spectra did not contain a protein profile and were thus not assessable (Table 1). For data set 1a normalisation factors as estimated by the apparatus-associated software of the assessable spectra were 0.67 to 3.39 (log -0.18 to 0.53) and for data set 1b 0.62 to 2.4 (log -0.21 to 0.38). For data set 2 values ranged from 0.33 to 4.8 (log -0.49 to 0.68). Since the spectra with aberrant normalisation factors (>2 SD from mean of log normalisation factor) were mostly not from the same samples for the two apparatus, none were excluded, to ensure an equal comparison of both machines. This concerned 13 and 11 spectra from data set 1a and 1b, and 14 from data set 2.

Protein profiling BC set

Following array reading with both the PCS 4000 and PBS-IIc apparatus, two spectra did not contain a protein profile. Along with their duplicate reading, these spectra were excluded from further analyses (Table 1). The normalisation factors for data set 1a were 0.52 to 2.15 (log -0.29 to 0.33). Using Ciphergen Express™ software, normalisation factors of all spectra ranged from 0.51 to 2.25 (log -0.29 to 0.35) for the PBS-IIc generated spectra and from 0.44 to 2.69 (log -0.36 to 0.43) for the PCS 4000 generated spectra. In total, 9 and 8 spectra from data set 1a and 1b respectively and 10 spectra from data set 2 had an aberrant normalisation factor (>2 SD from mean of log normalisation factor). As the majority of these spectra were from different samples for the 3 datasets, none were excluded, to ensure equal comparison of both apparatus. In data set 1a and 1b respectively, a total of 81 and 31 clusters were detected. The ProteinChip software detected 51 clusters that were not detected by Ciphergen Express™ in the same dataset. Except for one cluster (>100 kDa), these unique clusters were all < 10 kDa in mass and < 4 in intensity. In the data set 2, a total of 59 peak clusters was detected. Fifteen of these clusters (all > 9 kDa) were not detected in either data set 1a or 1b. Tables 2 and 3, respectively, provide an overview of peak clustering results and detailed peak cluster information.

Reproducibility CRC set

Reproducibility BC set

For the BC sample set, Spearman's rank correlation coefficient was calculated on peak intensities in each duplicate analysis for successive subsets of peaks including the 10%, 20%, 30% to 100% of peaks with lowest intensity, for all three data sets. As depicted in Figure 1, a correlation coefficient > 0.8 was only reached after inclusion of 80% of lowest peaks in data set 1a, while in the other two data sets, this coefficient was already reached at inclusion of < 20% of lowest peaks. Similar results were obtained when considering the significance of correlation (Figure 1). However, when considering only the common peaks detected across all three data sets, results obtained were highly similar for the three data sets (Figure 2).

Serum fractionation

The numbers of clusters detected on CM10 and IMAC arrays for each sample in each acquired fraction are summarised in Table 7. Some of the clusters are occurring in several fractions. Ignoring these overlapping clusters, on average twice as many peaks were detected in the PCS 4000 generated spectra compared to the PBS-IIc generated spectra (analysed either by the ProteinChip software or Ciphergen Express™). This is also illustrated in Figure 3.

Data set:	Serum sample 1						Serum sample 2						Serum sample 3
	CM10			IMAC30			CM10			IMAC30			CM10			IMAC30
	1a	1b	2	1a	1b	2	1a	1b	2	1a	1b	2	1a	1b	2	1a	1b	2
FT+pH9	43	24	60	23	22	52	59	23	57	28	22	43	29	19	47	24	15	47
pH7	16	6	45	9	9	30	28	9	46	9	7	24	9	5	51	7	7	26
pH5	42	19	51	15	15	32	37	16	49	24	16	29	21	15	56	17	11	28
pH4	22	22	46	24	22	53	19	17	50	34	24	54	23	19	54	23	21	46
pH3	20	20	46	16	14	42	10	8	48	16	13	36	19	16	58	18	13	38
Organic	22	17	58	31	30	40	17	12	61	22	15	36	24	20	53	19	20	42
Total	165	108	306	118	112	249	170	85	311	133	97	222	125	94	319	108	87	227
Unique	103	78	167	85	82	158	106	61	162	82	72	135	82	71	163	67	53	128

Peak detection

For the CRC sample set, most peaks were detected with the new PCS 4000 series using the Ciphergen Express™ software, indicating a better sensitivity and less detector saturation of this apparatus. The latter allows for the application of increased laser intensities, after which proteins will desorb more comprehensively, resulting in detection of more peaks. However, for the BC sample set, most peaks were detected with the PBS-IIc instrument using the ProteinChip software, indicating the opposite. Interestingly, in both sample sets, fewer peaks were detected by Ciphergen Express™ than by the ProteinChip software in the spectra generated with the PBS-IIc, despite the fact that both software packages use the same algorithm with similar settings to generate peak clusters. Apparently, the spectrum processing algorithms underlying the visible settings are different for both software packages.

In the BC set, all peaks detected in the PBS-IIc generated spectra by the ProteinChip software, but missed by Ciphergen Express™ were < 4 in intensity. As peaks are detected by means of their signal-to-noise ratio, detection of these low intensity peaks becomes critical when either the noise increases or the signal decreases due to over-estimation of the baseline. Conceivably, the algorithm for noise and/or baseline estimation between both software packages has been changed. Due to the detector attenuation of the PCS 4000 instrument, matrix blanking has improved compared to the PBS-IIc. Hence, less chemical noise is expected when measuring with the PCS 4000 instrument, to which the algorithm applied in noise calculation might have been adapted. As such, for spectra generated with the PBS-IIc (in which relatively more chemical noise is present), the Ciphergen Express™ software will estimate the noise too high or the signal too low, the latter being the consequence of the baseline being estimated too high. Either way results in fewer detected peaks.

The difference between peaks detected by either software package in the PBS-IIc generated spectra was more pronounced in the BC set than in the CRC set. These two datasets differed in their deflector/detector attenuation settings (CRC: 2000 Da, BC: 1000 Da), but in both sets, the noise was calculated between 2 and 200 kDa. However, as matrix peaks are generally observed up to 2000 Da, their contribution to the noise will most likely increase with decreasing deflector settings. Hence, the difference in deflector settings could have caused higher noise estimation in the BC set compared to the CRC set. Combined with the probable noise overestimation by Ciphergen Express™ in PBS-IIc generated spectra, and the fact that relative to the CRC data sets, the BC data sets contained more low intensity peaks (30 and 70%, respectively), which were mainly present in the <10 kDa range, this might explain the more pronounced difference in number of peaks detected in the PBS-IIc generated BC dataset by both software packages.

The difference in deflector/detector attenuation settings might also explain why, contrary to the CRC set, in the BC set more peaks were detected by the ProteinChip software in the PBS-IIc spectra than by Ciphergen Express™ in the PCS 4000 spectra. Compared to the ProteinChip software, the noise calculation algorithm in Ciphergen Express™ apparently is more sensitive to the noise in the low molecular weight range. Due to the difference in detector attenuation settings, this low molecular weight range will contain a higher signal in the BC spectra than in the CRC spectra. Consequently, the noise is estimated higher and less peaks are detected. This hypothesis is supported by the observation that all peaks detected in the PBS-IIc spectra, but not in the PCS 4000 spectra were < 3 in intensity.

One of the alleged improvements of the PCS 4000 compared to its PBS-IIc predecessor is its special configuration for sensitivity in the high mass range that allows detection of proteins above 100 kDa. Indeed, in the BC set, four > 100 kDa peaks were detected exclusively in the PCS 4000 generated spectra, compared to two peaks in the PBS-IIc generated spectra. Moreover, all peaks that were detected exclusively in the PCS 4000 spectra by Ciphergen Express™ were above 10 kDa. However, in none of the CRC data sets any proteins > 100 kDa were detected, indicating no better sensitivity for proteins in the higher mass range for the PCS 4000 series. Most peaks detected only in data set 2 were in the 2–10 kDa range. The differences in detection of high molecular weight peaks could, however, be caused by the different array types used for the analyses of both sample sets.

Classification

As the ultimate gain of the improved performance of the PCS 4000 instrument would be detection of more and better biomarker candidates, we also assessed the classification potential of the data sets generated by both machines. For the CRC set, the improved performance of the new instrument was indeed reflected in the classifiers constructed, as the best classification was obtained with the data set generated by the PCS 4000 instrument, using the total number of peaks detected. When using the subset of peaks detected in all three datasets, the performance of the classifier build on dataset 1b and 2 was similar. For the BC data set, results were less unambiguous. While for data set 1a and 1b only one classifier was applied in the different optimum decision trees constructed, best performance was achieved in data set 1b. Apparently, the different spectrum processing algorithms underlying both software packages also contribute to the alleged improved performance of the PCS 4000 instrument. However, application of both the PCS 4000 and Ciphergen Express™ yielded no better classifiers. Hence, for the BC set, the superior performance of the PCS 4000 instrument in providing better biomarker candidates could not be confirmed. It can, however, not be precluded that our data sets do not contain any real biomarkers.

Reproducibility

For the CRC set, the reproducibility of peak intensities was largely similar across data sets, although a non-significant trend could be seen to a lower CV for data set 2 compared to 1a and 1b. Thus, the spot scanning in a raster and the less detector saturation with the PCS 4000 series does not seem to result in a significant better reproducibility. The fact that significant differences in CV were seen when all peaks were considered indicates that the surplus of peaks detected in data set 2 consists of more robust peaks than the ones also detected in the other data sets, causing the median CV to drop. Reproducibility of the PCS 4000 instrument as measured by the CV has been stated to be < 20% using an external standard [13]. It is not known to us in which m/z range this reproducibility was obtained and whether this was with manual or robotic sample handling. However, our observed median CV is well in concordance with this value, especially taking the manual sample handling into account.

Reproducibility in the BC data sets was assessed by calculation of Spearman's rank correlation coefficient on duplicate intensities of the 10 to 100% peaks with lowest intensity. When all peaks detected were included in this calculation, usage of the PCS 4000 and Ciphergen Express™ software package led to a better performance, as statistically significantly (p < 0.05) good correlations (R > 0.8) were already achieved upon inclusion of only 20% of lowest peaks, compared to the 80% of lowest peaks necessary to achieve comparable results in the PBS-IIc generated data set. However, when correcting for the excess of low intensity peaks detected in data set 1a relative to data set 2 by considering only the peaks detected across all three data sets, results obtained were highly similar for the three data sets. Thus, the improved features of the PCS 4000 instrument relative to the PBS-IIc apparatus do not lead to an improved reproducibility, as already observed in the CRC data sets.

Serum fractionation

Analysis of the PBS-IIc generated spectra by Ciphergen Express™ generally yielded the lowest number of peaks detected. Hence, the performance of the PCS 4000 in serum fractionation is indeed superior compared to the PBS-IIc instrument, reflecting the improved spot coverage and increased detector sensitivity. These observations are highly similar to the results obtained following peak detection in the three CRC data sets.

Although deflector/detector attenuation settings were different for the fractionation spectra on IMAC and CM10 chips, peak clustering results were highly similar for the two array types used, contrary to the results obtained in the CRC and BC sample sets. This could be due to the fact that these spectra have a higher noise level than spectra from crude serum (data not shown), limiting the influence of the different noise estimation between both software packages. Moreover, the number of peaks < 10 kDa is similar in the fractionation spectra from the IMAC and CM10 chips, contrary to the spectra from the CRC and BC set, which could also cause less influence of the noise estimation on peak detection.