cdna or oligonucleotide microarrays have made possible the measurement of thousands of mrna levels in parallel , enabling researchers for the first time to generate comprehensive cellular gene - expression profiles . among several competing techniques ,
current chip layouts allow for the parallel measurement of > 12,000 gene - expression levels on a single array . in this approach ,
every gene is represented by a set of oligonucleotides perfectly matching the target sequence ( pm oligo ) and by a corresponding set with a 1 base - pair ( bp ) mismatch in a central position ( mm oligo ) .
relative transcript abundance is reported as the so - called ' average difference ' value , that is the average of all pm - mm differences across the gene - specific set of probes .
an alternative approach fits a linear model onto the differences between pm and mm hybridization intensities and takes a model - based expression value as a measure of transcript abundance .
the technique is standardized in such a way that generation of gene - expression data is straightforward and quite easy to do .
analysis of a typical microarray experiment involves the following steps : pre - scaling of the fluorescence intensity across the different arrays belonging to one experiment to correct for differences in probe labeling , probe concentration , hybridization efficiency , and potentially other factors ( in the context of microarray analysis , this process is generally termed normalization ) ; detection of differentially expressed genes ; in the case of experimental setups comparing more than two conditions , a clustering step to group together genes with similar expression patterns ; and higher - level analysis , for example by combining functional annotations of genes having predefined interesting expression patterns with previous knowledge about the experimental system under investigation .
most frequently , high - density oligonucleotide data are normalized by a simple ' global scaling ' procedure .
this involves multiplication of every gene - expression value with a constant factor so that the mean intensities of the arrays to be compared are identical .
a conceptually related approach involves fitting a linear regression model on the data and scaling the fluorescence intensities so that the resulting regression model has a slope of 1 and a y - intercept of 0 .
this approach suffers from two significant drawbacks : first , it relies on the implicit assumption that the total mrna content of different cell types compared is the same .
this is not always the case , especially if cell types of different size and/or cell - cycle status are compared .
control of this effect is attempted by loading identical amounts of crna onto the chips .
however , it has been shown that the mean expression level on any array can be subject to significant variation across arrays .
second , the normalization is linear and can not account for nonlinearity in the underlying data .
previous studies   show that simple linear regression models incompletely fit the data and that two or more linear models with different slope for different ranges of fluorescence - intensity values result in a better fit .
they assume that an ' invariant set ' exists , containing genes that do not change significantly between two experimental conditions .
this procedure can be performed either on the feature level ( taking raw fluorescence values as input )   or on the probe set level , taking average - difference values as input .
the easiest approach is to define genes as differentially expressed that change more than an arbitrarily chosen threshold .
more sophisticated analyses additionally apply statistical tests such as student 's t - test for comparisons between two experimental conditions . however , this and other parametric tests rely on certain assumptions , namely that the underlying data are normally distributed with equal variances across experimental conditions .
these assumptions must not necessarily be met , and analysis of our own ( t.s . and r.h .
, unpublished observations ) and other   datasets show that they are often not fulfilled .
non - parametric tests such as the mann - whitney test , in contrast , do not rely on such strong assumptions , but a larger number of replicate experiments is desirable . a particular problem is the analysis of ' multiclass experiments ' containing more than two experimental conditions , such as cellular developmental stages . many researchers carry out pairwise comparisons of all possible pairs of combinations , resulting in a list of genes that are detected as differentially expressed at least once .
this leads to increased type - i error rates , with the final data set having a type - i error rate up to 1-(1- ) , where  is the type - i error rate of individual pairwise comparisons and
five pairwise comparisons at the 95% confidence level thus result in a confidence level for the resulting dataset of 77% .
classical statistics offer anova algorithms for such problems . here , differential gene expression is detected by comparing variances within experimental conditions to variances across experimental conditions .
both parametric ( f ) and nonparametric ( h or kruskal - wallis ) tests exist , with the associated problems described above . recently , an alternative procedure for detection of differentially expressed genes , called significance analysis of microarrays ( sam ) , has been described . here , a relative difference in gene expression
next , the dataset is permuted several times , and the relative difference is computed again , on the basis of the permuted datasets .
for some genes , however , the difference between the two scores exceeds a certain threshold parameter .
a false - discovery rate   can be computed on the basis of how many genes are called in the permuted datasets with the given threshold .
obviously , there are a large number of analysis options for gene - expression data .
the influence of normalization and statistical analysis on the detection of differentially expressed genes has not been investigated to date . in this study
, we carry out a thorough comparison of different normalization and statistical procedures to define the key components for detection of differentially expressed genes in a multiclass experiment .
the aim of the present study was to evaluate different normalization and statistical analysis methods for their influence on detection of differentially expressed genes .
the dataset used comprises high - density oligonucleotide array - derived gene - expression data of five consecutive cellular populations of an ordered cellular differentiation pathway .
figure 1 shows signal intensity scatterplots of one randomly chosen array set , consisting of two arrays with different gene content .
not normalized values are on the x - axis , and the normalized counterparts on the y - axis . in all scatterplots ,
the suba and subb arrays can easily be distinguished by the different slopes of the respective data points .
this underlines the necessity for separate normalization of different subar - rays of one set .
the two normalization methods based on invariant features produce a significant amount of scatter compared to the not normalized data ( figure 1a,1b ) , especially the model - based expression values as compared to the traditional average difference values .
however , the model - based expression values calculated after invariant feature normalization ( y - axis in figure 1b ) differ by two factors from the not normalized average difference values ( x - axis in figure 1b ) .
the scatter in figures 1a and b reflects data processing on the fluorescence level with recalculation of gene - expression metrics after normalization , in contrast to the other methods .
comparing average difference and model - based expression values derived after invariant feature normalization ( figure 1c ) , it becomes evident that model - based expression values tend to be higher than average difference values in the low - signal area of the plot .
thus , low - abundance genes give higher signals when model - based expression values are used .
this might reflect either a greater sensitivity of the model - based approach or an overestimation of the expression level . the invariant set normalization method results in very similar slopes for the suba and subb arrays , respectively ( figure 1d ) .
since pre - computed average difference values are used , the normalization does not result in recalculation of the expression - level values from fluorescence data , resulting in less scatter than in figure 1a,1b .
the global scaling method , as expected , produces two ' straight lines ' of data points representing different normalization factors for the two array types ( figure 1e ) . to explore further the differential impact of the normalization procedures ,
we compared the normalization curves generated by the two nonlinear ( invariant feature and invariant set ) normalization schemes ( figure 2 ) . in figure 2a , all the approximately 400,000 feature intensities from two different arrays are plotted against each other ; the invariant features are displayed as red circles .
the position of the invariant features far off and below the diagonal ( blue line ) indicates substantial need for normalization .
figure 2b shows a normalization curve for the same two non - normalized arrays obtained by the invariant set method .
in contrast to figure 2a , the invariant sets are located very closely to or directly on the diagonal ( blue dots ) , indicating that normalization by the invariant set method will affect expression level values mildly .
this illustrates that different normalization methods will have a profound effect on the expression - level values , even if normalization has been carried out to the same baseline array .
the four normalized datasets were subjected to three popular methods for identifying differentially expressed genes , namely parametric and nonparametric anova models and the permutation - based sam procedure .
this resulted in 12 datasets containing data about probability of differential gene expression . at the 99% confidence level - a median false discovery rate ( fdr ) of 1% in sam ,
respectively - large sets of genes are detected by each of the combinations of normalization and statistical analysis methods .
however , the number of genes detected differs dramatically ( figure 3a ) . the combination of linear global scaling with parametric anova yields 1,526 differentially expressed genes .
in contrast , the combination of invariant feature normalization with calculation of average difference values and sam results in only 608 genes with a median fdr of 1% or lower
. looking at the average numbers of differentially expressed genes , it is highest with the parametric anova model and smallest with the sam procedure across different normalization procedures . similarly , linear global scaling yields the largest , and invariant feature normalization with calculation of average difference values smallest , set of differentially expressed genes across the different statistical evaluations performed ( figure 3a ) .
however , the confidence level alone is seldom used for detection of differentially expressed genes . usually , a fold - change criterion as well as an absolute difference criterion is added .
figure 3b shows what percentage of the genes shown in figure 3a also pass additional criteria ( twofold change and absolute difference of at least 100 units ) .
these additional criteria lead to a reduction of the number of detected genes in all datasets , but to a markedly different extent . on average , 78 , 82 and 88% of the genes detected using only the 99% confidence criterion
are still detected when applying the additional criteria in the parametric anova , nonparametric anova , and sam datasets , respectively ( figure 3b ) .
in contrast , this holds true for only 63% of the genes normalized with the invariant probe set method .
genes detected by the combination with parametric anova are most significantly affected , with only 54% fulfilling the additional criteria .
table 1 shows fold - change and difference statistics for different subsets of the gene - expression datasets .
considering all genes on the arrays ( table 1a ) , the dataset normalized by the invariant probe set method contains the smallest fold changes with the smallest differences between mean and median fold change .
notably , the highest fold change achieved by any gene is 72-fold in this dataset , as opposed to several hundred fold in the other datasets .
similarly , the mean fold change of genes detected with 99% confidence by any of the three methods is three- to fourfold in the invariant probe set normalized dataset , as opposed to around tenfold or higher in datasets employing the other normalization procedures .
also , this subset of genes shows the smallest difference between mean and median ( table 1b ) .
this indicates that the fold - change values are less skewed toward small values in the dataset normalized by the invariant probe set method .
analyzing the absolute differences between the maximum and minimum value across the five cellular stages examined , these effects are still present , but markedly reduced ( table 1c , d ) .
we next asked how well different combinations of normalization and statistical - analysis strategies agree in detecting differentially expressed genes .
figure 4 shows how many genes are detected as differentially expressed by how many data - analysis combinations .
all three criteria for filtering the data ( confidence , fold change , absolute difference ) were applied as above .
interestingly , a large proportion ( 382 genes ) is detected by all 12 combinations .
an almost equally large number is detected by one particular combination only . among 201 genes detected by two combinations ,
these two combinations involve the same normalization procedure in 155 cases , in contrast to 40 cases that involve the same statistical algorithm .
similarly , among 208 genes detected by three combinations , the same normalization is involved in all three detecting combinations in 145 cases ( whereas the same statistical algorithm is involved in 11 cases only ) .
this might indicate that the normalization procedure has a profound influence on which genes are detected as differentially expressed in a subsequent statistical analysis step .
the sharp drop between three- and four - method combinations is most likely due to the fact that three statistical algorithms have been employed .
the sharp rise between 11 and 12 method combinations is most likely due to the fact that genes detected by 11 combinations form a very stable subset already , and thus are more likely to be detected by the twelfth method as well .
we thus investigated how many genes are detected simultaneously when comparing any two different combinations of methods .
table 2 shows the percentage of genes identified as differentially expressed by the method combination defined by the column header out of the combination defined by the row designation .
values range from 41% ( of the genes identified by global scaling and parametric anova which are also found by application of invariant feature normalization with calculation of average difference values and sam ) to 100% . strikingly , the genes identified by sam are always a subset of the genes identified by parametric anova , if the same normalization procedure has been employed ( figures of 100% in table 2 )
. a high proportion ( > 93% ) of the genes identified by nonparametric anova are also identified by parametric anova , and a similar high proportion of the genes identified by sam are also identified by nonparametric anova using the same normalization ( table 2 ) .
thus , a type of hierarchy evolves : genes detected by sam generally are a subset of genes detected by nonparametric anova , and genes detected by nonparametric anova generally are a subset of genes detected by parametric anova .
interestingly , the genes detected after invariant probe set normalization generally are a subset of the genes detected after global scaling , with the hierarchy as described above ( values marked by asterisks in table 2 ) .
these results point to a pronounced role for the normalization process in the detection of differentially expressed genes .
we sought to determine which one of the two steps ( normalization versus statistical analysis ) has a greater influence on the detection of differentially expressed genes .
we thus calculated rank correlations for probabilities of differential expression for all genes represented on the arrays between pairs of all 12 combinations of normalization and statistical analysis procedures ( table 3 ) .
strikingly , a correlation of > 0.9 can only be achieved between two statistical analysis methods if the same normalization has been carried out .
rank correlations between probabilities for differential expression are markedly lower ( as low as 0.4 to 0.5 in some instances ) if two different normalization procedures are compared , irrespective whether the same statistical analysis method has been used .
this indicates that the normalization procedure employed has a tremendous effect on the subsequent detection of differentially expressed genes .
figure 1 shows signal intensity scatterplots of one randomly chosen array set , consisting of two arrays with different gene content .
not normalized values are on the x - axis , and the normalized counterparts on the y - axis . in all scatterplots , the suba and subb arrays can easily be distinguished by the different slopes of the respective data points .
this underlines the necessity for separate normalization of different subar - rays of one set .
the two normalization methods based on invariant features produce a significant amount of scatter compared to the not normalized data ( figure 1a,1b ) , especially the model - based expression values as compared to the traditional average difference values . however , the model - based expression values calculated after invariant feature normalization ( y - axis in figure 1b ) differ by two factors from the not normalized average difference values ( x - axis in figure 1b ) .
the scatter in figures 1a and b reflects data processing on the fluorescence level with recalculation of gene - expression metrics after normalization , in contrast to the other methods . comparing average difference and model - based expression values derived after invariant feature normalization ( figure 1c ) ,
it becomes evident that model - based expression values tend to be higher than average difference values in the low - signal area of the plot .
thus , low - abundance genes give higher signals when model - based expression values are used .
this might reflect either a greater sensitivity of the model - based approach or an overestimation of the expression level . the invariant set normalization method results in very similar slopes for the suba and subb arrays , respectively ( figure 1d ) .
since pre - computed average difference values are used , the normalization does not result in recalculation of the expression - level values from fluorescence data , resulting in less scatter than in figure 1a,1b .
the global scaling method , as expected , produces two ' straight lines ' of data points representing different normalization factors for the two array types ( figure 1e ) . to explore further the differential impact of the normalization procedures ,
we compared the normalization curves generated by the two nonlinear ( invariant feature and invariant set ) normalization schemes ( figure 2 ) . in figure 2a , all the approximately 400,000 feature intensities from two different arrays are plotted against each other ; the invariant features are displayed as red circles .
the position of the invariant features far off and below the diagonal ( blue line ) indicates substantial need for normalization .
figure 2b shows a normalization curve for the same two non - normalized arrays obtained by the invariant set method .
the invariant data points are indicated by red circles . in contrast to figure 2a , the invariant sets are located very closely to or directly on the diagonal ( blue dots ) , indicating that normalization by the invariant set method will affect expression level values mildly .
this illustrates that different normalization methods will have a profound effect on the expression - level values , even if normalization has been carried out to the same baseline array .
the four normalized datasets were subjected to three popular methods for identifying differentially expressed genes , namely parametric and nonparametric anova models and the permutation - based sam procedure .
this resulted in 12 datasets containing data about probability of differential gene expression . at the 99% confidence level - a median false discovery rate ( fdr ) of 1% in sam ,
respectively - large sets of genes are detected by each of the combinations of normalization and statistical analysis methods .
however , the number of genes detected differs dramatically ( figure 3a ) . the combination of linear global scaling with parametric anova yields 1,526 differentially expressed genes .
in contrast , the combination of invariant feature normalization with calculation of average difference values and sam results in only 608 genes with a median fdr of 1% or lower .
looking at the average numbers of differentially expressed genes , it is highest with the parametric anova model and smallest with the sam procedure across different normalization procedures . similarly , linear global scaling yields the largest , and invariant feature normalization with calculation of average difference values smallest , set of differentially expressed genes across the different statistical evaluations performed ( figure 3a ) .
however , the confidence level alone is seldom used for detection of differentially expressed genes . usually , a fold - change criterion as well as an absolute difference criterion is added .
figure 3b shows what percentage of the genes shown in figure 3a also pass additional criteria ( twofold change and absolute difference of at least 100 units ) .
these additional criteria lead to a reduction of the number of detected genes in all datasets , but to a markedly different extent . on average , 78 , 82 and 88% of the genes detected using only the 99% confidence criterion are still detected when applying the additional criteria in the parametric anova , nonparametric anova , and sam datasets , respectively ( figure 3b ) .
in contrast , this holds true for only 63% of the genes normalized with the invariant probe set method .
genes detected by the combination with parametric anova are most significantly affected , with only 54% fulfilling the additional criteria .
table 1 shows fold - change and difference statistics for different subsets of the gene - expression datasets .
considering all genes on the arrays ( table 1a ) , the dataset normalized by the invariant probe set method contains the smallest fold changes with the smallest differences between mean and median fold change .
notably , the highest fold change achieved by any gene is 72-fold in this dataset , as opposed to several hundred fold in the other datasets .
similarly , the mean fold change of genes detected with 99% confidence by any of the three methods is three- to fourfold in the invariant probe set normalized dataset , as opposed to around tenfold or higher in datasets employing the other normalization procedures .
also , this subset of genes shows the smallest difference between mean and median ( table 1b ) .
this indicates that the fold - change values are less skewed toward small values in the dataset normalized by the invariant probe set method .
analyzing the absolute differences between the maximum and minimum value across the five cellular stages examined , these effects are still present , but markedly reduced ( table 1c , d ) .
we next asked how well different combinations of normalization and statistical - analysis strategies agree in detecting differentially expressed genes .
figure 4 shows how many genes are detected as differentially expressed by how many data - analysis combinations .
all three criteria for filtering the data ( confidence , fold change , absolute difference ) were applied as above .
interestingly , a large proportion ( 382 genes ) is detected by all 12 combinations .
an almost equally large number is detected by one particular combination only . among 201 genes detected by two combinations ,
these two combinations involve the same normalization procedure in 155 cases , in contrast to 40 cases that involve the same statistical algorithm .
similarly , among 208 genes detected by three combinations , the same normalization is involved in all three detecting combinations in 145 cases ( whereas the same statistical algorithm is involved in 11 cases only ) .
this might indicate that the normalization procedure has a profound influence on which genes are detected as differentially expressed in a subsequent statistical analysis step .
the sharp drop between three- and four - method combinations is most likely due to the fact that three statistical algorithms have been employed .
the sharp rise between 11 and 12 method combinations is most likely due to the fact that genes detected by 11 combinations form a very stable subset already , and thus are more likely to be detected by the twelfth method as well .
we thus investigated how many genes are detected simultaneously when comparing any two different combinations of methods .
table 2 shows the percentage of genes identified as differentially expressed by the method combination defined by the column header out of the combination defined by the row designation .
values range from 41% ( of the genes identified by global scaling and parametric anova which are also found by application of invariant feature normalization with calculation of average difference values and sam ) to 100% .
strikingly , the genes identified by sam are always a subset of the genes identified by parametric anova , if the same normalization procedure has been employed ( figures of 100% in table 2 )
. a high proportion ( > 93% ) of the genes identified by nonparametric anova are also identified by parametric anova , and a similar high proportion of the genes identified by sam are also identified by nonparametric anova using the same normalization ( table 2 ) .
thus , a type of hierarchy evolves : genes detected by sam generally are a subset of genes detected by nonparametric anova , and genes detected by nonparametric anova generally are a subset of genes detected by parametric anova .
interestingly , the genes detected after invariant probe set normalization generally are a subset of the genes detected after global scaling , with the hierarchy as described above ( values marked by asterisks in table 2 ) .
these results point to a pronounced role for the normalization process in the detection of differentially expressed genes .
we sought to determine which one of the two steps ( normalization versus statistical analysis ) has a greater influence on the detection of differentially expressed genes .
we thus calculated rank correlations for probabilities of differential expression for all genes represented on the arrays between pairs of all 12 combinations of normalization and statistical analysis procedures ( table 3 ) .
strikingly , a correlation of > 0.9 can only be achieved between two statistical analysis methods if the same normalization has been carried out .
rank correlations between probabilities for differential expression are markedly lower ( as low as 0.4 to 0.5 in some instances ) if two different normalization procedures are compared , irrespective whether the same statistical analysis method has been used .
this indicates that the normalization procedure employed has a tremendous effect on the subsequent detection of differentially expressed genes .
the present study examined the influence of normalization and statistical analysis on detection of differentially expressed genes in a oligonucleotide microarray experiment .
a first striking observation is that the number of genes detected as differentially expressed varies by a factor of almost three , depending on which combination of normalization method and statistical analysis has been carried out .
the genes detected by a confidence criterion alone show large differences in mean and median fold changes , and thus show different susceptibility to the use of additional criteria for filtering .
this affects primarily the dataset normalized by the invariant probe set method . also , genes in this dataset show smaller fold changes , and the maximum fold change achieved is 6- to 11-fold lower than in the other datasets
. most probably , this is due to the shifting of data as a first step in the normalization so that only 2% of the raw values are below 20 .
this assigns a higher value to each data point while preserving the difference between any two data points , effectively reducing the ratio .
analysis of the identity of probe sets indicates that the normalization method has a very high influence of detection of differentially expressed genes . in those cases where
one gene is detected by two or more different combinations of normalization and statistical analysis algorithms , these combinations usually employ the same normalization .
moreover , genes are detected as differentially expressed to a high percentage in different combinations of analysis strategies only when the same normalization has been applied . applying the same statistical algorithms , in contrast ,
finally , a high rank correlation for probability of differential expression can only be achieved if the same normalization procedure has been applied .
thus , normalization appears to have a higher influence on the set of differentially expressed genes than the choice of statistical algorithm .
first , the dataset chosen consists of five independent replicate experiments , a situation rarely encountered in microarray experiments .
second , the dataset has been derived from highly purified cell populations . this situation is also different from most other microarray experiments .
it might therefore be that the effects described here are even more pronounced in different experimental settings investigating less well - defined materials .
however , we do not expect this to be a significant issue , as the hybridization behavior of amplified and standard probes has been shown by us and other groups   to be similar .
moreover , we have high confidence in our dataset , as many genes with known expression patterns are detected in concordance with earlier results .
fourth , the present analysis follows the general practice of detecting differentially expressed genes solely on the basis of differences in the average difference value .
this represents only a small proportion of the information generated by the analysis of the fluorescence images .
additional information about cross - hybridization , fractions of probe pairs contributing to the signal , and ' presence ' or ' absence ' of a transcript , among others , is readily available .
unfortunately , no consensus exists on how to incorporate this additional information in a setting that can not be handled by the manufacturer 's software .
anecdotally , individual groups apply their own , often arbitrarily chosen , criteria to increase confidence in the results .
data - analysis algorithms employing as much information as possible with incorporation of replicate experiments and the ability to analyze more than two conditions simultaneously are urgently needed .
finally , testing more than 13,000 hypotheses on only five different conditions constitutes a significant multiple - testing problem .
it is commonly accepted in multivariate statistics that the number of hypothesis should not exceed the number of parameters .
thus , when testing such a high number of hypotheses , the probability of at least one falsely rejected null hypothesis ( the so - called family - wise error rate ) is high .
although multiple solutions to this problem have been proposed ( like sam , controlling the false - discovery rate rather than the type - i error rate ) , to date no consensus exists on how to deal with that problem in the context of gene - expression analysis .
the question naturally arises of which combination of algorithms is ' best ' for analyzing gene - expression data .
one has to balance sensitivity , which attempts to detect all differentially expressed genes , against specificity , which attempts to reduce the number of false positives as much as possible .
this is nicely illustrated by the set of genes mentioned above that are known to change .
all of the 21 genes examined so far are detected in at least one method combination .
however , two genes are detected by only one combination , and only seven of the genes known to change during b - cell differentiation are detected by all 12 combinations of methods .
thus , the more specific an algorithm is , the more likely is a loss of sensitivity .
however , with the high number of differentially expressed genes typically detected in a microarray experiment , specificity might be a major issue .
as most of the genes detected by the permutation - based sam method are enclosed in the anova models , this algorithm appears to be inherently more specific than the classical ones .
regarding normalization , the invariant - feature method with calculation of average difference values yields the smallest set of genes .
however , this set is not a subclass of the genes detected after normalization with other methods ( table 2 ) , as is the case for sam .
ideally , a researcher would have a set of genes with known differential expression and a set known not to be differentially expressed .
this could be used to define the conditions for analysis . in the absence of such a training set ,
it is probably a wise decision to use the algorithms likely to result in the most specific analysis .
the b - cell precursor gene - expression dataset described here has been published in detail previously .
total femoral bone marrow cells of 5 - 6-week - old c57/bl6 mice ( n = 4 per experiment ) were divided into into three equal samples .
these stages were : pre - bi cells ( c - kit b220 ) , large pre - bii cells ( surface immunoglobulin ( sig ) cd25 b220 large ) , small pre - bii cells ( sig cd25 b220 small ) , immature b ( sigm b220 ) and mature b cells ( sig b220 ) .
a total of 50,000 ( pre - bi , large pre - bii ) or 150,000 ( small pre - bii , immature and mature b cells ) cells were sorted directly into trizol rna isolation reagent ( life technologies ) at 50,000 cells/500 l trizol .
rna was then subjected to two rounds of in vitro transcription - based rna amplification as described earlier .
affymetrix mu11k suba and subb genechip arrays were hybridized , washed , stained and scanned according to the manufacturer 's specifications .
five independent replicate experiments were performed ; thus , a total of 50 chips is included in the current analysis ( 5 conditions  5 replicates  2 chip layouts ) . scanned raw data images were processed with affymetrix genechip v3.2 software , resulting in processed image ( .cel ) and numerical ( .chp ) files .
after normalization , all gene - expression values below 20 were set to 20 to eliminate low - level signals .
suba and subb chip types were combined into one gene - expression matrix . for global scaling
all average difference values of every chip were summed up , and the mean of these sums across all chips of the same layout was calculated .
the ratio of the actual average difference sum for any given chip and the mean of all average difference sums across all chips with the same layout served as a correction factor for this chip , with which all the average difference values were multiplied . for invariant - feature normalization
processed image ( .cel ) files served as input , and normalization was carried out according to the developer 's specifications .
next , for every array , invariant features ( defined as all the features with similar ranks of fluorescence intensity between two arrays ) were identified .
finally , a piecewise linear running median line based on the invariant features was calculated and used as the normalization curve .
after normalization , both traditional average difference values and model - based expression values ( mbev ) were calculated and exported to microsoft excel .
for invariant - set normalization on the probe - set level , all average difference levels were extracted from the .chp files and imported into ' the equalizer ' .
briefly , all values were first shifted ( by adding a constant value ) so that only 2% of the data points had an intensity below 20 .
next , values with similar ranks (  15 ) between two arrays were identified , taking the first array of the set as baseline .
finally , all data points were shifted so that the original similar - rank subset has a slope of 1 .
three different methods were used for detection of differentially expressed genes . the f - test for parametric anova and the h
( kruskal - wallis ) test for nonparametric anova were implemented in microsoft excel using standard formulas .
the permutation - based method sam   is freely available to academic researchers as an add - in for microsoft excel . to enable comparisons between sam and the two anova approaches
, we considered a median false - detection rate of 1% in sam as comparable to a 99% confidence level in anova .
maximum fold changes and maximum differences in gene expression were calculated from the minimum and the maximum of the population - wise means across the five cellular populations examined .
the b - cell precursor gene - expression dataset described here has been published in detail previously .
total femoral bone marrow cells of 5 - 6-week - old c57/bl6 mice ( n = 4 per experiment ) were divided into into three equal samples .
these stages were : pre - bi cells ( c - kit b220 ) , large pre - bii cells ( surface immunoglobulin ( sig ) cd25 b220 large ) , small pre - bii cells ( sig cd25 b220 small ) , immature b ( sigm b220 ) and mature b cells ( sig b220 ) .
a total of 50,000 ( pre - bi , large pre - bii ) or 150,000 ( small pre - bii , immature and mature b cells ) cells were sorted directly into trizol rna isolation reagent ( life technologies ) at 50,000 cells/500 l trizol .
rna was then subjected to two rounds of in vitro transcription - based rna amplification as described earlier .
affymetrix mu11k suba and subb genechip arrays were hybridized , washed , stained and scanned according to the manufacturer 's specifications .
five independent replicate experiments were performed ; thus , a total of 50 chips is included in the current analysis ( 5 conditions  5 replicates  2 chip layouts ) . scanned raw data images were processed with affymetrix genechip v3.2 software , resulting in processed image ( .cel ) and numerical ( .chp ) files .
after normalization , all gene - expression values below 20 were set to 20 to eliminate low - level signals .
suba and subb chip types were combined into one gene - expression matrix . for global scaling
all average difference values of every chip were summed up , and the mean of these sums across all chips of the same layout was calculated .
the ratio of the actual average difference sum for any given chip and the mean of all average difference sums across all chips with the same layout served as a correction factor for this chip , with which all the average difference values were multiplied . for invariant - feature normalization
processed image ( .cel ) files served as input , and normalization was carried out according to the developer 's specifications .
next , for every array , invariant features ( defined as all the features with similar ranks of fluorescence intensity between two arrays ) were identified .
finally , a piecewise linear running median line based on the invariant features was calculated and used as the normalization curve .
after normalization , both traditional average difference values and model - based expression values ( mbev ) were calculated and exported to microsoft excel . for invariant - set normalization on the probe - set level ,
all average difference levels were extracted from the .chp files and imported into ' the equalizer ' .
briefly , all values were first shifted ( by adding a constant value ) so that only 2% of the data points had an intensity below 20 .
next , values with similar ranks (  15 ) between two arrays were identified , taking the first array of the set as baseline .
finally , all data points were shifted so that the original similar - rank subset has a slope of 1 .
all average difference values of every chip were summed up , and the mean of these sums across all chips of the same layout was calculated .
the ratio of the actual average difference sum for any given chip and the mean of all average difference sums across all chips with the same layout served as a correction factor for this chip , with which all the average difference values were multiplied .
processed image ( .cel ) files served as input , and normalization was carried out according to the developer 's specifications .
next , for every array , invariant features ( defined as all the features with similar ranks of fluorescence intensity between two arrays ) were identified .
finally , a piecewise linear running median line based on the invariant features was calculated and used as the normalization curve .
after normalization , both traditional average difference values and model - based expression values ( mbev ) were calculated and exported to microsoft excel .
for invariant - set normalization on the probe - set level , all average difference levels were extracted from the .chp files and imported into ' the equalizer ' .
briefly , all values were first shifted ( by adding a constant value ) so that only 2% of the data points had an intensity below 20 .
next , values with similar ranks (  15 ) between two arrays were identified , taking the first array of the set as baseline .
finally , all data points were shifted so that the original similar - rank subset has a slope of 1 .
three different methods were used for detection of differentially expressed genes . the f - test for parametric anova and the h
( kruskal - wallis ) test for nonparametric anova were implemented in microsoft excel using standard formulas . the permutation - based method sam   is freely available to academic researchers as an add - in for microsoft excel . to enable comparisons between sam and the two anova approaches
, we considered a median false - detection rate of 1% in sam as comparable to a 99% confidence level in anova .
maximum fold changes and maximum differences in gene expression were calculated from the minimum and the maximum of the population - wise means across the five cellular populations examined .
the x - axis in all panels except ( c ) represents the not normalized average difference values ( ad ) derived from the affymetrix genechip software after scanning .
( a ) y - axis is invariant - feature normalization with calculation of ad values .
( b ) y - axis is invariant - feature normalization with model - based expression values ( mbev ) .
( c ) x - axis is invariant - feature normalization with calculation of ad values ; y - axis is invariant - feature normalization with mbev .
invariant sets and normalization curves generated by nonlinear normalization methods using identical randomly chosen arrays , ( a ) invariant - feature method , x - axis , mature b cells , replicate 3 , not normalized ; y - axis , baseline : pre - bi cells , replicate 2 .
black dots , feature intensities ; red circles , invariant set of features ( connected by the green line ) .
the blue line forms the diagonal with slope of i. ( b ) invariant set method .
blue dots form the diagonal with slope of i. axes are labeled with average difference intensities .
( a ) numbers of genes reaching a 99% confidence level in all possible combinations of normalization and statistical analysis algorithms .
( b ) percentage of genes from figure 2a that additionally reach a ratio of at least 2 and an absolute difference of at least 100 units .
ad , average difference ; f , f - statistics ; kw , kruskal - wallis ; mbev , model - based expression values ; significance analysis of microarrays ( sam ) . numbers of genes detected by one or more of the 12 possible combinations between normalization and statistical analysis .
x - axis , count of different combinations between normalization and statistical algorithm , y - axis , number of genes detected as differentially expressed in the respective number of different analysis combinations .
a total of 12 different analysis combinations ( four normalization procedures times three methods to detect differentially expressed genes ) have been investigated
. results of different methods of statistical analysis ( a ) mean , median , and maximum fold changes ( fc ) of all genes on the two array types normalized with different methods .
the averages are given across all genes of the ratio between the maximum and the minimum value across the five different conditions , calculated after averaging replicate arrays .
( b ) mean and median fold changes ( fc ) of genes detected as differentially expressed with 99% confidence by all possible combinations of normalization and statistical analysis algorithms , calculated as in ( a ) .
f , f - statistics ; kw , kruskal - wallis ; significance analysis of microarrays ( sam ) , ( c ) mean , median , and maximum signal differences of all genes on the two array types normalized with different methods , calculated by analogy with the ratios in ( a ) .
( d ) mean and median signal differences of genes detected as differentially expressed with 99% confidence by all possible combinations of normalization and statistical analysis algorithms , calculated as in ( c ) .
percentage of genes identical among all genes detected by different combinations of normalization and statistical analysis percentages are relative to the method defined in the column headings .
percentage of genes identified after invariant probe set normalization that are also identified after global scaling ( see text for details ) .
rank correlations for probability of differential expression between all possible combinations of normalization and statistical analysis procedures values of > 0.9 are highlighted in bold .
 a number of procedures for normalization and detection of differentially expressed genes have been proposed . 
 four different normalization methods and all possible combinations with three different statistical algorithms have been used for detection of differentially expressed genes on a dataset . 
 the number of genes detected as differentially expressed differs by a factor of about three . 
Background
Results
Normalization
Statistical evaluation and detection of differentially expressed genes
Discussion
Materials and methods
Gene-expression dataset
Normalization
Global scaling
Invariant feature normalization and model-based expression values
Invariant set normalization
Statistical analysis
Figures and Tables