What is the right statistical test to use when attesting claims like "60% people switched over from group A to group B"?
My problem is this:
There are 52 people divided into 5 groups: A, B, C, D, and E. From stage 1 to stage 2, they either stay in the same groups, or change groups. The following table shows how many people change from their stage 1 group to stage 2 group for all pairs.
Stage 1                   Stage 2
Group A   Group B   Group C   Group D   Group E
Group A    18         0        0         0         2
Group B     0         6        0         0         3
Group C     4         0        2         0         2
Group D     0         0        0         0         0
Group E     4         2        0         1         8
E.g. 18 people change from group A to group A (i.e. they retain their group); 2 + 3 + 2 people change from groups A, B, C, D to group E.
Now, my claims are these:
65% people (18 + 6 + 2 + 0 + 8 = 34 out of the total 52, the diagonal) do not change groups.
15% people (8 out of 52) change from some other group to group A.
13% people (7 out of 52) change from some other group to group E.
I want to attest these claims with the correct statistical test. Which one is the right one to use? I'm aware the numbers are small, so any test might be unreliable, but I'm curious to know what test it will be any way.
hypothesis-testing classification statistical-significance
Mikhil Masli
Mikhil MasliMikhil Masli
$\begingroup$ My logic so far: From stage 1 to stage 2, every person has 5 options to choose from for their stage 2 group (A, B, C, D, E). So, the chance that he/she will retain his/her own group is 20%. So, the expected number of people who will retain their own group is 20%. Comparing 65% with 20% using a 1-sample proportions test with continuity correction (in R: prop.test(34, 52, 0.2)), I get: Chi^2 = 54.14, df = 1, p < 0.001, meaning that my claim 1 is statistically significant. $\endgroup$ – Mikhil Masli Feb 8 '11 at 2:25
$\begingroup$ I'm sorry, but when 65% stayed in their group it isn't very probable, that the chance for staying in your group is 20%. $\endgroup$ – user unknown Feb 8 '11 at 6:59
I think what you're probably looking for are the so-called mover-stayer models. The basic model is a loglinear model that allows for an "inflated" diagonal and usually involves some form of symmetry on the off-diagonals.
A good place to start is A. Agresti, Categorical Data Analysis, 2nd ed., pp. 423--428. (That's the big green book, not the smaller blue or white/brown one with almost the same name!)
You might also look at J. K. Lindsey, Applying generalized linear models, Springer, 1997, pp. 33--35.
Addendum: The log-linear version of the mover-stayer model is also often referred to as a quasi-independence model. Two forms are popular. The more general form is
$$ \log \mu_{ij} = \mu + \alpha_i + \beta_j + \delta_i \mathbf{1}_{(i = j)} , $$
where $\mathbf{1}_{(x \in A)}$ is the indicator function that $x \in A$.
Let's interpret this model. First off, if we drop the last term, then we recover the model for independence of the rows and columns. This would mean assuming that people change groups in such a way that the beginning and ending groups were independent of each other, even though each particular group (before and after) may have different marginal distributions.
By throwing in the $\delta_i$ into the model, we are instead assuming there are two groups in the population. There are the movers who will tend to move from one group to another distinct group and there are the stayers who tend to be happy with their first choice. When the population breaks down into two such groups, we expect to see an inflated diagonal due to the stayers.
The model is also called a quasi-independence model since it is a model for independence on the off-diagonals. This means that, among the movers, there is no interaction between the group they started in and the one they end up in ("drifters" might be a more evocative term).
Notice that if we have a separate $\delta_i$ for each $i$, then the diagonal cells will be fit perfectly by this model. A common variant is to replace $\delta_i$ by $\delta$, i.e., each group has a common affinity to "stay" versus "move".
Here is some $R$ code to fit such a model
# Mover-stayer model example
Y <- rbind( c(18,0,0,0,2),
c( 0,6,0,0,3),
c( 4,2,0,1,8))
grp <- c("A", "B", "C", "D", "E")
colnames(Y) <- grp
rownames(Y) <- grp
X <- expand.grid( row=grp, col=grp)
X <- cbind( X, sapply(grp, function(r) { X[,1]==r & X[,2]==r }) )
y <- as.vector(Y)
df <- data.frame(y,X)
# General model
move.stay <- glm(y~., family="poisson", data=df)
move.stay.summary <- summary(move.stay)
# Common diagonal term
move.stay.const <- glm(y~row+col+I(A+B+C+D+E), family="poisson", data=df)
move.stay.const.summary <- summary(move.stay.const)
At this point, you can do lots of things. If you need a formal hypothesis test, than a deviance test might be appropriate.
Your particular data certainly has the peculiarity of a row of all zeros. It's interesting that there was a "group" choice that contained no one in it to start with.
cardinalcardinal
$\begingroup$ Can you please give a little more information. For example, what are the hypothesis being tested with this model? Are there any R implementations using @Mikhil's data? $\endgroup$ – B_Miner Feb 8 '11 at 13:19
$\begingroup$ @user2040, ok. I'll try to post something more in a bit. I noticed the Lindsey book can be viewed on Google Books. A standard glm call using family="poisson" can be used for the fitting and testing. All it requires is to get the symmetry covariates coded properly. $\endgroup$ – cardinal Feb 8 '11 at 13:27
$\begingroup$ @B_Miner, @Mikhil, added model description and code. Hope this helps. $\endgroup$ – cardinal Feb 9 '11 at 4:24
$\begingroup$ This is great, thanks so much. I need to study it and look at the preceding chapters in Agresti to truly understand. $\endgroup$ – B_Miner Feb 9 '11 at 23:53
$\begingroup$ @B_Miner, I'm not sure what your familiarity with GLMs is, so forgive the suggestion if it is too elementary. But, if you want a good quick overview, look up the notes for log-linear models by G. Rodriguez---at Princeton, I believe. I came across them once and they seemed like a good way to get quickly up to speed for someone who hadn't had much previous exposure. $\endgroup$ – cardinal Feb 10 '11 at 4:12
I would start with building the sums:
A   B   C   D   E | Sum
Group A    18   0   0   0   2 |  20
Group B     0   6   0   0   3 |   9
Group C     4   0   2   0   2 |   8
Group D     0   0   0   0   0 |   0
Group E     4   2   0   1   8 |  15
Sum        26   8   2   1  15 |  52
As long as you don't tell us, that this is just a sample from a bigger population, I don't see any statistic involved, just frequency distributions.
From A to B went 0 person, not much to calculate here. 2 Persons went from A to E. A had initially 20 members (am I right?) and 2/20 is 10%. 10 Percent of the A group went to E. How many people have been later in group A, or where in total is not of interest here. If you ask 'How many from group E came from A, then you would use the total of E in stage 2; there are 15 people, two of them came from A: 2/15.
user unknownuser unknown
Not the answer you're looking for? Browse other questions tagged hypothesis-testing classification statistical-significance or ask your own question.
What statistical techniques to use to distinguish between two groups?
What statistical test allows comparison of the 3rd quartiles between groups?
what test to use for comparing count data across multiple groups
What test to use to compare two statistical distributions with 1 variable and many categories
Looking for a statistical test which will test whether the "spread" from a line is statistically significant