Checking consistency of answers within groups (e.g. household) in R I got some survey data, in which the respondents answered various questions about environmental conditions on a 1-5 Likert scale. The respondents are also assigend to a household by an household ID. So e.g. there are 1 person living in household A, 3 persons in household B, 2 person in household C, 1 person in household D, etc.
Now I would like to check, whether the answers to the questions about environmental conditions are consistent within households. Or at least if it's more likely for persons living within a household to give the same (or a similar) rating on environmental conditions than people living in different households. (The hypothesis behind this is that people living in the same household should face similar conditions).
But I am kinda stuck on how to test that (in R)... any ideas/advice? :)
Thank you very much!

Two approaches that you can use:

*

*Compare the proportion of variance within an between groups - this is called the intra-class correlation (ICC). An intraclass correlation of 1 means that there is no variance within a group (all the members of the group agree) and there is variance between groups.  An ICC of zero means that there is as much variance within groups as between groups - a person is no more likely to give a similar answer as someone in their household as they would be to someone outside their household.


*Use some flavor of Cohen's Kappa - which assesses the extent to which agreement is greater than would be expected by chance. (E.g. if everyone has a 90% chance of saying 'yes' then you expect 82% agreement by chance - 82% agreement would therefore be a kappa of 0, representing only chance agreement.
There are several packages for calculating each of these. Use your favorite search engine.
