Proper method for comparing rates between subsets of a population I've been handed a bunch of historical data on performance ratings of individuals in a number of organizations, and asked to identify any instances where there appear to be "abnormalities" in the distribution of ratings.  This is not to immediately throw a "bias flag" but to identify cases where more scrutiny may be called for.
For privacy reasons the data below is notional, but representative of one of the orgs in question.  Many of them are significantly larger than this, but I chose to represent one of the smaller ones thinking that a methodology would "scale up" better than down...
At this location, the racial breakdown of the ratings is




Race
Count
Percent




Black
47
5.0%


Hispanic
18
1.9%


Other
24
2.5%


White
855
90.6%




Individuals are given a rating of A, B, or C.  In the aggregate those broke down as




Rating
Count
Percent




A
363
38.5%


B
482
51.1%


C
2
0.2%




And finally, by racial category




Rating
Black
Hispanic
Other
White




A
17
4
7
335


B
24
11
15
432


C
0
1
0
1




My initial thought was to look first at the population proportions...so for instance black people were 5% of the population, so you should expect them to be roughly %5 of each rating.  And for this data they were 4.7% of the A's, 5% of the B's, and 0% of the C's but there were only 2 C's.  So for A's there's a discrepancy of 0.3% from expected and 0% for B's.
It was then put to me that I should start instead with the distribution of the ratings.  Since 38.5% of the ratings given were A's, roughly 38.5% of black ratings should have been an A.  And in this case 41.6% of black ratings were A's.  That's a discrepancy of 3.1% which, although not huge, is certainly more than the 0.3% from the other method.
And in either case, I have no idea how to talk about statistical significance.
So I'm hoping someone can point me to a resource or outline an appropriate methodology to make these comparisons in a defensible manner.

I think the best way to approach this would just be to use the last contingency table you provide and to utilize Pearson's Chi-squared test (or a similar test). Here, you're testing whether the rows in your column are independent of the columns, i.e. do these racial categories tell us anything about the rankings.
This is how you'd do it in R:
tab <- matrix(data = c(17, 24, 0, 4, 11, 1, 7, 15, 0, 335, 432, 1),
              nrow = 3,
              ncol = 4)

chisq.test(tab, simulate.p.value = TRUE)

Simulating p-values to avoid relying on the asymptotic approximation to the Chi-squared distribution, we get a p-value of 0.021.
