Using the mean of predict_proba outputs as an indicator of potential classifier accuracy for semi supervised learning Compare these code examples:
import lightgbm, numpy as np, pandas as pd
d = pd.DataFrame(np.random.randint(10, size=(10000,100)))
d['y'] = d.apply(lambda x: x[0] if np.random.random() < 0.55 else np.random.randint(10), axis = 1)
cl = lightgbm.LGBMClassifier().fit(d.iloc[:9000,0:10], d[:9000]['y'])
print(cl.score(d.iloc[9000:,0:10], d[9000:]['y']))
pr = cl.predict_proba(d.iloc[9000:,0:10])
print(np.mean([x.max() for x in pr]))



d = pd.DataFrame(np.random.randint(10, size=(10000,100)))
d['y'] = d.apply(lambda x: x[0] if np.random.random() < 0.95 else np.random.randint(10), axis = 1)
cl = lightgbm.LGBMClassifier().fit(d.iloc[:9000,0:10], d[:9000]['y'])
print(cl.score(d.iloc[9000:,0:10], d[9000:]['y']))
pr = cl.predict_proba(d.iloc[9000:,0:10])
print(np.mean([x.max() for x in pr]))

output
 0.59
 0.6095001350341738
 0.956
 0.9934019651423959

What I'm doing is training lgbm classifier with data that is very noisy, and taking the mean of the max predict_proba values.  I then do it again with data is that is only slightly noisy.
The mean of the predict proba max goes way up for the slightly noisy versus the very noisy data.
Is this a reasonable method of predicting the accuracy of a classifier without access to the labels?  I can't find any literature on this seemingly very important topic.  Papers, terminology, etc welcomed.
I understand this is a type of semi supervised learning, but can't find anything talking about this evaluation metric in particular.  I also understand that this might only roughly work under certain assumptions.  What would those assumptions be?

I may not be understanding your question correctly, so apologies if my answer is off topic.
Using predict_proba to construct a point prediction is acceptable.  However, to make a reliable probabilistic prediction (i.e. understand the accuracy of your prediction) you need to form a predictive p-value or construct a prediction interval. For example, a 75% prediction interval constructed from a sample of size $n$ will cover the next observed result 75% of the time in repeated sampling regardless of the unknown fixed true model parameters.
This type of probabilistic prediction is easiest to construct when the prediction target is a continuous endpoint.  When the sample space is discrete as in a logistic regression the possible error rates of the prediction interval will only come in discrete chunks.
