Question: I'm trying to understand the Bayes Classifier. I don't really understand its purpose or how to apply it, but I think I understand the parts of the formula:

$$P(Y = j \mid X = x\_{0})$$

If I'm correct, it's asking for the largest probability, depending on one of two conditions. If $Y$ is equal to some class $j$, or if $X$ is some data point $x\_{0}$.

How would I compute this with a data set $(x, y)$ where $x$ is just a number between 1 and 100, and $y$ is one of two classes ("blue" or "orange"), e.g. (5, "blue"), (51, "orange")? Does this data set even work to apply the classifier or should I consider making a new data set?

Sorry if it's a silly question, I'm out of touch with my statistics. Some pseudocode would be terrific, but I'll be applying this in R. I'm not interested in the R function to complete this. Some regular guidance with good ol' math would be great as well.

Thank you for any help!

Answer: 
The Bayes classifier is the one that classifies according to the most likely category given the predictor $x$, i.e.,

$$
\text{arg max}\_j P(Y = j \mid X = x) .
$$

Since these "true" probabilities are essentially never known, the Bayes classifier is more a theoretical concept and not something that you can actually use in practice. However, it's a helpful idea when doing simulation studies where you generate the data yourself and therefore know the probabilities. This allows you to compare a given classification rule to the Bayes classifier which has the lowest error rate among all classifiers.
Interpret the formula as follows: What is the probability of Y being equal to j, when we know X = x0. So in your dataset, the bayes classifier is effectively computing probabilities of achieving blue or orange when you define the value of x. If in your data, when x is greater than 75, if 90% of the balls are orange, then the classifier will choose orange whenever this happens.

This is a very "non-technical" explanation and I hope it helps you understand the basic idea.

So when someone chooses to use a Bayes classifier (or any other classifier for that matter) you use it to predict categorical outcomes based on one or more input variables that may be continuous or categorical.