Differing Definitions of K-Fold Cross-Validation Elements of Statistical Learning notes that the $K$-fold cross-validation error is defined as follows where $\hat{f}^{-\kappa(i)}$ is the fitted function with the fold $k$ corresponding to observation $i$ removed (7.48):
$$\mathrm{CV}(\hat{f}) = \frac{1}{N} \sum_{i=1}^N L\left(y_i, \hat{f}^{-\kappa(i)}(x_i)\right)$$
Introduction to Statistical Learning provides a different definition (5.3)
$$\mathrm{CV}_{(k)} = \frac{1}{k} \sum_{i=1}^K \mathrm{MSE}_i$$
They differ in that the former is the average across observations and the latter is across folds. These coincide in some simple cases (e.g. identically sized folds and mean-squared error), but are conceptually distinct and seem to differ in some cases. For example, if we were interested in the median test error, the latter definition would work fine (average of the median error across folds) but the former would not. Further, if the folds differ in size, they also diverge.
Is there any guidance on which one is "correct" or on where the difference in definition comes from?

Both are commonly used and accepted. As you can see, both books have a common writer. The first one predicts all the points and estimates the loss at the end. The second one averages among the folds. For some metrics, and for equal fold sizes, both give the same value (e.g. MSE); and they are close if fold sizes are approximately equal. Authors of Introduction to Statistical Learning mentions the approximate fold sizes as well. The second one is useful while describing a standard deviation around your test score, can be seen in many publications for a cheaper alternative to bootstrapping or repeat experiments.
