Prediction Intervals on Judgment Forecasts - Possible? Are there any R packages available or general methodologies for calculating prediction intervals on multi-step judgmental forecasts?
I've looked at Hyndman's text and the R forecast package - which will provide intervals for each period in time, but also a point estimate - which may differ from the judgment forecast, therefore the intervals aren't necessarily applicable.
I read the portion of his book on judgment forecasting, which speaks to the pervasiveness of this technique, as well as the fact that its, of course, entirely sound (though maybe not the best approach) to create a judgment forecast based off of a combination of historical data, domain knowledge, knowledge of changes in the market, etc.
He also discusses generating forecasts via a model (naive, mean, etc.), then tweaking those forecasts further with judgment -- but he doesn't go on to discuss how the PI itself would also evolve.
Just curious if anyone has an experience or advice on this front - generating prediction intervals on judgment forecasts, any advice/suggestions are much appreciated.
Text for reference if interested: https://otexts.com/fpp2/index.html

I see various possibilities.

*

*You could ask your judgemental forecasters to not only provide their "best guess", but also provide intervals that they are (say) 90% certain the actual will fall in. This is a frequently studied setup. Under laboratory conditions (so: no "real" forecasters, just undergrad students or Amazon Mechanical Turks), it invariably yields intervals that are too narrow. You could in principle post-process judgmental interval forecasts, but once your forecasters learn about this, they will game the post-processing.

*If you have system forecasts that do provide prediction intervals, you could shift the entire PI up or down by the difference between the judgmental forecast and the system expectation forecast. Essentially, if your system gives a 90% PI of $[1,10]$ with an expectation forecast of $3$, and your judgmental forecaster gives a forecast of $5$, then you just shift the system-generated PI up by $5-3=2$ to obtain a PI of $[3,12]$.

*If you have a sufficiently long history of judgmental forecasts and corresponding actuals, you can compare the two and get a distribution of the errors. You can then either directly take quantiles of the historical errors, or fit any distribution to the errors and take quantiles from that. Of course, this presupposes that your error distribution remains stationary.

