Thanks for your response Jung. This is not quite an adversarial example – I discovered a very similar one “in the wild” while analyzing my own implementation of beam search.
Since I’ve heard that increasing the beam width increases the probability of the best path, I attempted to test my implementation with that idea. If my implementation was correct, then it should show path probability monotonically increasing with beam width when searching through any random tree.
But instead I found that there is often dips in the path probabilities as beam width increases, especially for longer paths.
Screen Shot 2020-12-27 at 1.08.32 PM1352×1058 83.7 KB
Here’s my full implementation, tested with different lengths and a few different trees: https://colab.research.google.com/drive/1gfm61UiksXG3KGkBfkOjWdxikObCH3WA?usp=sharing 8