We cage students to protect ourselves.
Too many teachers (and others) are way too afraid of open-ended exploration for any number of reasons. Perhaps most commonly, the problem is not knowing the territory.
If I want to allow my students true open-ended exploration of a novel, then I have to know the territory. It's a big sprawling place, like a great forest, and if I'm going to let my students wander around in it, I need to know each stream, each hollow, each hidden pit, all the flora and fauna. I can only be effective as "the guide on the side" if I'm familiar with all the places we might go. I have to know the material far better than my students do. What if they ask questions I can't answer? What if they come up with ideas I'm not prepared to discuss?
There are other potential problems with open-ended instruction. Some are practical; if our discussion of symbolic threads in Song of Solomon is open-ended, we might run out of steam in twenty minutes, or the discussion might run for a week. How do I cover that in my lesson plans? How do I keep my class on a particular schedule and still allow the students to roam about the grounds till they're truly finished?
There's also the problem of letting go of The Answer. If your belief is that there's really only one way to see "Stopping by Woods on a Snowy Evening," you're going to stink at open-ended exploration. I know people who think they do this, who really believe that their students are free to explore any ideas about literature "as long as the students can rationally support them." But their measure of rational support is whether or not it leads to the conclusion they've already settled on.
Biggest problem? Standardization. If we are going to be able to measure and tag and compare across the country the responses and reactions to the literature, we can't have 143,257 responses scattered across the landscape. True open-ended answers are too diffuse to quantify.
So to make our lives easier, we leash the students. We build a trail and command them to stay on it. We drive them through the forest on a tour bus we won't let them leave. We build a big pavillion and tell them that's the only destination they are allowed to reach. We eliminate options, reduce their choices, reign in their ability to explore and discover.
For standardized testing, we do even worse. We lie. We say, "You can open-endedly explore this forest (well, on most tests, a small garden)." And then we judge their constructed response on whether they arrived at the one "correct" destination; a multiple choice question for which they have to write their own answers. Or we ignore the destination they reached and judge them on something simple that we can quantify. We don't care if they land on gold or in a pile of poop, as long as their boots are properly laced. On a "Give three reasons for..." question, we don't care if the reasons are brilliant or stupid; we just want them to have three of them. What pretends to be a reading question is really only testing compliance and counting.
The continued push to create teacher-proof programs, programs that can be taught by any content delivery specialist, ignores a crucial facet of this approach-- it lives or dies, rests completely upon the degree to which we limit the intellectual freedom of students. The best teachers are wiser and more knowledgeable than their students. The only system in which that is not true is a system in which students are not allowed to be smart, curious or knowledgeable.
Drones can only teach other drones. Intelligent human beings can only be taught by other intelligent human beings. When you create a system that tries to turn intelligent human beings into drones, everybody suffers.