Over engineering in the search for automated perfection thought of everything except the simplest of things.
Your technicians are aware of the protocol when there are no tasks to solve and since it is looming, like the whole y2k thing, they started looking over the source code. And while everything looks solid, one person noticed a simple oversight in conditional logic. Obfuscated, naturally, it appeared innocuous. A minor synchronization routine in place to keep all servers on current status hadn't factored in the condition of there being no tasks when the system updates. If a sync routine had begun the update process and was provided an empty queue it would time out and be subject to the automated termination, restart protocol for rogue applications. Normally this would be no problem as it is part of the design of the system. But because the sync routine was developed back when an empty queue was inconceivable, the update process condition for empty queue was left out. A simple comment in the code saying "continue update" was left in place, but no protocol defined. The update was not complete. When the service restarts, it hits an incomplete update. An error is logged, and the system falls back into the automated termination protocol. This one server in a cluster of millions is bricked.
This would pose no real threat except the scale of the system as a whole was built off efficiency and distribution. The load is sent in chunks to appropriate places in data sets such that no one service would single handedly complete a task. It is distributed, and distributed rapidly. 115
A queue could go from 7 million tasks to 0 in a matter of milliseconds. In the final queue, a mass distribution of an empty task would trigger a cloud wide incomplete update routine that bricks every server in the service. Preposterous notion to the engineers, but they have not been responsible for the maintenance of the system in decades. If they had to reinstall any part of the system they would need access to some central source and that, by this point, may be difficult to find.
While a resolution is not impossible, the threat of the whole system, or even the majority of the system going down for even one minute could cause a cascade failure in the automations the AI was responsible for. Something simple like the cooling procedure for a power source on an assembly line freezing the production of food for hundreds of thousands. The lack of cooling caused a rapid heating failure that melds the high friction components, requiring replacement. Just to name one. And the AI was appointed authority of so much that the act of restarting the system manually would require more knowledgable manpower than we as a comfortable society have to offer. Suddenly we need to be able to think again.
This idea came to me when I was fixing my 3D printer. The interface had an option to reset to factory defaults. However, a bug is known where if you reset to the factory conditions while the system is in reset mode by the automated software, it corrupts the installation and bricks the machine. Seems like a pretty stupid oversight to me, but it is possible. How possible on a distributed system, I don't know for sure, but it's all code. It does what it is told to do. Maybe someone just forgot to plug in a simple condition and thus, the end of society as we know it. 115