Why does the A320’s maximum allowable aileron deflection decrease, rather than increase, when the flaps are extended? According to this answer to why the A320 uses mainly spoilerons for roll control during landing, rather than conventional ailerons, the ailerons can’t extend as far when the flaps are extended, for fear of stalling the wing:

Downward aileron travel must be restricted when flaps are deflected. Since flaps change the local incidence on the flapped part of the wing, the outer wing will experience an increase in its local angle of attack. The increased suction over the inner wing will accelerate not only the air flowing over the inner wing, but also that air which will flow over the outer wing as well. Left to itself, the outer wing would stall if powerful Fowler flaps are deployed on the inner wing.
Adding slats will push the stall angle of attack up, and now the outer wing is back in business. However, if the aileron is deflected downwards, the stall angle of attack is reduced again. The outer wing could stall simply due to the aileron deflection!

Which would make sense - after all, if the trailing edge of the wing is already angled more down than usual, then it can’t safely extend as far further down for roll control than it would be able to do otherwise - if not for the fact that the ailerons aren’t located on the portion of the wing with the flaps on it (they’re located near the wingtip, where they’ll be more effective at rolling the aircraft, whereas the flaps are located near the wing roots, where there’s more space to store them and they won’t increase the bending stresses on the wing as much), and, since extending the flaps increases the amount of lift produced by the wings at a given speed and vertical load factor, thereby allowing the aircraft as a whole to fly at a lower angle of attack than it would be able to do with the flaps retracted, the A320’s allowable aileron deflection should increase when the flaps are extended, since the lower attack angle of the unflapped, aileron-containing portion of the wing should allow the ailerons to deflect further without stalling.  Indeed, the A320’s direct competitor, the 737, exhibits precisely this behaviour, where higher flap settings increase the allowable aileron deflection (and, thus, the lateral control authority available) for a given indicated airspeed and vertical load factor1 - so why does the A320 behave in precisely the opposite way?

1: This means, for instance, that the 737’s crossover airspeed, like that of essentially all aircraft, decreases as the flaps are lowered, whereas, if the aforementioned quote were accurate, the A320’s crossover airspeed would increase at higher flap settings!
You assume that


  the lower attack angle of the unflapped, aileron-containing portion of the wing should allow the ailerons to deflect further without stalling.


but that only looks at the geometric angle of attack. What counts, however, is the induced angle of attack. Since the flaps on the inner wing create much more lift, the induced angle of attack is much higher than for the clean configuration at the same geometric angle of attack. Consequently, the outer wing will stall at a much lower geometric angle of attack, and adding trailing-edge-down aileron deflection will stall the outer wing with only a medium angle.

Add to this that the spoilers are much more effective for creating rolling moments once the flaps are deployed and will avoid the adverse yaw of an aileron deflection due to their high drag on the down-going wing, and the spoilers are a much safer option for roll control that the ailerons are in landing configuration.