Though, even if it was actually something that applied here, I don't think your gym would be the last one before the Elite Four, I think you could probably go either way with it. As long as you give them a free heal between challenges, making every trainer defeat both of you before they got the badge might make things interesting.