For the warmaster part, I remember thinking about that at some point, too. Then I forgot.
Anyway, unless the organizers were really lazy, I'd go so far as giving them a bonus that equals what their opponent gets. It's only for the final battle, so you're only just taking the disadvantage away.
I suppose you could do something similar for brawlmasters... but I don't agree with doing it for the voting panel. Not the full amount, anyway.