I normally quit (not rage qit, just quit for efficiency purposes) against Hermes, Graviton, and Rainbow. Their damage speed is too much for me and then they have control sprinkled in too, in case my hand is good.
My personal least favorite is Divine Glory though, and I sometimes ragequit against him. To me, the only reason he is so hard is because of the double decks FGs have.
I quit the most against Hermes because he likes the most to blow up my shields. At least I can neurotoxin Rainbow.