It's hard for me to pin down to just one, but I'll give it a go.
I'm running a RoL/Hope deck at the moment. This means that I'm vulnerable to all creature control which affects all creatures at once (like Plague or Rain of Fire) and also Fire Shield and Spine Carapace. But I think, for me, the worst card to come up against is Steal. If my opponent has a few Steals, then they'll usually win because they'll take my Lobotomiser, along with a few Towers/Pendulums, and they'll use it to gradually chip away at my defence. They can also neuter my Light Nymph, preventing me from making my neutered cards useful again (as I could if they had a Mind Flayer that I had to wait a few rounds to lobotomise, and only then played my Nymph).
It's not quite as devastating to my deck as others can be, but it is more annoying than, say, a Fire Shield because it's using my own card against me, or can leave me with 3 unplayable cards in my hand - a big drawback if you're using Fractal.
What do you hate, and why?