I personally think that, as is, it doesn't make any sense for TU to work opposite the creature, since every spell is treated as the caster casting it. Making TU the exception would be silly, in my opinion.
If you are arguing that all spells cast by Pandemonium should work as if the opposite player was casting them, then that would make more sense to me at least, but I haven't seen Siphon Life, Fire Bolt, or Ice Bolt mentioned here, all of whose effects are also determined by the caster. I personally could live with all of them being flipped, which I think would satisfy the people who think it should only be negative effects, but I prefer it to work the way it does. Especially considering the comment someone made earlier about Chaos Seed. I don't think Chaos Seed needs the nerf, either, but it doesn't make any sense to make them different. I have Chaos Seed'd my own creatures many times. I think only once or twice have I actually gotten the PU out of it and swung the match, but I wouldn't like to see that strategy eliminated. Pandemonium just casts Chaos Seed on every creature, for all intents and purposes. I don't see why Chaos Seed has been fine for all this time and now Pandemonium somehow doesn't make sense. It makes perfect sense and it's completely consistent. It would be different than Chaos Seed but equally consistent if all effects were caused by the opposite player, but just flipping TU makes no sense to me at all.