No nerf needed, especially not immaterial removal
There are only few "common" permanents in the game...
Support:
Pillars - stacks, figments count as substitution
Soul Catcher - figments count as substitution
Mindgate
Hourglass
Cloak
Shards - can stack
Survival:
Empathic Bond, Sanctuary
Shards - can stack
Flooding
Damage:
Unstable Gas - UG decks usually have a secondary form of assault
Shards - can stack
Sundial
Catapult
Note 1: After the "common permanent" area is filled, Shatter only becomes as powerful as a second Deflagration.
Conclusion: Not only is the range of the swarming small, but also the extremely slow permanent swarming is negligent compared to the basic PC ability. Therefore, I do not believe that the swarming effect is enough to warrant a nerf.
Note 2: If it's really that bad, you can do something like "For 8 turns, generate 1 random quanta. Immaterial." Giving the opponent an option to waste PC on figments won't nerf it that badly.
Btw, since I hate hard PC, I'd rather have this "delay" (make unusable) the target for a few turns.