It's an interesting concept, but there's definitely a few issues with the card.
First of all, as OldTrees implied, the effect is really ambiguous. What if the RNG pulls out a spell? Would it immediately cast the spell? Would the spell's caster be based on the targeted creature's owner (i.e. if I target my own creature with the effect and the RNG replaced it with a spell, who would it be cast by)? Same goes for permanents -- who would control the permanent if I used Dimension Shift on my own creature?
Even if everything is explained a little clearly, it still would be quite difficult to code for, since the game would have to somehow choose a random card in your opponent's deck, and also make sure that any non-creatures chosen don't go into creature slots.
Here's a small modification to the effect that stays true to the spirit of the original, but would be a lot easier to code, and hopefully less confusing to understand. Feel free to consider it (or not) and modify it for balance:
: Dimension Shift -- Return target creature to your opponent's hand, then put a random creature from your opponent's hand into play under your control.
This would make for some interesting synergies with cards like Nightmare, and would also be an indirect way of 'reseting' a creature if your opponent has no creatures in their hand when the ability is used.