This was a desync. There was probably a yellow or red exclamation point next to the time counter. From the Glossary:
Desync: In PVP dueling, when the clients desynchronize and cause the players to drop the game. Often causes odd glitches, such as monsters disappearing and other ones appearing, marks changing, monster’s stats set to 0/0, or health changes.