Are the fish just knocking it over or are they nipping at it... I had a xenia on a small piece of rubble and it just started appearing all over the tank one day, I kept putting it back and it kept moving. Every time it moved it looked worse, I thought it was just stress, one day I walked in on my black back butterfly eating the xenia and dragging the rock all over the tank in the process!