Those worms only eat whats already dead.
Heres something that might help.See if it matches what you got going on.
Its in the disease section of Aquarium Corals by Eric Borneman.
Mushroom Condition 1 affects corallimorphs,primarily from the genus Actinodiscus,which fail to thrive.They shrink,their tissue becomes mottles and pale,and they seem unable to regain the ability to expand.This behavior seems remarkably like wasting of certian anemones,and the similarities in the biology of corllimorphs and anemones may be relevent.As in zoanthid condition 1,no amount of water chemistry manipulation seems to have a pronounced effect.The environment of the corallimorphs may have remained virtually unchanged in terms of normal additions,lighting,and water flow for long periods of time when this condition occurs.Individual mushrooms or whole colonies can be affected.Affected mushrooms may remain in such a state for many months,or they may die fairly quickly.Occasionally,individuals or colonies may spontaneously recover,though this is rare.