Those are adhesive caps and can be removed by using a pin or dental tool with a sharp point. Use a very thin coating of all-purpose glue (the kind preschool children might eat) coating to re-attach the sticky side or leave them off (I reattached mine).
just don't seem to want to come out unless I destroy them.
Someone might have used the wrong adhesive or glue.
Removed and Reinstalled