Gunter,
If you do that, it shouldn't be that much more trouble to go ahead and put a screw in to hold it. I think I would use some force to push it in for the loctight to set( C clamp?. If it was flush before and stands proud now, there must be some space behind it(you may want to "break the edge" at the rear end so it doesn't keep the bushinh from seating all the way.
Mike