The retainer which I think you're referring to is a theft deterrent which prevents a thief from using a screwdriver to start the vehicle if he removes the lock cylinder with a slide hammer or similar device.
The key and cylinder must be in the on position to align the end of the actuator shaft with the slot in the inside of the actual switch to remove it properly.
You must now remove this piece and the end of the shaft will be easily removed. You can then either reinstall it or discard it as it serves no other purpose other than as a theft deterrent.
