If you look at your second image, the rectangular hole pointing up at the camera is where the needle (aka "stylus") is pushed in. The plastic stylus carrier should be held between finger and thumb, with the stylus tip pointing towards you. There is a brass tube on the carrier the same size as the hole, so it's just a matter of pushing it in gently until it stops.
This looks like a Shure M75 cartridge (it's difficult to read the label on the bottom, but maybe there's identification on the front), for which you would need an N75-type replacement stylus. The original Shure replacement needles aren't made any more, as this is a very old model. However, "aftermarket" replacements are widely available from other suppliers.