I'd bet anything that it's just hitting that stuck detent when you either move the lever up or down. It will move about 3/4 of an inch or so before it hits the detent. It's good to be cautious, so I completely respect that. If it were my bike I would keep hitting it with Liquid Wrench or PB blaster, and keep letting it soak. Then tap it from underneath with a punch, then do some gentle tapping by pushing the shift lever either up or down, then adding a few additional taps with a small hammer. Just my humble opinion though, so do what you think is best. I would just keep trying that before I opened it up.

Yep, I think I'll let it soak some more and then try again

Can you pull it out with some hemostats you could I guess this might work is to spin the counter shaft sprocket while putting it in first gear and see if it engages  pictures would help

Could you pop the bung out of the drum end under the sprocket [without ruining it] & then get something expandable into the end of the drum & give it a bit of twisting force. Its pretty solid in there. By memory there's a bit of a lump in there maybe where the dowel pin fits in that locates the cam, so not a smooth round hole as such.
Here's a random google image but not a straight on image looking in there as such. 

I wonder if the right size screw extractor could be used to twist it, and break it free?

 

I like the screw extractor idea Tom, I don't have a set but I could buy an individual one 

The pin that is stuck is hardened steel and rounded on the bottom. Can't budge it and I think I'm just making it worse. Might as well be welded in there lol. I moved on to trying to start the bike. I have spark and fuel flows from the petcock but I'm sure the carb is gummed up and probably has a stuck float. I kicked it 20 times for fun but didn't expect anything. I should probably pull the carb next.

The plunger is definitely still in there?. I've heard of them found in the oil tray later. Early ones had a short spring & a slid plunger rounded one end. Later had the long plunger hollow inside & a long spring [happy to be corrected].
I very much feel it's very unlikely the plunger is rusted to the cam or rusted/corroded into the case. But gotta be something unusual going on. Might be selectors rusted to the drum. Get back in behind the clutch & force the drum with pliers & if it won't free then ya going to have top go in.

I wonder if it actually has a ball bearing detent, like the earlier bikes? Is it really a '73? Haven't seen a pic of the bike yet.

The sticker says 73. It looks more like a ball bearing up in the bore using my mirror but it's hard to say. Whatever it it it has zero play - I agree that I need to drain the oil and try another method to free up the shifter.