The ejector sear itself, the sear edge on it, may be worn so it releases the ejector early like Mike says.

The corresponding sear notch in the ejector hammer on that side can be worn so the sear doesn't engage securely and again releases the ejector early,,
Both of these conditions may exist together on the same side.

There may be nothing more than dirt and built up crud in the ejector hammer sear notch not letting the ejector sear engage properly

The ejector sear might be a replacement from some point in it's life and was just that,,replaced w/o any fitting and adjustment done.
It works but the bottom edge of the back end that you can see inside the forend iron 'shoe' is sitting too low.
That's where the thin blade exits the front of the frame upon firing and lifts the ejector sear. Engage & lift it too quickly and the ejector fires too soon,,,too late and it may not eject at all.
Closing the gun cams that thin blade back into the frame so if the bbl is not fired,,the bbl does not eject.

.
It can be fitted/corrected a couple of different ways to make it time correctly. It takes a close look at the mechanism as it is and good understanding of how it works before going about filing and bending things thinking it might correct the problem.
Best way to look at it is to take the forend iron assembly out of the wood. Then place the complete forend assembly back on the gun. With snap caps in the chambers, dry fire and very slowly open the breech. You will be able to see the blade engage the ejector sear and pivot it, then releasing the ejector itself. Usually you can figure out what needs to be done.
This is easiest to observe if you place the bbls in a padded bench vise sideways. This gives you leverage to easily open the gun very slowly and watch whats going on.

>>Is it just annoyingly a bit out of time, or does the left bbl eject the shell and it hits the standing breech instead of clearing it.