Bob,
A bullet needs gyroscopic stability to gain it's best accuracy. In single shot rifles bullets are generally seated back from engaging the lands by .001 to lessen bullet jump and maintain consistent accuracy and to breech seat it must engage and hold, then full bore length rifling engagement. Many rifle barrels are 'choked' tighter at the muzzle.
Are you shooting paper patched or lubed and sized bullets or as cast...?...For larger diameter bullets with your current mould use linotype as cast.
Cast bullets are usually cast and sized .001 or .002 larger than groove diameter of the bore, no more or the lead will create fins on the back of the bullet. For jacketed bullets .001 should be the max bore oversize.
If your bullets are not engaging the rifling as it leaves the chamber throat, then you need to increase the bullet diameter so it does engage the lands and grooves for accuracy.
As you describe the scenario above, it is the perfect formula for bullet key holing and poor accuracy.
I would definitely change moulds after slugging the bore with various diameter bullets.
Best of Luck with it,
