I would call and order two custom bullet sizing dies from Lee precision. One in .320 and one in .317. this would allow you to resize any standard .323 bullet.
I do that to go from .338 to .333 (.333 jeffery) and .330 (.318 WR).
It's easy enough and very cost effective.
One has to order the final die .001 less than final as the bullet springs back a bit after going through the die.
It takes about a month to get the dies.

If starting from .321, you probably need only the final die.
It's better to keep resizing at or below .004 in one pass.
Best regards,
WC-