The only way I have found that works is what SKB calls a gun drill. Mine were made with a jobber length drill bit welded onto a rod of the exact same size. It requires frequent pulling it back to clean the wood bits as the hole goes deeper. A true gun drill has just a small channel milled into the rod and is sharpened like a drill bit with only a single flute. The drill from both ends is a way hide any drift in an area with the most material to work with. 
It takes me a long time to drill a stock bolt hole. I am going 1/4”-3/8” or so at a time, then cleaning out and repeating over and over again. But you do not get more than the slightest drifting any. I have more time than good blanks and nothing will ruin your day like screwing up a nice blank.