I've collected and researched Bakers for quite a while and I agree with you that they compare favorably quality-wise with the other big name American doubles of the time. In my opinion they fell short in sales largely due to a lack of strong and consistent leadership for the firearm making portion of the Baker Gun & Forging Company.
If you look at the history of the company you will see multiple changes in leadership from about 1898 through its sale to Folsom in 1919. To go along with those changes, there seemed to be little consistency in their direction as well with models changing names or models being dropped and others being introduced too frequently (IMHO). But more than the inconsistent nature of their lineup, I think they faltered because they didn't seem to market themselves as aggressively as the other big makers.