Bores of many shapes have been tried .There was the thought that square bullets were more effective against infidels , but that was a very long time ago .

Never actually seen a square smooth bored gun but one glaring problem I could foresee would be the disadvantage of the strength of the barrel . Lets be honest have you ever seen a square hose pipe or pressure pipe? To make such a barrel the extra weight due to the thickness of the walls would be impractical in any sporting gun. Let alone the sheer impracticality of cartridges ,wadding etc .