Is accuracy affected by having a barrel band vs a sling swivel on the stock? I imagine it would affect it some, but, maybe not all that much the closer it is to the action.

Out to 100 yards, it probably doesn't affect it much.