A man who was jailed after his Audi hit another car while he was on the wrong side of the A46 near Market Rasen has lost his bid to overturn his driving ban.