Residents and business owners said groups of people congregating on a busy road in Blackburn are 'driving customers away'.