New councillors, school governors and civil servants would be expected to say the oath, the UK Communities Secretary Sajid Javid has proposed.
He said it would highlight values of democracy, equality and free speech.
But Scottish Lib Dem MSP Alex Cole-Hamilton said there was no place for the oath in British society.
Mr Javid's proposal comes after a report by Dame Louise Casey warned of increasing ethnic segregation in some UK communities.
He said he wanted public servants to set an example to newly arrived migrants.
Writing in The Sunday Times, he said: "We can't expect new arrivals to embrace British values if those of us who are already here don't do so ourselves, and such an oath would go a long way to making that happen."
His view was supported by Mohammed Amin, the chairman of the Conservative Muslim Forum, during a discussion on BBC Radio Scotland.
Speaking in a personal capacity, he told Good Morning Scotland that a councillor who was not willing to commit to individual liberty or the rule of law was "not fit" to hold public office.
However, he was challenged by Mr Cole-Hamilton who said: "This is just absolute gimmick politics and I don't think it has any place in our society.
"If you aren't as tolerant as us, if you're not willing to say you're as tolerant as us, then you're not welcome here.
"I think there's a cold irony about that."
Mr Amin suggested that the move could prevent extremists from holding public office but Mr Cole-Hamilton warned that it could encroach on freedom of speech.
"If you're saying if you have those beliefs then you can't hold public office, then you're pushing people into an underground," he said.
"If you can't defeat the arguments of people like that in open democracy, then this isn't a democracy and you've lost the battle already."
Mr Javid said he did not want to see a "government-approved, one-size-fits-all identity" where everyone "drinks tea, watches cricket and bobs up and down at the Last Night of the Proms".
But, he added, people would struggle to play a positive role in British life if they did not accept the "building blocks of our society".
His plans have been supported by former chancellor George Osbourne and ex-culture secretary John Whittingdale.
Plans to require public servants to swear an oath to British values have been described as "absolute gimmick politics".