Mr Perks, who runs property renovations company Changing Rooms, failed in a three year battle with HM Revenue and Customs after they ruled he was VAT registered when he thought he was not.
Mr Perks is also standing for UKIP in next week's Dudley Council elections for Upper Gornal and Woodsetton.
Leader of Sandwell Council Councillor Darren Cooper urged the UKIP candidate to immediately withdraw from the parliamentary and council races.
He said: "Both MPs and councillors have a great responsibility to make important decisions on public money so how can Mr Perks have any credibility when he cannot even run his own finances without getting in trouble over VAT.
"He should stand down or UKIP should deselect him."
Cllr Cooper added: "This is the man who said he looked up to Enoch Powell and believes Islamic Sharia Law has a place in our society, he is a laughing stock.
"And this joke candidate is indicative of all the stories across the country about the kind of people that UKIP attract."
Chairman of Halesowen and Rowley Regis Conservatives councillor Jeff Hill also questioned Mr Perks candidacy. He said: “As Conservatives, we want taxes to be as low as possible for everybody, but that means that everyone - businesses and individuals - must pay their fair share.
“For a parliamentary candidate to have attempted to gain an advantage over law-abiding competitors, by avoiding paying tax that he properly owed, is bound to raise questions over his suitability.”
Mr Perks told the News the episode was down to him being misinformed by his accountant and not through trying to avoid tax.
He said: "I was never guilty of not paying VAT, I have never refused to pay a bill back in my life. I am paying back what I owe now.
"I was misinformed by my accountant who told me I was deregistered when I was in fact not, so when I had a tax inspection a couple of years after they clearly wanted their chunk of my earnings.
"I went through several tribunals and appeals as I thought it was a genuine mistake, I even asked Dudley North MP Ian Austin for help but got none."
He added: "I have now received a lump sum from my accountant concerning the matter, and I am paying back what I owe."
"It is a disgrace Councillor Dean Cooper is poking around in my finances and I shall be seeking legal advice about his comments.
"It is an obvious case of attacking the candidate because Labour know they are going to get a kicking at the polls."
The full HMRC tribunal concerning Mr Perks can be found at http://www.financeandtaxtribunals.gov.uk/judgmentfiles/j6351/TC01930.pdf.