Communities Secretary Eric Pickles has warned councils against using "under-the-counter pay-offs to silence departing staff".
Nearly 5,000 public servants may have been given pay-offs involving gagging orders, says The Daily Telegraph.
About 200 civil servants in Whitehall and 4,562 council workers have signed "compromise agreements", many of which involved confidentiality clauses.
The total cost over two years was £14m, according to the newspaper.
A Freedom of Information request found 256 councils in Britain signed compromise agreements with former staff between 2005 and 2010.
Mr Pickles said: "For too long, local government has made departing staff sign gagging orders, often with big pay-offs attached, away from the eyes of those who get left with the bill: the taxpayer.
"When leaving a job, councils and their employees need to part ways fairly.
"Giving out thousands in under-the-counter pay-offs to silence departing staff is not the way to achieve this.
"Councils have a responsibility to the public and transparency is at the heart of that.
"By shining a light on these activities and introducing new democratic checks and balances to stop gagging orders being abused we are helping councils improve accountability in local government."
Mr Pickles claimed new checks and balances would help stop gagging orders being abused.