"There was not much point in spending a couple of million pounds more just to have a one in three chance of winning," Doughty said.
"We might have got our money back if we had won, but there were no guarantees and the Ministry of Defence was delaying."
Costain did not reveal how much it had spent on the scheme.
Fellow contractor Bouygues and facilities management firm Sodexho have also pulled out of the consortium bidding for the two-pronged scheme, called Allenby and Connaught. They are understood to have been replaced by Alfred McAlpine and American FM group Aramark. Others in the team include quantity surveyor Gleeds and architect HCM.
The Jarvis consortium is up against two other consortiums – one led by Bovis Lend Lease and Amey, the other by Mowlem and Brown & Root.
A team that included Atkins, Balfour Beatty and Amec pulled out earlier this year.