Last Wednesday the protesters' request for legal aid to take their case to the Court of Appeal was turned down by the Legal Services Commission.
A consortium headed by developer Dawnay Day and housebuilder Barratt will begin work on the Terry Farrell and Partners-designed leisure centre and housing scheme in April.
A local pressure group called the Friends of Swiss Cottage had been at the centre of the protests against planning consent. It claimed that the development would result in a reduction in leisure facilities.
The protesters, funded by legal aid, tried to push for a judicial review of Camden council's planning approval of the scheme in December, but presiding judge Mr Justice Ouseley declined to grant it.
Ian McNicol, director of leisure at Camden council, said the protest had delayed the project by 11 months at a cost of more than £300,000 to the council, including legal fees.
He said: "We have wasted time with people complaining about a reduction in sports facilities. But by the time the project is complete in 2005 there will be a massive increase in sports halls and pitches."
The project includes the redevelopment of a leisure centre, the refurbishment of Hampstead Theatre and Basil Spence's Swiss Cottage library, and the redesign of public space.
The main part of the scheme will be the building of 169 homes, 41 of which will be owned by a housing association. The proposals include a community centre and a doctor's surgery.
The Barratt–Dawnay Day consortium edged out strong competition – including a consortium of Carillion, Sir Robert McAlpine and St George – to win the scheme in April 2000.
As part of the scheme, John McAslan & Partners has been appointed architect on the £3m library refurbishment and Bennetts Associates is heading a £9.86m project to rebuild Hampstead Theatre.