The move is the latest blow for the leisure centre and housing scheme, which was stalled for a year while its developers fought off a legal challenge from local pressure groups.
A spokesperson for consortium partner Barratt Homes said Farrell's decision was unexpected. He said: "We are very surprised and disappointed to learn that the architect has unilaterally chosen to leave the project at this stage without notification to any of the joint developers."
Officials at Camden council and S&P Architects, the firm designing the interior of the leisure facilities on the scheme, are also thought to have been taken by surprise by the move, which was announced by Farrell on Tuesday.
But council officers said they would make efforts to persuade Farrell to return. One officer conceded that the loss of a "name" architect would be a blow to the project.
Farrell's announcement accused developer Dawnay Day and Barratt Homes of scaling down the practice's role. It said: "The developers have confirmed that they will not agree to Terry Farrell & Partners continuing with the full architectural service as was originally agreed.
Terry Farrell & Partners will not accept that their role should be reduced to a partial service in scope and have withdrawn from the development."
Farrell said his practice's more limited role would be unlikely to produce building designs of high quality.
After the failure of local groups' efforts to overturn planning consent, the consortium had been due to begin work on site this month. The development is due to for completion by 2005. The main part of the scheme will be the construction of 169 new homes, 41 of which will be owned by a housing association.
The proposals include a community centre and a GP surgery.
As part of the scheme, John McAslan + Partners has been appointed architect on a £3m library refurbishment and Bennetts Associates is heading a £9.86m project to rebuild Hampstead The Barratt-Dawnay Day consortium edged out strong competition – including a consortium of Carillion, Sir Robert McAlpine and St George – to win the scheme in April 2000.