Bovis Lend Lease, Carillion, Sir Robert McAlpine, Skanska and Mace are also vying for the scheme, called the Grand Union Building. One rival said Laing O'Rourke was "pushing hard" to win the contract.
The six firms attended a briefing on the scheme, which will be tendered in two stages, last week.
A revised planning application for the project was submitted by Chelsfield last November.
The building's height was reduced by 12 storeys to 30 storeys last summer after objections from the Royal Parks Agency and English Heritage that the scheme was too high and visually intrusive. The scheme now includes six interlinked office blocks of nine, 12, 15, 18, 24 and 30 storeys.