London Town bought the site in 2000 and submitted proposals for a 32-storey block of flats. Faced with overwhelming local opposition, the plans were withdrawn.
The developer then submitted a proposal in June 2002 for a 20-storey tower, but Southwark council refused planning permission. London Town appealed against the decision and, after a public inquiry last May, the planning inspector granted permission.
David Lough, the chairman of Bankside Residents for Appropriate Development, said: "We have been advised that the inspector's decision is unlawful and have agreed to pool our savings to overturn it."
Tate Modern cannot challenge the decision because of its charitable status, but director Sir Nicholas Serota said: "We support BROAD in its campaign to try and stop this development, which will cast an ugly shadow across our entrance."