The birth of the Australian landmark caused its architect much suffering, but Jørn Utzon must have known it would be what he was remembered for

The hazard of being a noted builder or architect is that you risk being remembered solely for your most famous project. Most, though, manage to build at least two. When Lord Rogers eventually goes to the great River Café in the sky, will the Millennium Dome, Heathrow Terminal 5 or the Lloyds Registry be on the front page of the Times? Picture editors will at least have a choice.

There are those, though, who will be defined in memoriam by a single building. Cesar Pelli will, at least on these shores, be remembered for No. 1 Canada Square. I predict that Rod Sheard will forever be the Man Who Designed the London Olympic Stadium - even if most of it is to be taken down after the Games finish.

Jørn Utzon, who died this weekend, is in a different league. The story of the Sydney Opera House is the story of his life - a drama that spanned 20 years and saw a promising young architect disgraced, and his designs bastardised by bureaucrats.

The story of the Sydney Opera House is a drama that spanned 20 years and saw a promising young architect disgraced, and his designs bastardised by bureaucrats

That Utzon won the Sydney Opera House competition in the first place is astonishing. The story goes that Utzon's entry had been discarded before the Finnish-American architect Eero Saarinen picked it up from the rejects pile, announcing it as the winner.

There follows what must be the most disastrous construction story of the past 100 years. Wembley stadium has absolutely nothing on Sydney Opera House.

It took 15 years to build instead of the planned five years, and went roughly 1,400% over budget - ending up the wrong side of A$100m. The architects that were employed to take over after Utzon resigned in 1966 designed an interior with terrible acoustics that, in spite of the A$56m cost - as much as half the budget for the whole building - had to be redesigned in 2004.

This must be the most disastrous construction story of the past 100 years. Wembley stadium has absolutely nothing on Sydney Opera House

Utzon suffered the indignity of having his name dragged through the mud, even after walking away from the project. Rumours even persisted that he was deported from the country after being found with a suitcase full of pornography - even though this was in fact the unhappy fate of Sir Eugene Goossens, then conductor of the Sydney Symphony Orchestra, and not true of Utzon at all.

When Queen Elizabeth II opened the building in 1977, Utzon was not invited and his contribution was left unrecognised. He never returned to Australia after 1966, and never saw the finished product.

Moves were made by subsequent Australian governments to repair its relationship with the architect of its most famous building. He was appointed to the Order of Australia in 1985, and prime minister Kevin Rudd yesterday led an affectionate tribute to the architect, calling him a “son of Australia”.

Receiving the RIBA Gold Medal in 1978, he said: 'If there was a tiny scar on my soul from the Sydney tragedy it is hereby completely removed'

Utzon, for his part, had moved on. Upon receiving the RIBA Gold Medal in 1978, he said: “If there was a tiny scar on my soul from the Sydney tragedy it is hereby completely removed.” So removed was it that Utzon and his practice returned to the opera house in work if not in person - both to redesign the interior and to create a western extension to the building.

No doubt the Danish architect realised that, whatever else he built, the cathedral-like spires of Sydney Opera House would always be his legacy. It might as well be the legacy he had intended for it 50 years before.