The pigeons have left Trafalgar Square, but a new menace has arrived – contractors causing chaos. And yet the British public has such low expectations of builders, it hasn't logged a single complaint. With this going on, what hope do we have of attracting talent to our industry?
The Fitzpatrick subbie is angry. "F*** off," he grunts. "Stop taking f***ing photos." He picks up a long metal spirit level and brandishes it menacingly, as if it were an axe.

This is Trafalgar Square on a mid-week morning. I've come down with my camera following reports that the world-famous square has been transformed into a filthy, confusing building site as workers rush to complete a major pedestrianisation and landscaping scheme. Given the industry's need to improved its image and attract more bright young people, you might think the highest profile job in the country would be striving to impress members of the public.

Instead, I was threatened with physical violence.

"Stop that or I'll f***ing have you," snarls the Fitzpatrick subbie. He's only a few feet away from me but, fortunately, we are separated by a waist-high site fence. He sprays spittle as he speaks and his face is covered in food particles. "I'll f***ing kill you."

He sounds like he means it. I back off and he returns to his work, emptying his nostrils onto the ground as he goes.

This incident happened two weeks ago. The man – I was too scared to ask his name – is one of the dozens of men working on the £150m "World Squares for All" project, designed by Foster and Partners. Works started in November 2001 and are due to end in July this year. They include removing traffic from many parts of the square, installing boulevard-style landscaping in the surrounding streets, and building a new grand staircase linking the National Gallery to the square itself.

When people see a filthy site, they aren’t going to want their children to work in construction

Peter Rogers, chairman, strategic forum

Fitzpatrick is the main contractor, so I call to report the incident. "Obviously we can't condone that," says project manager David Bailey. "But I'm also not very happy that you've gone to the site to take photos without a permit. I'm a bit gobsmacked really that you've gone down without an invite. We've had nothing but praise from the Considerate Constructors' Scheme and from Westminster council. I'm not answering any more questions."

I called the CCS and spoke to Edward Hardy who, initially, takes the same defensive line.

"Did you have permission from the site to take photos?" asks Hardy. "No? That's not on."

I point out that the Fitzpatrick employee in question was laying paving slabs in Trafalgar Square – a public space and one of the most photographed locations in Britain. Hardy accepts the point and adopts a more conciliatory stance. "A guy threatening to kill you is outrageous. You don't go to the supermarket and get abused by the staff so it shouldn't happen at a construction site. It doesn't do our industry much good if tourists are wandering around experiencing this sort of thing."

A few minutes later Hardy calls me back. "I've spoken to a couple of guys on the site and they're appalled. They were incredibly shocked. I wouldn't imagine the guy's got much of a future with Fitzpatrick."

The delayed reaction at least treated the spirit-level incident seriously. But it doesn't address why the worker felt able to behave as he did. It is almost as if normal rules of civility and appropriate behaviour are waived for construction workers.

Multiple inconveniences await visitors to Trafalgar Square. At the entrance to a pedestrian underpass on the south side, an Onyx worker, hand-rolled cigarette hanging from his lips, nonchalantly sprinkles grit from a shovel using just one hand. He seems oblivious to the people emerging from the underpass, who repeatedly bump into him. "Sorry," they say, in typical British fashion.

Outside the church of St Martin in the Fields, three workers down tools to grin, point and guffaw as an attractive girl passes by. In front of the Canadian Embassy, a worker squats inside a phone booth as he eats his sandwiches, trying to hide his face from my camera.

I was told by our architect the other day that our site was a bit messy, and I thought that was a bit unfair

David Bailey, project manager, Fitzpatrick

On the north side of the square, a JCB is parked across the pavement, its engine running and its cab door open. The driver is several metres away, talking to a colleague. In front of the National Portrait Gallery, a row of bollards on the pavement is forcing pedestrians to walk single file on a narrow ridge of exposed kerbstone, inches from passing traffic.

In St Martin's Place, a pile of refuse sacks, metal barriers and more bollards creates a baffling chicane that has pedestrians bumping into each other. In several places around the square, pedestrians are forced to walk in the busy road to avoid materials, plant and workers.

There are stray traffic cones in the middle of traffic lanes, on zebra crossings and even blocking the door to a functioning phone booth. In a fenced-off marshalling yard in Charing Cross Road, cable drums, rolls of plastic piping and other materials lie in a higgledy-piggledy mound that seems to double as a tip for waste materials.

In fact, there is rubbish everywhere. A swath of smashed crates and ripped-up frost matting lies behind a barrier at the southern end of Northumberland Avenue. Wind-blown litter lies thickly in the crevices between newly laid paving. Split sandbags and broken traffic cones stand in sorry heaps.

I try to grab a couple of passers-by to ask them what they think, but they are all in too much of a hurry. Instead, I speak to Michael Theodorou, who sells souvenirs from a kiosk on the east side of the square. "It's been horrendous," he says. "The noise can be quite horrendous. I've been working here with JCBs right next to me."

Sure enough, there is a digger juddering away about two metres from his kiosk. "You can't do it – you have to walk away every hour or so to clear your head."

It's ugly and inconsiderate, but none of it amounts to breaches of regulations or rules. Neither the Considerate Constructors Scheme, which operates a voluntary code, nor Westminster council's inspectors have received a single complaint from members of the public about the works. "It's always been the British way not to complain," says the CCS's Hardy. "They're conditioned to it; they're used to it."

So, Building lodges its own informal complaint, and shows Hardy the photographs the site worker didn't want us to take. "There are things that we would want to see improved," he says. "Some are things that would just be there for 10 minutes, but other things are bad housekeeping."

But Peter Rogers, chairman of the strategic forum, points out that poor housekeeping on such a high-profile site can do real damage to the industry. "The first thing the strategic forum is tackling is site conditions, because of the effect they have on everybody. Messy sites set a poor image and inconvenience people. When parents or teachers walk past a filthy site, the first thing they'll think is: 'There's no way I'm going to encourage my children or students to join the construction industry'."

A few days later, I meet up with Fitzpatrick's Bailey at Trafalgar Square. Many of the areas are looking much tidier now. Refurbishing Trafalgar Square while keeping the traffic and pedestrians flowing is a monumental undertaking, he explains. He is rightly proud that the work – which is of undeniably high quality – is ahead of programme. Even London mayor Ken Livingstone has publicly praised the contractor, he points out.