The group, which is known as the Church of Euthanasia, believes people ought to kill themselves to reduce the world's human population.
A number of workers on the T5 site have been approached by cult members, who have tried to persuade them to become full members of the "church".
One T5 worker said that three men had been hanging around the site gates speaking to workers. He said the cult members tried to persuade workers to go to nearby cafes so they could explain why they wanted them to join the cult.
The worker said: "I have taken what the cult members have been saying with a pinch of salt and found it all to be quite funny but the worry is that some of the lads on site may be taken in by it and join."
The worker, who lodges in the Heathrow area during the week, said that cult members were also targeting areas where staff live. He said that some of the workers who live away from their families might be particularly vulnerable to the advances of the cult members, as they sometimes suffered from boredom and loneliness, and sought company.
He said: "I know these kind of people often hang around university halls of residence, where they prey on homesick, impressionable students in their first year of studies. It is not a too dissimilar situation to T5 at the moment."
The disclosure that the T5 site has been targeted by cultists comes after Building revealed in June that about 200 construction workers commit suicide every year.
UCATT regional secretary Bryan Rye, who is investigating the subject of suicide in construction, said that the cult members' actions were "wholly inappropriate".
He said: "Every effort should be made to prevent these people from gaining access to construction sites and preying on vulnerable workers."