Amicus believes that many employers are defining agency workers as self-employed tradesmen, so removing their entitlement to benefits such as holiday and sick pay, and protections such as limited working hours.
The union is calling for legislation to define explicitly the rights of agency workers. It believes that this should take place at European and national levels.
Roger Lyons, Amicus' joint general secretary, said: "About 500,000 people in the UK are having the wool pulled over their eyes. There needs to be a European directive for temporary workers."
He added that part of the problem was that agency workers were not asserting rights they may have because they were not in touch with any group offering information about them.
Lyons said this lack of information meant that agency staff in construction were failing to exploit a legal victory in the Court of Appeal last month. This found that carmaker Ford had failed to recognise agency workers' rights to holiday pay under the European Working Time Directive.
About 500,000 people in the UK are having the wool pulled over their eyes
Roger Lyons, Amicus
The government is currently re-examining the Employment Relations Act 1999, which could result in more protection and rights for part-time, agency and some casual workers.
Building revealed last week that the DTI is to host a construction round table meeting next month to examine the use of labour agencies.
Industry leaders are to be given an opportunity to voice concerns over agencies that use untrained workers, or do not employ workers properly.
n Derek Simpson was finally ratified this week as joint general secretary of Amicus.