More POE means better buildings

Post-occupancy evaluation (POE)
Any reader of this journal cannot fail to be aware of the "PROBE" series of POEs, which began in 1995(1). Supported by the DTI's Partners in Innovation (PII) programme, Probe revisits projects typically three years after they were initially featured in BSJ, and studies how they are really working and what occupants and management think of them. The published results extract lessons of interest to designers and their clients generally, for wider application.

Probe is just one example of a broad spectrum of UK POE activity, going back many decades (2) (3), and has covered a wide range of issues including technical performance, energy and environmental performance, cost-effectiveness and client and occupant satisfaction.

POE techniques include questionnaires, focus groups, interviews, observational checks and physical measurements. These can be applied separately or together for many different purposes, for example to:

• review how a building is performing in relation to its design brief - repeat clients can also consider how successful the brief has been and areas which may need to be changed;
  
• identify whether environmental systems and/or facilities provision are satisfactory from the staff's perspective, and where improvements should be directed;
  
• evaluate the performance of facilities, energy, or environmental management;
  
• demonstrate to project sponsors that money (including public funds) has been well spent;
  
• help a project team to see how they have done and to learn lessons for their next project;
  
• evaluate the impact of a building on business performance, and whether an organisation has achieved the culture change it was aiming for: for this one must understand both the building and the business using it (4)(5)(6).

POE has been shown to lead to improvements in building design and operation. It is of increasing interest internationally: in the UK this has been boosted by the Egan review of the construction industry (7).

Why doesn't POE happen more often?
If POE is so useful and effective, why isn't it standard of every project? A study by BRE for CRISP (2) revealed several barriers. For example:

• CLIENTS believe they spend enough time and money procuring a building and should not pay more after it is supposed to be finished. They think the industry should get things right first time. They consider that the design team and its future clients will reap most of the benefits of a POE anyway, while a bad result might even reduce the value of their own building.
  
• THE DESIGN AND BUILDING TEAM believe they are on a hiding to nothing and cannot afford to do it themselves. They need permission from the client and the occupier. If they then unearth a problem, they could attract responsibility for sorting it out and might even be sued. They believe their PI insurance premium could rise to cover the increased risk; and their insurance could even be invalidated.
  
• OCCUPIERS believe that moving in is disruptive enough - they now need to get on with their work. They are not convinced of POE's cost-effectiveness. It may also get them into trouble with their staff, particularly if they have already been complaining about something.

Until recently, the institutions did not actively promote routine POE either. But times are changing:

• RIBA Practice Services suggest (8) that in terms of the interaction between clients and users, the greatest improvement would come through 'the provision of systemised feedback and in instituting POE'.
  
• The Confederation of Construction Clients (CCC) has a mission to 'secure major measurable and consistent improvement in performance across the industry'. Its Clients' Charter (9) obliges signatories to obtain feedback on building performance.
  
• The Construction Best Practice Programme (www.cbpp.org.uk) is administering a series of performance measurement tools including key performance indicators for customer satisfaction with the construction product and the construction process.
  
• BREEAM 2002 (10 ) gives a credit for having a means of measuring staff satisfaction, and a further credit for using it to set performance improvement targets.

Getting started on POE
Most POEs are done several years into the life of a building - long after the project team has dispersed. However, a lesson from Probe (11) and other projects is that assessing building performance during the first year of occupancy can bring important benefits. In particular:

• The original design and building team can review how well things are working, undertake fine-tuning, and learn lessons for future projects.
  
• Occupiers can better understand how the building is intended to operate and how best they or their agents can operate and maintain it, improving performance and reducing whole life operating costs.
  
• Any emerging problems can be identified and dealt with more quickly and effectively in a spirit of collaboration and resolution, not confrontation and buck-passing.
  
• Better mutual understanding develops between the providers and users of buildings.
  
• Rapid feedback is obtained, both for the parties involved and for wider benefits.

If reviews are not planned, buildings are very likely to add less value than they were intended and designed to.

POE in the first year of occupancy
In 2000, a team led by BRE received DTI PII funding to investigate the opportunities for POE in the first year of occupancy, and specifically to:

• Explore the liability issues with designers, professional bodies and insurance companies.
  
• Seek ways to remove the culture of fear, blame, and conflict.
  
• Develop a method for use by designers offering a post-handover service, or by experienced clients, building on available techniques where possible.
  
• Focus on issues that regularly cause difficulties, where the design team can work with clients and occupiers to propose an action plan to improve building performance.
  
• Encourage designers to offer these services to their clients, and clients to request them.

Progress to date – liability issues
The impact of carrying out POEs on a consultancy's PI premiums is not straightforward:

• Strategically, insurance companies appreciate that better feedback should help designers to learn faster and get better at managing risks and avoiding problems.
  
• In practice, the reaction of PI insurers to the carrying out of POEs may be variable.
  
• Surveys in many industries show that while clients like everything to go smoothly, they are even more impressed by teams that did encounter some problems, but then dealt with them swiftly and professionally.
  
• Extreme defects will normally make themselves evident in other ways and are unlikely to be exposed purely as a result of a POE service.
  
• Most clients are unlikely to sue, and then only in exasperation. Since POE keeps communication and discussion going, it could well reduce the likelihood of litigation.
  
• It may be possible to institute no-blame agreements beforehand. For example, research led by the Estate Management and Building Service at Cambridge University is currently exploring options for what it calls "soft landings" contracts.

Definitive answers to issues of liability may not be possible until POE becomes more widespread and claims histories arise. However, further study is being undertaken. Anyone wishing to be involved in the discussion is invited to contact the BRE Project Manager.

Progress to date - what can easily be done?
It has proved unrealistic to propose a full-blown POE procedure at this stage, given the big difference between what might reasonably be done and how little normally happens today – a situation further exacerbated by the remaining uncertainties about liability. Instead, the team has decided to develop a voluntary Code of Practice, suitable for the first year of occupancy (which usually coincides with the defects liability period).

The main tool is a proposed checklist, intended to:

• Ease less experienced or one-off clients into POE by providing a quick, simple and inexpensive means of holding a post handover dialogue with their project teams.
  
• Allow more experienced or repeat clients to adopt or adapt it as part of their standard project management procedure.
  
• Help the supply side to develop POE as a standard part of the services they offer.
  
• Work towards problem reduction and resolution by confirming that key actions have been undertaken in areas known to be critical, or that plans have been made to achieve them.

The six critical areas identified are:

1. Commissioning and fine-tuning of building services and controls. Often there are loose ends. The checklist will help to tie them up.
   
2. Initial technical information, support and training. Frequently problems arise because O&M manuals are incomplete, inappropriate or even absent, inadequate arrangements are made for operation and maintenance, and training of technical staff is absent, inappropriate or incomplete. The checklist helps to avoid this.
   
3. Information and training for non-technical staff. Very often, nobody explains to managers and individual occupants how the building and its systems are supposed to work, what conditions they can expect, what adjustments they can make, and what to do if there is still a problem. The checklist identifies what needs to be done.
   
4. Keeping information and support up to date. Good record information is often supplied for a building but ends up in a shambles: badly organised and not kept up to date. In addition, simple tabs are not kept on the performance of systems, e.g. by BMS trend logging and by monitoring and targeting of utility consumption. The checklist will help to ensure that appropriate records and systems are put in place.
   
5. Client feedback and learning. This is where traditionally POE has begun. The checklist identifies services that may be required. We expect that just going through such a list will encourage more clients and occupiers to undertake feedback activities, reinforced – for example – by the CCC's Client's Charter requirement.
   
6. Client/design team communication. This confirms the routine arrangements that would apply to any project but are sometimes not clear to the occupier. It also identifies some opportunities for the design team to work more closely with the occupier during the first year of occupancy, so that both will learn from each other and achieve better results. Of course, this extra work will have to be paid for. To start with, this may well be partly by enlightened clients and partly by design and building teams wishing to improve their levels of performance, understanding and customer service.

In the long run, these activities are likely to cover their costs many times over, as building performance is enhanced, problems and disputes avoided, and everyone learns faster.

The final version of the checklist will be accompanied by a series of explanatory notes on:

• techniques available for anyone who wants to take things further
  
• topic areas, e.g. thermal, aural, visual comfort, energy and environmental performance, controls, impact of facilities on business performance;
  
• appropriateness of techniques, e.g. advantages, costs, value of information gained;
  
• technical issues, e.g. questionnaire design, running focus groups; interviewing;
  
• who to get involved, e.g. end users, facilities managers etc;
  
• how to develop plans for taking any necessary remedial actions;
  
• sources of further information, e.g. benchmarking data.

What do you think?
Please refer to the "method checklist" below). When commenting you may wish to consider the issues raised in the feedback form (below).

Contact
Contact the BRE Project Manager, Denice Jaunzens on 01923 664522 (e-mail jaunzensd@bre.co.uk), or any other member of the project team if you wish to know more about this project.

Further sources of information on post occupancy evaluation
Other sources of information on post occupancy evaluation methods and studies include:

• www.usablebuildings.co.uk (link to PROBE studies).
  
• Post Project Building Appraisal Toolkit, a DTI Framework project, contact John Reid at BRE, East Kilbride, 01355 233001.
  
• Office productivity network – www.officeproductivity.co.uk
  
• www.KPIZone.com
  
• 'Facilities management – towards best practice', edited by Peter Barrett, Blackwell Science, 1995.
  
• 'Construction Procurement Guidance No 8 Project Evaluation and Feedback', by the Office of Government Commerce – www.ogc.gov.uk.