Behavioral Research Team Releases Final Report
The HVAC Behavioral Research Initiative (HBRI) is excited to announce the release of its final report on the contractor and technician behavior study. The WCEC led the field observation part of the study, which looked directly at what technicians do when they respond to an HVAC maintenance service call. The study was conducted using an innovative method of undisclosed participant observation, followed by semi-structured interviews.
Posing as homeowners, HBRI’s lead Claudia Barriga and Verified, Inc’s lead trainer Robert Eshom called a number of contractors, and scheduled maintenance services for their “home”. The house used belonged to a research ally, and had a perfectly functioning HVAC system. When a technician came to provide the maintenance service, the “homeowners” showed him the AC system and explained that they didn’t know if the system was working correctly or efficiently. Then, they proceeded to observe the kind of service, diagnostics, explanations and repair offers that the technician provided. After the work was completed, (which did not include any actual modifications to the system, only recommendations for service), the technicians were paid, and then the “homeowners” revealed themselves to be researchers. The technicians were offered $100 for consenting to let the researchers use the observation for the research, and participate in a post-observation interview. Of the 16 technicians who were called, 13 allowed the researchers to use the observations, and 9 gave the researchers in-depth interviews.
While a sample size of 13 is certainly not sufficient to generalize to all technicians, it was an eye-opening study.
None of the technicians provided a service that remotely resembled ACCA Standard 4—the industry standard for what maintenance services are supposed to accomplish. In fact, none of them knew what ACCA Standard 4 was.
The figure shows which ACCA 4 tasks were completed correctly, which were attempted but not completed correctly, and which were not attempted at all. One technician did do several of the tasks that were expected, but most technicians did not attempt many of the tasks, or completed them incorrectly. One technician did not attempt to do any of the expected tasks. There was no relationship between the number of tasks completed correctly and the cost of the service (which ranged from $60 to $180), nor to the years of experience of the technician (which ranged from 1 to 26 years). There was something of a correlation to the duration of the service: it is simply not possible to do many of the ACCA 4 tasks within the 23 – 90 minutes most technicians had available.
Technicians were also observed to be consistently discouraging discussions about energy efficient upgrades. As “homeowners”, the researchers made increasing efforts to ask for energy efficiency optimization services, recommendations or possible upgrades. These efforts did not result in more thorough maintenance services. Moreover, technicians often stated that such things were “not necessary” or “not worth it”.
Maintenance is key to achieving cooling related energy goals in California. It has been shown to have the potential to save up to 30% of a typical air conditioner’s energy use and peak demand. But just as significantly, understanding the relationship between building owners and their service technicians sheds light on that magic moment when an HVAC system is not performing well and should be replaced with something, hopefully something efficient. The penetration of energy efficient technologies and services is significantly influenced by service technicians and contractors. In this case, the service that was provided would not have resulted in direct energy savings, and it did not provide some of the relationship-building elements that can lead to energy efficient choices. Were the technicians less than competent, or worse, lazy? Findings from the interviews indicate that this was not the case. Technicians were well trained and had the ability to perform at higher levels than they did. They were also proud of their abilities and training, and interested in providing good services to customers.
They simply did not have the time to do the job correctly. The way the industry and their specific job is structured provides no incentives for them to perform services up to ACCA 4 standards, or to promote energy efficient solutions and upgrades.
Overall, the conclusions drawn from this study were:
- Technical performance by the observed technicians was below the standards of ACCA Standard 4. Yet, the technicians seemed to be more knowledgeable than their technical performance scores would suggest, and all seemed confident and proud of their work.
- In general, technicians work hard to achieve their perceived company and customer goals: accomplish as many service calls as possible in one day. Industry transformation should include a way to make technicians see the non-traditional goals of quality technical performance as consistent with the goals of their employers and their customers.
- There is a need to structure an industry system in which hard-won skills can be put to use, requiring competent technicians to “push” well-documented benefits, as well as early-adopter customers to value and “pull” the benefits. Technicians and contractors may also need to develop the skills to recognize and respond to this customer pull, a skill that was completely absent in our observations.
This study was funded by the California IOUs and the CPUC, and the prime contractor was Energy Market Innovations (EMI). The overall project included also a major survey of HVAC contractors throughout California (led by EMI), which provides a very interesting contrast with the technician observation part of the study. Complete findings and analyses for both parts of the project are documented in a report entitled “California HVAC Contractor & Technician Behavior Study–Final Report”, released in September and available at CALMAC Study ID SCE0323.01