Choice 3: Metrics

Once you've determined what impacts you want to focus on and how far up and down the product's lifecycle you want to assess, the final decision is how accurately you need to measure your selected impacts across your chosen lifecycle stages. Once you've determined your choice of metrics, you'll be able to identify the types of impact assessment tools and techniques that will be most useful.

 

 

Most metrics fall into one of four categories:

• Comments
• Checkmarks
• Scores
• Measurements

 

Comments

The most qualitative, and usually most subjective, way impacts are expressed is through text alone. People can generally describe what they believe an impact will look like, its severity, and so forth at a high level based on their understanding of the product. Comparisons read more like product reviews than detailed technical analyses. This form might be appropriate for a first-pass assessment or as a basis for narrowing down alternatives to be compared. It is not a useful format if continuity and standardization is important because it's so subjective.

Checkmarks

In some cases, evaluations are based on checklists. The assessment will have certain criteria for each of the categories, which are either met, or not. Is mercury present? Is it certified organic? Is it FSC (Forestry Stewardship Council) certified? Does at least 25% of the energy used come from renewable resources? Checklists like this have the advantage of resulting in evaluations that are easy to compare across a wide range of products. They can be used relatively (i.e., seeing which of the products has more checkmarks) or absolutely (i.e., all of the parts we use must meet a certain threshold). While the checkmarks don't reflect many details or degrees of difference (i.e., the product that uses 100% renewable energy gets the same checkmark as the one that uses 25% if that's the threshold), they may provide enough information to support relevant decisions.

Scores

Whether in the form of grades, number scales, smiley face icons, or stars, scoring systems have the advantage of the at-a-glance nature of checklists, while also reflecting a more nuanced evaluation of a product's impact. One of the challenges that comes with nuance however is that someone needs to decide whether something gets an A or a B, 3 stars or 4. In many cases, scoring systems lay out guidelines for what qualifies as an A versus a B so that there is some consistency across evaluators and products. Even so, scores can be subjective and, in some cases, political. Still, a balanced and transparent evaluation process can produce a helpful assessment of the scale of a product's environmental impacts. Such scoring systems are especially useful when a quick assessment is needed to initiate the first discussion across a multistakeholder group.

"These are probably beneficial for initial assessment," said Tom, "but that's not what I'm after. I need to get a sense of my actual carbon footprint."

"I agree," said Priscilla. "I've actually used a couple scorecards, where I learned about issues like manufacturing and eventually recycling PET. Now I'd like to put some numbers to this process... you know, some real measurements."

Measurements

The most precise and objective metrics come in the form of specific numbers representing impact levels. These usually take two forms, one impact-specific and the other a standardized conversion into a single proxy number.

Impact-Specific

The impact-specific metric is usually expressed in equivalencies of a certain key component of that impact, such as kilograms of CO2 for global warming. In this case, no matter what the source of the impact on global warming, it would be converted into the equivalent kilograms of CO2 (often written as "kg CO2e," "kgeq CO2", "kg-eq CO2", etc.) using standardized equations.[1]

Other common equivalency units for several environmental indicators are listed in the table below.[2]

 

The next challenge is to determine the impact profiles of substances. For instance, what impact does silver have on ozone layer depletion, eutrification, etc.? There are actually well over a dozen methods for classifying substances.[3] Each maps materials to impacts based on scientific research, with many materials having impacts in multiple categories. The assessment is usually facilitated by software that can take component inputs and calculate allocated impacts based on either actual data gathered or standardized data tables. While there are pros and cons to each assessment tool, some have been adopted more broadly than others. A 2006 survey of 65 lifecycle assessment (LCA) practitioners[4] reported that:

58%* used GaBi (PE International)

31%* used SimaPro (PRé Consultants)

11%* used TEAM (Ecobilan)

Other tools cited:

• BEES (NIST)
• Umberto (ifu Hamburg)
• ECO-IT (PRé Consultants)
• Excel-based spreadsheets
• Math package (e.g. MATLAB, Mathematica

*percentages include overlap due to usage of multiple tools

Single Proxy

Because it is difficult to compare the impact of 1 kg-eq CO2 and 1 kg-eq chloroethylene, for instance, it can be useful to convert all impacts into a single proxy metric. All of the impact-specific equivalencies can be translated into a universal impact factor, often expressed in terms of "millipoints," sometimes after being normalized based on a national or global reference model. Such single-number impact factors are therefore a weighted measurement showing relative impacts across multiple categories. While there are some standard sets of factors, each represents a specific perspective on what to use as a reference model and how to calculate the conversions. Several of the most widely-used data sets are Eco-Indicator 99 (EI99), EcoInvent, U.S. Life-Cycle Inventory, and CML.

 

Weighting

Whenever multiple factors are combined and represented by a single number, some sort of weighting takes place. Sometimes all of the inputs are considered of equal value, but in many cases some inputs are given more influence over the final result than others, reflecting a certain prioritization of the importance of each type of impact. Weighting is more of a political (social, cultural) than a scientific process -- giving, say, more weight to the global warming indicator than to acidification is a values-based decision. Stakeholders may differ significantly on their views about the importance of impacts, as shown in the chart below.[5]

 

 

Many practitioners choose to leave the impact scores broken out into categories, with no weighting at all. Although this approach creates a more complicated report, it enables impact comparisons between products on a more granular level.

Weighted "single score" assessments have the advantage of generating one, easy-to-communicate impact number. However, even within the community that supports this approach there are two schools of thought. Some believe that there should be a standard weighting, while others feel that companies should be free to weight impacts as they see fit. One of the advantages of a standard weighting, as is used in the Okala approach among others, is that products can be compared to each other more easily since the single impact scores are only meaningful if compared among products with the same weighting. A second benefit is that companies can't "game" the assessment to make their products look better than they are by emphasizing the areas in which the product does well and decreasing the effect of categories in which the product has problems.

The advantage of variable weighting approaches is that they can be customized to fit an organizations goals and values. For instance, if an organization is making global warming a priority, it may want to weight that category much more heavily as it's assessing the impacts of its products. As long as the weighting remains constant within its own assessments, the disproportionate weight it gives to this category is fine. In some cases, there may be external reasons for giving some impacts priority. For instance, there are some sustainability accounting and reporting standards that focus almost exclusively on greenhouse gas emissions, making it useful for organizations using them to put most, if not all, of the weight on that subset of impact factors.[6]

"Well, I've chosen five indicators, one from each of the domains of impact. I don't think I want to weight these results together, because I want to choose which ones I'm optimizing for in each design," said Priscilla.

"And I'm just measuring a single indicator, carbon, so I won't use any weighted single-proxy results either.

"So now that we know our three choices, what tools can we use?"

 

[1] The EPA's Greenhouse Gas Equivalencies Calculator is available online at http://www.epa.gov/cleanenergy/energy-resources/calculator.html

 

[2] Adapted from "IMPACT 2002+" LCIA methodology / Dr. Olivia Jolliet, Univ. of Michigan

http://www.sph.umich.edu/riskcenter/jolliet/impact2002+.htm

[3] For a good overview of many of the major methods, see Appendix B: LCA and LCI Software Tools in "Life Cycle Assessment: Principles and Practice," Scientific Applications International Corporation, EPA/600/R-06/060 (May 2006), pp. 74-77.

[4] Cooper, J.S.; Fava, J. (2006), "Life Cycle Assessment Practitioner Survey: Summary of Results", Journal of Industrial Ecology

[5] T.G. Gloria, B.C. Lippiatt, and J. Cooper, "Life Cycle Impact Assessment Weights to Support Environmentally Preferable Purchasing in the United States," Environmental Science and Technology, November/December 2007.

[6] Carbon footprint standards such as PAS 2050 and the GHG Protocol fit this description.

<<< Previous: Choice 2: Scope	>>> Next: Chapter 4: Putting It All Together