EME 807
Technologies for Sustainability Systems

4.2. Frameworks for assessment of alternatives

So, what does it take to change the conventional practice with possibly hazardous or harmful chemicals or processes to a more sustainable solution?

The major step is assessment of alternative solutions, which takes into account a wide range of criteria. When thinking about replacing the existing process with an innovative alternative, chemists and engineers try to avoid so-called "regrettable substitutions". In other words, avoid switching to an alternative process or chemical that either transfers risk to another point in the production chain or lifecycle or contains unknown future risks.

Ideally, in the concluded assessment, chosen alternatives must:

  • be technologically feasible;
  • provide the same or better value in performance and cost;
  • have an improved profile for human health and environment;
  • account for economic and social considerations;
  • have potential to be sustainable over long period of time (look out for restrictions that may arise in the future; for example shortage of rare elements, etc.).

We see that choosing the best alternative requires careful investigation! Such investigation must be comprehensive, i.e., would require cross-disciplinary expertise, and based on high quality data.

Let us look at some typical criteria that may be used in chemical industry and research for evaluation of various processes and reactions. In the "green chemistry" context, the main emphasis is put on the environmental profile of a chemical alternative, while economic feasibility is included in the picture at the stage of technology transfer.

Evaluation criteria

Table 4.1 below represents the set of criteria that can be effectively used for assessing alternatives in chemical and material manufacturing. On the left, the top-level criteria are listed, which are key points of concern when introducing new chemicals to the manufacturing process. The middle column lists some sub-criteria, which show how the impacts can be distributed. The right column specifies specific measures for each type of impact, which basically become guides for data search and analysis.

Table 4.1. Criteria and measures used to assess green chemistry alternatives to existing processes (adapted from UCLA Sustainable Technology & Policy Program, 2011)
Criteria Sub-Criteria Measures
Physical Chemical Hazards - Flammability
Flashpoint
Explosivity limits
Auto-ignitability temperature
Oxidizing properties
Human Health Impact Toxicity Acute toxicity
Carcinogenicity
Developmental toxicity
Endocrine toxicity
Endocrine disruption
Epigenetic toxicity
Genotoxicity
Organ, tissue, cell toxicity
Human Exposure Volume in manufacturing
Volume in consumer use
Extent in dispersive use
Sensitive sub-populations
Persistence
Bioaccumulation
Ecological Impacts Adverse Impacts Aquatic, animal or plant species
Aquatic and terrestrial ecosystems
Endangered or threatened species
Environmentally sensitive habits
Exposure Volume in manufacturing
Volume in consumer use
Extent in dispersive use
Persistence
Bioaccumulation
Environmental Impacts Adverse Air Quality Impacts Nitrogen oxide
Sulfur oxides
Greenhouse gases
Ozone-depleting compounds
Photochemically reactive compounds
Particulate matter
Fine particle matter
Adverse Water Quality Impacts Biological oxygen demand
Total dissolved solids
Thermal pollution
Adverse Soil Quality Impacts Chemical contamination
Biological contamination
Loss of organic matter
Erosion
Natural Resource Use Impacts Non-renewable material use
Renewable material use
Water Use
Energy Use
Waste generation and end-of-life disposal
Reusability and recyclability
Technical Feasibility - Functionality
Reliability
Usability
Maintainability
Efficiency
Economic Feasibility - Manufacturer impact
Purchaser impact

The list of criteria given in Table 4.1 has been proved effective for some case studies. While it puts the main emphasis on the hazard assessment and environmental impact, the technical and economic criteria are also included and can play a significant role even at the stage of selection of particular chemical reagents for the process. Note that the above list of criteria and sub-criteria is not something written in stone. It is presented here as an illustration. For each specific assessment project, choice of criteria needs to be justified through expert and stakeholder involvement and will depend on the goals of assessment. Depending on the assessment team decision, some criteria can be added, some – removed, and weights of all factors can be tuned. Clear identification and justification of the selected criteria is critical.

Data collection

In any assessment project, clear and consistent requirements should be set for the sources of data to be used. Information should meet specific data quality criteria for inclusion into the assessment. Quality of data will determine their utility. Data selection should follow the internationally recognized definition for reliable information: "Reliable information is from studies or data generated according to valid accepted testing protocols in which the test parameters documented are based on specific testing guidelines or in which all parameters described are comparable to a guideline method. Where such studies or data are not available, the results from accepted models and quantitative structure activity relationship (QSAR) approaches may be considered. The methodology by Organization for Economic Cooperation and Development (OECD) can be used for the determination of reliable studies." (Principles of Alternative Assessment, 2012)

Preferably data should be obtained from authoritative bodies, those referenced by US government agencies (e.g. EPA). The following are links to some of such resources:

Technical data sources

Information should be obtained from published studies or directly from technical experts or users of the alternatives. In other cases, information can be requested from product manufacturers. The specific performance information (reactions, energy effects, thermodynamic analysis) available from experimental labs may be needed to draw conclusions about technical feasibility for each individual application. Clear referencing of the data sources is important.

Economic data sources

Data sources for financial information may include manufacturers, stakeholders, the Chemical Economics Handbook, and other standard reference sources. For many emerging alternatives, hard cost information may be unavailable. Cost comparisons today may not be directly extrapolated to emerging technologies, because learning curves, scaling, and other factors can affect costs over time. Assumptions and use of surrogate data should be clearly explained in the assessment.

Ranking the alternatives

Quantification (scoring) of the impacts based on the criteria listed above is typically done via a multi-criteria analysis (MCA) model, appropriately build for the project. MCA provides techniques for comparing and ranking different outcomes of existing and alternative processes. When setting up an assessment project, it is important that the scoring system is transparent and is consistently applied to all scenarios under consideration.

MCA is a great tool for comparison of different options, but it is hardly objective because choice of criteria and metrics to quantify impacts varies from case to case. In contrast, cost analysis is aimed at providing objective measure of economic feasibility based on predicted cash flow. Cost analysis requires impacts to be expressed in monetary terms. MCA can use both monetary and non-monetary measures as well as both quantitative and qualitative measures.

In MCA, ranking of chemicals or processes with respect to the listed criteria can be done in a variety of ways. One way is to assign each criterion a score that spans from 0 to 1, with the value of 1 corresponding to the best (most preferable) choice and value of 0 corresponding to the worst (least preferable) choice among the available. The rest of the choices would score in between.

Example

For example, if substance A performs better than substances B and C on acute toxicity criterion, and substance B performed the worst of the three choices, then A will receive a score of 1, B will receive a score of 0. In case of qualitative assessment, substance C receives a score of 0.5 (linear dependence). In case of quantitative assessment, the utility values may be connected to the acute toxicity measure and will place substance C on the relative scale (i.e. taking into account how much more toxic it is compared to substance A and how much less toxic it is compared to substance B). This approach will be illustrated in one of the case studies further in this lesson.

Another possible approach for assigning scores is outranking. There is no relative scoring, but instead, alternatives are compared by each criteria in pairs (two at a time). This way we try to identify the extent to which one alternative out-performs the other. In the end, the dual performance scores (1 - "win"; 0 - "lose") are aggregated and the preference index is calculated for each alternative.

Example

For example, substance A out-performs B and C by acute toxicity thus getting the cumulative score of 2 (1 point for each "win"). Respectively, substance C receives a score of 1 for beating B, and B is left with 0. One of the case studies described further in this lesson uses both approaches in order to compare the outcomes.

Evaluation of the economic impacts associated with the implementation of a new product or practice generally focuses on the changes in capital and operational costs and revenues. (These terms of cost analysis were overviewed in Lesson 3.). The main areas where impact is expected are:

  • cost of new equipment or production process;
  • operation and maintenance costs (labor costs, energy costs, etc.);
  • cost differences for different substances;
  • cost of transportation;
  • cost of design, monitoring, and training;
  • regulatory costs.

The data on economic impacts is collected in consultation with relevant supply chain actors and possibly trade associations. Evaluation can be an iterative process starting from qualitative comparison of the old and new scenarios and ending at quantification of impacts with monetary values.

The European Chemicals Agency (ECHA) website provides a more detailed guide to economic assessment of alternatives and can be used as a resource for this task. There are some documents linked that you are not required to read unless you're specifically interested in the socio-economic assessment.

Weighing factors

In most situations, decision-makers are not equally concerned about all highlighted criteria. For instance, a particular decision-maker may place more importance on whether a household cleaner causes cancer than on whether it contributes to smog formation. Thus, the decision-making method should account for respective “weight" of each criterion in the evaluation process. Since different stakeholders may place different weights upon criteria, the weighting raises significant questions in the context of a regulatory program. For example, can we consistently compare the alternatives without regulating the weight of factors? This is something to watch out for.

The criteria weights can be established by three methods:

  1. Using generic or recommended weights;
  2. Calculating the weights based from objective measures; and
  3. Eliciting weights from stakeholders or experts.

Method (1) is exemplified by Table 4.2 which lists several sets of generic weights recommended by National Institute of Standards and Technology (NIST) based on the data of Environmental Protection Agency (EPA) and Harvard Study for a set of criteria usually used in life cycle assessment (LCA).

Table 4.2. Suggested weights on assessment criteria from various regulatory studies normalized to 100% (source: UCLA Sustainable Technology & Policy Program, 2011)
Criteria NIST EPA Harvard Equal Weights
Global warming 29.3 16 11 7.7
Fossil Fuel Depletion 9.7 5 7 7.7
Air pollutants 8.9 6 10 7.7
Water intake 7.8 3 9 7.7
Human health cancerous 7.6 11 6 7.7
Human health non-cancerous 5.3 11 6 7.7
Ecological toxicity 7.5 11 6 7.7
Eutrophication 6.2 5 9 7.7
Habitat alteration 6.1 16 6 7.7
Smog 3.5 6 9 7.7
Indoor air quality 3.3 11 7 7.7
Acidification 3.0 5 9 7.7
Ozone Depletion 2.1 5 11 7.7

In the above table, the NIST panel generated weights from stakeholder consulting that involved 7 building product manufacturers, 7 product users, and 5 LCA experts. EPA weights and Harvard weights were derived by NIST from sets of qualitative rankings of impacts developed respectively by EPA’s Science Advisory Board in 1990 and Harvard researchers in 1992.

Method (2) of calculating corresponding weights can be based on distance-to-target approach, when each criterion is weighted by the variance between the existing and desired conditions. For example, if the global community is further away from achieving goal for global warming than it is for ozone depletion, then greater weight is given to the global warming potential. Another way to such calculation is monetary evaluation, when weighing is done based on the cost of environmental consequences.

Method (3), which assumes obtaining weights from stakeholders directly, may be based on public opinion surveys, community working group decisions, and different multi-criteria analysis models. The main types of stakeholders to consider: (1) Environmental Non-Government Organizations, Industry, Policymakers, and Consumers (Public). Weight assignments collected through surveys are then averaged across the board of stakeholders and then normalized to 100%.

Use of any of the methods depends on the goals of the assessment project, its scope, resources, and timeline. When building an assessment project, the weighing process should be transparent and well justified. When comparing different cases within one study, keep the weighing scale the same across the evaluation criteria.

Within the MCA approach, the final score (Si) of a particular option (alternative) with respect to any major top-level criterion i is estimated as an average of all sub-criteria scores under that criterion:

Si=j=1nsjn

where n is number of sub-criteria or metrics used to assess the option under top-level criterion i. The final total score (Stot) is the weighted sum of all top-level criteria scores:

Stot=i=1NSiwi

where N is the number of top-level criteria considered in assessment; wi is the weight factor of a particular criterion. The example study presented in the next section of this lesson demonstrates how the MCA scores are calculated and compared.

Consider the following supplemental reading materials on this topic:

Supplemental (Optional) Reading - Alternative Assignment Methodology

These recommendations were developed by the Industry Coalition on how the assessment of chemical alternatives should be conducted.

This website provides some advice on socio-economic analysis of chemical alternatives under REACH regulation program. 

Supplemental (Optional) Reading - Multi-Criteria Analysis

  • Linkov, I, Moberg, E., Multi-Criteria Decision Analysis: Environmental Applications and Case Studies, CRC Press 2011.

This book is available online through Penn State Library system. It provides in-depth explanation on MCA methods and shows its applications to environmental science.