Comparing Coating Equality Through Performance Data

Specifiers, contractors, and owners should thoroughly understand the coating tests.

Selecting the optimal coating system for a given application can make you the hero on a project, while choosing an inferior product can lead to reduced efficiencies, lower performance, and higher lifecycle costs. Whether a specifier, contractor, or asset owner, careful and informed coatings selections are imperative. Unfortunately, the selection process may be tougher than expected. It is possible to predict the application environment and conditions within reason and to rely on experience; however, comparing products may be confusing.

Consider ASTM International’s testing methods. Some tests and data points offer reasonably accurate comparisons for certain products in certain applications. However, ASTM does not intend for its tests to provide numerical comparisons of stand-alone data. Direct product performance comparisons should be made only when testing similar systems at the same time, in the same lab, using the same technician. Making correlations without a controlled, side-by-side test creates the potential for invalid conclusions about the most suitable coating for a project. While ASTM tests that provide repeatable, reproducible determinations can be highly relevant to the coating selection process, irrelevant test data can create confusion and may lead to product selections based on incorrect assumptions. The trick for specifiers, contractors, and owners is to understand which tests are most relevant.

Understanding Repeatability, Reproducibility, and Acceptable Variances

Numerous coating-related ASTM standards can help in evaluating physical and chemical characteristics to make sound product selections. Their relevance relates primarily to the repeatability and reproducibility of each testing method and the resulting acceptable levels of variance. Mechanical tests, such as those using pneumatic testing devices, have lower acceptable levels of variance, while tests relying on human factors, such as manual adhesion tests, have higher acceptable variances due to their subjective nature.

The repeatability of a test method measures the maximum difference calculated among multiple tests that are run on the same piece of equipment in the same lab. The reproducibility of a test method measures the maximum difference calculated among multiple tests that are run in different labs. In either scenario, the labs involved should perform round-robin testing to capture multiple values; and averages, using the greater and lesser results, should be used to determine an allowable variance. Actual application results are expected to fall somewhere within a product’s determined variance.

Intra-lab repeatability results are typically very accurate, as the conditions and test operators are the same. However, inter-lab reproducibility results can be suspect due to lab variances. For example, a coatings manufacturer’s lab could have greater accreditation than a third-party lab. Therefore, the third-party lab’s results—while unbiased—may not be as accurate as expected when comparing them to results from other labs. A direct comparison of two or more products tested in the same lab under identical conditions will provide useful, comparative data to help with product selections.

Detecting Nonrelevant Data

Even with credible testing available, product specifiers should be wary of the best testing result achieved from any lab rather than an average of multiple test results. This artificially inflated rating could lead to selecting a less-than-optimal product.

Unfortunately, any shortcomings related to inaccurate data may not be evident, as the final coating application may look great. However, any deficiencies will be evident when a compliance engineer evaluates the coating system’s physical properties against the specification. If the classification of the application is determined to be noncompliant, who is at fault? The contractor isn’t to blame because the specified product was used. The specifier is off the hook because the selection was based on published data. The manufacturer could be absolved based on the allowable variances for published data. Ultimately, the application may be noncompliant per the specification with no one to blame.

Another misleading practice is publishing data based on exorbitant hours of testing. For example, if 50,000 h of ASTM B1171 salt fog testing is done on one product and a competitive product has been tested for only 5,000 h, is the 10-fold factor any more relevant? The competitive product may perform just as well, but it simply hasn’t undergone the longer testing. The inflated 50,000-h testing mark may prevent a broader selection of products if the specifier publishes this data point as a requirement. Doing so may limit the choice to a sole product, which may not be in the best interest of the project.

Identifying Misleading Tests

While most tests provide reasonable, reproducible determinations, some hold little relevance for selecting products and others are highly subjective. In addition, coatings may not perform in real life as they did in a controlled environment. Specifiers, contractors, and owners should thoroughly understand the tests.

Salt Fog Test

This sample panel underwent the ASTM D5894 cyclic corrosion weathering test. Users can expect the coating to perform similar to this test panel when applied to a steel substrate. Photo courtesy of The Sherwin-Williams Co.The ASTM B117 standard, which measures corrosion potential by subjecting coated samples to a salt spray (fog) test environment, may not correlate with real-world performance. ASTM B117 states, “Prediction of performance in natural environments has seldom been correlated with salt spray results when used as standalone data.”1 Therefore, the test may not predict a coating’s performance accurately. For example, specifiers should be especially wary when presented with ASTM B117 data for product selection with an application where salt fog would never be present in the environment (i.e., a drinking water storage tank).

A more representative test is the ASTM D58942 cyclic corrosion weathering test, which uses a modified salt fog test chamber similar to ASTM B117 in conjunction with a separate fluorescent ultraviolet/condensation cabinet (Figure 1). The ASTM D5894 test attempts to emulate real-world conditions by alternating between wet and dry, and dark and light exposure conditions, as opposed to a constant salt fog atmosphere. Yet even with its more realistic testing protocols, the ASTM D5894 standard says, “No single exposure test can be specified as a complete simulation of actual use conditions in outdoor environments.”2 In other words, there is no ideal substitute for real-world application testing.


The ASTM D45413 test determines the pull-off strength—or adhesion—of a coating from a metal substrate and provides a value for the “perpendicular force … a surface area can bear before a plug of material is detached.”3 However, results can vary when using different substrates. For example, the adhesion rating will likely be lower on a 1/8-in (3-mm) steel plate compared to a ½-in (13-mm) plate because the thinner steel may deform and contribute to the plug peeling off. In addition, the test may be subject to manipulation, as a product can perform differently depending on the rate at which the manually controlled testing wheel is turned and how steadily the device is held.

Compressive Strength

The ASTM D6954 test typically involves casting a 2-in (51-mm) cube of a coating material and determining how much force can be applied before it is crushed. However, the result may not correlate with a real-world application of the coating applied at a normal thickness. Consider a specification for a high-strength floor coating applied at 125 mils (3 mm) with a compressive strength of 124 MPa (18,000 psi). A tested rating of 124 MPa on a 2-in thick sample will not transfer to the coating applied at only 125 mils. In addition, when the coating is applied to a concrete substrate, which only has a compressive strength of ~28 to 34 MPa (4,000 to 5,000 psi), the concrete theoretically will be crushed sooner than the coating. To keep it from cracking under higher pressures, the coating must have sufficient elongation properties so it can bridge cracks that will form in the substrate. Therefore, this specification would be better stated in this example as requiring a product with sufficient elongation properties and a compressive strength that is at least above the concrete substrate’s maximum strength.

Abrasion Resistance

The ASTM D40605 test evaluates a coating’s abrasion resistance using an automated Taber abraser to scuff the surface of a rigid coated surface. The reproducibility and repeatability of this test is highly subjective, as results can vary by as much as 95% between labs and even up to 45% within the same lab. The primary culprit for this variability, per the standard,5 is “due to changes in the abrasive characteristics of the wheel during testing.” In addition, the smoothness of the finish can cause varied results. Coatings with greater textures may demonstrate higher abrasion potential simply because the abraser will grind off the coating’s peaks before reaching a smoother area. Even if the smooth area of the coating has low abrasion potential, the loss of the original peaks will suggest a lower overall abrasion resistance.

Determining Optimal Product Selections

Given the subjectivity of certain tests and the potential for receiving nonrelevant or misleading data, how can specifiers, contractors, and owners make sound product selections for specifications and applications? Here are a few tips:

• Use industry standards that feature pass/fail criteria. While ASTM tests provide reasonable, reproducible determinations, they don’t provide pass/fail criteria, so there’s no benchmark to meet. Look to tests like SSPC Paint 36,6 which provides various levels of pass/fail criteria based on multiple test methods agreed upon by industry peers.

• Rely on relevant side-by-side testing. For reliable product comparisons, make sure the products are tested in the same manner at the same time in the same lab and with the same technician. Doing so ensures objectivity and standardization. Also, ensure the tests are relevant to the application and that they are run in a round-robin series to determine a rate of repeatability and reproducibility.

• Review verifiable case histories. Some products that perform the worst in accelerated testing environments actually work the best in real applications—and vice versa. There is no substitute for real-world performance, so look to application case histories, preferably ones similar to the given application, to verify product performance.

• Avoid sole-source specifications. Whether explicitly or by default because no other product can meet the published requirements, specifications listing a single product can artificially inflate job costs with no extended service life benefit. Specifiers should consider expanding the specification. Contractors should push for viable alternatives that can make bids more competitive or enhance the project’s lifecycle.

• Get the full story. Question the relevance and accuracy of data points, and ask product manufacturers to explain why the data are important for the application.

Following these tips can help improve the validity and relevance of specification requirements and lead to optimal coating system selections, which ensures asset owners can realize long service lives and minimal lifecycle costs for their coatings.

Trade name.


1 ASTM B117-16, “Standard Practice for Operating Salt Spray (Fog) Apparatus” (West Conshohocken, PA: ASTM International, 2016).

2 ASTM D5894-16, “Standard Practice for Cyclic Salt Fog/UV Exposure of Painted Metal (Alternating Exposures in a Fog/Dry Cabinet and a UV/Condensation Cabinet)” (West Conshohocken, PA: ASTM, 2016).

3 ASTM D4541-09e1, “Standard Test Method for Pull-Off Strength of Coatings Using Portable Adhesion Testers” (West Conshohocken, PA: ASTM, 2009).

4 ASTM D695-15, “Standard Test Method for Compressive Properties of Rigid Plastics” (West Conshohocken, PA: ASTM, 2009).

5 ASTM D4060-14, “Standard Test Method for Abrasion Resistance of Organic Coatings by the Taber Abraser” (West Conshohocken, PA: ASTM, 2014).

6 SSPC Paint 36, “Two-Component Weatherable Aliphatic Polyurethane Topcoat” (Pittsburgh, PA: SSPC, 2013).

Related Articles