A study released Thursday suggests there may be flaws in insurers' cost-profiling methods of physicians to establish tiered networks.

Santa Monica, Calif.-based RAND Corp. found that about one-fourth of 13,788 physicians studied would be misclassified under the system of cost-ranking commonly used by insurers, according to findings published Thursday in the New England Journal of Medicine.

Insurers increasingly have been using cost-profiling of physicians to decide which to include in tiered health plan networks, such as “high-performance networks,” that often provide incentives to plan members who use them.

Researchers analyzed information from insurance claims for 2004 and 2005 from four health plans in Massachusetts that provide coverage to about 80% of the nonelderly population with private insurance. They used commercially available software to examine the costs of treating episodes of common illnesses such as diabetes and heart attacks, assigning each episode of care to a physician and creating a cost profile for each physician based on all similar episodes of care.

Researchers evaluated the reliability of physician cost scores by considering factors such as the number and types of patients the physicians treated. The results show that the reliability of cost-profiling scores was unacceptably low for physicians in most of the specialty groups.

In particular, researchers found that only about 40% of physicians in 28 specialty groups had cost-profile scores that were at least 70% reliable—a common threshold for reliability—and that fewer than 10% of those physicians had cost profiles that were at least 90% reliable.

Among physicians in a hypothetical two-tiered insurance plan, nearly 40% of internists and nearly two-thirds of vascular surgeons that had been labeled as low-cost providers were not, in fact, lower cost, according to the RAND study. Researchers also found that physicians in surgical specialties were most likely to have low reliability cost-profile scores, while dermatologists' cost-profile scores were the most reliable.

“These ranking systems may be useful for some purposes, but they are not reliable enough at this point to make decisions about encouraging patients to see certain providers or excluding some doctors from insurance networks,” John L. Adams, the study's lead author and a senior statistician at RAND, said in a statement.

American Medical Assn. President J. James Rohack also weighed in on the findings, issuing a statement that said the research “verified the AMA's longstanding contention that there are serious flaws in health insurer programs that attempt to rate physicians based on cost of care. The RAND study shows that physician ratings conducted by insurers can be wrong up to two-thirds of the time for some groups of physicians.”

Insurer physician ranking programs came under fire after a 2007 probe by New York Attorney General Andrew Cuomo that resulted in an agreement with Bloomfield, Conn.-based CIGNA Corp. and several other insurers to revamp their doctor-ranking programs. The revised programs include national standards to measure quality in addition to cost and discloses to consumers and physicians how the rankings are determined, breaking them down by cost, quality and what proportion is based on cost vs. quality when a combined score is given. The insurers also agreed to submit to outside oversight.

In response to the settlement, the Washington-based National Committee for Quality Assurance was selected by the National Consumer-Purchaser Disclosure Project in August 2008 to review and develop standards for physician performance measurement by insurers. The Disclosure Project, a collaboration of employers, consumers, labor organizations, insurers and providers, has been working to establish a set of principles to rate provider performance that would satisfy all of their competing concerns.

“CIGNA has always adhered to the highest available standards in physician profiling,” said Dr. Dick Salmon, CIGNA's medical director for physician performance measurements programs based in Bloomfield, Conn. And since the NYAG settlement, “CIGNA was the first health plan to go through certification under those revised standards in January 2009,” he said.

“NCQA strongly supports efforts to measure cost as well as quality and appropriateness in health care,” said NCQA President Margaret E. O'Kane in an e-mailed statement. “All measures need to be reliable and based on a statistically significant sample if they are to be used to measure performance.”

The Disclosure Project declined to comment.