ICD-10 and the problem of interobserver variability

I first heard about the ICD-10 when I was working at a small start-up, trying to develop an EMR for a string of dialysis clinics. It was always spoken of with a certain gravity, like the ominous visit from an aunt that nobody in the family likes, but feels obligated to see.  Practical (read: business) people hate ICD-10. It’s giant and unwieldy. Doctors think it’ll be an excuse to bilk them out of payments.  They dread the day that they get a “false coding” note for a visit for a broken arm because they didn’t specify the patient fell of their bicycle or down a flight of stairs.  So who’s driving this?

I can only assume that it’s research.  ICD-10 must be an epidemiologist’s dream. Want to prove something inane, like the fact that waterskiing accidents are more common in the summer?  ICD-10 is your tool.  If you can collate all the insurance billing from the entire country, you can begin to pull out these vanishingly rare instances and analyze them.

Admittedly, as this article points out, some of the events the ICD-10 tries to capture are so vanishingly rare that they actually, well, vanish.  They’re literally unheard of or actually impossible. But what about some of the other widely panned codes, like falling off a chicken coop?  Theoretically, we could begin to perform real time monitoring of safety conditions in all kind of industries.  These events are rare, which means that if we see a cluster of them occurring in a particular geographic area, an investigation might be warranted. Maybe building inspectors aren’t performing their inspections. Maybe a certain company isn’t enforcing proper safety standards. Again, theoretically, the giant index of ICD-10 codes could drive meaningful data collect and interventions.

The problem is the observer. Interobserver variability is a problem in all sorts of medical fields, from reading chest x-rays to interpreting physical exam findings. For the ICD-10 to be useful for research, you need to code these rare events correctly.  And, with the endless array of options, the chances of this happening seem, to me, to be vanishingly small.  Maybe there’s a good technical solution to this, where an EMR scans the history of present illness and offers a variety of appropriate billing codes (writing “chicken coop” should be a dead giveaway).

The validation and implementation of this for all 155,000 codes is, however, a monumental task at best.  Such an undertaking can (and should) be done by those who created the codes in the first place. Unfortunately, something tells me they can be less than thorough.

Michael Slade is a medical student who blogs at Inside Looking In: Healthcare and the Transformation of Self.

Comments are moderated before they are published. Please read the comment policy.

  • ninguem

    Have you noticed that the ONLY people in favor of ICD-10 are those who stand to profit from it?

    I have this product. It’s crap. It’s useless.

    I know, I’ll FORCE people to buy it.

    • Dr. Drake Ramoray

      You could make the exact same post if you exchanged EMRs for ICD-10

      • ninguem


  • buzzkillerjsmith

    ICD-10 has gone into remission for a year so we don’t have to start complaining about it again until this fall at the earliest. I fully expected to have won the lottery by then.

    • Dr. Drake Ramoray

      On my way home from my most recent conference, I’m looking for early retirement advice. What is your choice game of chance?

    • Ava Marie Wensko George

      Actually, you should start documenting in ICD-10 now, rather than wait. Otherwise you will become further and further behind.

  • ninguem

    More appropriate might be ICD-9 569.42

  • Markus

    I have read that some countries like Germany, France, and Canada are using the icd-10 already. Anybody out there from a nation using the system?

    • Ava Marie Wensko George

      Every other industrialized nation in the world has been using it. ICD-11 is being trialed in Australia, and will be reported back to the WHO in 2017, for approval to roll out in all countries using ICD-10. We like to say how amazing our health system is….when in reality, we fall so far behind even some moderately industrialized countries. Our health reporting is shameful.

      • ninguem

        And every single one uses a system more abbreviated than the one anticipated in the USA.

  • Ava Marie Wensko George

    Actually Ninguem, that is not true. We need this kind of research to advance medicine. It continues to astound me that those who comment on ICD-10 have no idea about it at all. Capturing mortality and morbidity information as well as epidemiological information is part of keeping our country healthy. We can use it to track immunizations, track breakouts, reinforce and influence public health policy. ICD-10 will help us more than hurt you. This is the time to make an argument for adding more coders so that the correct information can be captured and let physicians focus on what it is that they do best – Take care of our patients.

    • Michael Slade


      Thanks for all your comments. My exposure to the ICD-10 was in the context of writing software in the US, so I wasn’t familiar with it’s wider use. I appreciate the opportunity to learn! And I agree that most of the arguments people use to delay its implementation are pretty suspect.

      My point isn’t that the ICD-10 is evil and needs to be buried. My point is that its incredibly complex and human are generally bad at complex things. If we’re going to implement it, let’s do it in a smart way that allows for convenient, consistent use, so it’s not a burden on healthcare providers and actually provides good data.

    • ninguem

      “ICD-10 will help us more than hurt you.”

      Thank you for proving my point.

      The ONLY people in favor on ICD-10 are those who stand to profit from it.

      If that ICD-10 is so important……YOU do it.

    • southerndoc1

      Tell us why you think practicing physicians should bear the enormous expense of all this data collection. If it’s that important, shouldn’t I be paid for my time and effort?

    • JR

      We can’t even trust birth certificates or death certificates to be accurate, so now we’ve implementing a coding nightmare that is guaranteed to be inaccurate. How does inaccurate documentation lead to reliable research? (hint: it doesn’t)

  • Ava Marie Wensko George

    Every other industrialized nation in the world has been using it for decades. The problem in America is that we tie codes to reimbursement. The DSM-5 plays a big part in ICD-11, which the rest of the world will be going to in 2017. We are so incredibly behind that our reporting to the World Health Organization with respect to morbidity and mortality numbers as well as public health is stunted as we do not use the same code structure. We must change so that we can report accurately and be counted amongst the world population health information that is reported out by other countries.

    • ninguem

      “…..The problem in America is that we tie codes to reimbursement…..”

      As do other countries. The Germans as one example. As do other countries,

      Here’s the dissatisfaction the Germans are having with their implementation of ICD-10.

      Wockenfuss R, Frese T

      Three- and four-digit ICD-10 is not a reliable classification system in primary care

      Scandinavian Journal of Primary Health Care, 2009; 27: 131136


      • Dr. Drake Ramoray

        Your only supposed to gather the data for the research projects, not present conflicting data for work that other people want you to do for free (or for a loss when you code incorrectly). /s

        • ninguem

          Indeed. The most vocal advocates of ICD-10 are those who stand to profit from it.

        • ninguem

          The French tie the ICD-10 to payment as well.


          “…..Physicians are required to perform the coding manually or with the help of a navigation tool within a nomenclature or of a lexical research tool. This requires trained staff with a good knowledge of the economic coding rules and of the classification used to code. As a result, coding is a time consuming activity and must be performed together with patient care that is and should remain a priority for medical staff…..”

          Oh, yeah, they love it in Europe.

          • Dr. Drake Ramoray

            Yup you have pointed out that only the US wants to use most/all of ICD-10 for billing (the index of the darn thing is 421 pages. The index.)

            And I have previously pointed out that the US is also the only country that seems to be trying to eliminate fee for service entirely. Every other industrialized nation has at least some fee for service structure.

            Add the legal climate in this country and I wonder how long it will be until there is an exodus of physicians (at least of the primary care non-procedural specialist types.)

            Happy times ahead for all. /s

          • ninguem

            Here’s the Canadian experience. At least what the CODERS went through.


            One of the most frightening aspects of Canada’s switch from paper ICD-9 coding to digital ICD-10 was the significant drop in coder productivity that followed the transition. According to a chart prepared by the American Health Information Management Association (AHIMA), Canadian medical coders working with inpatient charts were completing 4.62 charts per hour in April of 2002, while using the Canadian version of ICD-9. At the start of ICD-10 in July 2002, that number dropped to 2.15, and only rebounded to 3.75 in April 2003. The results for day surgery charts were even worse: 10.68 charts per hour before ICD-10 implementation, and 3.82 immediately after.

            This is NOT a learning curve, it is a PERMANENT decrease in productivity.

          • ninguem

            Here’s what the AAOS had to say.


            The physician loss of productivity, same as in Canada, is expected to be PERMANENT, not just a learning curve.

            It will cause cash flow disruption that may kill the small practices that do not have a professional coder.

            When you look at the actual experience of what other countries go through, they do in fact get a loss of productivity, and it persists. They do not go back to the productivity they had under the old system.

          • Dr. Drake Ramoray

            Spend 83k per provider in my group to be permanently less efficient. Brilliant.

          • ninguem

            See, Drake, you exist to serve the coders.

          • Dr. Drake Ramoray

            And the insurance companies. And the hospitals. Yet another reason to go direct pay.

  • Ava Marie Wensko George

    Dear Bob,
    I cannot agree with you more. Really, some of the people commenting here really need to get out of the US and visit other countries, like Germany. You cannot get a real appreciation for how far behind and failing we are until you see other countries and really research how well universal health care and ICD-10 is working for them.

    • ninguem


  • http://www.hdmedicalcoding.com HD Medical Solutions

    Another important clarification to consider when discussing the International Classification of Disease (ICD) is how the U.S. did adopt the Tenth Revision in 1999. The U.S. continues to use ICD-10 to this day along with over 100 countries for death reporting and statistics to the World Health Organization.

    ICD-10-CM (International Classification of Disease, Tenth Revision, Clinical Modification) is the U.S. modification of the original code set for reimbursement and resource allocation. Australia modified ICD-10 for their reimbursement system and uses ICD-10-AM (Australian Modification) just as Germany made their own modifications and currently uses ICD-10-GM (German Modification). Other countries made modifications of the original ICD-10 for their use as well.

    It should be noted how the U.S. was able to acknowledge the benefits and importance of implementing ICD-10 in 1999 and has successfully used it for reporting statistics and specific codes for the deceased. http://www.cdc.gov/nchs/icd/icd10.htm

    The unfortunate fact is how the clinical modification was introduced in 2002. Twelve years later, we are still debating the benefits and important necessity of accurately coding with a higher level of specificity for our living.

    • ninguem

      And every country that has adopted ICD-10 uses a much more abbreviated version than the one anticipated in the USA.

      Or so I’ve heard.

      Is that accurate?

      • http://www.hdmedicalcoding.com HD Medical Solutions

        Based on the CDC website for ICD-10-CM, the U.S. modification has 69,823 code sets. I’m not familiar with ICD-10-GM but the site I found states “The electronic version of the Alphabetical Index currently contains more than 75,000 such texts.”

        I’m sure one would have to consider differences in language although mention is made to point out how changes in word order or splitting the term into components might increase the “approx. 75,000 diagnostic texts [to] turn into almost twice as many search entries in the print edition.” For example, “Thorakaler Bandscheibenvorfall” can be searched for under “thorakal” (thoracic), “Bandscheibe” (intervertebral disc) , “Vorfall” (displacement)”.

        The U.S. modification directs the coder with instruction to “see condition” or “see disease” etc. rather than duplicating entries. In my opinion, this lessens the opportunity for potential error compared to the German version which has been in use since 2005.

  • Michael Slade

    I’m not trying to say ICD-10 shouldn’t be implemented. My point is that if, like you say, we want to use ICD-10 to improve patient care and do research, we need to implement it thoughtfully. If using or not using specific codes is difficult and variable, any data derived from them is going to be suspect.

    • ninguem

      By definition, if there are nation-specific adaptations of ICD-10, there will be differences in data reporting between countries.

      There is, and always has been, significant differences between countries in how they report health statistics. Infant mortality is the classic example.

  • ninguem

    Even with ICD-10, there are significant variations between countries in the reporting of health statistics.

    The classic example is infant mortality.


    See the section on “measuring IMR”

    Then there’s the nation-specific adaptation of modifications of ICD-10.

    We will get ICD-10 eventually. There’s no reason to make it as difficult as it is, as costly to individual physicians as it is here. Why are we being forced to use the biggest, most cumbersome version of ICD-10?

  • T H


    ICD-11′s roll out is 2017.


  • Dr. Drake Ramoray

    I’m late to the party but lets take a simple example of a real world problem and not weird codes that nobody will ever use.

    Lets say you have an uncontrolled type II diabetic. In the past if there were no complications all you had to code was 250.02. If they had retinopathy you had to code 250.50 and 362.01. The first code for diabetes uncontrolled and the second for retinopathy. Now there are other codes for retinopathy but they are not mandatory and nobody uses them.

    ICD-10 has 16 codes for uncontrolled type II diabetes well controlled with retinopathy. Each one of them is unique so using them incorrectly will disrupt billing.

    E11.351Type 2 diabetes with proliferative diabetic retinopathy with macular edema.

    E11.359 Type 2 diabetes with proliferative diabetic retinopathy without macular edema.

    14 more just like it.

    Now lets say you have to use ICD-10 and you have a patient with a history of a pressure ulcer or other poor circulation or some other reason that they need shoes. You used to be able to just have well controlled diabetes 250.80 and even without one of the supporting codes the RX would go through. Even this level of specificity and note documentation with ICD-9 is burdensome and is one of my least favorite forms to fill out.

    But with ICD-10. 250.80 becomes goes from that to 20 different unique codes.

    E11.618 Type II diabetes with other diabetic arhtrophathy

    E11.620 Type II Diabetes with diabetic dermatitis

    E11.621 Type II Diabetes with foot ulcer.

    17 more just like it.

    Diabetes goes from a coding schema that most primary care doctors already don’t do correctly 250.x0, 250.x1, 250.x2, 250.x3 with a secondary supporting diagnosis that can fit on a que card to 168 individual codes on ICD-10 all of which are required to be correct for billing.

    If these research argument people want me to help you with your research and data gathering then pay me to collect your data. My job is to take care of patients not be a research intern.