Evaluation and Comparison of Diagnostic Test Performance Based on Information Theory

A fundamental concept of information theory, relative entropy and mutual information, is directly applicable to evaluation of diagnostic test performance. The aim of this study is to demonstrate how basic concepts in information theory apply to the problem of quantifying major depressive disorder diagnostic test performance. In this study, the performances of the Dexamethasone Suppression Test-DST and the Thyroid-Stimulating Hormone Test-TSH, two of the diagnosis tests of Major Depressive Disorder, are evaluated with the method of Information Theory. The amount of information gained by performing a diagnostic test can be quantified by calculating the relative entropy between the posttest and pretest probability distributions. And also demonstrates that diagnostic test performance can be quantified as the average amount of information the test result provides about the disease state. It is aimed that this study will hopefully give various points of view to the researchers who want to make research on this subject by explaining how the tests used for the diagnosis of various diseases are evaluated with this way.


Introduction
Diagnostic tests are widely used in many areas. In particular, these tests have a huge importance in medicine sector. By courtesy of early and accurate diagnosis can decrease the morbidity and mortality of disease. It is often important to compare various diagnostics test with each other for specific clinical conditions in order to determine which one is the best to use. To evaluate the performance of a diagnostic test within a given population, we ideally determine both the test result and the disease state for every individual in that population. One of the approaches used to analyze the performance of diagnostic tests in recent years, information theory. This theory of knowledge and uncertainty for the measurement is based on a mathematical basis. In 1973, Metz, Goodenough and.Rossmann have developed a formula used in assessing the performance of diagnostic tests using information theory [1]. After this work, Somoza and Mosmann developed new mathematical and graphical method to evaluate and compare the performance of diagnostic tests for the value of any prevalence (P) using the properties of the ROC analysis and information theory approach [2]. In 1999, Lee was obtained the distance between patients and healthy distributions using the concept of relative entropy [3]. In 2002, Benish was investigated the concept of relative entropy with a different perspective [4].
The aim of this study is to demonstrate how basic concepts in information theory apply to the problem of quantifying diagnostic test performance.

Basic Concepts of Diagnosis Test
A fundamental concept of information theory, relative entropy and mutual information, is directly applicable to evaluation of diagnostic test performance. In this study the amount of information gained by performing a diagnostic test can be quantified by calculating the relative entropy between the posttest and pretest probability distributions. And also demonstrates that diagnostic test performance can be quantified as the average amount of information the test result provides about the disease state. While evaluating diagnosis test performance creating sample is the first thing that should be done. A sample group is selected, the diagnosis is made and the group is tested. On the condition that there is a bilateral class result as disease positive / negative and the test result positive / negative, a crosstab is created. The crosstab is as below. We are going to mention about 5 criteria for the calculation of the test quality. These criteria will also be used in the Information Theory.
Sensitivity: The possibility of finding sick people among actually sick people.

Entropy, Relative Entropy and Mutual Information
A fundamental concept of information theory, relative entropy and mutual information, is directly applicable to evaluation of diagnostic test performance. This section briefly defines Shannon entropy, relative entropy (Kullback-Leibler divergence) and mutual information. The entropy is a measure of the average uncertainty in a data set. It is the number of bits on average required to describe the random variable. Let X be a discrete random variable, in the works Cover and Thomas [5]. While evaluating the performance of the diagnosis test using the information theory, we need to explain the concepts of test results and disease statement. Disease statement is shown with D . On the condition that there are two statements such as the existence or the non-existence of a disease, we can specify the Disease Statement as follows. { The probability distribution of the disease statement before the test is defined with Prevalence ( ( ) P D+ ) and 1-Prevalence ( ( ) P D− ) values. In this case, the entropy before the test is calculated with this formula: The mutual information of two random variables is a quantity that measures the mutual dependence of the two variables. The interpretation is that when mutual information is absent, marginal distributions are independent and their entropies add up to total entropy. Mutual information (10) See Cover and Thomas [5]. After the diagnosis test is applied, the uncertainty of the disease statement changes. On the condition that the diagnosis test results are known, the entropy of the disease statement is called conditional entropy and is calculated according to the formula below.
The properties of the relative entropy equation (12) make it non-negative and it is zero if both distributions are equivalent namely, p q = . The smaller the relative entropy is the more similar the distribution of the two variables and vice versa, see Cover and Thomas [5].
In diagnosis tests, the distance between positive posttest entropy and pretest entropy is relative entropy. This distance is called as the information content which is provided with positive result and it is shown with . I pos Similarly, the distance between negative posttest entropy and pretest entropy is called information content which is provided with negative result and it is shown with . I neg The concepts of . I pos and . I neg are calculated according to the formulas below.

Application and Results
Major Depressive Disorder is to feel so sad, bored, distressed and depressed that it ruins daily routine and social functions of the person suffering. Major Depressive Disorder patients can not even manage their daily routines. In this study, the performances of the Dexamethasone Suppression Test-DST and the Thyroid-Stimulating Hormone Test-TSH, two of the diagnosis tests of Major Depressive Disorder, are evaluated with the method of Information Theory. In this application, data of 200 patients who applied DEU. medical faculty department of psychiatry are studied; The results of DST and TSH diagnosis tests are evaluated below. According to the results deduced from the first 4 tables for application, the DST and the TSH have the same prevalence however, the level of the test values are different.
Since SE value of DST is bigger than SE value of TSH; it can be said that DST can select actually ill people better than TSH from the sample. Since SP value of TSH is bigger than SP value of DST; it can be said that TSH can select actually healthy people better than DST from the sample. Since the EFF value of DST is higher than the EFF value of TSH, we can conclude that DST is more efficient and has less error than TSH.
When we have a look at the legitimacy of the test, we can see that both tests are legitimate. Since both Chi-square values are high and p values − of both tests are smaller than alpha ( α ), we call both tests legitimate. Besides we see that (1, 0) κ , and (0, 0) κ parallel to SE and SP values respectively.    We see that PVP and PVN values on Table 3 Table 5 respectively. On the condition that the test is positive, the possibility to diagnose the person as ill is higher for the TSH test; however, if the test is negative, the possibility to diagnose the person as healthy is higher for the DST test. If we take the disease statement as the random variable, the random variable is indicated either as disease exists before diagnosis test or disease non-exists before diagnosis test. According to this situation the entropy of the disease is only affected with the possibility of disease existence or disease non-existence. Since these possibilities are equal in both tests, the entropy of the disease is the same. When we have a look at Table 6, the entropy value is the same but ( ; ) I D T values are different. According to the result of ( ; ) I D T , DST provides more diagnostic information than TSH for value of P =0.6. Therefore DST dominates TSH.  [6].
According to the results of Table 7, a positive test result provides more information than a negative test result for both tests when P is 0.6 (also P is equal for both tests). The positive test result of TSH provides more information than the positive test result of DST. On the contrary, the negative result of DST provides more information than the negative result of TSH. Therefore, we can conclude that . I pos is related to PVP and . I neg is related to PVN . Another result that can be deduced from Table 7

Discussion and Results
In recent years, for the evaluation of the diagnosis tests, in addition to the other methods, measures calculated according to the information theory have become important.
In this study, it has been explained that how these measures are calculated for the diagnosis tests used in physic and the performances of two tests used for the diagnosis of the major depressive disorder are interpreted by calculating with these methods.
It is aimed that this study will hopefully give various points of view to the researchers who want to make research on this subject by explaining how the tests used for the diagnosis of various diseases are evaluated with this way.