Evaluation of methodological quality of clinical studies

Размещено на http: //www. allbest. ru/

Department of clinical pharmacology and evidence-based medicine

Discipline “Basics of evidence-based medicine”

Theme: «Evaluation of methodological quality of clinical studies»

1. Reliability of research results

Any study based on the reliability and validity of the results and their applicability in clinical practice, characterized with two entries - credibility (internal validity), and generalizability (external validity, applicability).

Random error - occurs because of deflection resulting from a single observation or measurement of its true value, which was caused by an accident. Random variations occur at any stage of the study and are associated with individual variability of biological properties studied in humans or animals, random measurement errors and inadequate sample size.

In contrast to systematic errors in the random errors can not be eliminated but can be minimized. This is done the proper planning of the study, the increasing number of patients in the study, repeating the measurements several times and, in addition, by estimating the probability of random error, using statistical methods. That is the minimization of random errors is one of the main objectives of the statistical analysis of results obtained in biomedical research.

In cases where systematic errors are detected during the formation of different types of samples, created the so-called bias (unrepresentative) sample. This sample is systematically different from the population, representing the object of study, or from the population against which to apply research results.

Reliability (internal validity) of research results depends on how the structure of the study to meet the requirements, and to what extent the data obtained are valid for the sampling. On this basis, should be considered a reliable study in which the possibility of systematic and random errors is minimized.

Commonly used level of statistical significance (p value) reflects the probability of validity of the hypothesis that there is no statistically significant differences in estimated effects. If as a result of statistical comparison of experimental and control groups is set to p <0.05, the probability of differences arising as a result of chance is no more than 5%. Differences between the values of having a probability of error, considered to be statistically significant.

When conducting the CI generally use the following basic criteria for assessing the value of p:

p > 0.05 - statistically significant difference;

p <0.05 - statistically significant difference;

p <0.01 - statistically highly significant difference.

For example, if the relative risk (RR, ratio of the probability of adverse outcome in the intervention group to the likelihood of the control group) is 0.8 p <0.05, then such a difference between groups statistically significant.

The range within which may lay the true value of rate in the population from which the sample was formed of the study, called the confidence interval (CI). It is calculated to compare the magnitude of the effect, evaluating the degree of equivalence of the two interventions, which rye characterize the range of parameters within the boundaries down the errors of definitions, measures the statistical relationship. Usually at this point the boundaries within which the true effect is 95% (95-percent confidence limits relatively interval). For example, in the example above, point score is 0.8, and 95% CI can be 0.6 - 0.99. By potentially clinically significant effect - reducing the frequency does not a favorable outcome, for example, phlebothrombosis, 20% - could be provided with such a presentation of the survey is not very convincing, because its value can only be to 2-3%.

Evaluation of statistical significance and CI refer only to random error. The systematic error arises from errors in design and analysis of research data, and usually can not be estimated by statistical means.

It is the validity of the research is determined by its value. In TBE developed recommendations on the best options for the structure of research conducted for a variety of tasks which are researchers.

It should be noted that in the following table lists the types of research that are optimal for a particular case, but in practice allow for derogation from this scheme, primarily because in many cases the "optimal" design is not available. For example, in relation to rare diseases is not possible RCTs, but with respect to the possible harmful effects of RCTs and cohort studies are limited. It should be critical of the study, whose structure does not meet its objectives.

Generalizability (external validity) - the extent to which the results of this study apply to other groups of patients, for example, both sexes, populations, etc. Since there is a representation of the general properties of patients with a disease, the possibility to treat them similar means, is it possible to carry out research on a limited group of patients, and then based on the results of research to treat such patients.

Table 1 Main variants of optimal structure of study depending on it task

In other words, allowed the validity of the fact that the subjects included in the study are comparable to others like them. The structure of scientific research implies that participants in the study correspond to patients who are expected to be treated based on the results of the study. For example, participants are selected from the study at random. In practice it is difficult to perform, such as in the study may include new, not yet treated patients, and the doctor are treated mostly patients who are suffering for many years. So in most cases to increase the generalizability of the results tend to ensure that the sample was at least representational, i.e., corresponded to the main characteristics of the study population. For this we would avoid working with groups that differ significantly from the general population. However, manufacturers of drugs may specifically to conduct research in a way that most clearly demonstrate the effectiveness of its drugs, for example, selecting patients with special characteristics. As a result, some quite good-quality studies have a low generalizability.

In order to increase generalizability also apply multicenter study to include patients from different geographical regions, i.e. sample is represent commutative with respect to a wider geographical area. Therefore, the results of these studies can then be more reasonably applied to the population of this area.

At the critical reading, be sure to evaluate the representativeness of the sample described in the article.

Usually the quality of generalizability of research results achieved by minimizing the systematic errors.

2. Systematic and accidental errors

Systematic error - the systematic (non-random, one-way), the deviation of the results of studies from the true values. Identify several basic types of systematic errors.

Systematic error due to violation of the rules of selection of patients (selection bias). It most often occurs at the stage of the study groups as a result of selection for inclusion in the study of persons who are not representative of the general population of patients. This systematic error is generated due to the fact that the compared groups of subjects differ not only on the main priorities, but also on other factors affecting the results of a study, i.e. participants actually drawn from different populations.

Example: in the case when, as a control group used previously recruited patients, and methods of their household survey over time has evolved, there comes a chronological shift.

Example: In a study included volunteers, themselves responding to the announcement of the study.

Selection bias may lead to the formation of the ICS control group, poorly comparable with the core group. For example, the formation of a control group of patients with other diseases interfere with incoming factors associated with this disease. On the other hand, if the control group is formed from the general population, the results may be incompatible with the core group, for example, by age and sex. To prevent these errors is necessary to select pairs of patients in the control and study group on multiple grounds, potentially affecting the studied parameters. Another option to prevent a mistake - to use multiple control groups.

Error identification, more typical of claim may also arise in RCTs, for example, if the control group lost the most severe patients.

Systematic error during the measurement, due to poorly chosen method of evaluation research. This error occurs when patients in the two groups are examined differently (different methods of diagnosis, the frequency of surveys) or used non-standardized schema of the data and subjective assessments.

Subjective assessment, in most cases gives the result of an overvalued compared with the estimate of the independent expert and / or objective methods.

Example: The error due to differences in details of anamnesis in groups of patients and healthy subjects.

Example: radiologists, if assess radiographs, knowing more about the patient, may more closely and critically evaluate the "control" patients, compared with receiving active treatment. "

Systematic error due to the influence of confounding factors (confounding), occurs when the studied factors are interrelated, and some of them distort the effects of others. This may be due to systematic errors in the selection, under the influence of chance or because the real interaction of factors that should be considered when analyzing the results of the study.

Example: if you study the influence of vegetable consumption on the occurrence of the disease was not taken into account different propagation of the second risk factor (eg, smoking) in the two groups.

Systematic error due to a placebo effect. "The effect of pacifiers" - the systematic improvement of patients during treatment simulation. If the control group being treated, outwardly indistinguishable from the active to the intervention group, the difference between these groups eliminates the placebo effect.

During the observation of patients they have seen improvement. Part of this effect is explained by the natural course of disease, and some - a nonspecific effect of treatment (placebo effect) and the difference between the groups corresponds to the additional benefits brought by active treatment. RCTs specifically planned so as to weed out all the effects, except for the actual effect of active treatment.

Figure 1 Revealing of effects of active treatment on comparison with placebo

3. Ways to eliminate systematic errors

The most common sources of errors in the conduct of CI are the expectations of researchers and subjects, whose influence can be reduced by using standard control methods using:

competent selection of subjects in the control group;

methods of "blinding (masking interference);

randomisation (stratified or not) in the formation of different groups of subjects;

methods of statistical modeling.

Test with self-control - for the experimental and control groups involved the same object, for example, a patient on separate days receiving treatment in the other - a placebo.

Cross-testing - some patients are selected for the experimental group, the other - for control, after stopping the treatment in the new period treatment group becomes the control, and control - a group of treatment. When a generalization of the results obtained, that each patient was his own control.

Figure 2 Sources of systematic errors and methods of struggle with it

Tests with the chosen control - by adjusting the controls to each case so that they do not differ on any of the suspected factors. This avoids the differences between the groups associated with known factors that are not interesting in this study. For example, when studying the relation of the disease with features of power by adjusting the control persons can exclude the effect of income on health and smoking. In the selection compared the differences are not between all cases and controls, and the set of differences within the individual pairs.

4. Method of masking of interventions ("blind" study, blinding)

Non-maskable (open) method of doing trials - the subject and the researcher knows about the treatment that gets the subject. In this case, for example, the subject in the control group may begin to be treated by other means and the difference between the groups disappears.

A simple blind method - the subject did not know what treatment he receives. Method is fraught with errors related to the fact that physicians and other medical workers will be treated differently to the management of patients receiving active treatment and placebo groups (old and new intervention).

Double-blind method - the researcher and the patient does not know what treatment he receives, or group.

Triple-blind method - the researcher, the patient and guide drivers CR, initiate studies and analyzing the results, do not know which treatment group receives.

Randomization - method of distribution of subjects in the group in random order - using a table of random numbers or other proper method. Randomization - required property right of the CR, which in this case is called randomized. Using random numbers ensures that the probability that a specific group treatment is the same for all subjects. Randomization is used not only during the CR, but also for studies in experimental animals.

Currently, RCTs have become the standard clinical tests. Developed different methods of randomization, randomization of patients into groups, pair randomization factor, adaptive, and several others.

Correct methods of randomization are used transform tables of random numbers and computer programs, as well as sometimes a coin toss, i.e. methods that generate a random sequence distribution of patients into groups.

Размещено на http: //www. allbest. ru/

Figure 3 Schematic presentation of RCT

However, it should be noted that, despite the general acceptance of randomization are often misunderstood and instead of a random distribution of the subjects resorted to a simplified way (in alphabetical order, dates of birth, day of week, etc.) and even allow an arbitrary distribution of the group. Such "psevdorandomization" does not give the expected results.

Stratification is used to ensure equal distribution of subjects by treatment groups with regard to the factors significantly affecting the outcome, such as age, duration of illness, etc. In other words, for example, male patients randomized, regardless of women. Stratification ensures equal distribution of these factors in the treatment groups.

Statistical modeling is used to estimate the power of communication and interaction effect with simultaneous consideration of many variables. The most common method of statistical modeling of the probability of high-quality events (hospitalization, death) is a multiple logistic regression.

5. Hypothesis Testing

clinical error placebo treatment

Using statistical methods are closely related to ideas about the process of cognition. When confronted with unknown phenomena, scientists first described them, classify, observe. The result is a plausible idea about possible causes of observed events, such as unusual disease, or possible ways to manage these events, such as treatment. Knowledge of the mechanisms of disease - the usual source of hypotheses about how they can affect the progression of the disease.

Scientific hypothesis - a hypothesis or a statement calling for a review to confirm or refute it based on the results of the study.

Hypothesis testing is carried out in the study, the results of which then become the material of the statistical proof of the conjecture, for example, the equality of average values or an association.

Usually, the scientist begins with the formulation of hypotheses, seeming to him the most plausible (basic hypothesis). For example, it can be hypothesized that under equal conditions, the result of the treatment is better than no treatment. In order to use the tools of statistical proof of binding formulated null hypothesis, which assumes no difference between the two representative samples from the same population, receive different treatment. Researchers adhere to this hypothesis, prior experience, i.e. until then, until it is disproved by the results of research. Deviation of the null hypothesis means accepting the alternative hypothesis, the essence of which is that the identified differences between groups are not random, i.e. basic hypothesis of the researcher.

Alpha - error is the probability of erroneous rejection of the null hypothesis, and a beta error - the probability of erroneous acceptance of the null hypothesis.

As the basic tool for testing hypotheses, is a statistical analysis, he must answer the question "Are the findings the null hypothesis? »

Statistical analysis of results of studies using a variety of statistical criteria, the selection of which depends on the characteristics of the object, the study design and measured characteristics. The results of applying different criteria can be expressed as an index of magnitude of difference (distance) or the binding forces in the form of the probability of random occurrence of such differences, or a power connection. Last chance is called statistically significant and p value is expressed. If the value of p is small enough (conventionally taken less than 0.05), then found a difference, a trend that is recognized statistically significant. The very magnitude of the criterion (z - a criterion, the criterion of %2, Fisher, Mann-Whitney McNamara, the correlation coefficient, etc.) said the magnitude of the differences, the power of communication. The value of p depends on the number of patients studied. For example, on a small sample of communication, as measured by the correlation coefficient may be close (if there are only two objects, the correlation coefficient is always equal to one!), But it will be statistically significant.

Conversely, the smallest difference between the groups could be shown with a high level of statistical significance if the groups are sufficiently uniform, and their number is large. For example, middle-aged men at high risk of cardiovascular disease statins reduces cardiovascular mortality, but the absolute risk reduction is very small.

A more detailed discussion of statistical analysis in this guide are given. However, those who are interested in this problem can find the necessary and available information in other sources.

Recently, many researchers have come to understand the limitations of hypothesis testing. Becoming an increasingly popular alternative approach to the comparison of data obtained in the study, namely, the calculation of point values and confidence intervals.

Размещено на Аllbest.ru