Non-triviality of the results of Milgram field experiment in Moscow and New York subway

Note. Statistical significance of prognoses differences from real results was tested using binomial statistics: * .05 >p > .01; **p < .Of; all other prognoses are not significantly differing from real results.

There are only 3 cases of prognosis which are higher than the experimental results. All these 3 cases are statistically insignificant and belong to group of informed respondents. In these three cases an experimenter asked an old man to give (to him or her) a seat. And male respondents overestimated the situation concerning a male experimenter (in New York and Moscow) and female respondent overestimated the situation concerning a female experimenter (in Moscow). Because of insignificance of differences in these cases we will not to discuss the possible reasons of such results.

As it has been expected, the most distant from reality results came out in the group of uninformed respondents: 79.7% (51 of 64) of their prognoses differ significantly from experimental results; the levels of significance of these differences tested by using binomial statistics (here and below “binomial statistics” just will be designated in abbreviated form as “binst”) werep < .05 for 7 of 51 (13.7%) predictions andp < .01 for 44 of 51 (86.3%) ones; young women from Tashkent (subgroup of uninformed respondents), in contrast to all of other five subgroups, made wrong predictions for all 16 positions (8 in New York and 8 in Moscow) with the lowest p-values < .01 (binst). Each of these 16 predictions was the lowest among all 6 subgroups of respondents. At the same time their analogous 8 predictions for Tashkent are considerably higher than for Moscow and New York, i.e. for these respondents the location of the experiment was the most important factor.

For comparison, in the group of informed respondents we have only 43. 8% (14 of 32) of predictions differing significantly from experimental results; all (14 of14) of the levels of significance of these differences hadp < .01 (binst). We haven't found distinctions between gender subgroups of informed respondents concerning distribution of statistically significant and non-significant differences between all of predictions and corresponding frequencies of positive responses of subjects in all of 16 experimental situations.

The level of significance of the total distinction in prognoses exactness between uninformed respondents and informed ones is p = .004 (Mann-Whitney U Test).

Perhaps, during thinking about their prognostic decisions respondents identify themselves not only with experimenters but with subjects as well. One can think about this kind of identification in the cases when gender, age, and residence of respondent coincide with the corresponding characteristics of a subject. We call it complete identification. In the case of partial coincidence of these characteristics we could talk about partial identification of a respondent with a subject. In these two cases (complete and partial identification of a respondent with a subject) we could expect higher predictions as compared with predictions made by respondents free from identification with a subject: the phenomenon of the fundamental error of attribution (I am more personally oriented than other typical person of my gender, age, place of living etc.). This effect must be more pronounced when identification is more complete. Our data confirm these suggestions in 8 of 12 cases of complete identification and in 34 of 60 cases of partial identification.

The situation when a young man (experimenter) asks a woman (subject-passenger) over 40 to yield him a seat is the strongest misconduct in a subway. Such a situation is the most stressful and therefore it is the most difficult for the participants (for the young male experimenter and for the female subject-passenger). If our respondents thought similarly while making their predictions then we could assume that the minimal predicted frequencies ofpassengers' positive responses should be for such a situation; our assumption was confirmed by all 6 groups of respondents for New York subway and by 5 from 6 groups for Moscow subway.

In the real experiment this minimum of passenger response in such a situation was observed only in New York -- 42.9%. In Moscow, such a minimum is 30.0% occurred in another “scene” when the young male experimenter made his unexpected request not to a female passenger over 40 years of age, but to a male passenger over 40 years of age. Such an experimental result in Moscow can be explained by the fact that men older than 40 years are the least healthy part of the Moscow population; another explanation can be revealed from breaking within gender “concurrent” subordination (analogically it was expected that older women will not yield a seat to younger female experimenters, but really it did not happen). Only the subgroup of uninformed (!) female respondents from Moscow (i.e., only one of all 6 subgroups of our respondents) correctly predicted the result of this “real Moscow minimum”.

In a group of 8 real Moscow experiments the result (53.9%) of the theoretically minimal in passengers' response to the experimental situation (a young male experimenter and a female passenger over 40 years of age) follows immediately after 30% response in the situation of the “real Moscow minimum”.

In New York, 44.4% of the response of male passengers over the age of 40 to the request of the young male experimenter (the “real Moscow minimum” situation) follows immediately after the theoretical and practical minimum of the American passengers' responses (see 42.9% above).

Now let us discuss the predictions about experiments in Tashkent (which has never been carried out) made by the same 6 subgroups of respondents. The respondents of 4 subgroups predicted minimal number of positive responses for the situation when an experimenter is a young man and a subject is a woman older than 40 years old. Almost all respondents (except informed women from Moscow) predicted sufficiently low percent positive results for the situation when the experimenter is a young man and the subject is a man older than 40.

Some gender aspects of the predictions of the results obtained in subway experiments

In 70 of total 72 prognoses (6 subgroups of respondents x 3 cities x 4 categories of subjects-passengers) of the frequencies of positive reactions on the request to give a seat to the young female experimenter these frequencies are higher than the analogical prognostic frequencies but related to the young male experimenter. This result can be explained by the fact that a such behavior of a young woman (an experimenter) breaks informal social norms of behavior much less than a similar request made by a young man (such a request of a young man is a challenge to accepted norms of behavior of men in a public place). One can think that this gender effect reveals itselfwhen a respondent makes predictions. Really (in the experiments), this gender effect manifests in both cities (Moscow and New York) for all 4 categories of subjects-passengers.

Uninformed respondents predicted the behavior of female subjects less accurately than of male subjects: the group of these respondents gave 31 predictions statistically significantly differing from reality from total 32 predictions of behavior of female subjects (compare similar values for male subjects: 20 from 32). The level of significance of this gender distinction isp < .001 (^-criterion with 9*emp = 3.86).

The same gender phenomenon was also observed for the group ofinformed respondents: these respondents gave 10 predictions statistically significantly differing from reality from total 16 predictions of behavior of female subjects in comparison with 4 from 16 for male subjects; the level of significance is p = .014 (^-criterion with 9*emp = 2.195).

This second gender effect (the first one was described above) may be probably explained by unexpectedly high frequencies of real positive responses of female subjects after the experimenter's request.

The cross-cultural aspect of the accuracy of prediction
of the results of subway experiments

The integrated sample of all 6 subgroups of respondents gave more precise predictions of results of the New York experiments than of the Moscow ones. They gave (from total 48 for each town) 22 (45. 8%) and 9 (18. 8%) predictions non-significantly statistically differing from reality for New York and for Moscow, respectively. The level of significance of this cross-cultural distinction isp < .001 (^-criterion with 9*emp = 2.89).

This cross-cultural difference between the accuracy of prognoses data (New York vs. Moscow) was more obviously [p < .001 (^-criterion with 9*emp = 3.881)] for the group of informed respondents: they gave 14 (87.5%) and 4 (25%) predictions (from total 16 for each town) statistically non-significantly differing from reality for New York and for Moscow, respectively; probably the awareness of this group was based only on the published US data [for example, in English (Milgram & Sabini, 1978) and in Russian (Milgram, 2001)].

We found no distinctions between gender subgroups of the group of informed respondents concerning distribution of statistically significant and non-significant prognoses. Concretely: in each of these gender subgroups statistically significant distinctions of predicted results from real ones were noticed in the identical 1 of 8 (12.5%) and in the identical 6 of 8 (75%) cases for New York and Moscow, respectively; statistical significance of each of these 14 (7+7) distinctions had also the same level:p < .01 (bins).

The group of uninformed respondents gave 8 (25%) and 5 (15.6%) predictions (from total 32 for each town) statistically non-significantly differing from reality for New York and for Moscow, respectively. However, this cross-cultural effect was statistically non-significant: p > .1 (^-criterion with 9*emp = .94).

The distinction between cross-cultural (Moscow vs. Tashkent) subgroups ofuninformed respondents concerning analogous distribution of statistically significant (10 vs. 3) and non-significant (22 vs. 29) differences amongst all of their 64 predictions (32 for Moscow respondents plus 32 for Tashkent ones) was at the level of statistical significance p = .012 (9*-criterion with 9*emp = 2.256).

The using SEM for analyzing all of the results

Using SEM (Bentler, 2000) allows us to summarize separate findings made in previsions section, to bring together all differences and build a complete picture of individual fragments just as a large mosaic is made.

Fig. 1. The theoretical model describing the influence of all possible combinations of levels of realization of factors determining an experiment on the results of predictions

The predictions made under various combinations of factors determining an experimental situation were considered observable variables. The latent dummy variables were created corresponding to each possible value of all variables determining the experimental situations. For variable city latent dummy variables Moscow, New York, Tashkent were created, for variable experimenter gender latent dummy variables Female and Male were created and so on.

According to the theoretical model, each latent dummy variable, being the level of realization of the factor, determines observable variables (situations) with a corresponding level of realization of this factor (see Fig. 1).

Thus, all measured variables (the predictions about behavior of subjects) in New York are determined by the latent variable “New York”. All predictions about behavior of subjects with respect to an experimenter being a young man are determined by the latent variable “young man” and so on.

Since each situation is determined by four factors, in the full model there are four arrows from any four latent variables entering in each rectangle (a dependent variable). The calculated model can differ from the full model because some determinations can be insignificant.

The analysis will allow us to talk about convergent influence of the latent dummy variable corresponding to this or that value of each variable deterring experimental situation if the majority of predictions corresponding to this latent variable is confirmed. At the same time a “non-relevant” latent variable for this or that measured variable should not give significant predictions. Apart from this, the variables should not be completely synonymous or antonymous (i.e. the absolute value of the coefficient of correlation between them has to be less than 1).

The structural model includes correlations between all latent dummy variables: characteristics of experimental situations, characteristics of respondents making predictions (their gender, age, residence).

Having used the SEM we have found out:

— which of the factor loadings are significant;

— which of the correlations between factors determining an experimental situation are significant;

— which of the correlations between factors determining an experimental situation and characteristics of respondents are significant.

We should note that there are no cases when any loading should not be according the theoretical model but they are according to the data.

The model is good (CFI = .919).

If a latent variable loads a measured variable significantly, this means that the corresponding information (the value of this latent variable) affects significantly the corresponding prediction.

The analysis of Table 5 allows us to make the following conclusions

The predictions are mostly dependent on the information about a city where an experiment was carried out. However, in Tashkent the most important information was the fact that the experimenter was a young man and the place of the experiment affected the predictions much less. Therefore, we can assert that there are convergent and divergent impacts of these latent variables. First, almost all measured variables were loaded by corresponding latent variables. Second, the latent variables corresponding to the cities where the experiments were carried out correlate between themselves, however, this correlation differs from 1 significantly (See Table 6).

The information that the experimenter is a young man affects significantly the predictions in Tashkent, while in New York the predictions are affected significantly by the fact that the experimenter is a young woman.

The latent variable “a subject is younger than 40” affects the predictions significantly in Moscow and New York, while the latent variable “a subject is older than 40” is very important for predictions in Tashkent.

Table 5