in the sense of signal-detection theory). It may be that increased profundity ratings are associated with lower reflective thinking (for example), regardless of the presented content.
The goal of Study 3 was to test the possibility that some
people may be particularly insensitive to pseudo-profound
bullshit, presumably because they are less capable of detecting conflict during reasoning. For this, we created a
scale using ten motivational quotations that are conventionally considered to be profound (e.g., “A river cuts through
a rock, not because of its power but its persistence”) in that
they are written in plain language and do not contain the
vague buzzwords that are characteristic of the statements
used in Studies 1 and 2. The difference between profundity ratings between legitimately meaningful quotations and
pseudo-profound bullshit will serve as our measures of bullshit sensitivity. Secondarily, we also included mundane
statements that contained clear meaning but that would not
be considered conventionally profound (e.g., “Most people
enjoy some sort of music”). If the association between analytic thinking and profundity ratings for pseudo-profound
bullshit is due to bullshit detection in particular, analytic
thinking should not be associated with profundity ratings for
mundane statements.
12Method
12.1Participants
A total of 125 participants (52 male, 73 female, Mage = 36.4,
SDage = 13.3) were recruited from Amazon’s Mechanical
Turk in return for pay. Only American residents were permitted to sign up for the study. All participants reported
speaking fluent English. Given the strength (and accumulating cost) of the previous findings, 125 participants was
deemed a sufficient sample. These data were not analyzed
until the full sample was completed.
Eleven participants were removed because they responded affirmatively when asked if they responded randomly at any time during the study. Fourteen participants
failed an attention check question but were retained, as in
Studies 1 and 2.
12.2Materials
We created four 10-item scales. For the BSR, we used the
original 10 items from Study 1 and the 10 Chopra-Twitter
items from Study 2. We created a scale with 10 statements
that convey meaning, but that are mundane (e.g., “Newborn
babies require constant attention”; see Table S4 for full list).
Finally, ten motivational quotations were found through an
internet search and used to form a second scale (e.g., “A wet
person does not fear the rain”; see Table S5 for full list). Participants completed the heuristics and biases measure from
Studies 1 and 2 (α = .61).
12.3Procedure
The four types of statements were intermixed in a unique
random order for each participant. The statements were presented at the beginning of the study. Participants then completed the heuristics and biases battery.
13Results
Of the 114 participants, 47 (41.2%) indicated that they know
who Deepak Chopra is (“uncertain”: N = 7, 6.1%; “no”: N =
60, 52.6%). This knowledge was not associated with lower
profoundness ratings for bullshit or Chopra-Twitter items,
t’s < 1.4, p’s > .17. Nonetheless, we report our correlational
analyses with the full and restricted sample.
Focusing on the full sample, profoundness ratings for the
BSR items (α = .91) and for Deepak Chopra’s actual tweets
(α = .93) were very highly correlated (r = .89). We combined the two sets of items into a single BSR scale, which
had excellent internal consistency (α = .96). The motivational quotation scale had acceptable internal consistency
(α= .82) and the mundane statement scale was also reliable
(α= .93). However, the distribution of profoundness ratings
for each of the mundane statements was highly skewed (see
Table S4). Further inspection revealed that the vast majority of ratings (80.1%) for mundane statements were 1 (not
at all profound) and many participants (N = 52, 46%) responded with 1 for every statement. Three standard deviations above the mean for the mundane statement scale was
not larger than 5, indicating that there were outliers. There
were no outliers for the other scales. A recursive outlier
analysis revealed 22 participants who had profoundness ratings for mundane statements that were statistical outliers.
Evidently, these participants found the ostensibly mundane
statements at least somewhat profound. This may reflect a
response bias toward excess profundity among some participants. Indeed, relative to the remainder of the sample, the
22 outlying participants had higher profundity ratings for the
pseudo-profound bullshit, t(112) = 2.50, SE = .21, p = .014,
and (marginally) the motivational quotations, t(112) = 1.83,
SE = .16, p = .071. Moreover, the outlying participants also
scored lower on the heuristics and biases task, t(112) = 3.23,
SE = .13, p = .002. Key analyses below are reported with
outliers both retained and removed for the mundane statement scale. The mundane statement scale had low reliability (α= .35) when the outlying participants were removed,
as would be expected given the low variability in ratings.
The mean profoundness rating was lower for the BSR
items (M = 2.72, SD = .90) than for the motivational quotations (M = 3.05, SD = .69), participant-level: t(113) = 3.90,
