Attention and Performance

Split your attention and your score drops by 1.68 points. That is the headline from a small attention experiment in the seaborn data collection (load("attention"), sourced via seaborn-data): 20 people, 6 conditions, 60 rows. The focused condition averaged 6.80; the divided condition averaged 5.12. The gap is 1.68 points on a score that runs from 2 to 9, and it is not noise. A Welch t-test gives t(53.9) = 4.68, p = 0.00002, with a Cohen’s d of 1.21.

That d above 1 is large, and the headline is tidy: focus helps, go close your Slack. The flagship figure is where the story turns. Focus does not pay off evenly. It pays off most when the task is hardest, and it almost stops mattering when the task is easy. That is the opposite of what most people would guess.

The setup, honestly

The data is balanced and clean. Two attention conditions, divided and focused. A solutions column taking the values 1, 2, and 3, the number of worked solutions a subject saw before scoring. And the score itself. Every one of the six cells holds exactly 10 observations. No missing values, no cleaning.

That balance hides the real constraint. This is a between-subjects design with 20 people. The 30-versus-30 in the t-test is not 30 independent people per side. It is the same 20 subjects measured across solution levels, which four columns do not let me disentangle without subject-level modeling I will not oversell. When I say n, I mean 20 people. Read every p-value below through that lens.

The group-means figure shows the split cleanly, and it shows the catch. The divided cloud is wider (SD 1.57 versus 1.19 for focused), and a couple of divided subjects outscore the focused mean. The separation is real on average. It is not two non-overlapping blobs. Sixty points let you see the overlap with your eyes, which is the right amount of humility for a sample this size.

Score climbs with the number of solutions

Pool both conditions and look at score by solution count: 5.35 at one solution, 5.98 at two, 6.55 at three. Monotonic, in the direction you would guess. See more worked solutions, score higher. A Spearman correlation puts the trend at rho = 0.28, p = 0.031. In the two-way ANOVA the solutions main effect lands at F = 4.70, p = 0.013.

So both things clear the usual p < 0.05 bar. Focus helps, and seeing more solutions helps. Two significant main effects, both pointing the sensible way. This is where most write-ups would wrap.

The turn: the focused edge collapses

Here is what made me re-run the cell. The focused advantage is not constant across difficulty. At one solution, focused beats divided by 2.7 points. At two solutions, by 2.05. At three solutions, by 0.3. The edge does not grow with material to work from. It nearly vanishes.

Picture the two lines in the flagship as two runners. The focused runner starts near the finish and barely moves: 6.7, 7.0, 6.7 across the three levels. The divided runner starts far back and sprints: 4.0, then 4.95, then 6.4, nearly drawing level by the easy task. The whole gap is the divided runner catching up, not the focused one pulling away. My read is that focus buys you the most when the task is sparse, one solution to reason from. Give divided-attention people more material and they recover most of the gap.

The two-way ANOVA backs this up. The attention-by-solutions interaction comes in at F = 5.02, p = 0.010, clearing the same bar the main effects did. But I want to be careful about how hard I lean on that 0.010. An interaction term is a difference of differences, and each cell mean rests on 10 observations. The interaction lives on the smallest, noisiest quantities in the dataset. A single odd subject in the divided-three-solutions cell could swing the 0.3 gap. The cross-check I ran on the main effect (Mann-Whitney p = 0.00005, matching the t-test) has no clean rank-based equivalent for “is this cell pattern non-additive” that I would trust at n = 10 per cell. The interaction is significant. It is not robust.

What I would actually claim

The focused-versus-divided main effect, I will stand behind: 1.68 points, d = 1.21, p = 0.00002, and a nonparametric test that agrees. That is about as solid as a 20-person dataset gets.

The interaction I will report and not bank on. It is statistically significant and directionally clean, focused stays flat while divided catches up. It is also the kind of finding a study this size exists to generate, not to confirm. If someone handed me a budget, the interaction is the hypothesis I would power a real experiment around: does focus mainly matter when there is little to work from? This data suggests yes. It does not get to say yes.

There is also a confound I cannot rule out. solutions is not randomly assigned within subject in a way I can verify from four columns. Treating it as a clean manipulation versus a covariate changes how much the interaction means. I am reading it as suggestive structure in a small dataset, which is the most these 60 rows can honestly support.

The number I keep coming back to is the 0.3. At three solutions, the entire focused advantage, the thing worth 2.7 points when the task was bare, is down to almost nothing. Whether that is a real ceiling effect or ten people having a good day, I cannot tell from here. A significant p-value on a tiny study tells you where to look next, not what is true.