Background: Researchers and practitioners have been using code complexity metrics for decades to predict how developers comprehend a program. While it is plausible and tempting to use them for this purpose, their validity is debated, since they rely on code properties and rarely consider particularities of human cognition.

Aims: We investigate whether and how code complexity metrics reflect difficulty of program comprehension.

Method: We conducted a functional magnetic resonance imaging (fMRI) study with 19 participants observing program comprehension of short code snippets at varying complexity levels. We dissected four classes of code complexity metrics and their relationship to neuronal, behavioral, and subjective correlates of program comprehension, overall analyzing more than 41 metrics.

Results: While we could corroborate that complexity metrics can—to a limited degree—explain programmers’ cognition in program comprehension, fMRI allowed us to gain more insights into why some properties of code can be difficult to process. In particular, the code’s textual size drives programmers’ attention and vocabulary size burdens programmers’ working memory.

Conclusion: Our results provide neuro-scientific evidence that supports warnings of prior research questioning the validity of code complexity metrics and reveal factors relevant to program comprehension.

Future Work: We outline a number of follow-up experiments investigating fine-grained effects of code complexity and describe possible refinements to complexity metrics.