Social isolation is associated with higher mortality in studies comprising mostly white adults, yet associations among black adults are unclear. In this prospective cohort study, we evaluated whether associations of social isolation with all-cause, cardiovascular disease, and cancer mortality differed by race and sex. Adults enrolled in Cancer Prevention Study II in 1982/1983 were followed for mortality through 2012 (n = 580,182). Sex- and race-specific multivariable-adjusted hazard ratios and 95% confidence intervals were estimated for associations of a 5-point social isolation score with risk of death. Social isolation was associated with all-cause mortality in all subgroups (P for trend ≤ 0.005); for the most isolated versus the least isolated, the hazard ratios were 2.34 (95% confidence interval (CI): 1.58, 3.46) and 1.60 (95% CI: 1.41, 1.82) among black men and white men, respectively (P for interaction = 0.40) and 2.13 (95% CI: 1.44, 3.15) and 1.84 (95% CI: 1.68, 2.01) among black women and white women, respectively (P for interaction = 0.89). The association did not differ between black men and black women (P for interaction = 0.33) but was slightly stronger in white women than in white men (P for interaction = 0.01). Social isolation was associated with cardiovascular disease mortality in each subgroup (P for trend < 0.03) but with cancer mortality only among whites (P for trend < 0.0001). Subgroup differences in the influence of specific social isolation components were identified. Identifying and intervening with socially isolated adults could improve health outcomes.