Recent meta-analyses and simulation studies have suggested that the relationship between soil resource heterogeneity and plant diversity (heterogeneity–diversity relationship; HDR) may be negative when heterogeneity occurs at small spatial scales. To explore different mechanisms that can explain a negative HDR, we conducted a mesocosm experiment combining a gradient of soil nutrient availability (low, medium, high) and scale of heterogeneity (homogeneous, large-scale heterogeneous, small-scale heterogeneous). The two heterogeneous treatments were created using chessboard combinations of low and high fertility patches, and had the same overall fertility as the homogeneous medium treatment. Soil patches were designed to be relatively larger (156 cm2) and smaller (39 cm2) than plant root extent. We found plant diversity was significantly lower in the small-scale heterogeneous treatment compared to the homogeneous treatment of the same fertility. Additionally, low fertility patches in the small-scale heterogeneous treatment had lower diversity than patches of the same size in the low fertility treatment. Shoot and root biomass were larger in the small-scale heterogeneous treatment than in the homogeneous treatment of the same fertility. Further, we found that soil resource heterogeneity may reduce diversity indirectly by increasing shoot biomass, thereby enhancing asymmetric competition for light resources. When soil resource heterogeneity occurs at small spatial scales it can lower plant diversity by increasing asymmetric competition belowground, since plants with large root systems can forage among patches and exploit soil resources. Additionally, small-scale soil heterogeneity may lower diversity indirectly, through increasing light competition, when nutrient uptake by competitive species increases shoot biomass production.