Climate change represents a threat to the productivity of staple crops which are sensitive to heat stress. While heat stress during grain filling is known to reduce rice (Oryza sativa) yields, the impact on grain quality traits such as head rice yield (HRY) has not been fully resolved. The impact of a 10 d heat stress event (31/23 °C, 33/24 °C and 35/25 °C: day/night, 12 h/12 h) from 10 days after anthesis (DAA) to 20 DAA on grain yields and HRY was investigated in two medium grain (M205, Baru) and one long grain (Lemont) japonica cultivars and one medium grain indica cultivar (Teqing). Losses during milling were partitioned into husk losses or grain breakage during de-husking, and bran losses or grain breakage during polishing. While the highest heat stress (35/25 °C) reduced grain yields by around 20% compared to control plants across the four cultivars, reductions in HRY were greater than 50%. In the control treatment, husk losses during milling ranged from 21% (Teqing) to 40% (Lemont) while breakage of white rice during milling ranged from less than 5% in M205 and Lemont to 23% in Teqing. High levels of heat stress led to increased grain breakage during dehusking and/or polishing in cultivars M205, Baru and Lemont. However, heat stress reduced cultivar Lemont HRY not by increased grain breakage during polishing but predominantly due to a 59% increase in husk loss. Given husk tissue is not a net carbon source for developing rice grains, this suggests heat stress in Lemont increased carbon partitioning from vegetative tissues to the husk at the expense of the developing caryopsis. Ultimately, our results indicate reduction in HRY due to heat stress during grain filling is complex and cannot be attributed solely to greater grain breakage during de-husking and polishing. While HRY is typically treated as a single trait, our data suggest quantifying the individual process losses that contribute to HRY as separate traits will improve the accuracy of HRY QTL analysis and selection in breeding programs.