Highly alkaline soils are widespread throughout south-eastern Australia. High pH limits the accumulation and retention of organic carbon (C), and thus the availability of nutrients to crops and is also associated with various elemental deficiencies and toxicities that restrict crop productivity. Applying gypsum and increasing legume production reduce pH and improve soil chemical and physical properties in alkaline, sodic soils, but this strategy has not been explored in alkaline soils in the region where many of the topsoils are non-sodic. Five short-term rotation experiments were conducted to examine the potential to reduce pH, improve soil organic C and increase crop production. In the first year, three legumes—field peas (Pisum sativum), vetch (Vicia sativa) and a medic pasture (Medicago spp.)—were grown at low or high inputs of seed and fertilizer and with three rates of gypsum (0, 2.5 t/ha and 5.0 t/ha). In the following seasons, wheat (Triticum aestivum) and barley (Hordeum vulgare) were grown. Using high sowing and fertilizer rates to increase legume biomass had little effect on soil properties; however, applying gypsum consistently resulted in a lower soil pH and dissolved organic C and higher organic C compared with the unamended control to a depth of 0.20–0.30 m. The largest difference occurred with 2.5 t gypsum/ha with little further change at 5 t/ha. Gypsum significantly increased biomass and grain yield at sites with low concentrations of calcium carbonate. Low rates of gypsum may be effective in reducing pH, maintaining soil organic C and improving crop production in regions with alkaline soils low in calcium carbonate.