Numerous culture-based diagnostics are available on the Australian and international markets for on-farm detection of bacterial pathogens in milk. Use of such diagnostics may provide an opportunity to improve the prudent use of antimicrobials in udder health management. Farms are low-resource settings in terms of diagnostic microbiology capacity. The World Health Organisation has identified criteria for the evaluation of diagnostic tests in low resource settings based on Accuracy, Sensitivity, Specificity, User-friendliness, being Rapid or Robust, Equipment-free and being Deliverable (ASSURED). Here, we review how those criteria can be interpreted in the context of microbiological diagnosis of mastitis pathogens, and how on-farm diagnostics that are currently available in Australia perform relative to ASSURED criteria. This evaluation identifies multiple trade-offs, both with regard to scientific criteria and with regards to convenience criteria. More importantly, the purpose of testing may differ between farms, and test performance should be evaluated relative to its intended use. The ability of on-farm mastitis diagnostics to inform mastitis treatment decision-making in a timely and cost-effective manner depends not just on test characteristics but also on farm-specific pathogen prevalence, and on the farm enterprise's priorities and the farm manager's potential courses of action. With most assay evaluations to date conducted in professional laboratories, there is a surprising dearth of information on how well any of the diagnostic tests perform on-farm and, indeed, of the on-farm decision-making processes that they aim to inform.