The time-based resource-sharing (TBRS) model of working memory indicates that secondary tasks that capture attention for relatively long periods can result in the interference of working memory processing and maintenance. The current study investigates if discrete and continuous movements have differing effects on a concurrent, verbal serial recall task. In the listening condition, participants were asked to recall spoken words presented in lists of six. In the drawing conditions, participants performed the same task while producing discrete (star) or continuous (circle) movements. As hypothesised, participants recalled more words overall in the listening condition compared to the combined drawing conditions. The prediction that the continuous movement condition would reduce recall compared to listening was also supported. Fine-grained analysis at each serial position revealed significantly more words were recalled at mid serial positions in the listening condition, with worst recall for the continuous condition at position 5 compared to the listening and discrete conditions. Kinematic analysis showed that participants increased the size and speed of the continuous movements resulting in a similar duration and number of strokes for each condition. The duration of brief pauses in the discrete condition was associated with the number of words recalled. The results indicate that fine motor movements reduced working memory performance; however, it was not merely performing a movement but the type of the movement that determined how resources were diverted. In the context of the TBRS, continuous movements could be capturing attention for longer periods relative to discrete movements, reducing verbal serial recall.