The use of a binary classifier like the sigmoid function and loss functions reduces the accuracy of deep learning algorithms. This research aims to increase the accuracy of detecting and classifying oral tumours within a reduced processing time. The proposed system consists of a Convolutional neural network with a modified loss function to minimise the error in predicting and classifying oral tumours by reducing the overfitting of the data and supporting multi-class classification. The proposed solution was tested on data samples from multiple datasets with four kinds of oral tumours. The averages of the different accuracy values and processing times were calculated to derive the overall accuracy. Based on the obtained results, the proposed solution achieved an overall accuracy of 96.5%, which was almost 2.0% higher than the state-of-the-art solution with 94.5% accuracy. Similarly, the processing time has been reduced by 30–40 milliseconds against the state-of-the-art solution. The proposed system is focused on detecting oral tumours in the given magnetic resonance imaging (MRI) scan and classifying whether the tumours are benign or malignant. This study solves the issue of over fitting data during the training of neural networks and provides a method for multi-class classification.