Mobile Ad hoc Networks (MANETs) are vulnerable to external threats due to their open access and lack of central point of administration. Black hole attack is a well-known routing attack, in which an attacker node replies to the Route Requests (RREQs) by pretending itself as a neighbor of the destination node in order to get the data. These days, it has become very challenging to secure a network from such attacks. In this paper, we propose a Detection and Prevention System (DPS) to detect black hole attack in MANETs. For this purpose, we deploy some special nodes in the network called DPS nodes, which continuously monitor RREQs broadcasted by other nodes. DPS nodes detect the malicious nodes by observing the behavior of their neighbors. When a node with suspicious behavior is found, DPS node declares that suspicious node as black hole node by broadcasting a threat message. Hence, the black hole node is isolated from the network by rejecting all types of data from it. The simulations in NS-2 show that our proposed DPS mechanism considerably reduces the packet drop ratio with a very low false positive rate.