She has also terminated the services of four policemen accused in the case.
The two ministers - Lal Singh and Chander Prakash Ganga - had caused an outrage by their presence at a rally in support of the accused arrested in connection with the rape and murder of an eight-year-old girl in Kathua.
On January 23, the government had handed over the case to the Crime Branch which formed a special investigation team and arrested eight people, including two Special Police Officers (SPOs) and a head constable, who was charged with destroying evidence.
Former chief minister and leader of the opposition, Omar Abdullah, criticised Mehbooba for not taking action against these ministers.
She said a sense of justice had been reinforced in the state and urged "New Delhi to wake up to the Kashmir cauldron".
The rape and killing of the girl, who belonged to the nomadic Bakerwal Muslim community, has become a national issue. "Therefore, keeping in mind the perception and general situation, the two BJP leaders, after holding consultations with the prime minister and other senior leaders of the party, chose to submit their resignations".
"I want to assure the entire nation that I stand committed not just to ensure justice for Ashifa but also seek exemplary punishment for those responsible for a crime whose brutal savagery has shamed humanity", Ms Mehbooba wrote on micro-blogging site twitter.
Madhav, who stitched the alliance along with the PDP's Haseeb Drabu, has arrived in Jammu and Kashmir today and he will choose fresh faces from the BJP for the cabinet after the unceremonious exit of Ganga and Lal Singh.
Ahead of the crucial PDP meeting, senior party leader Nayeema Mehjoor, who is also the state's women commission chief, has spoken out and demanded the Chief Minister to pull out from the coalition with the Bharatiya Janata Party (BJP).
"This serves as a cue that the people of Jammu and Kashmir can hold a beacon of hope for the whole country amid the harshly polarized discourse", she added.
The girl was kidnapped on January 10 and her body was found on January 17.