This comes after the president officially informed Prime Minister Ranil Wickremesinghe on Monday that he would be stepping down from the post.
Wickremesinghe's office said that the entire Cabinet would resign and hand over their responsibilities to the new all-party government once it is formed.
"All the ministers who participated in the discussion were of the opinion that as soon as there is an agreement to form an all-party government, they are ready to hand over their responsibilities to that government," the PM's office said.
As per reports, Opposition parties conducted a meetings to come to an understanding regarding forming a government.
Wickremesinghe had met leaders of Opposition parties in their latest meeting on Monday to discuss government formation.
"We agreed in principle to form a government of unity with all parties' participation for an interim period. This will be a government where all parties are represented," Wimal Weerawansa, a member of the Sri Lanka Podujana Peramuna (SLPP) party's breakaway group said, as per news agency PTI.
Unprecedented protests had rocked the capital city Colombo on Saturday, compelling the PM Wickremesinghe to offer his resignation.
'Interim Govt for a Short Period, Then Parliamentary Election'
Another member of the SLPP, Vasudeva Nanayakkara, said that they may even form a government before Rajapaksa's resignation on Wednesday.
Rajapaksa on Saturday told Parliament Speaker Mahinda Yapa Abeywardena that he would be resigning.
A representative of the primary Opposition party, Samagi Jana Balawegaya (SJB), said that extensive discussions had been held in this regard.
"We aim for an interim government of all parties for a limited period and then go for a parliamentary election," SJB general secretary Ranjith Madduma Bandara said.
As per Sri Lanka's Constitution, the incumbent PM would be appointed the acting president for a certain period of time, until Parliament holds a vote to appoint a new president.
PM Wickremesinghe had also expressed willingness to step down when a new government comes to power. In that case, the Speaker would become the president until a new one is assigned.
Protests In Sri Lanka
On Saturday, thousands of protesters breached police barricades and stormed into the residence of President Rajapaksa in Colombo.
Rajapaksa had fled from his residence a day before the protest, and is said to have fled the country itself.
At least 30 people, including two police officials, were injured amid clashes between security forces and protesters, in one of the biggest anti-government rallies the country has seen in recent months. The police also fired tear gas shells to disperse them.
Several protesters could also be seen taking a dip in a pool in Rajapaksa's residence.
Wickremesinghe's private residence in Colombo was also set on fire by the protesters on Saturday – hours after he announced that he would resign from his post.
Five Cabinet ministers had also announced their resignation in the aftermath of the protests.
Amid the protests, Sri Lanka's Chief of Defence Staff Shavedra Silva refuted claims on social media that military personnel were heading towards the Galle Face protest site in Colombo on Monday, ANI reported.
(With inputs from PTI and ANI.)