The unit treats severe cases of COVID-19, and at least 28 of the patients killed were on ventilators, CBS reports.
Officials are still trying to figure out how many were wounded in the explosion at Ibn al-Khatib Hospital, but they say there are "many".
The blaze spread quickly across multiple floors in the middle of the night, as dozens of relatives were at the bedsides of the 30 patients in the hospital's intensive care unit where the most severe Covid-19 cases are treated, a medical source said.
Iraq is in the midst of a severe COVID-19 wave.
After the fire first broke out, Mr al-Khadhimi held an emergency meeting at the headquarters of the Baghdad Operations Command, which co-ordinates Iraqi security forces, according to a statement on his Twitter account. The flames overwhelmed the hospital's second floor isolation hall within three to four minutes of the oxygen cylinder exploding, he said.
The incident took place at Ibn al-Khatib hospital in the Diyala Bridge area after oxygen tanks exploded, two health officials at the facility told CNN. Others died of smoke inhalation. Following a special cabinet meeting to discuss the blaze, the government suspended key officials, including the health minister and the governor of Baghdad province.
Videos on social media show firefighters scrambling to extinguish the flames as people flee the building.
Baghdad governor Mohammed Jaber also called on the health ministry "to establish a commission of enquiry so that those who did not do their jobs may be brought to justice".
Salih supported "the decisions adopted by the Iraqi government and the Parliament to call for an immediate investigation into the cause of the blaze until those at fault have been brought to justice", the statement added. He also fired the director of Ibh al-Khatib Hospital and its director of engineering and maintenance, according to a statement from the Health Ministry and his office.
He gave Iraqi authorities 24 hours to present the results of an investigation.
The Special Representative calls for stronger protection measures to ensure that such a disaster can not reoccur.
The health ministry has since recorded a total of 1,025,288 cases of the disease and 15,217 deaths.
As of Wednesday, 274,343 people had received at least one dose, the ministry said.
Iraq rolled out its vaccination campaign last month and has received almost 650,000 doses of different vaccines - the majority by donation or through the Covax scheme for low and middle income nations.