In 2014, his forces allied with the Houthis - despite the fact that as president he had gone to war with them on more than one occasion - to fight the internationally recognised government which has the support of a Saudi-led coalition.
It was a bitter end for the former president who had ruled the north of Yemen and then a united north and south for almost 34 years.
A video circulating online purported to show Saleh's body, his eyes open but glassy, motionless with a gaping head wound, as he was being carried in a blanket by rebel fighters.
As news broke of Saleh's death on Monday, the United Nations humanitarian coordinator for Yemen, Jamie McGoldrick, called for a pause in the fighting in Sanaa, noting that dozens of people had reportedly been killed and hundreds injured, including civilians.
Saleh's rule over Yemen extended for more than three decades. After a short exile in Saudi Arabia he returned to the country, and continued to wield power from behind the scenes.
Local broadcast journalist Ra'afat Al Yarrisi said the Houthis had arrested several commanders of the Republican Guard, which is led by Saleh's nephew Col Tariq Mohamed Abdullah Saleh.
Smoke billows in Sanaa on 3 December 2017 as Houthis and Saleh supporters clash
"In addition, the fault lines of the conflict pre-date Saleh's involvement with the Houthis and numerous foreign powers involved in the Yemeni conflict are driven by their own strategic interests in the region", he added.
The Saudi-led coalition imposed a total blockade in October on the country, where almost 80 percent of the people need humanitarian aid to survive. This week's fighting reportedly left aid workers trapped inside a building and unable to administer support to civilians.
The attack took place following a six-day fight in Yemen's capital between the Houthi rebels and the forces supporting ex-President Saleh. The rivalry between Riyadh and Tehran, which has increased significantly in recent months, has exacerbated the conflict in Yemen.
The realignment of Saleh's forces with the Saudis would mark a significant turn in a war that is part of a wider struggle between regional powers Saudi Arabia and Iran.
"Fighting is restricting the movement of people and life-saving services within Sana'a city".
Almost one million people have been infected by cholera in Yemen this year, including more than 2,200 people who have died, according to the World Health Organization. "This latest outbreak of violence could not come at a worse time for the Yemeni people, who are already caught up in the world's largest humanitarian crisis".