The leader of the governing party in Romania (the Social Democrats) Liviu Dragnea has announced that the decision is final and that further steps are being taken.
Iohannis said the decision was "not based on firm, wide-ranging evaluations".
Romania is now the fourth country to have announced the relocation of its embassy from Tel Aviv to Jerusalem.
Romania's government has backed moving its embassy in Israel to Jerusalem, drawing a warning from the president - who has the final say - that the move could break global law.
Romania's Government has adopted a memorandum to start the procedure of moving its embassy in Israel from Tel Aviv to Jerusalem. "Moving the Embassy may and I believe would bring great benefits in the short, medium and long run, and we must use this huge opportunit", he said.
Haaretz said the proposal to relocate the embassy was made following a visit to Bucharest by Deputy Foreign Minister Tzipi Hotovely, who is a member of the centre-right ruling Likud party, where she met Romanian Foreign Minister Teodor Melescanu.
Palestinian Prime Minister Rami Hamdallah urged Romania on Thursday during a meeting with Romanian representative to Palestine Catalin Tirlea not to move its embassy.
Dragnea came out in favor of the idea past year, soon after U.S. President Donald Trump recognized Jerusalem as the capital of Israel and said the U.S. embassy would be relocated there.
Earlier, on December 6, U.S. President Donald Trump announced the relocation of the American embassy from Tel Aviv to Jerusalem, marking it an official recognition of Jerusalem as Israel's capital city.