The radical Islamist preacher, who is in his early 50s, was driven out of the high-security Long Lartin prison in central England in a black van at high speed, television pictures showed.
Abu Qatada was expected to be taken to his home in northwest London where he will be under a curfew 16 hours a day but can leave his home between 8:00 am and 4:00 pm.
He will have to wear an electronic tag and who he meets will be restricted.
The heavily bearded Jordanian of Palestinian origin has been in jail in Britain for the past seven years fighting extradition, and also spent much of the time between 2002 and 2005 in detention or under house arrest.
Abu Qatada was convicted in absentia in Jordan in 1998 for involvement in terror attacks.
The decision by senior judges on Monday to uphold his appeal against extradition to Jordan was a major blow for the British government, which has fought for seven years to deport him.
Deputy Prime Minister Nick Clegg said on Tuesday that the government was determined to deport Abu Qatada.
"We strongly disagree with the court ruling. We are going to challenge it, we are going to take it to appeal," Clegg told ITV television.
"We are absolutely determined to see this man get on a plane and go back to Jordan, he does not belong here," Clegg said.
Home Secretary Theresa May has said had ordered Abu Qatada's extradition after she was given assurances by Jordan that no evidence gained through the torture of two other men would be used against him in a retrial.
But the Special Immigration Appeals Commission -- a semi-secret panel of British judges that deals with decisions on national security -- said that could not be guaranteed.
May told parliament after Monday's decision: "Qatada is a dangerous man, a suspected terrorist, who is accused of serious crime in his home country of Jordan."
She said she believed that the judge who made the final decision had applied the "wrong legal test" in finding in Abu Qatada's favour, adding: "It is deeply unsatisfactory that Abu Qatada has not already been deported to Jordan.
"Successive governments have tried to remove him since December 2001. He has a long-standing association with Al-Qaeda. British courts have found that he 'provides a religious justification for acts of violence and terror'."
The European Court of Human Rights had ruled earlier this year that Abu Qatada could not be deported while there was a "real risk that evidence obtained by torture will be used against him" in a possible retrial.
May then travelled to Jordan to secure guarantees from Amman that he would receive a fair trial and the European court subsequently gave the go-ahead for him to be extradited.
But the immigration tribunal ruled Monday that statements from Abu Qatada's former co-defendants Al-Hamasher and Abu Hawsher, which were alleged to have been obtained by torture, created a risk that any trial would be unfair.
The cleric, a father of five whose real name is Omar Mohammed Othman, arrived in Britain in 1993 claiming asylum and has been a thorn in the side of successive British governments.
Videos of his sermons were found in the Hamburg flat used by some of the hijackers involved in the September 11, 2001 attacks. He has also defended the killing of Jews and attacks on Americans.
A Spanish judge once branded him late Al-Qaeda leader Osama bin Laden's right-hand man in Europe, although Abu Qatada denies ever having met bin Laden.
Britain first ordered his deportation in 2005 and his appeal against that order was rejected in 2009. May then signed a fresh deportation order and Abu Qatada appealed to the European court.
He was briefly freed on bail earlier this year but then re-arrested.
In October Britain extradited another radical Islamist preacher, Abu Hamza, and four other terror suspects to the United States.