Some of the criticism of his appointment has been political in nature, such as questions about his role inthe Greensill lobbying controversy, or objections to the fact he will not have to answer questions in the House of Commons.
It is also fair to say Eurosceptic Tory MPs are not exactly dancing for joy either.
But more pertinently, Lord Cameron's foreign policy record from his time in No 10 is also coming under scrutiny.
After all, he is the man whoheld the referendum that led to Britain leaving the European Union, andpushed for a "golden era"of closer relations between Britain and China, a policy now long abandoned.
As prime minister, hebacked military action in Libyawhich brought down Muammar Gaddafi - but left a near failed state in its wake.
And in 2013 hecalled and lost a Commons vote on whether to bomb Syrian forcesafter they used chemical weapons, a political failure that many believe opened the way for Russia to play a bigger role in the Middle East.
That said, Lord Cameron does bring political heft to the Foreign Office.
He may be the fourth foreign secretary in as many years - and the eighth in total since 2010 when the Tories took power  - but Lord Cameron is well known on the international stage.
Lord Cameron has relationships with key leaders that can be leveraged to Britain's gain, and may be seen by allies as a political grown up they can do business with.
Many Foreign Office staff were enthusiastic about his arrival. One senior official told me: "To bring someone of that firepower and experience to head up our diplomacy and development work can only aid the impact we want.
"Most counterparts will be pleased and impressed to see someone of his calibre across the table."
While European leaders will remember Lord Cameron opened the door to Brexit, they also know he campaigned against it, and European nations are increasingly looking to rebuild bridges with the UK.
China hawks will worry about Lord Cameron being too close to Beijing - financially and politically - but the government's policy of "protect, align and engage" with China is pretty settled and laid down in the "integrated review refresh" of UK foreign policy published earlier this year.
Even so, there is much for Lord Cameron's opponents at home and abroad to get their teeth into.
Since leaving Downing Street - and aside from time spent cloistered in his shepherd's hut at home writing his memoirs - Lord Cameron's business activities have been global and controversial.
They include his stint as the vice-chairman of a failed attempt to build a Â£1bn China-UK investment fund, which raised concerns about his close links with Beijing.
He has also given speeches supporting investment in a new port in Sri Lanka that has been heavily backed by China.
Lord Cameron was photographed taking tea with Lex Greensill in Saudi Arabia on a trip where it was reported they met Mohamed Bin Salman, the country's de facto leader who was shunned by many over human rights concerns.
Amid these various business endeavours, charity work, teaching and lucrative speaking engagements, the former prime minister never found a substantive role after No 10 - until, perhaps, now.
The next question will be whether Lord Cameron will seek to use his undoubted political heft to shift UK foreign policy.
He has long been a friend of Israel but has in the past been willing to be candid and critical too.
As prime minister, he was willing to urge Israel to restrain its military operations and do more to protect civilians, and once called Gaza a "giant open prison".
He has strong views on international development, and legislated to ensure the UK always gave 0.7% of its national income on foreign aid - a commitment Rishi Sunak cut to 0.5%.
In his statement on Monday, Lord Cameron pointedly chose to praise the UK's "aid and development capabilities as some of the finest assets of their kind anywhere in the world".
A first test will come next week when the Foreign Office is expected to publish a White Paper setting out its development policy for the next seven years.
Lord Cameron's appointment is not without historical and constitutional precedent.
Alec Douglas-Home and Arthur Balfour both became foreign secretary after their time as prime minister - and Margaret Thatcher also had a foreign secretary drawn from the House of Lords.
But despite the potential benefits, Lord Cameron's return to government also comes with risks.
One of his predecessors in Downing Street, Lord Rosebery, once said having a former prime minister in cabinet was "a fleeting and dangerous luxury".
Mr Sunak, the Conservative Party and the rest of the country are about to find out if that is true.
David Cameron is not the first former prime minister to take over at the Foreign Office - but he comes to the role with some baggage.