The activists demonstrated with a large hand-painted sign which read "Stop arming Israel".
A large Palestinian flag was also attached to the front of the building in a prominent position.
Police Scotland officers closed off the area in front of the building and spoke to the protesters.
The force confirmed the arrests after the group was escorted from the roof at about 16:00.
The incident came ahead of First Minister's Questions in the Holyrood chamber.
As reporters shouted questions to the protesters, one member of the group said: "We're here to put pressure on the Scottish government to take action against arms exports to Israel.
"The occupation of Palestine is illegal, what's happening right now is genocide."
The group of people, who had their faces covered, did not say if they were part of any organisation.
A Scottish Parliament spokesperson said there was no disruption to parliament business and public access to the building was unaffected.
The protest ended with the protesters led away by police officers. They were questioned and searched by officers in a car park area of the Scottish Parliament before being driven away in a police van.
According to theCampaign Against Arms Trade, the UK government has licensed more than Â£400m worth of arms to Israel since 2015.
In response, a Department for Business and Trade spokesperson said: "We can and do respond quickly and flexibly to changing international circumstances.â¯
"All licences are kept under careful and continual review and we are able to amend, suspend, refuse or revoke licences as circumstances require."
Large pro-Palestinian protests have been held in recent days in both Glasgow and Edinburgh, with activists occupying both Glasgow Queen Street and Edinburgh Waverley train stations.
Marches have been taking place across the UK to urge an end to Israeli attacks in Gaza.
First Minister Humza Yousaf, whose parents-in-law escaped from Gaza last week after four weeks under Israeli siege, has backed the right of protesters to hold further events on Armistice Day on Saturday.
It came after Home Secretary Suella Braverman described a planned pro-Palestinian event in London on Armistice Day as a "hate march".
Border crossings in and out of Gaza had been closed since 7 October, when Hamas, which is a proscribed terrorist organisation in the UK, attacked Israel, killing more than 1,400 people and taking more than 240 hostage.
Since then Israel has been carrying out military action in Gaza. The Hamas-run health ministry says more than 10,000 people have died.
Five people have been arrested after a group of pro-Palestinian protesters occupied the roof of the Scottish Parliament in Edinburgh.