Dozens of severely injured Palestinians were taken in ambulances and hundreds of foreign passport holders made the crossing on Wednesday.
Egypt said it would initially allow in more than 80 injured and more than 500 foreign and dual nationals in total, in a deal said to have been brokered by Qatar.
This is the first such evacuation since the conflict began more than three weeks ago. It is not clear how long the border will stay open.
It has already been open for more than 10 days to allow aid into Gaza. So far, more than 200 trucks have crossed the border, but much more is needed.
It's the southernmost post of exit from Gaza and borders Egypt's Sinai peninsula.
There are only two other border crossings from and into the Gaza Strip - Erez, a crossing into Israel in northern Gaza, which is for people, and Kerem Shalom, a solely commercial goods junction with Israel in southern Gaza. Both are shut.
Gunmen from the Palestinian militant group Hamas, which governs Gaza, attacked the Erez crossing on 7 October during an unprecedented assault on southern Israel which killed more than 1,300 people.
Days later, Israel declared Erez and Kerem Shalom closed until further notice, leaving the Rafah border as the only way into and out of the Strip for people.
Rafah is now also the only crossing point for humanitarian aid.
Hamas and Egypt exercise control over who can pass through, but operations have been disrupted since Israel began conducting waves of air strikes on Gaza in retaliation for Hamas's assault.
Egyptian media said the crossing was shut down following three Israeli strikes on 9 and 10 October, which it said left injuries on the Egyptian and Palestinian sides of the border.
On 12 October, the Egyptian government asked Israel to halt strikes near the Rafah border crossing so it could serve as a "support lifeline" for people in Gaza, and made it clear that it would not open the passage until there were guarantees for the safety of its staff.
Three weeks later, people are gathering at the border following reports that Qatar has mediated an agreement to allow some people out of Gaza.
Reuters news agency says the Gulf state has been holding talks between Egypt, Israel and Hamas, in coordination with the US.
It is this deal that is allowing some critically injured people and foreign passport holders out of the Gaza Strip.
Qatar is home to the political leadership of Hamas - which has had an office in the capital, Doha, since 2012.
Israel and Egypt have restricted the movement of goods and people in and out of Gaza since Hamas took control of the territory in 2007. The two countries say their blockade is needed for security reasons.
As part of its response to Hamas's deadly attack, Israel's defence minister ordered a "complete siege" of Gaza on 9 October, adding: "There will be no electricity, no food, no fuel, everything is closed."
Meanwhile Egypt fears a massive influx of Palestinian refugees fleeing the war.
Egypt's president warned on 12 October that an exodus from Gaza risked "liquidating" the Palestinian cause and called on Palestinians to "remain steadfast on their land".
It is also concerned about the possibility of Islamist militants coming into the country, having faced a jihadist insurgency in Sinai for almost a decade.
It is not easy for Palestinians to leave Gaza via Rafah. Palestinians wishing to use the crossing must register with the local Palestinian authorities two to four weeks in advance and may be rejected by either the Palestinian or Egyptian authorities with little warning or explanation.
According to the UN, in August 2023, the Egyptian authorities allowed 19,608 exits from Gaza and denied entry to 314 people.
The first civilians from the Gaza Strip have crossed the Rafah border into Egypt since the Israel-Gaza war started.