At the opening of a new exhibition, Gabriele Finaldi said he wanted to make the area around the gallery "a space that works for us", The Times reports.
The paved area has become a popular place for street artists, especially people dressed as "levitating Yodas".
The council said it wanted to "find a solution that appeals to everyone".
During the opening of Delacroix and the Rise of Modern Art, Mr Finaldi recalled how when he worked for the gallery in the 1990s the North Terrace was a street.
"I don't remember the bagpipes or the Yodas or the oompah band player who shoots flames out of his tuba or the Elvis impersonator or the west African evangelist preacher," he said.
The area was pedestrianised in 2003 and attracts performers who arrive early each morning to claim the best spot.
Hannah Rothschild, chairwoman of the the gallery's trustees, added that meetings sometimes took place "to the sound of Bob Dylan being played on the bagpipes".
She suggested turning the area into "one of London's great parks... with shade and respite and calm".
A National Gallery spokesperson said it believed "there is room for improvement" and wanted to work "in partnership with Westminster Council and the Mayor of London".
Trafalgar Square is looked after by Westminster City Council.
A spokesperson for the council said it would look to "deal with concerns" but wanted to make sure "we keep a vibrant feel to the area".
The north end of Trafalgar Square should be cleared of buskers and "Yoda" street performers, according to the National Gallery's new director.