Although the other answers may be right when they say that it's a reference to the longstanding Brooklyn/Queens rivalry, as a native New Yorker, I think it is simpler than that.
In the geography of New York City, Brooklyn and Queens are neighbors.
They're both located on the western tip of Long Island.
As we can see, Peter 1 and Steve 2 have lived most of their lives less than 10 miles apart from one another .
It sounds like Steve's reaction to hearing that Peter is from Queens is simply a "Hey, we're neighbors!"
kind of thing, not a reference to any deeper significance.
Update:
My mother was born and raised in Williamsburg, Brooklyn, and my dad was born and raised in Long Island City, Queens.
I ran the scene by them and asked them to explain why Cap chuckles - they both agreed: "Because they're neighbors".
1
For those who are interested, here's 20 Ingram Street, Spider-Man's house.
A family named Parker lives there in real life, and their next-door neighbors are the Osbournes 2 Unlike Spider-Man
, Marvel never gave Captain America a real-world address.
We know he was born in Red Hook, Brooklyn, but we don't know where.
He later moved into an apartment in Brooklyn Heights, but the apartment address - 569 Leaman Place - doesn't actually exist.
movies.stackexchange.com/users/20680/Wad Cheber stands with Monica
It's just to show us that Steve and Peter have a natural rapport, even when fighting on opposite sides.
This is likely just intended to be set-up for a later film when they will be required to work together; establishing a common ground (such as the fact that they both come from New York) early on helps that relationship build naturally.
It also plays into a joke in the second post-credits sequence; When asked by Aunt May how he got a black eye, Peter replies "From Steve".
Aunt May asks if it's a Steve she knows and Peter says "No, he's from Brookyln.
And his friend was huge.
Like, huge".
The joke here is that Peter is 100% telling the truth, even if Aunt May can't possible know what really happened from his line.
movies.stackexchange.com/users/8686/Dr R Dizzle
Lived all my life in Queens.
There's a bit of rivalry/respect between Queens and Brooklyn.
Kevin Smith pointed out in his podcast that Cap most likely recognized Spidey's New York/Queens accent even amidst fighting all the way in Germany.
I saw the movie in Astoria, Queens.
It was definitely a crowd-pleaser moment.
movies.stackexchange.com/users/34380/alex
I found this on instagram and was amazed to see the reference (backward compatibility).
Adding to all the good answers and as @michael also mentions, it's a reference to the first Captain America movie.
Another example is the last fight scene between Cap, Bucky and Iron Man, in which Captain says, I can do this all day.
This exact same line was said by Steve (he wasn't Captain America yet), when he was being bullied in the first movie.
movies.stackexchange.com/users/21081/ABcDexter
You got to be from New York City to understand that reference.
I'm from New York and that's my favorite line in the film; in fact in the entire MCU.
It's a very local thing, specially if you were born or raised in New York City.
For example if you grew up in Brooklyn, you never hung out in Queens, etc.
movies.stackexchange.com/users/34713/cassiosnyc
It's very simple.
Stop reading too much into it.
I'm from New York.
If I'm in an airport in Germany, and I ask someone where they're from, and they reply "Queens", the odds of meeting another NYC resident would be rare, so a chuckle and disbelief would be about the right reaction.
So Steve is basically acknowledging the innate New York brotherhood and Spider-Man's heart, basically saying, "Hey you kick butt.
Oh you're from Queens?
That's cool.
I kick butt too, neighbor, I'm from Brooklyn!".
movies.stackexchange.com/users/36776/Mark Roman
In Captain America: The First Avenger Steve Rogers asks the doctor where he was from and he said Queens.
Then the doctor asked Steve where he was from and he said Brooklyn.
Maybe it was a reference to the first movie.
movies.stackexchange.com/users/34831/michael 
It's not about "superiority", Brooklyn over Queens, so much as it is about the rivalry and the fact that they're fighting (Queens and Brooklyn) literally on the OTHER SIDE IF THE WORLD and the rivalry STILL rears its head.
That's what makes it funny.
But if you didn't perceive that then it's probably "a New York thing".
movies.stackexchange.com/users/38267/user38267
This is a conversation between Captain America and Spider-Man from the movie  Captain America: Civil War :   Captain America: You got heart, kid, where are you from?  Spider-Man: [Straining] Queens!  Captain America: [Chuckles in mild disbelief] Brooklyn!   I don't understand the real meaning of this conversation. Is it supposed to be funny? Is the "Queens" and "Brooklyn" means anything different here?
Why does Captain America chuckle when Spider-Man tells him which borough he's from?
movies.stackexchange.com
<dialogue><marvel-cinematic-universe><captain-america-civil-war>
movies.stackexchange.com/questions/52753
movies.stackexchange.com/users/4925/Ramaraj T
The reference to Spider-Man being from Queens and Captain Americs'a reaction is really only undertandable to people who grew up in New York. It could be a reference to them being from neighboring zones or it could be to do with Queens/Brooklyn rivalry. There is some mutual dislike/disrespect between the Queens and the Brooklyn. It's a very local thing, specially if you were born or raised in New York City"
New York City, Brooklyn and Queens are neighbors. There is some mutual dislike/disrespect between the Queens and the Brooklyn. It's not about "superiority", as opposed to "being on the world's surface", more than it is about those two boroughs battling. You got to be from New York City to understand that reference.
It's a very local thing, specially if you were born or raised in New York Cit