The word you're looking for is probably eavesdrop : Secretly listen to a conversation.
‘my father eavesdropped on my phone calls’ (source: Oxford Dictionaries ) @Zikato reminded me of this scene from The Fellowship of the Ring where Sam is caught eavesdropping (around 2:45) on a secret conversation between Gandalf and Frodo.
ell.stackexchange.com/users/18673/Glorfindel
While I think eavesdrop is probably your best choice, spy would also be an appropriate descriptor for the action of stealthily listening though it does include all kinds of observation, so watching (
and I suppose smelling?)
as well.
Observe (someone) furtively.
(Source: Oxford Dictionary )
ell.stackexchange.com/users/4887/aslum
You can use to listen in .
It can be used either without an object, or with one.
In that case, you listen in on something.
Definition and examples from the Oxford dictionary : listen in Listen to a private conversation, especially secretly.
Seth listened in as his father chatted philosophy with friends
I hurried him into my room and shut the door behind us; I didn't want any eavesdroppers listening in on our conversation.
At this point, I decided to stop listening in on their private conversation.
ell.stackexchange.com/users/30156/Fabio says Reinstate Monica
This is more of a visual, (perhaps even nasal ), term, but it doesn't rule out listening: snoop intransitive verb : to look or pry especially in a sneaking or meddlesome manner
She locks up her diary to keep her brother from snooping. -- MW
ell.stackexchange.com/users/36151/agc
A word that means "listening to someone or a group of people without having him/them know" but not "listening stealthily" is overhear Hear (someone or something) without meaning to or without the knowledge of the speaker.
‘I overheard two doctors discussing my case.’
(source: usage from collins and definition from oxford )
ell.stackexchange.com/users/62104/Hazkaz
I agree that eavesdrop is a good word for this.
An alternative synonym is earwig Oxford Dictionaries definition British informal Secretly listen to a conversation.
‘he looked behind him to see if anyone was earwigging’
ell.stackexchange.com/users/1430/uɐɪ
In this case ‘spy’ could do as well, though that indicates more than just listening, it’s more ‘observing stealthily’
ell.stackexchange.com/users/51836/patrick
I am wondering if there's a verb that means "listening stealthily" or "listening to someone or a group of people without having him/them know".  I am highly confident that there's a verb, but I can't seem to remember what it was.  Just so that we're in the same page, here's an example:   John put his ears over the door to ___ at the private conversation between the king and the queen.
Is there a verb for listening stealthily?
ell.stackexchange.com
<word-request>
ell.stackexchange.com/questions/206488
ell.stackexchange.com/users/93377/blackbird
The word you're looking for is probably  eavesdrop :   Secretly listen to a conversation. An alternative synonym is  earwig  Oxford Dictionaries  definition    British informal  Secretly listen to a conversation.  The following can be used for "listening stealthily": "eavesdropping", "listening in", "earwigging", "spying", "overhearing" and "snooping". 
The following can be used for "listening stealthily": "eavesdropping", "listening in", "earwigging", "spying", "overhearing" and "snooping". 