user-1: I have 5V -> button -> 10K resistor -> PIN2 setup with Arduino UNO. 
My code is very simple: 
int count = 0;

void setup() {
  pinMode(2, INPUT);
  Serial.begin(9600); 
}

void loop() {
  int state = digitalRead(2);
  Serial.println(state);
  if (state == HIGH) {
    count++;
    Serial.print("Hi");
    Serial.println(count);
  }
  delay(1000); 
}

If the button is pressed print "HiX", but what I get is very strange. When I press the button only once I get this:
0
0
0
0
0
1
Hi1
1
Hi2
1
Hi3
1
Hi4
1
Hi5
1
Hi6
1
Hi7
1
Hi8
1
Hi9
1
Hi10
0

The button press is registered 10 times. 
What am I doing wrong ?

user30005: there could be some stray capacitance that's holding the charge.
generally for cases like that, put the buttons active low + pull-up to avoid cases like yours.

user51380: I think it's not a problem of debouncing...
There is a delay(1000) in the loop so after 1 second, the switch is not bouncing anymore.
But switching to VCC is the problem. Connect the switch to GND and use the internal pullup resistors (pinMode(2, PULLUP).
The switch then pulls it down to ground. Now it is floating and probably still HIGH after opening the switch....

user-1: I can't comment that's why I'm writing this as an answer but,
Are you doing exactly this this to wire the button?

I'd like to see your schematic but from what you describe there's a pull-down connection missing

-1: <p>I have 5V -> button -> 10K resistor -> PIN2 setup with Arduino UNO. </p>

<p>My code is very simple: </p>

<pre><code>int count = 0;

void setup() {
  pinMode(2, INPUT);
  Serial.begin(9600); 
}

void loop() {
  int state = digitalRead(2);
  Serial.println(state);
  if (state == HIGH) {
    count++;
    Serial.print("Hi");
    Serial.println(count);
  }
  delay(1000); 
}
</code></pre>

<p>If the button is pressed print "HiX", but what I get is very strange. When I press the button only once I get this:</p>

<pre><code>0
0
0
0
0
1
Hi1
1
Hi2
1
Hi3
1
Hi4
1
Hi5
1
Hi6
1
Hi7
1
Hi8
1
Hi9
1
Hi10
0
</code></pre>

<p>The button press is registered 10 times. </p>

<p>What am I doing wrong ?</p>
 30005: <p>there could be some stray capacitance that's holding the charge.</p>

<p>generally for cases like that, put the buttons active low + pull-up to avoid cases like yours.</p>
 51380: <p>I think it's not a problem of debouncing...
There is a delay(1000) in the loop so after 1 second, the switch is not bouncing anymore.
But switching to VCC is the problem. Connect the switch to GND and use the internal pullup resistors (pinMode(2, PULLUP).
The switch then pulls it down to ground. Now it is floating and probably still HIGH after opening the switch....</p>
 -1: <p>I can't comment that's why I'm writing this as an answer but,</p>

<p>Are you doing exactly this this to wire the button?</p>

<p><a href="https://i.stack.imgur.com/d2ei5.png" rel="noreferrer"><img src="https://i.stack.imgur.com/d2ei5.png" alt="https://www.arduino.cc/en/uploads/Tutorial/button.png"></a>
I'd like to see your schematic but from what you describe there's a pull-down connection missing</p>
