title: GitLab SSH keys stopped working
body: <p>OK, I've stumbled on this topic many times, but this is the first time that none of the regular solutions worked.</p>
  
  <p>I have a CentOS 6.4 server running <a href="http://gitlab.org/" rel="noreferrer">GitLab</a>. It's been working great with 20+ users and 60+ projects, but about 5 hours ago, my main staging server was unable to connect to the GitLab machine for the first time using key authentication and it prompted for password. I regenerated the RSA key and added it to my deploy keys, but that failed as well.</p>
  
  <p>Next, I tried to create a new user on the staging server, create a key for it, and add it to GitLab but it still fails.</p>
  
  <p>Permissions:</p>
  
  <pre><code>drwxr-x---  22 root root 4.0K Oct 28 13:20 root
  </code></pre>
  
  <p>Inside root:</p>
  
  <pre><code>drwx------   2 root root     4096 Oct 28 11:49 .ssh
  </code></pre>
  
  <p>Inside .ssh:</p>
  
  <pre><code>-rw-------  1 root root  227 Oct 28 11:48 authorized_keys
  -rw-------  1 root root 1675 Oct 28 13:09 id_rsa
  -rw-------  1 root root  398 Oct 28 13:09 id_rsa.pub
  -rw-r--r--  1 root root  413 Oct 28 11:49 known_hosts
  </code></pre>
  
  <p>When I try to connect to the git machine:</p>
  
  <pre><code>OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
  debug1: Reading configuration data /etc/ssh/ssh_config
  debug1: Applying options for *
  debug1: Connecting to git.mygitlab.com [212.29.122.24] port 22.
  debug1: Connection established.
  debug1: permanently_set_uid: 0/0
  debug1: identity file /root/.ssh/identity type -1
  debug1: identity file /root/.ssh/id_rsa type 1
  debug1: identity file /root/.ssh/id_dsa type -1
  debug1: loaded 3 keys
  debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
  debug1: match: OpenSSH_5.3 pat OpenSSH*
  debug1: Enabling compatibility mode for protocol 2.0
  debug1: Local version string SSH-2.0-OpenSSH_4.3
  debug1: SSH2_MSG_KEXINIT sent
  debug1: SSH2_MSG_KEXINIT received
  debug1: kex: server-&gt;client aes128-cbc hmac-md5 none
  debug1: kex: client-&gt;server aes128-cbc hmac-md5 none
  debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024&lt;1024&lt;8192) sent
  debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
  debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
  debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
  debug1: Host 'git.mygitlab.com' is known and matches the RSA host key.
  debug1: Found key in /root/.ssh/known_hosts:1
  debug1: ssh_rsa_verify: signature correct
  debug1: SSH2_MSG_NEWKEYS sent
  debug1: expecting SSH2_MSG_NEWKEYS
  debug1: SSH2_MSG_NEWKEYS received
  debug1: SSH2_MSG_SERVICE_REQUEST sent
  debug1: SSH2_MSG_SERVICE_ACCEPT received
  debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-    mic,password
  debug1: Next authentication method: gssapi-with-mic
  debug1: Unspecified GSS failure.  Minor code may provide more information
  No credentials cache found
  
  debug1: Unspecified GSS failure.  Minor code may provide more information
  No credentials cache found
  
  debug1: Unspecified GSS failure.  Minor code may provide more information
  No credentials cache found
  
  debug1: Next authentication method: publickey
  debug1: Offering public key: /root/.ssh/id_rsa
  debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-    mic,password
  debug1: Trying private key: /root/.ssh/identity
  debug1: Trying private key: /root/.ssh/id_dsa
  debug1: Next authentication method: password
  git@git.mygitlab.com's password:
  </code></pre>
  
  <p>When I add SSH keys through the web interface, they aren't added to <code>.ssh/authorized_keys</code>.</p>
  
  <p>I don't really know what to try next :(</p>