<p><strong><em>Please see edit below for added information.</em></strong></p>

<p>So long story short I'm trying to get the Unifi Controller to run on my home Kubernetes cluster. In doing so, I have needed to decentralize the MongoDB database since having a MongoDB instance bundled with each replica on Kubernetes causes the database to crash. Here is my project this far: <a href="https://github.com/zimmertr/Kubernetes-Manifests/tree/unifi_mongodb_separation/Unifi_Controller" rel="nofollow noreferrer">https://github.com/zimmertr/Kubernetes-Manifests/tree/unifi_mongodb_separation/Unifi_Controller</a></p>

<p>In doing so, I have written the following script that runs at provision time for the <a href="https://hub.docker.com/_/mongo" rel="nofollow noreferrer">MongoDB container</a>:</p>

<pre><code>mongo \
    --username ubnt \
    --password "{{ mongodb_password }}" \
    --authenticationDatabase admin \
    --eval 'db.getSiblingDB("unifi").createUser({user: "ubnt", pwd: "{{ mongodb_password }}", roles: [{role: "readWrite", db: "unifi"}]})'

mongo \
    --username ubnt \
    --password "{{ mongodb_password }}" \
    --authenticationDatabase admin \
    --eval 'db.getSiblingDB("unifi_stat").createUser({user: "ubnt", pwd: "{{ mongodb_password }}", roles: [{role: "readWrite", db: "unifi_stat"}]})'
</code></pre>

<p>And then I have configured the Unifi Controller to talk to the database through a volume mounted <code>system.properties</code> file configured like so:</p>

<pre><code># Inform IP Address
system_ip={{ load_balancer_ip }}

# Autobackup directory
autobackup.dir=/backups

# External MongoDB information
db.mongo.local=false
db.mongo.uri=mongodb://ubnt:{{ mongodb_password }}@unifi-controller-mongodb:27017/unifi
statdb.mongo.uri=mongodb://ubnt:{{ mongodb_password }}@unifi-controller-mongodb:27017/unifi_stat
unifi.db.name=unifi
</code></pre>

<p>This is configured as <a href="https://community.ubnt.com/t5/UniFi-Wireless/External-MongoDB-Server/m-p/2669795" rel="nofollow noreferrer">instructed by Ubiquiti.</a> </p>

<p>This all works, and when the Kubernetes deployments start up I see that the Unifi Controller connects to the MongoDB instance in the logs. Furthermore, if I manually connect to the MongoDB databases and run a <code>show collections</code> I can see that many new collections have been created. However, the Unifi Controller stops producing logs here.</p>

<p>If I manually stop the <code>jar</code> file that is running the Unifi Controller in the background on the container, and then restart it, the following stack trace is produced:</p>

<pre><code>$&gt; s6-setuidgid abc java -Xmx1024M -jar /usr/lib/unifi/lib/ace.jar start
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter INFO: destroy called
Exception in thread "launcher" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'Ò00000' defined in class com.ubnt.service.AppContext: Instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [com.ubnt.service.P.D com.ubnt.service.AppContext.Ò00000()] threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dbService' defined in class com.ubnt.service.AppContext: Invocation of init method failed; nested exception is com.mongodb.CommandFailureException: { "serverUsed" : "unifi-controller-mongodb:27017" , "ok" : 0.0 , "errmsg" : "not authorized on unifi to execute command { dropDatabase: 1 }" , "code" : 13 , "codeName" : "Unauthorized"}
</code></pre>

<p>The key element here is</p>

<blockquote>
  <p>not authorized on unifi to execute command { dropDatabase: 1 }"</p>
</blockquote>

<p>And this is where my understanding of MongoDB comes to an end. And my question comes to a beginning. I believe that the reason the Unifi Controller does not continue to start or log any additional messages after connecting to the database and creating the collections is that it is silently failing to perform an action on the MongoDB database. For which it does not have the requisite permissions to do.</p>

<p>When I provision the MongoDB Docker container, I specify the <code>MONGO_INITDB_ROOT_USERNAME</code> &amp; <code>MONGO_INITDB_ROOT_PASSWORD</code> environment variables. Which, I believe, turns <code>auth</code> mode on. This belief is reiterated by the fact that I can connect to MongoDB's <code>admin</code> authentication database via the username and password I provide to these variables.</p>

<p>However, based on the script I posted above that creates my databases and assigns the role <code>readWrite</code> to the <code>ubnt</code> user, I'm curious how I should go about giving the <code>ubnt</code> user the requisite permissions required to drop a database. If I swap out <code>readWrite</code> for other roles like <code>root</code> and <code>dbAdminAnyDatabase</code> the commands fail.</p>

<p>What do I have to do to make it so my ubnt user can drop the <code>unifi</code> and <code>unifi_stat</code> databases? Or what do I have to change about my connection strings to prevent this from happening? I'm a bit of a database admin noob.</p>

<p><strong><em>Continuation Edit:</em></strong></p>

<p>I have updated the Role that is attributed to the <code>ubnt</code> user on <code>unifi</code> and <code>unifi_stat</code> to be <code>dbAdmin</code> instead of <code>readWrite</code>. And got a little further.</p>

<pre><code>#!/bin/bash
mongo \
    --username ubnt \
    --password "{{ mongodb_password }}" \
    --authenticationDatabase admin \
    --eval 'db.createUser({user: "ubnt", pwd: "{{ mongodb_password }}", roles: [{role: "dbAdmin", db: "unifi"}]})'

mongo \
    --username ubnt \
    --password "{{ mongodb_password }}" \
    --authenticationDatabase admin \
    --eval 'db.createUser({user: "ubnt", pwd: "{{ mongodb_password }}", roles: [{role: "dbAdmin", db: "unifi_stat"}]})'
</code></pre>

<p>However, the Unifi Controller is still acting strange. It is now simply looping this in the log files:</p>

<pre><code>2019-02-10 22:33:45,449] &lt;launcher&gt; INFO  system - ======================================================================
[2019-02-10 22:33:45,450] &lt;launcher&gt; INFO  system - UniFi 5.6.40 (build atag_5.6.40_10370 - release) is started
[2019-02-10 22:33:45,450] &lt;launcher&gt; INFO  system - ======================================================================
[2019-02-10 22:33:45,457] &lt;launcher&gt; INFO  system - BASE dir:/usr/lib/unifi
[2019-02-10 22:33:45,464] &lt;launcher&gt; INFO  system - Current System IP: 192.168.0.1
[2019-02-10 22:33:45,465] &lt;launcher&gt; INFO  system - Hostname: unifi-controller-5bb95c7688-bzp4z
[2019-02-10 22:33:48,635] &lt;launcher&gt; INFO  db     - waiting for db connection...
[2019-02-10 22:33:49,173] &lt;launcher&gt; INFO  db     - Connecting to mongodb://ubnt:PASSWORD@unifi-controller-mongodb:27017/unifi
[2019-02-10 22:33:49,526] &lt;launcher&gt; DEBUG db     - db connected (3.4.19@unifi-controller-mongodb:27017)
[2019-02-10 22:33:49,534] &lt;launcher&gt; INFO  db     - *** Factory Default *** Database exists. Drop it
[2019-02-10 22:33:52,391] &lt;launcher&gt; INFO  db     - waiting for db connection...
[2019-02-10 22:33:52,896] &lt;launcher&gt; DEBUG db     - db connected (3.4.19@unifi-controller-mongodb:27017)
[2019-02-10 22:34:13,292] &lt;launcher&gt; INFO  system - ======================================================================
[2019-02-10 22:34:13,295] &lt;launcher&gt; INFO  system - UniFi 5.6.40 (build atag_5.6.40_10370 - release) is started
[2019-02-10 22:34:13,295] &lt;launcher&gt; INFO  system - ======================================================================
[2019-02-10 22:34:13,303] &lt;launcher&gt; INFO  system - BASE dir:/usr/lib/unifi
[2019-02-10 22:34:13,312] &lt;launcher&gt; INFO  system - Current System IP: 192.168.0.1
[2019-02-10 22:34:13,313] &lt;launcher&gt; INFO  system - Hostname: unifi-controller-5bb95c7688-bzp4z
[2019-02-10 22:34:16,781] &lt;launcher&gt; INFO  db     - waiting for db connection...
[2019-02-10 22:34:17,300] &lt;launcher&gt; INFO  db     - Connecting to mongodb://ubnt:PASSWORD@unifi-controller-mongodb:27017/unifi
[2019-02-10 22:34:17,640] &lt;launcher&gt; DEBUG db     - db connected (3.4.19@unifi-controller-mongodb:27017)
[2019-02-10 22:34:17,656] &lt;launcher&gt; INFO  db     - *** Factory Default *** Database exists. Drop it
[2019-02-10 22:34:20,463] &lt;launcher&gt; INFO  db     - waiting for db connection...
[2019-02-10 22:34:20,969] &lt;launcher&gt; DEBUG db     - db connected (3.4.19@unifi-controller-mongodb:27017)
</code></pre>

<p>So I'm at a loss here. Not sure why it's trying to drop the database? Is Unifi trying to create the databases from scratch on the MongoDB instance?</p>

<ol>
<li><p>If I DON'T create the <code>unifi</code> and <code>unifi_stat</code> databases at provision time for MongoDB, then the Unifi Controller fails to ever connect to them and the logs stall there.</p></li>
<li><p>If I DO create the databases and give the <code>ubnt</code> user <code>dbAdmin</code> over them, then Unifi just appears to drop them over and over. Shown above.</p></li>
<li><p>If I DO create the databases and give the <code>ubnt</code> user <code>readWrite</code> over them, then Unifi just connects to the databases, creates the collections, and silently stalls for no apparent reason. And If I try and manually execute the <code>jar</code> file mentioned above then it leaves the stacktrace describing the lack of necessary permissions to drop a database.</p></li>
</ol>

<p>If someone could please provide some documentation on how an external MongoDB database should be prepared for the Unifi Controller to use it would be greatly beneficial for me. I've only been able to track down a forum post  whereby an employee discussed how to configure the controller's system.properties to point to an external instance. </p>

TJ Zimmerman
<p>Turns out this was a bug in MetalLB, the service with which I expose my bare metal kubernetes services to my network.</p>

<p><a href="https://github.com/google/metallb/issues/399" rel="nofollow noreferrer">https://github.com/google/metallb/issues/399</a></p>

<p>All is working now. But decentralizing MongoDB did little to resolve the problems introduced by multiple replicas, unfortunately. </p>

<p>However I still think something is amiss in the container as it reports that <code>service unifi status</code> is <code>unifi is not running</code>. Despite the fact that it is actually running.</p>
