title: How to include MongoDB client, common middleware and other 'bits and pieces' within separate-route, MVC-style Express.js application?
body: <p>I'm refactoring a big (~8,000 line) single file app into a 'separate-route', 'MVC-style' app.</p>
  <p>I want to apply 'best practises' to the folder/file structure and the contents of <code>app.js</code>.</p>
  <p>I'm trying to keep things simple, intuitive and minimal.</p>
  <p>I've referred to several video tutorials, articles, posts and github repo's and grabbed what seems to be the most sensible parts of each (see links at end of post).</p>
  <p>These learning resources have got me 90% of the way, but there are nuances in their examples and my implementation that don't quite match up.</p>
  <p>They also don't seem to cover how to handle these requirements:</p>
  <ol>
  <li>MongoDB client configuration</li>
  <li><a href="https://socket.io" rel="nofollow noreferrer">Socket.io</a> implementation</li>
  <li><code>app.use()</code> middleware used in all requests</li>
  <li><a href="https://jwt.io" rel="nofollow noreferrer">JWT</a> token verification</li>
  <li><a href="https://expressjs.com/en/advanced/developing-template-engines.html" rel="nofollow noreferrer">Simple template engine</a> implementation</li>
  </ol>
  <p>These are fairly common features, so I am hoping there are standard approaches to handling these requirements and also that this question will be valuable to others in the same boat.</p>
  <p>Below is what I have so far.</p>
  <p>Everything seems OK until I get to <code>app.js</code>.</p>
  <p><strong>structure</strong></p>
  <pre class="lang-js prettyprint-override"><code>app.js &lt;---- entry file  
  routes
  - index.js
  - resource_01_route.js
  - resource_02_route.js
  - resource_03_route.js
  // etc
  middleware
  - middleware_01.js
  - middleware_02.js
  - middleware_03.js
  // etc  
  views
  - index.ntl  
  config
  - db_connection.js
  - environment_variables.js
  - local_settings.js    
  </code></pre>
  <p><strong>routes/index.js</strong></p>
  <pre class="lang-js prettyprint-override"><code>const express = require('express');
  const router = express.Router();
  
  const resource_01_route = require('./resource_01_route');
  const resource_02_route = require('./resource_02_route');
  const resource_03_route = require('./resource_03_route');
  
  router.use('/api/:api_version/resource_01', resource_01_route);
  router.use('/api/:api_version/resource_02', resource_02_route);
  router.use('/api/:api_version/resource_03', resource_03_route);
  
  module.exports = router;
  </code></pre>
  <p><strong>routes/resource_01_route.js</strong></p>
  <pre class="lang-js prettyprint-override"><code>const express = require('express');
  const router = express.Router();
  
  const middleware_01 = require('../middleware/middleware_01');
  const middleware_02 = require('../middleware/middleware_02');
  const middleware_03 = require('../middleware/middleware_03');
  
  router
      .route('/')
      .get(middleware_01)
      .post(middleware_02)
      .delete(middleware_03);
  
  // etc  
  
  module.exports = router;
  </code></pre>
  <p><strong>middleware/middleware_01.js</strong></p>
  <pre class="lang-js prettyprint-override"><code>const { db_connection } = require('../config/db_connection');
  db_connection.open();
  
  const middleware_01 = async (req, res) =&gt; {
      try {
          const query = { username: 'my name' };
          const collection = db_connection.conn.db('users').collection('users');
          const result = await collection.findOne(query);
          res.json({ result: result });
      } catch (error) {
          console.log(error);
          res.status(500).json({ error });
      }
  };
  
  module.exports = middleware_01;
  </code></pre>
  <p><strong>config/db_connection.js</strong></p>
  <pre class="lang-js prettyprint-override"><code>const MongoClient = require('mongodb').MongoClient;  
  const { connection_string } = require('./environment_variables');
  
  class db_connection {
    static async open() {
      if (this.conn) return this.conn;
      this.conn = await MongoClient.connect(connection_string);
      return this.conn;
    }
  }
  
  db_connection.conn = null;
  db_connection.url = connection_string;
  
  module.exports = { db_connection };
  </code></pre>
  <p>The logic for the class above is from <a href="https://stackoverflow.com/a/70135909">here</a>.</p>
  <p><strong>config/environment_variables.js</strong></p>
  <pre class="lang-js prettyprint-override"><code>let online_status, 
      PORT, 
      var_01, 
      var_02, 
      var_03, 
      var_04; // etc
  
  if (process.env.MONGODB_USER) {
      online_status = 'online';
  } else {
      online_status = 'offline';
      local_settings = require('./local_settings');
  }
  
  if (online_status === 'online') {
      PORT = 8080;
      var_01 = process.env.VAR_01;
      var_02 = process.env.VAR_02;
      var_03 = process.env.VAR_03;
      var_04 = process.env.VAR_04;
  } else if (online_status === 'offline') {
      PORT = 3000;
      var_01 = local_settings.VAR_01;
      var_02 = local_settings.VAR_02;
      var_03 = local_settings.VAR_03;
      var_04 = local_settings.VAR_04;
  }
  
  module.exports = {
      online_status,
      PORT,
      var_01,
      var_02,
      var_03,
      var_04,
  };
  </code></pre>
  <p><strong>config/local_settings.js</strong>  <em>ignored by <code>.gitignore</code></em></p>
  <pre class="lang-js prettyprint-override"><code>module.exports = {
      VAR_01: 'VAR_01',
      VAR_02: 'VAR_02',
      VAR_03: 'VAR_03',
      VAR_04: 'VAR_04',
  };
  </code></pre>
  <p>My next step is to create a clean and minimal <strong>app.js</strong>, eg ideally something like:</p>
  <pre class="lang-js prettyprint-override"><code>const express = require('express');
  const app = express();
  const routes = require('./routes');
  
  app.use('/', routes); // &lt;---- call in the routes/index.js file   
    
  app.listen(PORT, () =&gt; {
      console.log(`the app is listening`); 
  });
  </code></pre>
  <p>This is where I am getting stuck.</p>
  <p>I have, for the most part, got the <code>route</code> and <code>middleware</code> management out of <code>app.js</code>.</p>
  <p>However, I still have a significant number of 'bits' left over in <code>app.js</code>.</p>
  <p>I don't know how to include them (either within <code>app.js</code> or somewhere else to keep <code>app.js</code> clean).</p>
  <p>This includes:</p>
  <p><s><strong>1)</strong> A reference to <code>mongo_client</code> that is used in multiple middleware functions</s><br />
  <em>solution found, as shown above</em></p>
  <p><strong>2)</strong>  <a href="https://socket.io" rel="nofollow noreferrer">socket.io</a> logic, ie:</p>
  <pre class="lang-js prettyprint-override"><code>io.on('connection', async (socket) =&gt; {
  
  /*
  
  define LOTS of socket.io logic
  
  needs access to mongo_client
  
  */
  
  });
  </code></pre>
  <p><strong>3)</strong>  <code>app.use()</code> middleware that is used on all requests, ie:</p>
  <pre class="lang-js prettyprint-override"><code>app.use(
    helmet({
      contentSecurityPolicy: {
        useDefaults: true,
        directives: {
          defaultSrc: [&quot;'self'&quot;],
          scriptSrc: [&quot;'self'&quot;, 'https://static.codepen.io'],
          styleSrc: [&quot;'self'&quot;, 'fonts.googleapis.com', &quot;'unsafe-inline'&quot;],
          fontSrc: [&quot;'self'&quot;, 'fonts.gstatic.com'],
          frameSrc: [&quot;'self'&quot;, 'https://codepen.io'],
        },
      },
    })
  );
  
  app.use(bodyParser.urlencoded({ extended: true }));
  app.use(bodyParser.json());
  app.use(express.static('dist'));
  </code></pre>
  <p><strong>4)</strong>  <a href="https://jwt.io" rel="nofollow noreferrer">JWT</a> verification logic, ie:</p>
  <pre class="lang-js prettyprint-override"><code>app.use((req, res, next) =&gt; {
  
  /*
  
  define all JWT verification logic  
  
  needs access to mongo_client
  
  */
  
  next();
  
  });
  </code></pre>
  <p><strong>5)</strong>  <a href="https://expressjs.com/en/advanced/developing-template-engines.html" rel="nofollow noreferrer">Simple template engine</a> definition and <code>res.render</code>, ie:</p>
  <pre class="lang-js prettyprint-override"><code>app.engine('ntl', (filePath, options, callback) =&gt; {
    fs.readFile(filePath, (err, content) =&gt; {
      if (err) return callback(err);
      var rendered = content
        .toString()
        .replace('#page_html#', options.page_html)
        .replace(/#site_title#/g, site_title)
        .replace(/#site_description#/g, site_description)
        .replace(/#og_image#/g, og_image)
        .replace(/#og_url#/g, og_url);
      return callback(null, rendered);
    });
  });
  
  app.set('views', './views');
  app.set('view engine', 'ntl');
  </code></pre>
  <p>For full context, below is the remaining <code>app.js</code> file I am trying to refactor, in all its glory (please ignore combo ES6/common module handling):</p>
  <pre class="lang-js prettyprint-override"><code>import express from 'express';
  import cors from 'cors';
  import bodyParser from 'body-parser';
  import bcrypt from 'bcrypt';
  import jwt from 'jsonwebtoken';  
  
  // needed for template engine
  const fs = require('fs');
  
  // needed for security
  const helmet = require('helmet');
  
  // needed for socket.io 
  const app = express();
  const http = require('http');
  const server = http.createServer(app);
  const io = require('socket.io')(server);
  app.set('socketio', io);
  
  // can't remember what this is for  
  const path = require('path');
  
  io.on('connection', async (socket) =&gt; {
  
  /*
  
  define LOTS of socket.io logic
  
  needs access to mongo_client
  
   */
  
  });
  
  app.use(
    helmet({
      contentSecurityPolicy: {
        useDefaults: true,
        directives: {
          defaultSrc: [&quot;'self'&quot;],
          scriptSrc: [&quot;'self'&quot;, 'https://static.codepen.io'],
          styleSrc: [&quot;'self'&quot;, 'fonts.googleapis.com', &quot;'unsafe-inline'&quot;],
          fontSrc: [&quot;'self'&quot;, 'fonts.gstatic.com'],
          frameSrc: [&quot;'self'&quot;, 'https://codepen.io'],
        },
      },
    })
  );
  
  
  app.use(bodyParser.urlencoded({ extended: true }));
  app.use(bodyParser.json());
  app.use(express.static('dist'));
  
  app.use((req, res, next) =&gt; {
  
  
  /*
  
  define all JWT verification logic  
  
  needs access to mongo_client
  
   */
  
  next();
  
  });
  
  // BEGIN simple templating engine
  // see:  https://expressjs.com/en/advanced/developing-template-engines.html
  
  app.engine('ntl', (filePath, options, callback) =&gt; {
    fs.readFile(filePath, (err, content) =&gt; {
      if (err) return callback(err);
      var rendered = content
        .toString()
        .replace('#page_html#', options.page_html)
        .replace(/#site_title#/g, site_title)
        .replace(/#site_description#/g, site_description)
        .replace(/#og_image#/g, og_image)
        .replace(/#og_url#/g, og_url);
      return callback(null, rendered);
    });
  });
  
  app.set('views', './views');
  app.set('view engine', 'ntl');
  
  // END simple templating engine
  
  server.listen(PORT, () =&gt; {
    console.log(`running on port ${PORT}`);
  });
  </code></pre>
  <p><strong>Research</strong></p>
  <p><a href="https://www.youtube.com/watch?v=v-Va0fm2R18" rel="nofollow noreferrer">Video: How to Add a Route to an Express Server in Node.js</a></p>
  <p><a href="https://www.youtube.com/watch?v=ozHl73XBROc" rel="nofollow noreferrer">Video: Project Folder Structure in Express</a></p>
  <p><a href="https://www.youtube.com/watch?v=TQU6BWtc3yc" rel="nofollow noreferrer">Video:  NodeJS REST: How to Refactor Code into Multiple Files with Router</a></p>
  <p><a href="https://caffeinecoding.com/better-express-routing-for-nodejs" rel="nofollow noreferrer">Article:  Better Express Routing &amp; Architecture for NodeJS</a></p>
  <p><a href="https://expressjs.com/en/guide/routing.html" rel="nofollow noreferrer">Article:  Express.js - Documentation - Routing</a></p>
  <p><a href="https://github.com/expressjs/express/tree/master/examples/route-separation" rel="nofollow noreferrer">Repo:  Express.js - GitHub Repo - Route Separation Example</a></p>
  <p><a href="https://expressjs.com/en/guide/writing-middleware.html" rel="nofollow noreferrer">Article:  Express.js - Documentation - Writing middleware for use in Express apps</a></p>
  <p><a href="https://expressjs.com/en/guide/using-middleware.html" rel="nofollow noreferrer">Article:  Express.js - Documentation - Using middleware</a></p>
  <p><a href="https://stackoverflow.com/questions/23923365/how-to-separate-routes-on-node-js-and-express-4">Question:  How to separate routes on Node.js and Express 4?</a></p>
  <p><a href="https://stackoverflow.com/questions/6059246/how-to-include-route-handlers-in-multiple-files-in-express">Question:  How to include route handlers in multiple files in Express?</a></p>
  <p><a href="https://stackoverflow.com/questions/47125420/express-jsmultiple-route-files-into-single-file">Question:  Express js:multiple route files into single file</a></p>