title: Collaborating on a new filebeat module
body: <p>I am building a new filebeat module for a custom application log and I wish to collaborate on it with a colleague of mine. I understood that a clone is a local copy for me only, without a chance for making my changes visible to my colleague without having them first merged to the <a href="https://github.com/elastic/beats/" rel="nofollow noreferrer">https://github.com/elastic/beats/</a> repo, so I opted to fork the beats repository instead of cloning it and then cloned that fork.</p>
  
  <p>I set up my dev environment by following the <a href="https://www.elastic.co/guide/en/beats/devguide/current/beats-contributing.html#setting-up-dev-environment" rel="nofollow noreferrer">instructions</a>. Running <code>make</code> under filebeat gives no output so I assume everything is a-ok.</p>
  
  <p>I generate the module, fileset, and set up my ingest pipeline for the fileset. My <code>pipeline.json</code> looks like this:</p>
  
  <pre><code>{
    "description": "Pipeline for parsing CA Service Desk Manager stdlogs",
    "processors": [
      {
          "grok": {
              "field": "message",
              "patterns": [
                  "%{TIMESTAMP:casdm.stdlog.timestamp} %{HOSTNAME:casdm.stdlog.hostname} %{PROCESS:casdm.stdlog.process.name} %{PID:casdm.stdlog.process.id} %{LOGLEVEL:casdm.stdlog.level} %{FILENAME:casdm.stdlog.file.name} %{POSINT:casd.stdlog.file.line} %{DATA:casdm.stdlog.message}"
              ],
              "pattern_definitions": {
                  "TIMESTAMP": "%{MONTHNUM2}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}",
                  "PROCESS": "%{USERNAME}",
                  "PID": "%{POSINT}",
                  "LOGLEVEL": "(FATAL|EXIT|SIGNIFICANT|SEVERE_ERROR|ERROR|WARNING|INFORMATION|MILESTONE|TRACE|VERBOSE)",
                  "FILENAME": "(?:[A-Za-z0-9_. -]+)"
              }
          }
      }
    ],
    "on_failure" : [{
      "set" : {
        "field" : "error.message",
        "value" : "{{ _ingest.on_failure_message }}"
      }
    }]
  }
  </code></pre>
  
  <p>It's not all the way there but being new to filebeat module creation I am excluding all the special cases of the log format for now.</p>
  
  <p>I am then <a href="https://www.elastic.co/guide/en/beats/devguide/current/filebeat-modules-devguide.html#_meta_fields_yml_2" rel="nofollow noreferrer">instructed to generate the fields based on the pipeline configuration</a>. Everything looks good:</p>
  
  <pre><code>~/go/src/github.com/jvalkonen/beats/filebeat $ make create-fields MODULE=casdm FILESET=stdlog
  Fields.yml generated for casdm/stdlog
  </code></pre>
  
  <p>However, no <code>fields.yml</code> (regardless of the case) is found under the <code>module/casdm/stdlog/_meta/</code> and the module level <code>module/casdm/_meta/fields.yml</code> doesn't contain any of my fields set in the <code>module/casdm/stdlog/ingest/pipeline.json</code>. This already indicates there is something wrong, but I don't know what as the output indicates all is well. If I then run the <code>make update</code> to generate documentation and configuration I get an error, which doesn't seem to indicate issues with the <code>pipeline.json</code> but the development setup itself and possibly the fact that I forked the project instead of just cloning it:</p>
  
  <pre><code>~/go/src/github.com/jvalkonen/beats/filebeat $ make update
  mage update
  Error: failed to find github.com/elastic/beats/dev-tools/mage in the project's vendor
  failed to find github.com/elastic/beats/dev-tools/mage in the project's vendor
  make: *** [update] Error 1
  </code></pre>
  
  <p>Something along the build pipeline is referring to the elastic github path instead of my forked repo. I could go search and replace all of those references but am I trying to do this completely backwards and is there an easier way to achieve what I'm trying to do?</p>
  
  <p>My questions are:</p>
  
  <ol>
  <li>On a general level, can one collaborate on a cloned repo or does it need to be forked first?</li>
  <li>Being rather new to git and GitHub, is there an easier way of collaborating on a GitHub repo without becoming a contributor or messing up the source repo?</li>
  <li>Are my assumptions about the source of my make errors even close? If yes, what's the proper solution to it?</li>
  </ol>