{"action":"opened","issue":{"url":"https://api.github.com/repos/elastic/beats/issues/657","labels_url":"https://api.github.com/repos/elastic/beats/issues/657/labels{/name}","comments_url":"https://api.github.com/repos/elastic/beats/issues/657/comments","events_url":"https://api.github.com/repos/elastic/beats/issues/657/events","html_url":"https://github.com/elastic/beats/issues/657","id":125367370,"number":657,"title":"Configuration handling","user":{"login":"urso","id":75886,"avatar_url":"https://avatars.githubusercontent.com/u/75886?v=3","gravatar_id":"","url":"https://api.github.com/users/urso","html_url":"https://github.com/urso","followers_url":"https://api.github.com/users/urso/followers","following_url":"https://api.github.com/users/urso/following{/other_user}","gists_url":"https://api.github.com/users/urso/gists{/gist_id}","starred_url":"https://api.github.com/users/urso/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/urso/subscriptions","organizations_url":"https://api.github.com/users/urso/orgs","repos_url":"https://api.github.com/users/urso/repos","events_url":"https://api.github.com/users/urso/events{/privacy}","received_events_url":"https://api.github.com/users/urso/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/beats/labels/discuss","name":"discuss","color":"ffa500"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-01-07T10:32:17Z","updated_at":"2016-01-07T10:32:17Z","closed_at":null,"body":"I think it makes sense to parse configs into an intermediate representation (independent of file format or source), which can be used populate the structs.\r\n\r\nMain advantage will be support for per module decoupled configuration definitions and handling. E.g. for plugin-like modules we've got in packetbeat and metricsbeat.\r\n\r\n    type Config {\r\n        ...\r\n    }\r\n    \r\n    var (\r\n        ErrMissing = errors.New(\"field name missing\")\r\n        ErrTypeMismatch = errors.New(\"...\")\r\n    )\r\n    \r\n    func NewConfigYAML(in io.Reader) (*Config, error) { ... }\r\n    func NewConfigYAMLFile(path string) (*Config, error) { ... }\r\n    \r\n    func (c *Config) GetFields() []string { ... }\r\n    func (c *Config) HasField(name string) (bool) { ... }\r\n    \r\n    // ******************************************************************************\r\n    // Low level getters and setters (do we actually need this?)\r\n    // ******************************************************************************\r\n    \r\n    // number of elements for this field. If config value is a list, returns number\r\n    // of elements in list\r\n    func (c *Config) CountField(name string) (int, error) { ... }\r\n    \r\n    // Config getters. If config value is actually a list, idx can be used to access\r\n    // values at said index\r\n    func (c *Config) Bool(name string, idx int) (bool, error) { ... }\r\n    func (c *Config) Str(name string, idx int) (string, error) { ... }\r\n    func (c *Config) Int(name string, idx int) (int, error) { ... }\r\n    func (c *Confog) Child(name string, idx int) (*Config, error) { ... }\r\n    \r\n    func (c *Config) SetBool(name string, idx int, value bool) { ... }\r\n    func (c *Config) SetStr(name string, idx int, value string) { ... }\r\n    func (c *Config) SetInt(name string, idx int, value int) { ... }\r\n    func (c *Config) SetChild(name string, idx int, value int) { ... }\r\n    \r\n    // ******************************************************************************\r\n    // High level interface\r\n    // ******************************************************************************\r\n    \r\n    type reifyOptions func(opts options) options\r\n    \r\n    // check if config has options not known to type to be filled\r\n    func CheckUnknown(opts options) options\r\n    \r\n    // all fields given in input type must be present in config file\r\n    func CheckRequired(opts options) options\r\n    \r\n    // (optional)\r\n    // func (c *Config) Validate(t reflect.Type, opts ..reifyOptions)\r\n    \r\n    // unpack config into 'to'. Default values will already be set in 'to'.\r\n    func (c *Config) Materialize(to interface{}, opts ...reifyOptions)\r\n    \r\n    // add and overwrite fields in config by input (e.g. set defaults)\r\n    func (c *Config) Incorporate(input interface{})\r\n\r\nHaving an API like this, we can easily re-interpret a configuration:\r\n\r\n    config, _ := NewConfigYAMLFile(\"config.yml\")\r\n\r\n    var genericConfig struct {\r\n        Period int `config:\"period\"`\r\n    }\r\n    err := config.Materialize(&genericConfig, CheckRequired)\r\n\r\n    ...\r\n\r\n    var myModuleConfig struct {\r\n        Hostnames []string `config:\"hosts\"`\r\n        Port int `config:\"port\"`\r\n    } {\r\n        Port: 12345, // set some defaults\r\n    }\r\n    err := config.Materialize(&myModuleConfig)\r\n    ...\r\n\r\nThe struct-tags to be supported:\r\n\r\n    -  : ignore field in config\r\n    omitempty: ignore if not present in config\r\n    inline: include fields from inner struct in outer struct\r\n "}}