{"action":"created","issue":{"url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/696","labels_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/696/labels{/name}","comments_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/696/comments","events_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/696/events","html_url":"https://github.com/SnabbCo/snabbswitch/issues/696","id":124707109,"number":696,"title":"Unified configuration tree with YANG models","user":{"login":"lukego","id":13791,"avatar_url":"https://avatars.githubusercontent.com/u/13791?v=3","gravatar_id":"","url":"https://api.github.com/users/lukego","html_url":"https://github.com/lukego","followers_url":"https://api.github.com/users/lukego/followers","following_url":"https://api.github.com/users/lukego/following{/other_user}","gists_url":"https://api.github.com/users/lukego/gists{/gist_id}","starred_url":"https://api.github.com/users/lukego/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lukego/subscriptions","organizations_url":"https://api.github.com/users/lukego/orgs","repos_url":"https://api.github.com/users/lukego/repos","events_url":"https://api.github.com/users/lukego/events{/privacy}","received_events_url":"https://api.github.com/users/lukego/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/SnabbCo/snabbswitch/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-01-04T06:59:00Z","updated_at":"2016-01-14T12:43:05Z","closed_at":null,"body":"Snabb Switch should unify its configuration parameters and monitoring counters into a tree structure that can be modelled in YANG [RFC 6020](https://tools.ietf.org/html/rfc6020).\r\n\r\nThis would cover the counters and parameters for the engine, the app network layout, the individual apps, and application-level abstractions like [netmod](https://datatracker.ietf.org/wg/netmod/documents/) models that are being directly implemented by Snabb applications.\r\n\r\nThis will be key to making Snabb applications easy to configure and manage, especially in large networks that require automation and standardized interfaces.\r\n\r\nThere are some aspects to work out:\r\n\r\n1. Create a configuration tree API.\r\n2. Move all existing configuration/statistics objects into the configuration tree.\r\n3. Create YANG models as formal reference documentation.\r\n\r\nSo, does anybody have a bright idea about how to go about this? :-)\r\n\r\nQuick braindump: I can imagine that our `core.config` model could be updated to be hierarchical and to become comprehensive (e.g. include engine parameters). I think we could either write YANG models by hand (and parse them for validating configurations) or we could generate them from Lua code (ensuring it is isomorphic). We have to think about how to make this work well in a multiprocess/multiserver setup.\r\n\r\nIn principle we could push the YANG models away from the apps and treat them as external \"northbound\" abstractions, but to me it seems like we would benefit from introducing these abstractions pervasively as an upgrade from only informal documentation in READMEs.\r\n\r\ncc @plajjan @sleinen @alexandergall @andywingo @mwiget @everybody :)"},"comment":{"url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/comments/171634370","html_url":"https://github.com/SnabbCo/snabbswitch/issues/696#issuecomment-171634370","issue_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/696","id":171634370,"user":{"login":"andywingo","id":1745903,"avatar_url":"https://avatars.githubusercontent.com/u/1745903?v=3","gravatar_id":"","url":"https://api.github.com/users/andywingo","html_url":"https://github.com/andywingo","followers_url":"https://api.github.com/users/andywingo/followers","following_url":"https://api.github.com/users/andywingo/following{/other_user}","gists_url":"https://api.github.com/users/andywingo/gists{/gist_id}","starred_url":"https://api.github.com/users/andywingo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andywingo/subscriptions","organizations_url":"https://api.github.com/users/andywingo/orgs","repos_url":"https://api.github.com/users/andywingo/repos","events_url":"https://api.github.com/users/andywingo/events{/privacy}","received_events_url":"https://api.github.com/users/andywingo/received_events","type":"User","site_admin":false},"created_at":"2016-01-14T12:43:05Z","updated_at":"2016-01-14T12:43:05Z","body":"It seems that having external Yang modules written in Yang syntax with inline documentation would make operators the happiest in terms of saying \"this is what this network appliance does\" and \"this is how I configure this device\" and \"this is what information I can get out of it\".  Of course it helps Snabb people answer these questions too :)\r\n\r\nYang is fine as far as describing the data model goes, but from what I can tell the standard way to do things is to use XML for describing instances of Yang data, which is not very Snabby.  I would be happy if we could avoid XML in Snabb.  On the other hand there is a JSON representation these days, https://tools.ietf.org/html/draft-ietf-netmod-yang-json-06 / https://github.com/mbj4668/pyang/wiki/XmlJson; perhaps that would be not so terrible.  I don't know.  Most JSON tools can't do 64-bit numbers though, so perhaps that's not so useful, though given that a JSON document would have a Yang schema, representing numbers as strings is a valid solution.\r\n\r\nI must admit I much prefer a language that looks like Yang over some generic JSON thing, much less XML.  The Yang language itself is semantically rich but easy to parse, which is the exact opposite of XML.  (I say this as someone who maintains a full XML parser.)\r\n\r\nSomeone will eventually want to hook up Snabb+Yang with Netconf, and there of course we'd need some pretty good XML capabilities.  I think that's not a near-term thing though, so I leave it a bit unspecified.  A first crack at Snabb and Yang would not do XML.\r\n\r\nI think you'd want a Lua interface to look something like this:\r\n\r\n```lua\r\nlocal yang = require('yang')\r\nlocal schema = yang.load_module('acme-system')\r\n-- Throw an error if the config is invalid.\r\nlocal program_config = schema:load_config[[\r\n  foo bar;\r\n  baz {\r\n    qux 42;\r\n  }\r\n]]\r\n-- Use the value of the foo leaf\r\nprint(program_config.nodes.foo.value)\r\nprint(program_config.nodes.baz.qux.value)\r\nprogram_config.nodes.baz.qux.value = 10\r\n```\r\n\r\nThis is assuming some kind of Yang-like syntax for configuration instances.  Whatever.  I don't know.  I'm pretty sure you will want the config to be available as a normal Lua data structure, nicely nested, but I don't know how the API could be -- you want to be able to nicely reference child nodes and value but also query things like the node's schema.  Not sure.  I think I would use some kind of \"active record\" pattern where writing to a value will validate that value as well.  Of course there are validation steps to take that need the whole tree, so we'd need an additional step there.\r\n\r\nThe yang modules would be somehow compiled into the Snabb binary.  They could be compiled in as source files, or some preprocessor could run on them so they validate at build-time.  The output of the preprocessor would be Lua code that builds a schema using the Lua API of the Yang module.\r\n\r\nThat's fine enough as far as configuration goes, but you also want data as well.  Here it gets a bit more irritating.  The solution we have right now with counters is nice.  I think ideally we'd find a way to write both configuration and state into the trees in the SHM file system and then use a Yang model to interpret those trees.  That way you can still monitor a Snabb process from the outside.  It's attractive too from an eventual Netconf perspective too -- a candidate configuration can be an additional tree.  We can actually store all configurations in the same way NixOS does to enable rollback to any configuration state in the past.\r\n\r\nThere's more work to spec out here: we'd need a standard yang data model for many parts of Snabb (links, counters, etc, even app networks).  I don't fully understand reconfiguration.  And specific programs like the NFV or the lwAFTR will have their own Yang modules too.  So, lots to do here.  Yang is pretty big!"}}