LXD Container Home Server Networking For Dummies
If you're going to operate a fleet of LXD containers for home entertainment, you probably want some of them exposed with their own ip addresses on your home network, so that you can use them
LXD Container Home Server Networking For Dummies
If you're going to operate a fleet of LXD containers for home entertainment, you probably want some of them exposed with their own ip addresses on your home network, so that you can use them as containerized servers for various applications.
Others containers, you might want to be inaccessable from the lan, in a natted subnet, where they can solicit connections to the outside world from within their natted subnet, but are not addressable from the outside. A database server that you connect a web app to, for instance, or a web app that you have a reverse proxy in front of.
But these are two separate address spaces, so ideally all of the containers would have a second interface of their own, by which they could connect to a third network, that would be a private network that all of the containers can use to talk directly to each other (or the host machine).
It's pretty straightforward, you just have to glue all the pieces together.
Three Part Overview.
Define and create some bridges.
Define profiles that combine the network interfaces in different combinations. In addition to two bridges you will have a macvlan with which to expose the containers that you want exposed, but the macvlan doesn't come into play until here in step two when you define profiles.
Assign each container which profile it should use, and then configure the containers to use the included network interfaces correctly.
Build Sum Moar Bridges
The containers will all have two network interfaces from their own internal point of view, eth0 and eth1.
In this scheme we create a bridge for a natted subnet and a bridge for a non-natted subnet. All of the containers will connect to the non-natted subnet on their second interface, eth1, and some of the containers will connect to the natted subnet on their first interface eth0. The containers that don't connect to the natted subnet will instead connect to a macvlan on their first interface eth0, but that isn't part of this step.
bridge for a natted subnet
If you haven't used lxd before, you'll want to run the command
By default this creates exactly the bridge we want, called lxdbr0.
Otherwise you would use the following command to create lxdbr0.
lxc network create lxdbr0
To generate a table of all the existing interfaces.
lxd network list
This bridge is for our natted subnet, so we just want to go with the default configuration.
lxc network show lxdbr0
This cats a yaml file where you can see the randomly generated network for lxdbr0.
config: ipv4.address: 10.99.153.1/24 ipv4.nat: "true" ipv6.address: fd42:211e:e008:954b::1/64 ipv6.nat: "true" description: "" name: lxdbr0 type: bridge used_by:  managed: true
bridge for a non-natted subnet
lxc network create lxdbr1
Use the following commands to remove nat from lxdbr1.
lxc network set lxdbr1 ipv4.nat false lxc network set lxdbr1 ipv6.nat false
Of if you use this next command, your favourite text editor will pop open, preloaded with the complete yaml file and you can edit the configuration there.
lxc network edit lxdbr1
Either way you're looking for a result such as the following. Notice that the randomly generated address space is different that the one for lxdbr0, and that the *nat keys are set to "false".
config: ipv4.address: 10.151.18.1/24 ipv4.nat: "false" ipv6.address: fd42:89d4:f465:1b20::1/64 ipv6.nat: "false" description: "" name: lxdbr1 type: bridge used_by:  managed: true
recycle the default
When you first ran
lxd init, that created a default profile.
Confirm with the following.
lxc profile list
To see what the default profile looks like.
lxc profile show default
config: environment.http_proxy: "" security.privileged: "true" user.network_mode: "" description: Default LXD profile devices: eth0: nictype: bridged parent: lxdbr0 type: nic root: path: / pool: default type: disk name: default used_by: 
profile the natted
The easiest way to create a new profile is start by copying another one.
lxc profile copy default natted
edit the new natted profile
lxc profile edit natted
And add an eth1 interface attached to lxdbr1. eth0 and eth1 will be the interfaces visible from the container's point of view.
config: environment.http_proxy: "" security.privileged: "true" user.network_mode: "" description: Natted LXD profile devices: eth0: nictype: bridged parent: lxdbr0 type: nic eth1: nictype: bridged parent: lxdbr1 type: nic root: path: / pool: default type: disk name: natted used_by: 
Any container assigned to the natted profile, will have an interface eth0 connected to a natted subnet, and a second interface eth1 connected to a non-natted subnet, with a static ip on which it will be able to talk directly to the other containers and the host machine.
profile the exposed
Create the exposed profile
lxc profile