FreeBSD Jail Networking

Networking FreeBSD Jails

FreeBSD Jails: Networking

containerization as a workshop, instead of a toolbox

Some assumptions

This article assumes an understanding of:

Basic networking

FreeBSD has a refined form of containerization called jails which, while simple to install, are not quite as simple to network. We’ll distill the core concepts of jail networking here to make them easier to understand for implementation. By default, you must assign a jail an ip address, which attaches to and shares the host’s physical network adapter. This is good 'nough for government work but for more advanced deployments, you’re going to need more advanced configurations (go figure). This is achieved by assigning a jail a vnet. This gives the jail control over a virtual networking device created on the host.

One of the more confusing concepts to wrap my head around was ifconfig and how it manages network devices. I’m not used to dealing with networking on a unix system in such an elegant way. Imagine using one utility to list and manage devices?? Wowie!! ifconfig is a great tool. network-manager and all the other dogshit that you can find on linux pales in comparisson to the simplicity of configuring your network with a FUCKING API. Not through bullshit config files, hidden in bullshit places which have changed endless times over the years leading to fragmented, confusing, and enraging experiences while trying to use the web to do networking things on linux. But with an actual cli/api. WOW. This is great.

A few things are implied here, which tripped me up. I’ll note them:

ifconfig vlan1000 create vlan 1000 vlandev em0 is the same thing as ifconfig em0.1000 create so save yourself the trouble and use the latter syntax.
vnets are attached to running jails, and do not appear when using ifconfig -a

Lets first list the tools at our disposal before we start playing around with them.

the bridge. Think of this as a sort of virtual switch. Devices on a bridge can talk to one another.
the virtual device. ifconfig em0.1000 create creates a virtual device connected to the physical interface em0 that tags packets with vlan 1000. This virtual device can be created/destroyed without impactinv the parent (physical) device.
the epair. This is, essentially, a virtual crossover cable. This plugs two devices into each other directly (virtually) allowing them to speak.

So with the tools listed above, we can begin to think about what’s going on here. When configuring a jail to use a vnet, we’re giving it access to the whole virtual device. We can’t give it em0, but every device described above is in fact a virtual network device passable to a jail. Giving one access to a vlan directly is fine, but blocks that vlan from being shared by other jails. So we must architect the virtual network stack on a freebsd host to allow for such things.

This is how I’ve achieved it.

Note: Something about the order in which you do these things matters. I’m not quite sure about this as I haven’t done digging. Anyway…

For this example, our vlan will be vlan 10 and our physical network device will be em0. We assume the network is laid out as such: Each vlan is assigned to a subnet of /16 on the ips in 10.0.0.0/8, with each vlan denoted by the second octet.

We assume a fresh configuration that has a single physical network device present. This device may have an ip assigned directly to it.

ifconfig em0.10 create up will create and attach a virtual device on em0, using vlan 10, and designated as “up” for communication.

ifconfig bridge0 create addm em0.10 inet 10.10.0.1/16 up creates a bridge device, assigns it an ip on vlan 10, and connects it to em0.10.

ifconfig epair10 create up creates an epair device which will be passed to the jail.

ifconfig bridge0 addm epair10a up will attach the epair device to the bridge. We do not assign A an ip address.

Finally, we may pass epair10b to the jail in /etc/jail.conf. Using ifconfig on the jail, we may assign the jail an ip address (say, 10.10.0.10/16).

This method allows us to create any arbitrary amount of jails attached to vlan10, so long as we create an epair for it and attach it appropriately.