0x0035 - lxc / lxd VM walkthrough

A complete alternative to docker.io is lxc.

To install:

sudo apt-get install lxc

And from there you will have a list of possible commands:

lxc is more 'tool like' than standard docker, and you can see how it installed with lxc-checkconfig

With a detailed information about the lxc containerization service:

Due to lxc unprivileged nature - most containers will not work due to the mapping of sub ids to uids.  Effectively:

Create a ~/.config/lxc/default.conf and load the following:

NOTE: YOU HAVE TO DO THIS. LXC DOES NOT WORK WITHOUT A PERSONAL SETUP.

lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx
lxc.idmap = u 0 100000 65536  # You MUST have these commands for $USER access
lxc.idmap = g 0 100000 65536  # You MUST have these commands for $USER access

And then execute:

echo "$(id -un) veth lxcbr0 10" | sudo tee -a /etc/lxc/lxc-usernet

If you are working from Ubuntu:

export DOWNLOAD_KEYSERVER="hkp://keyserver.ubuntu.com"

Additionally you may require squashfs support:

sudo apt install linux-image-generic-hwe-20.04 linux-headers-generic-hwe-20.04
sudo reboot

We now must set lxc to recognize the root disk:

lxc profile device add default root disk path=/ pool=default

Now we need to install the lxd manager:

sudo apt install snap -y 
sudo snap install lxd

And then we initialize the lxc setup using the lxd manager:

lxd init

Inspecting the first running container shows it is network isolated, and it is recommended to install lxd:

sudo snap install lxd

After we have installed lxd we then can call the initializer which will make your storage backend etc:

Once we are done this we can follow the guide to start one.  Oddly they are calling from a systemd call:

sudo lxc-create -t download -n bob -- --dist ubuntu --release focal --arch amd64 --no-validate

The container automatically is stopped on creation and there seems to be some issues in making / keeping it running. Attempting to attach to it:

Broken out of the gate:

As the guide covers the one linux that is inherently and properly setup is Ubuntu.  Since we are on backbox, we were finally able to get a container to run - as root.

sudo lxc-create -t download -n privilege

If you have the type download it will give you a option to download a bunch of pre-built image types.

sudo lxc-start -n privilege
sudo lxc-info -n privilege
sudo lxc-attach <vm>

Networking:

lxc launch ubuntu:22.04 u1

Now it is much more managable:

lxc storage list

As we learn more effectively lxc by and of itself is pretty limited and to use it you need its lxd companion:

Whoa!  Here is where lxd/lxc just left docker in the dust - X11..

This was run out of the gate from inside the running instance:

sudo apt install x11-apps --fix-missing

And then simply ssh -X into the box:

xclock

Try setting up a X11 to a docker and it has been my experience you will be fumbling around for days configuring ports etc etc etc.  This just works.  Also it seems to properly setup a bridge interface for all your VM's that have proper port/ip bindings.  QEMU was found to be quite problematic in this regard (requiring lots of network plumbing)  but was treated like 'whole VM's' with their own desktop etc.  Very impressed.

So the next question becomes can one have a lxd/lxc load a .iso image on boot to install an entire VM machine like a normal virtual machine?

Looking around we find an awesome guide - so simple one would have to try it:

1. Create a VM.

2. Give it more disk space:

3. Create a virtual CDROM and add it to the 'u1' instance:

IMPORTANT! You must now disable autoboot - simply if the image is not correctly configured it will boot automatically into a disabled format (aka bad virtual bios setup)

lxc config set <vm> boot.autostart=false

4. Start the image (wow it worked!)

lxc start u1 --console=vga

It's not perfect but it tried:

What was missing? We needed to turn off 'secure-boot'

Nope that did not work either. But it is like the 'cdrom' is not being seen properly.

Finally we tried another .iso different from backbox - dragonOS

Solved! Thanks to here  As the machine boots treat it like a normal computer with it's own bios, hit escape inside the boot screen quickly you will get a 'virtual bios'!

The moment we changed the boot manager presto - we were up!

Summary - lxc/lxd wins hands down.

• QEMU was fantastic - minor one limitation, it never seemed to map a proper external IP to your VM without a pile of plumbing.  It was very mangly to ssh into a running VM.
• Docker just seemed to have no examples that worked properly with X11 I welcome anyone to advise otherwise.
• lxd/lxc answered both of these problems - allowing for full desktop VM's while allowing for proper network plumbing to reach them!
• NONE of this is possible in Windows.
• I used to utilize QEMU exclusively for local VM's but because of the proper network mapping of devices under the lxd/lxc I have now switched. It works.