Last week, I had to configure a new KVM hypervisor running on a Red Hat Enterprise Linux 8.5 server that would host VMs from multiple VLANs/subnets. This type of configuration is pretty easy to do in VMWare ESX, having a virtual switch trunked to the physical world. But in RHEL-based distributions, if you want to have full control over what you’re doing, that’s a bit more complicated.
I went over a couple references, either directly from Red Hat, stackoverflow and others, but no setup was exactly like the one I was doing. So I improvised a bit. The following schema represents the setup I had to implement:
Essentially:
- The hypervisor is a Dell PowerEdge R750 server with an RJ-45 iDRAC interface and a 2 ports SFP+ network card
- The iDRAC and hypervisor management address are in the same network (192.168.10.0/24);
- A pfSense VM will have an interface in all subnets and will route traffic between them;
- The Cisco Catalyst 2960XR trunks all VLANs (for use on other interfaces to link to the physical world)
In Linux, well on RHEL / CentOS, the best way to have full control over your trunked network connections is through config files.
The network config files can be found in
/etc/sysconfig/network-scripts/
Physical interface
The following file is that of the physical interface on which you will trunk your VLANs, in our case here eno12409np1. The file is named ifcfg-eno12409np1 and should already exist in /etc/sysconfig/network-scripts:
TYPE=Ethernet
NAME=eno12409np1
UUID=58cf950b-21ab[…]
DEVICE=eno12409np1
ONBOOT=yes
NM_CONTROLLED=yes
MTU=”9000″
VLANs
Next, we have to create the config files that will instruct to tag VLANs on the physical interface. You create them manually in /etc/sysconfig/network-scripts and are named as follows
- ifcfg-eno12409np1.10
- ifcfg-eno12409np1.20
- ifcfg-eno12409np1.30
- ifcfg-eno12409np1.40
The simple fact to have .10 at the end is an indicator to the OS that the VLAN ID will be 10. The content of each file is as follows (example is given with ifcfg-eno12409np1.10):
NAME=eno12409np1.10
DEVICE=eno12409np1.10
ONPARENT=yes
BOOTPROTO=none
VLAN=yes
BRIDGE=virbr10 (this refers to the bridge config files, see next section)
NM_CONTROLLED=yes
MTU=”9000″
Bridges
Then come the bridges that will be used to link the KVM VMs to the VLANs we just defined. Bridges in Linux / KVM are just like virtual switches inside your hypervisor. You can link multiple VMs to a bridge on the same subnet, and they can communicate just as if they were around a workgroup switch on the same VLAN.
The bridge that you want to link your hypervisor to has to have an IP configuration on it, and the others should not have an IP configuration (unless you really know what you’re doing…).
You have to create the following config file for the bridge where the hypervisor has its IP address. The file name is ifcfg-virbr10:
STP=yes
BRIDGING_OPTS=priority=32768
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.10.2
PREFIX=24
GATEWAY=192.168.10.1
DNS1=192.168.10.10 (type your own DNS servers here…)
DNS2=192.168.10.11
DOMAIN=local.local (Type your own domain name here)
DEFROUTE=yes
NAME=virbr10
DEVICE=virbr10
ONBOOT=yes
MTU=”9000″
NM_CONTROLLED=yes
Finally you have to create the config files for the bridges where the hypervisor has no IP address. In this example, the file name is ifcfg-virbr20:
STP=yes
BRIDGING_OPTS=priority=32768
TYPE=Bridge
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
NAME=virbr20
DEVICE=virbr20
ONBOOT=yes
MTU=”9000″
NM_CONTROLLED=yes
Apply configuration
Once you’re done, you should reboot your physical machine to apply the configuration. I’ve seen many tutorials where they use
# systemctl restart network
but I’ve never been able to use it. The only thing that does the trick in RHEL is a server reboot.
Test this configuration
The fastest way to test this configuration is to ping the iDRAC interface from the hypervisor. In your setup, it may not be possible, but you’ll have to improvise according to your environment. In my case, the first test I did was this:
$ ping 192.168.10.3
64 bytes from 192.168.10.3: icmp_seq=1 ttl=64 time=0.452ms
64 bytes from 192.168.10.3: icmp_seq=1 ttl=64 time=0.498ms
64 bytes from 192.168.10.3: icmp_seq=1 ttl=64 time=0.521ms
This means success!
Add the networks to KVM
If you need to add these bridges to KVM for use with your virtual machines, you have to define them with XML files and add them using virsh.
First, create a file in your home directory using the vi text editor, with the following content:
# vi virbr10.xml
<network>
<forward mode=”bridge”/>
<bridge name=”virbr1″ />
</network>
Then define the network using the virsh command:
# virsh net-define /tmp/virbr10.xml
Repeat for every bridge you need to define in your KVM hypervisor.
Once done, you’re god to go!