Tap Interfaces and Linux Bridge

In the previous blog of this series we saw that using Linux bridge we can connect a virtual Ethernet port of a VM to the physical Ethernet port of the hypervisor server. Let us now focus a bit more on these virtual ports to see what happens behind the scenes to make virtual networking actually work.

Physical vs. Virtual Networking

The network data traffic is handled by the physical Ethernet ports on physical machines. Similarly for virtual machines this traffic needs to be handled by the virtual ethernet ports. Eventually this traffic from virtual ports needs to be sent to the physical network for external connectivity. How does this happen? As seen in the image below the following components are key for physical networking:

  1. Ethernet Port on the server – commonly called pNIC (physical NIC)
  2. RJ45 Cable
  3. Ethernet Port on the physical switch
  4. Uplink Port on the physical switch – connects to external network.

Physical Network Components

Since the goal of virtualization is to emulate physical entities in software, it must support a construct where “a virtual machine’s virtual Ethernet port is connected to a virtual switch”.

Switch Ports

As mentioned in my previous post, Linux bridge is really a switch implemented inside Linux kernel. And like any switch, it requires ports or interfaces to carry data traffic in and out of the switch. We have also seen how to add a physical interface to the bridge. Linux bridge also supports virtual ports. Since virtual ports are actually software entities,  other software entities can use them to send Ethernet frames to the virtual switch for further processing. For Ethernet traffic, these Linux virtual ports are called tap interfaces. Tap interfaces can be added to a Linux bridge just like physical interfaces. With this approach a Linux bridge can forward packets from virtualized world to a physical world (and vice-versa).

Virtual Network Components with tap interfaces

As seen in the image below, when virtualization comes into the picture, the following components play a key role in networking:

  1. Ethernet Port on VM (emulates the pNIC) – commonly knows as vNIC (Virtual NIC). Virtual port is emulated with help from KVM/QEMU.
  2. Virtual RJ45 Cable – we will see how this is created.
  3. Ethernet Port on Virtual Switch – for Linux Bridge, this is represented by the tap interface since it connects to a vNIC.
  4. Uplink Port on the Virtual Switch – this is usually the interface that represent the pNIC. In Linux world eth0, eth1 etc are the software interfaces that represent the physical ethernet port.
  5. In case of a physical server, the Uplink port on physical switch provided external network connectivity. In case of virtual machines, the Uplink port on virtual switch provides external network connectivity.

Tap interfaces – Why do we need them?

Like their physical counterparts, virtual machines network ports can only process Ethernet frames. In non-virtualized environments, the physical NIC interface will receive and process the Ethernet frames. It will strip out the Ethernet related overhead bytes and forward the payload (usually IP packets) further up to the OS. With virtualization however, this will not work since the virtual NICs would expect Ethernet frames. This is where tap interfaces come into picture. Tap interfaces are special software entities which tell the Linux bridge to forward Ethernet frames as it is. In other words, the virtual machines connected to tap interfaces will be able to receive raw Ethernet frames. And due to this virtual machines can continue to emulate physical machines from a networking perspective.

Is there a virtual RJ45 cable as well?

The short answer is no. But there is a need for connecting a virtual Ethernet port of a VM to the tap interface on a Linux bridge. This connection is achieved programmatically. Applications such as libvirt create a ‘file descriptor’ using the tap interface. When Linux bridge sends Ethernet frames to a tap interface, it actually is sending the bytes to a file descriptor. Emulators like QEMU, read the bytes from this file descriptor and pass it onto the ‘guest operating system’ inside the VM, via the virtual network port on the VM. Note: Tap interfaces are listed as part of the ifconfig Linux command.

Further Reading – Tun/Tap interface tutorial

  • kthomas

    I’m a newbie but am very interested in a command or technique to correlate the VM’s interface with the Tap (vnet) and bridge to which it connects. I have found ways to list the VM name and a list of vnets names to which it connects (at the host), but no way to identify the VM’s interface name that is associated with each tap connection.

    • Sorry for the late reply. Yes, it does not look that there is a direct way to identify VM’s virtual NIC to the Tap on the bridge.

      But there is a way to find which bridge the VM connects to. You know the mac address of the VM’s virtual NIC. You can then use brctl showmacs one by one on each bridge and in the response search for your VM’s MAC.

  • Madhu

    Nice post, Sriram!

    Please continue to edify us with posts on Linux, networking and virtualization.

    Thanks!

  • Junwei Nie

    An excellent post which helps me a lot, Sriram!

    There is one thing i am not quite understand. Why there is a MAC address for tap interface which is different from the MAC address on VM?

  • Mallesham Jatharakonda

    Thank you for posting this. Your blog is informative.

  • Nav

    I have seen VMs have vnet interface in ifconfig and also some VMs will have tap interface listed in ifconfig. Is there any different between vtap and vnet interfaces ?

  • Pingback: ip vs. ifconfig - WILT - Blogs by Sriram()

  • Pingback: Create a virtual network with QEMU/KVM | werewblog()

  • Pingback: Getting started with OpenVSwitch | Tech and me.()

  • Pingback: Notes on Switching – Sheridan ICT-SDN Project()

  • alex

    if libvirt connects VM to tap devices then what does Linux veth cable do?

  • rajasekar bonthala

    Very clear to understand the Virtualization basics. Awesome description. By the way, do you have any plans to introduce concepts on inter-vm communication and intra-vm communication using non-network methods like shared memory etc….
    Thanks for your contribution.

  • Kelly Clarke

    Thank you!!! Great post. Please continue with your very informative posts. I found your posts while researching concepts for networking virtual containers and bridging. I found these posts to be a good read without getting bogged down in too much detail. This is exactly what I needed to get started. The images really helped a lot too.