These instructions are based on lxc, but a similar procedure can be used for Docker or other container-based virtualization systems.
The guest OS to be used in the containers can be built by using these scripts: https://gitlab.retis.santannapisa.it/l.abeni/BuildCore:
git clone https://gitlab.retis.santannapisa.it/l.abeni/BuildCore
mkdir Build
cd Build
sh ../BuildCore/buildcore.sh $(pwd)/test.gz
sudo mkdir /var/lib/lxc/NewTest
sudo cp -a bb_build-1.29.3/_install /var/lib/lxc/NewTest/rootfs
wget http://retis.sssup.it/luca/DPDK_exp/config
sudo cp config /var/lib/lxc/NewTest
sudo vi /var/lib/lxc/NewTest/config # Adjust the "lxc.rootfs.path" entry
sudo mkdir /var/lib/lxc/NewTest/rootfs/var
sudo mkdir /var/lib/lxc/NewTest/rootfs/var/run
sudo mkdir /var/lib/lxc/NewTest/rootfs/var/run/VIO
sudo mkdir /var/lib/lxc/NewTest/rootfs/var/run/hugepages
Now, build the DPDK-based test application (first, you need to build DPDK):
wget http://fast.dpdk.org/rel/dpdk-18.11.tar.xz
tar xvf dpdk-18.11.tar.xz
make -C dpdk-18.11 -j 24 install RTE_SDK=$(pwd)/dpdk-18.11 RTE_OUTPUT=$(pwd)/Build/dpdk_build-18.11 T=x86_64-native-linuxapp-gcc EXTRA_CFLAGS=-g DESTDIR=$(pwd)/I
export RTE_SDK=$(pwd)/I/share/dpdk/
wget http://retis.sssup.it/luca/DPDK_exp/test-app.tgz
tar xvf test-app.tgz
cd Test/
make
sudo cp build/app/my_test /var/lib/lxc/NewTest/rootfs
sudo cp /usr/lib/x86_64-linux-gnu/libnuma.so.1 /var/lib/lxc/NewTest/rootfs/lib
The last item is needed to copy in the container the library/libraries
used by the test application. To get a list of such libraries, use
"ldd build/app/my_test
".
At this point, you need to create a second container, for running the
testpmd
instance that will receive the packets. This
second container can be identical to the first one (only the used
CPUs might need to be changed), so it can be easily created with:
sudo mkdir /var/lib/lxc/Client2
sudo cp /var/lib/lxc/NewTest/config /var/lib/lxc/Client2
sudo vi /var/lib/lxc/Client2/config # Adjust the "lxc.cgroup.cpuset.cpus" entry
In order to use the DPDK virtio_user and vhost drivers, you have to
properly setup the host hugepages, by adding
"hugepagesz=1G hugepages=8 intel_iommu=on
" to the kernel
command line. To do this, edit "/etc/default/grub
" adding
the parameters to "GRUB_CMDLINE_LINUX
" or
"GRUB_CMDLINE_LINUX_DEFAULT
" and regenerate
"/boot/grub/grub.cfg
", or edit
"/boot/grub/grub.cfg
" directly. Then, reboot the system.
Your testbed is now ready for the first experiments.