mirror-linux/tools/testing/selftests
Quentin Monnet a2b5944fb4 selftests/bpf: Check consistency between bpftool source, doc, completion
Whenever the eBPF subsystem gains new elements, such as new program or
map types, it is necessary to update bpftool if we want it able to
handle the new items.

In addition to the main arrays containing the names of these elements in
the source code, there are also multiple locations to update:

- The help message in the do_help() functions in bpftool's source code.
- The RST documentation files.
- The bash completion file.

This has led to omissions multiple times in the past. This patch
attempts to address this issue by adding consistency checks for all
these different locations. It also verifies that the bpf_prog_type,
bpf_map_type and bpf_attach_type enums from the UAPI BPF header have all
their members present in bpftool.

The script requires no argument to run, it reads and parses the
different files to check, and prints the mismatches, if any. It
currently reports a number of missing elements, which will be fixed in a
later patch:

  $ ./test_bpftool_synctypes.py
  Comparing [...]/linux/tools/bpf/bpftool/map.c (map_type_name) and [...]/linux/tools/bpf/bpftool/bash-completion/bpftool (BPFTOOL_MAP_CREATE_TYPES): {'ringbuf'}
  Comparing BPF header (enum bpf_attach_type) and [...]/linux/tools/bpf/bpftool/common.c (attach_type_name): {'BPF_TRACE_ITER', 'BPF_XDP_DEVMAP', 'BPF_XDP', 'BPF_SK_REUSEPORT_SELECT', 'BPF_XDP_CPUMAP', 'BPF_SK_REUSEPORT_SELECT_OR_MIGRATE'}
  Comparing [...]/linux/tools/bpf/bpftool/prog.c (attach_type_strings) and [...]/linux/tools/bpf/bpftool/prog.c (do_help() ATTACH_TYPE): {'skb_verdict'}
  Comparing [...]/linux/tools/bpf/bpftool/prog.c (attach_type_strings) and [...]/linux/tools/bpf/bpftool/Documentation/bpftool-prog.rst (ATTACH_TYPE): {'skb_verdict'}
  Comparing [...]/linux/tools/bpf/bpftool/prog.c (attach_type_strings) and [...]/linux/tools/bpf/bpftool/bash-completion/bpftool (BPFTOOL_PROG_ATTACH_TYPES): {'skb_verdict'}

Note that the script does NOT check for consistency between the list of
program types that bpftool claims it accepts and the actual list of
keywords that can be used. This is because bpftool does not "see" them,
they are ELF section names parsed by libbpf. It is not hard to parse the
section_defs[] array in libbpf, but some section names are associated
with program types that bpftool cannot load at the moment. For example,
some programs require a BTF target and an attach target that bpftool
cannot handle. The script may be extended to parse the array and check
only relevant values in the future.

The script is not added to the selftests' Makefile, because doing so
would require all patches with BPF UAPI change to also update bpftool.
Instead it is to be added to the CI.

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20210730215435.7095-3-quentin@isovalent.com
2021-07-30 15:40:27 -07:00
..
arm64 kselftest/arm64: Add missing newline to SVE test skipping output 2021-05-25 19:19:29 +01:00
bpf selftests/bpf: Check consistency between bpftool source, doc, completion 2021-07-30 15:40:27 -07:00
breakpoints selftests: breakpoints: Use correct error messages in breakpoint_test_arm64.c 2021-02-08 17:04:41 -07:00
capabilities
cgroup tests/cgroup: test cgroup.kill 2021-05-10 10:41:11 -04:00
clone3
core selftests/core: add regression test for CLOSE_RANGE_UNSHARE | CLOSE_RANGE_CLOEXEC 2020-12-19 16:23:19 +01:00
cpu-hotplug
cpufreq
dma dma-mapping: benchmark: Add support for multi-pages map/unmap 2021-04-02 16:41:08 +02:00
dmabuf-heaps kselftests: dmabuf-heaps: Add extra checking that allocated buffers are zeroed 2021-02-08 16:25:53 -07:00
drivers testing: selftests: drivers: net: netdevsim: devlink: add test case for hard drop statistics 2021-06-14 13:04:25 -07:00
efivarfs
exec tools/testing/selftests/exec: fix link error 2021-05-22 15:09:07 -10:00
filesystems kselftest: introduce new epoll test case 2021-05-06 19:24:13 -07:00
firmware firmware_loader: Remove unnecessary conversion to bool 2021-03-23 14:51:50 +01:00
fpu
ftrace Tracing fix for histograms and a clean up in ftrace 2021-07-09 11:15:09 -07:00
futex selftests: futex: Add futex compare requeue test 2021-06-22 11:20:16 +02:00
gpio selftests: gpio: update .gitignore 2021-03-08 11:59:16 +01:00
ia64
intel_pstate
ipc selftests/ipc: remove unneeded semicolon 2021-02-08 16:32:43 -07:00
ir
kcmp
kexec
kmod
kselftest
kvm ARM: 2021-06-28 15:40:51 -07:00
landlock landlock: Enable user space to infer supported features 2021-04-22 12:22:11 -07:00
lib selftests: lib: Add wrapper script for test_scanf 2021-05-19 15:05:11 +02:00
livepatch
lkdtm lkdtm/heap: Add init_on_alloc tests 2021-06-24 15:32:08 +02:00
locking
media_tests
membarrier
memfd
memory-hotplug
mincore selftests: remove duplicate include 2021-05-07 00:26:33 -07:00
mount
mount_setattr tests: test MOUNT_ATTR_NOSYMFOLLOW with mount_setattr() 2021-06-01 15:06:51 +02:00
mqueue
nci selftests: Add .gitignore for nci test suite 2021-05-20 12:51:24 -07:00
net Networking fixes for 5.14-rc2, including fixes from bpf and netfilter. 2021-07-14 09:24:32 -07:00
netfilter selftest: netfilter: add test case for unreplied tcp connections 2021-07-02 02:05:59 +02:00
nsfs
ntb
openat2 test: add openat2() test for invalid upper 32 bit flag value 2021-05-28 17:44:37 +02:00
perf_events signal: Deliver all of the siginfo perf data in _perf 2021-05-18 16:20:54 -05:00
pid_namespace
pidfd
powerpc selftests/powerpc: Use req_max_processed_len from sysfs NX capabilities 2021-06-25 14:47:19 +10:00
prctl
proc proc: add .gitignore for proc-subset-pid selftest 2021-06-05 08:58:11 -07:00
pstore
ptp
ptrace
rcutorture Merge branches 'bitmaprange.2021.05.10c', 'doc.2021.05.10c', 'fixes.2021.05.13a', 'kvfree_rcu.2021.05.10c', 'mmdumpobj.2021.05.10c', 'nocb.2021.05.12a', 'srcu.2021.05.12a', 'tasks.2021.05.18a' and 'torture.2021.05.10c' into HEAD 2021-05-18 10:56:19 -07:00
resctrl selftests/resctrl: Fix incorrect parsing of option "-t" 2021-06-07 18:38:58 -06:00
rlimits kselftests: Add test to check for rlimit changes in different user namespaces 2021-04-30 14:14:03 -05:00
rseq
rtc
safesetid
sched kselftest: Add test for core sched prctl interface 2021-05-12 11:43:32 +02:00
seccomp seccomp updates for v5.14-rc1 2021-06-28 19:49:37 -07:00
sgx selftests/sgx: remove checks for file execute permissions 2021-06-23 18:38:04 -06:00
sigaltstack selftest/sigaltstack: Use the AT_MINSIGSTKSZ aux vector if available 2021-05-19 12:38:17 +02:00
size
sparc64
splice selftests: splice: Adjust for handler fallback removal 2021-06-07 18:39:43 -06:00
static_keys
sync
syscall_user_dispatch entry: Use different define for selector variable in SUD 2021-02-06 00:21:42 +01:00
sysctl
tc-testing tc-testing: add test for ct DNAT tuple collision 2021-06-22 10:52:39 -07:00
timens selftests/timens: Fix gettime_perf to work on powerpc 2021-04-21 22:52:32 +10:00
timers selftests: timers: rtcpie: skip test if default RTC device does not exist 2021-06-07 19:18:52 -06:00
tmpfs
tpm2
uevent
user
vDSO selftests/vDSO: fix ABI selftest on riscv 2021-02-08 16:38:34 -07:00
vm selftest/mremap_test: avoid crash with static build 2021-07-08 11:48:22 -07:00
watchdog
wireguard wireguard: selftests: make sure rp_filter is disabled on vethc 2021-06-04 14:25:14 -07:00
x86 Fixes and improvements for FPU handling on x86: 2021-07-07 11:12:01 -07:00
zram
.gitignore
Makefile Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2021-06-28 20:39:26 -07:00
gen_kselftest_tar.sh
kselftest.h
kselftest_deps.sh selftests: remove obsolete gpio references from kselftest_deps.sh 2021-02-15 11:43:28 +01:00
kselftest_harness.h selftests: kselftest_harness.h: partially fix kernel-doc markups 2021-01-21 14:06:00 -07:00
kselftest_install.sh
kselftest_module.h kselftest: add support for skipped tests 2021-02-15 11:07:42 +01:00
lib.mk selftests: lib.mk: Also install "config" and "settings" 2021-06-07 19:18:28 -06:00
run_kselftest.sh