From: Lorenz Bauer <lmb@cloudflare.com>
Date: Tue, 28 Sep 2021 10:30:59 +0100
Subject: bpf: Do not invoke the XDP dispatcher for PROG_RUN with single repeat
Patch-mainline: v5.16-rc1
Git-commit: de21d8bf777240c6d6dfefa39b4925729e32c0fd
References: jsc#PED-1377

We have a unit test that invokes an XDP program with 1m different
inputs, aka 1m BPF_PROG_RUN syscalls. We run this test concurrently
with slight variations in how we generated the input.

Since commit f23c4b3924d2 ("bpf: Start using the BPF dispatcher in BPF_TEST_RUN")
the unit test has slowed down significantly. Digging deeper reveals that
the concurrent tests are serialised in the kernel on the XDP dispatcher.
This is a global resource that is protected by a mutex, on which we contend.

Fix this by not calling into the XDP dispatcher if we only want to perform
a single run of the BPF program.

See: https://lore.kernel.org/bpf/CACAyw9_y4QumOW35qpgTbLsJ532uGq-kVW-VESJzGyiZkypnvw@mail.gmail.com/

Signed-off-by: Lorenz Bauer <lmb@cloudflare.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20210928093100.27124-1-lmb@cloudflare.com
Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
---
 net/bpf/test_run.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -807,7 +807,8 @@ int bpf_prog_test_run_xdp(struct bpf_pro
 	if (ret)
 		goto free_data;
 
-	bpf_prog_change_xdp(NULL, prog);
+	if (repeat > 1)
+		bpf_prog_change_xdp(NULL, prog);
 	ret = bpf_test_run(prog, &xdp, repeat, &retval, &duration, true);
 	/* We convert the xdp_buff back to an xdp_md before checking the return
 	 * code so the reference count of any held netdevice will be decremented
@@ -828,7 +829,8 @@ int bpf_prog_test_run_xdp(struct bpf_pro
 				     sizeof(struct xdp_md));
 
 out:
-	bpf_prog_change_xdp(prog, NULL);
+	if (repeat > 1)
+		bpf_prog_change_xdp(prog, NULL);
 free_data:
  	kfree(data);
 free_ctx: