From: =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= <bjorn.topel@intel.com>
Date: Mon, 20 Jan 2020 10:29:17 +0100
Subject: xsk, net: Make sock_def_readable() have external linkage
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Patch-mainline: v5.6-rc1
Git-commit: 43a825afc91e2b06af1e8e7422198e759c2c5e20
References: jsc#SLE-13706
XDP sockets use the default implementation of struct sock's
sk_data_ready callback, which is sock_def_readable(). This function
is called in the XDP socket fast-path, and involves a retpoline. By
letting sock_def_readable() have external linkage, and being called
directly, the retpoline can be avoided.
Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20200120092917.13949-1-bjorn.topel@gmail.com
Acked-by: Thomas Bogendoerfer <tbogendoerfer@suse.de>
---
include/net/sock.h | 2 ++
net/core/sock.c | 2 +-
net/xdp/xsk.c | 2 +-
3 files changed, 4 insertions(+), 2 deletions(-)
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2613,4 +2613,6 @@ static inline bool sk_dev_equal_l3scope(
return false;
}
+void sock_def_readable(struct sock *sk);
+
#endif /* _SOCK_H */
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -2786,7 +2786,7 @@ static void sock_def_error_report(struct
rcu_read_unlock();
}
-static void sock_def_readable(struct sock *sk)
+void sock_def_readable(struct sock *sk)
{
struct socket_wq *wq;
--- a/net/xdp/xsk.c
+++ b/net/xdp/xsk.c
@@ -217,7 +217,7 @@ static int xsk_rcv(struct xdp_sock *xs,
static void xsk_flush(struct xdp_sock *xs)
{
xskq_prod_submit(xs->rx);
- xs->sk.sk_data_ready(&xs->sk);
+ sock_def_readable(&xs->sk);
}
int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp)