From: Stanislav Fomichev <sdf@google.com>
Date: Wed, 18 Aug 2021 16:52:15 -0700
Subject: bpf: Use kvmalloc for map values in syscall
Patch-mainline: v5.15-rc1
Git-commit: f0dce1d9b7c81fc3dc9d0cc0bc7ef9b3eae22584
References: sable-5.14.16

Use kvmalloc/kvfree for temporary value when manipulating a map via
syscall. kmalloc might not be sufficient for percpu maps where the value
is big (and further multiplied by hundreds of CPUs).

Can be reproduced with netcnt test on qemu with "-smp 255".

Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20210818235216.1159202-1-sdf@google.com
Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
---
syu:
- Drop changes in generic_map_update_batch() and generic_map_lockup_batch()
  since we did not backport c60f2d286177 ("bpf: Add lookup and update batch ops
  to arraymap") and friends.
- Drop changes in map_lookup_and_delete_elem() since we did not backport
  bd513cd08f10 ("bpf: add MAP_LOOKUP_AND_DELETE_ELEM syscall")
---
 kernel/bpf/syscall.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -648,7 +648,7 @@ static int map_lookup_elem(union bpf_att
 		value_size = map->value_size;
 
 	err = -ENOMEM;
-	value = kmalloc(value_size, GFP_USER | __GFP_NOWARN);
+	value = kvmalloc(value_size, GFP_USER | __GFP_NOWARN);
 	if (!value)
 		goto free_key;
 
@@ -700,7 +700,7 @@ done:
 	err = 0;
 
 free_value:
-	kfree(value);
+	kvfree(value);
 free_key:
 	kfree(key);
 err_put:
@@ -759,7 +759,7 @@ static int map_update_elem(union bpf_att
 		value_size = map->value_size;
 
 	err = -ENOMEM;
-	value = kmalloc(value_size, GFP_USER | __GFP_NOWARN);
+	value = kvmalloc(value_size, GFP_USER | __GFP_NOWARN);
 	if (!value)
 		goto free_key;
 
@@ -808,7 +808,7 @@ static int map_update_elem(union bpf_att
 	maybe_wait_bpf_programs(map);
 out:
 free_value:
-	kfree(value);
+	kvfree(value);
 free_key:
 	kfree(key);
 err_put: