linux/arch/arm64/lib
Peter Collingbourne 339a43c9bd BACKPORT: FROMGIT: arm64: mte: handle tags zeroing at page allocation time
Currently, on an anonymous page fault, the kernel allocates a zeroed
page and maps it in user space. If the mapping is tagged (PROT_MTE),
set_pte_at() additionally clears the tags. It is, however, more
efficient to clear the tags at the same time as zeroing the data on
allocation. To avoid clearing the tags on any page (which may not be
mapped as tagged), only do this if the vma flags contain VM_MTE. This
requires introducing a new GFP flag that is used to determine whether
to clear the tags.

The DC GZVA instruction with a 0 top byte (and 0 tag) requires
top-byte-ignore. Set the TCR_EL1.{TBI1,TBID1} bits irrespective of
whether KASAN_HW is enabled.

Signed-off-by: Peter Collingbourne <pcc@google.com>
Co-developed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://linux-review.googlesource.com/id/Id46dc94e30fe11474f7e54f5d65e7658dbdddb26
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Link: https://lore.kernel.org/r/20210602235230.3928842-4-pcc@google.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 013bb59dbb
 https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/mte)
[pcc: adjust definition of new GFP flag for compatibility with GFP_CMA; rename TCR_KASAN_HW_FLAGS to TCR_MTE_FLAGS in the old location]
Change-Id: Id46dc94e30fe11474f7e54f5d65e7658dbdddb26
Bug: 186816853
2021-06-17 14:39:37 -07:00
..
clear_page.S FROMLIST: arm64: lib: Annotate {clear, copy}_page() as position-independent 2021-03-17 18:48:47 +00:00
clear_user.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
copy_from_user.S arm64: Reorder the macro arguments in the copy routines 2020-04-29 21:50:01 +01:00
copy_in_user.S arm64: Reorder the macro arguments in the copy routines 2020-04-29 21:50:01 +01:00
copy_page.S FROMLIST: arm64: lib: Annotate {clear, copy}_page() as position-independent 2021-03-17 18:48:47 +00:00
copy_template.S treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
copy_to_user.S arm64: Reorder the macro arguments in the copy routines 2020-04-29 21:50:01 +01:00
crc32.S arm64: lib: Consistently enable crc32 extension 2020-04-28 14:36:32 +01:00
csum.c arm64: csum: Disable KASAN for do_csum() 2020-04-15 21:36:41 +01:00
delay.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
error-inject.c arm64: Add support for function error injection 2019-08-07 13:53:09 +01:00
Makefile arm64: mte: Clear the tags when a page is mapped in user-space with PROT_MTE 2020-09-04 12:46:06 +01:00
memchr.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
memcmp.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
memcpy.S arm64: Change .weak to SYM_FUNC_START_WEAK_PI for arch/arm64/lib/mem*.S 2020-10-30 08:32:31 +00:00
memmove.S arm64: Change .weak to SYM_FUNC_START_WEAK_PI for arch/arm64/lib/mem*.S 2020-10-30 08:32:31 +00:00
memset.S arm64: Change .weak to SYM_FUNC_START_WEAK_PI for arch/arm64/lib/mem*.S 2020-10-30 08:32:31 +00:00
mte.S BACKPORT: FROMGIT: arm64: mte: handle tags zeroing at page allocation time 2021-06-17 14:39:37 -07:00
strchr.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
strcmp.S arm64: fix spelling mistake "ca not" -> "cannot" 2020-03-17 18:22:40 +00:00
strlen.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
strncmp.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
strnlen.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
strrchr.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
tishift.S arm64: lib: Use modern annotations for assembly functions 2020-01-08 12:23:02 +00:00
uaccess_flushcache.c arm64: uaccess: Remove uaccess_*_not_uao asm macros 2019-11-20 18:51:54 +00:00
xor-neon.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00