nsfs: check the trailing bytes of a file handle in bytes, not words

open_by_handle_at(2) gets the wrong answer for a namespace file handle
longer than struct nsfs_file_handle: a 16-byte handle padded with
zeros to 48 bytes or more, which is what the comment above the check
promises to accept, is refused with -ESTALE, while a 20-byte handle
whose extra word is garbage is accepted. Reproduced on v7.0 and on
mainline with the handle of /proc/self/ns/net from
name_to_handle_at(AT_SYMLINK_FOLLOW); no privilege is needed, as
nsfs_fh_to_dentry() admits the caller's own namespaces. Since v6.18.

nsfs_fh_to_dentry() receives fh_len in 32-bit words, like every
fh_to_dentry(), and NSFS_FID_SIZE_U32_LATEST is the struct size in
words (4). The check that the bytes past the struct are zero adds that
word count to a byte pointer and passes the word difference to
memchr_inv() as a byte count: it inspects bytes 4 .. fh_len - 1 of the
handle instead of bytes 16 .. 4 * fh_len - 1. Up to a 44-byte handle
that range lies inside ns_id, ns_type and ns_inum, so the outcome
depends on the values of those fields (a padded net namespace handle
passes because the upper bytes of its ns_id and ns_type are zero) and
the real trailing bytes are never looked at; from 48 bytes on the range
always covers a non-zero field byte and every padded handle is refused.

Start NSFS_FILE_HANDLE_SIZE_LATEST bytes into the handle and check
(fh_len - NSFS_FID_SIZE_U32_LATEST) * sizeof(u32) bytes.

Tested with a program that encodes /proc/self/ns/net and opens the
handle with handle_bytes 16, 20, 36, 48 and 128 (zero padding), and 20
and 48 with a non-zero last byte. Before: 48 and 128 refused, the
20-byte garbage handle accepted; after: every padded handle opens, both
garbage handles are refused.

Fixes: 5222470b2f ("nsfs: support file handles")
Signed-off-by: Nickolai Zeldovich <nickolai@csail.mit.edu>
Link: https://patch.msgid.link/20261018030342.799383-1-nickolai@csail.mit.edu
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>

fs/nsfs.c

@@ -529,8 +529,8 @@ static struct dentry *nsfs_fh_to_dentry(struct super_block *sb, struct fid *fh,
 
 	/* Check that any trailing bytes are zero. */
 	if ((fh_len > NSFS_FID_SIZE_U32_LATEST) &&
-	    memchr_inv((void *)fid + NSFS_FID_SIZE_U32_LATEST, 0,
-		       fh_len - NSFS_FID_SIZE_U32_LATEST))
+	    memchr_inv((void *)fid + NSFS_FILE_HANDLE_SIZE_LATEST, 0,
+		       (fh_len - NSFS_FID_SIZE_U32_LATEST) * sizeof(u32)))
 		return NULL;
 
 	switch (fh_type) {