diff --git a/Documentation/filesystems/fuse/fuse-caches.rst b/Documentation/filesystems/fuse/fuse-caches.rst new file mode 100644 index 000000000000..0778dd3d357f --- /dev/null +++ b/Documentation/filesystems/fuse/fuse-caches.rst @@ -0,0 +1,198 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=========== +FUSE Caches +=========== + +Introduction +============ + +This document summarises the different types of caches used in FUSE. For each +cache type, it documents the rules to insert data into it. It also documents the +rules for validating and invalidating data in the cache. + +symlink caching +=============== + +Whenever there's a link resolution request for a FUSE filesystem, the VFS will +call into ``fuse_get_link()``, the ``->get_link()`` inode operation. This +function will then send a ``FUSE_READLINK`` request to the user-space FUSE +server. + +The server can ask the kernel to cache all link resolutions by setting the +``FUSE_CACHE_SYMLINKS`` flag during the ``FUSE_INIT`` negotiation. If this flag +is set, when the VFS calls into the ``->get_link()`` operation, FUSE will +immediately call ``__page_get_link()``. The first time this is done for a +specific inode, it will result in sending the ``FUSE_READLINK`` request to +user-space. But the result returned from this request will then be added into +the page-cache. The next time this link needs to be resolved, it will use the +link resolution already cached, and will only fallback to user-space if the +folio isn't up-to-date. + +Attributes caching +================== + +Inode attributes may be obtained from user-space by different FUSE operations. +For example, ``FUSE_LOOKUP``, ``FUSE_GETATTR``, and also several other +operations that create file system objects (e.g. ``FUSE_MKDIR``). These +attributes obtained from user-space are cached by the kernel. They have, +however, a timeout associated and once it expires, they are invalidated. The +next time the attributes are needed, a request (``FUSE_GETATTR``) will be sent +to the FUSE server. + +The ``FUSE_GETATTR`` request can be sent to user-space in three different +scenarios: + +#. if the attributes for the inode aren't yet available in the kernel; +#. if they are not valid any more (timed-out, or have been invalidated), or +#. if there is an explicit request for forcing the request to be sent (for + example, by using the ``AT_STATX_FORCE_SYNC`` flag in ``statx``). + +Regarding the attributes invalidation, they may happen in several occasions. For +example, upon a user-space request for invalidation, through +``FUSE_NOTIFY_INVAL_INODE``, ``FUSE_NOTIFY_INVAL_ENTRY``, or +``FUSE_NOTIFY_DELETE`` requests. + +FUSE uses fine-grained invalidation masks rather than invalidating all +attributes at once. The principle is that each operation only invalidates the +specific attributes that the operation could have changed on the server. The +masks used are: + +- ``STATX_ATIME`` - after reads and readlink, since the server may update access + time +- ``STATX_CTIME`` - after xattr changes (including ACL set/remove) and rename +- ``STATX_BLOCKS`` - after a successful flush with writeback cache, since the + server's block count may differ from the local one +- ``FUSE_STATX_MODIFY`` (``STATX_MTIME | STATX_CTIME | STATX_BLOCKS``) - after + writeback completion (without writeback cache), since the server may have + updated modification metadata +- ``FUSE_STATX_MODSIZE`` (``FUSE_STATX_MODIFY | STATX_SIZE``) - after writes, + truncate-on-open, and fallocate, since the server's size and modification + metadata may have changed +- ``FUSE_STATX_MODDIR`` (``FUSE_STATX_MODSIZE | STATX_NLINK``) - after directory + modifications (create, unlink, mkdir, rmdir, rename), since the server may + have updated the directory's size, timestamps, and link count +- ``STATX_BASIC_STATS`` - as a full invalidation, used for server-initiated + invalidation (``FUSE_NOTIFY_INVAL_INODE``), interrupted setattr, and + interrupted link + +The full set of invalidation points can be found by searching for +``fuse_invalidate_attr_mask()`` in the FUSE source. + +ACL caching +=========== + +FUSE has allowed the usage of POSIX Access Control Lists (ACLs) for a long time, +as they can be set and accessed simply as extended attributes. However, it was +only with the introduction of the ``FUSE_POSIX_ACL`` flag that ACLs started to +be fully supported. Without this flag being set during the ``FUSE_INIT`` +negotiation, ACLs can still be set, but the VFS won't use them for performing +permission checks - that would be the user-space server's responsibility. + +Without setting ``FUSE_POSIX_ACL``, ACLs will not be cached by the kernel. In +this case, new inodes ``i_acl`` and ``i_default_acl`` fields will be set to +``ACL_DONT_CACHE``. + +If the ``FUSE_POSIX_ACL`` flag is set, when an inode ACL is accessed VFS will +first check if it's already cached. If it is not, FUSE ``->get_acl()`` operation +(``fuse_get_acl()``) is called, which will eventually send a user-space request. +Future accesses to this inode ACL will use the cached data. + +Setting an ACL in an inode will also result in sending a request to the FUSE +server for setting it. But this operation won't immediately cache the ACL -- it +will only be cached after it is accessed again and requested from user-space. + +ACLs will be removed from the cache in the following situations: + +- When setting an ACL in an inode (and the ``FUSE_POSIX_ACL`` flag is set), + previously cached ACLs for this inode will be invalidated. +- When invalidating an inode through the ``FUSE_NOTIFY_INVAL_INODE`` operation. +- After setting an inode attribute (i.e. operation ``FUSE_SETATTR`` is sent to + user-space), the user-space server may have also updated the ACLs. Thus, any + cached ACLs for this inode are also invalidated. +- Whenever attributes are refreshed from the server. For example, when + revalidating a dentry (``->d_revalidate()``), or when updating a dentry while + processing a ``FUSE_READDIRPLUS``. +- In general, when there is the need to send a ``FUSE_STATX`` or + ``FUSE_GETATTR`` to user-space (e.g. when attributes expired). + +readdir caching +=============== + +When opening a directory a ``FUSE_OPENDIR`` will be sent to the FUSE server, and +server will be responsible for setting the open flags related with caching, +namely ``FOPEN_KEEP_CACHE`` and ``FOPEN_CACHE_DIR``. + +``FOPEN_CACHE_DIR`` determines if readdir results of this open will be cached +and if readdir cache will be used to return readdir results during the current +open. If ``FOPEN_KEEP_CACHE`` is set, any readdir cache from previous opens is +preserved when the directory is opened. Otherwise, the old readdir cache is +invalidated on open. + +The readdir cache will also expire and reset if the inode's ``mtime`` or +``iversion`` don't match the cached values, or if the FUSE connection ``epoch`` +doesn't match the cache ``epoch``. + +dentry caching +============== + +FUSE keeps track of all its file systems dentries in a set of rbtrees. These +trees will keep the dentries sorted by their expiry time, so that it is easy to +quickly invalidate expired dentries. + +This set of rbtrees is protected through hashed locks to reduce contention while +doing trees traversal. + +When FUSE is initialised, an array of ``struct dentry_bucket`` is initialised +with a predefined (``FUSE_HASH_SIZE``) number of elements. Each element of this +array (``dentry_hash``) holds an rbtree (initially empty) of dentries. Every +time a new dentry is created it will be added to one of the elements of this +array's rbtree, where it will be sorted by its expiry time (``timeout``). The +selection of the array element where a dentry is inserted is done through the +``hash_ptr()`` hash function. + +A dentry expiry time will be set to a valid value (provided by the FUSE server) +in several occasions: + +- When a new file system object is created (e.g. ``FUSE_CREATE``, + ``FUSE_MKDIR``, etc). +- When a lookup is sucessfully performed. +- When a dentry is sucessfully revalidated (``->d_revalidate()``). + +A dentry expiry time can be set to 0, indicating that it is stale. This means +that a ``FUSE_LOOKUP`` will be sent to user-space then next time it needs to be +looked up. It can be set to 0 in several occasions: + +- When the user-space FUSE server explicitly requests a dentry to be invalidated + (``FUSE_NOTIFY_INVAL_ENTRY``). +- When a file system object is being deleted (e.g. ``FUSE_UNLINK``) or renamed. +- When a filesystem object is being created and fails with ``-EEXIST``. +- When a lookup fails with ``-ENOENT``. + +Cleaning up expired dentries can happen in different ways. One scenario is when +the VFS requests a revalidation. If the timeout value is 0 (expired), a new +lookup is sent to user-space. If this lookup results in ``-ENOENT``, the dentry +is invalidated. If the lookup is successful, the dentry is re-validated and the +expiry time will be updated. (There's a special case where the lookup is +successful but the nodeid returned is different from the expected one; in this +case a ``FUSE_FORGET`` needs to be sent to user-space and the dentry is +invalidated.) + +Another way of cleaning-up expired dentries is through a mechanism that can be +enabled through the ``inval_wq`` FUSE module parameter. Setting this parameter +to a value >= 5 will create a workqueue that will periodically walk through all +the dentries rbtrees in the ``dentry_hash`` array and invalidate those dentries +that have already expired. Once all the rbtrees have been checked, the workqueue +reschedules itself to run again after ``inval_wq`` seconds. By default, this +workqueue is disabled (``inval_wq`` is set to 0). + +There is yet another mechanism that allows to clean all the cached dentries for +a mounted file system, which is by incrementing the 'epoch' +(``FUSE_NOTIFY_INC_EPOCH``). Each FUSE connection will have its epoch value set +when it is created, and every new dentry have its ``->d_time`` set to the +current connection epoch value. By incrementing the connection epoch value, all +dentries that go through ``->d_revalidate()`` will be automatically invalidated +because their ``->d_time`` is be smaller than the connection epoch. If +``inval_wq`` is set, the workqueue to invalidate the dentries will be triggered +immediately after the epoch is incremented. + diff --git a/Documentation/filesystems/fuse/index.rst b/Documentation/filesystems/fuse/index.rst index 3dada6c4057a..c03c8b7095ed 100644 --- a/Documentation/filesystems/fuse/index.rst +++ b/Documentation/filesystems/fuse/index.rst @@ -12,4 +12,5 @@ FUSE (Filesystem in Userspace) Technical Documentation fuse-io fuse-io-uring fuse-passthrough + fuse-caches uapi/fuse-uapi-io-uring diff --git a/drivers/dax/fsdev.c b/drivers/dax/fsdev.c index 598604bf5ac5..910665c1f492 100644 --- a/drivers/dax/fsdev.c +++ b/drivers/dax/fsdev.c @@ -234,6 +234,29 @@ static const struct file_operations fsdev_fops = { .release = fsdev_release, }; +/** + * fsdev_dax_from_file - get dax_device from an open file + * @file: open device file + * + * Returns a dax_device pointer if @file refers to a fsdev_dax device. + * Otherwise return NULL. + * + * Caller must put_dax() the returned device when done. + */ +struct dax_device *fsdev_dax_from_file(struct file *file) +{ + struct dax_device *dax_dev; + + if (file->f_op != &fsdev_fops) + return NULL; + + dax_dev = inode_dax(file_inode(file)); + ihold(dax_inode(dax_dev)); + + return dax_dev; +} +EXPORT_SYMBOL_GPL(fsdev_dax_from_file); + /* * Acquire the dev_pagemap for probe: the static (pre-populated) one if * present, or a devm-allocated one for the dynamic case. Note that diff --git a/drivers/dax/super.c b/drivers/dax/super.c index 210f4af479f0..c35a7a27583a 100644 --- a/drivers/dax/super.c +++ b/drivers/dax/super.c @@ -565,7 +565,7 @@ static int dax_set(struct inode *inode, void *data) return 0; } -struct dax_device *dax_dev_get(dev_t devt) +static struct dax_device *dax_dev_get(dev_t devt) { struct dax_device *dax_dev; struct inode *inode; @@ -588,7 +588,6 @@ struct dax_device *dax_dev_get(dev_t devt) return dax_dev; } -EXPORT_SYMBOL_GPL(dax_dev_get); struct dax_device *alloc_dax(void *private, const struct dax_operations *ops) { @@ -643,7 +642,7 @@ EXPORT_SYMBOL_GPL(put_dax); */ void *dax_holder(struct dax_device *dax_dev) { - return dax_dev->holder_data; + return READ_ONCE(dax_dev->holder_data); } EXPORT_SYMBOL_GPL(dax_holder); diff --git a/fs/file.c b/fs/file.c index 88b7a5340815..4cd40dfa6129 100644 --- a/fs/file.c +++ b/fs/file.c @@ -1362,6 +1362,7 @@ struct fd fdget_raw(unsigned int fd) { return __fget_light(fd, 0); } +EXPORT_SYMBOL(fdget_raw); /* * Try to avoid f_pos locking. We only need it if the diff --git a/fs/fuse/Kconfig b/fs/fuse/Kconfig index 3a4ae632c94a..3d1e52896b87 100644 --- a/fs/fuse/Kconfig +++ b/fs/fuse/Kconfig @@ -40,9 +40,9 @@ config VIRTIO_FS If you want to share files between guests or with the host, answer Y or M. -config FUSE_DAX +config FUSE_VDAX bool "Virtio Filesystem Direct Host Memory Access support" - default y + default FUSE_DAX select INTERVAL_TREE depends on VIRTIO_FS depends on FS_DAX @@ -54,6 +54,10 @@ config FUSE_DAX If you want to allow mounting a Virtio Filesystem with the "dax" option, answer Y. +config FUSE_DAX + bool + transitional + config FUSE_PASSTHROUGH bool "FUSE passthrough operations support" default y diff --git a/fs/fuse/Makefile b/fs/fuse/Makefile index 245e67852b03..da9e802d7ae1 100644 --- a/fs/fuse/Makefile +++ b/fs/fuse/Makefile @@ -14,8 +14,8 @@ fuse-y := trace.o # put trace.o first so we see ftrace errors sooner fuse-y += dev.o dir.o file.o inode.o control.o xattr.o acl.o readdir.o ioctl.o req_timeout.o req.o fuse-y += poll.o notify.o fuse-y += iomode.o -fuse-$(CONFIG_FUSE_DAX) += dax.o -fuse-$(CONFIG_FUSE_PASSTHROUGH) += passthrough.o backing.o +fuse-$(CONFIG_FUSE_VDAX) += dax.o +fuse-$(CONFIG_FUSE_PASSTHROUGH) += passthrough.o backing.o ext_map.o fuse-$(CONFIG_SYSCTL) += sysctl.o fuse-$(CONFIG_FUSE_IO_URING) += dev_uring.o diff --git a/fs/fuse/backing.c b/fs/fuse/backing.c index 472b6afa7dff..8c21a8a74efb 100644 --- a/fs/fuse/backing.c +++ b/fs/fuse/backing.c @@ -9,8 +9,10 @@ #include "fuse_i.h" #include +#include +#include -struct fuse_backing *fuse_backing_get(struct fuse_backing *fb) +static struct fuse_backing *fuse_backing_get(struct fuse_backing *fb) { if (fb && refcount_inc_not_zero(&fb->count)) return fb; @@ -21,9 +23,20 @@ static void fuse_backing_free(struct fuse_backing *fb) { pr_debug("%s: fb=0x%p\n", __func__, fb); - if (fb->file) - fput(fb->file); - put_cred(fb->cred); + switch (fb->type) { + case FUSE_BACKING_PATH: + path_put(&fb->path); + put_cred(fb->cred); + break; + + case FUSE_BACKING_DAXDEV: + fs_put_dax(fb->dax_dev, fb); + break; + + case FUSE_BACKING_EXTMAP: + fuse_ext_map_destroy(fb); + break; + } kfree_rcu(fb, rcu); } @@ -48,6 +61,8 @@ static int fuse_backing_id_alloc(struct fuse_conn *fc, struct fuse_backing *fb) id = idr_alloc_cyclic(&fc->backing_files_map, fb, 1, 0, GFP_ATOMIC); spin_unlock(&fc->lock); idr_preload_end(); + if (id > 0) + fb->backing_id = id; WARN_ON_ONCE(id == 0); return id; @@ -61,81 +76,184 @@ static struct fuse_backing *fuse_backing_id_remove(struct fuse_conn *fc, spin_lock(&fc->lock); fb = idr_remove(&fc->backing_files_map, id); spin_unlock(&fc->lock); + if (fb) + fb->backing_id = 0; return fb; } -static int fuse_backing_id_free(int id, void *p, void *data) -{ - struct fuse_backing *fb = p; +static const struct rhashtable_params fuse_backing_prm = { + .head_offset = offsetof(struct fuse_backing, hash_node), + .key_offset = offsetof(struct fuse_backing, backing_id), + .key_len = sizeof_field(struct fuse_backing, backing_id), +}; + +int fuse_backing_add_64(struct fuse_conn *fc, struct fuse_backing *fb) +{ + return rhashtable_lookup_insert_fast(&fc->backing_64_ht, &fb->hash_node, fuse_backing_prm); +} + +int fuse_backing_close_64(struct fuse_conn *fc, u64 backing_id) +{ + struct fuse_backing *fb; + int err; + + if (!fc->backing_id_64) + return -EINVAL; + + scoped_guard(spinlock, &fc->lock) { + fb = rhashtable_lookup_fast(&fc->backing_64_ht, &backing_id, fuse_backing_prm); + if (!fb) + return -ENOENT; + + err = rhashtable_remove_fast(&fc->backing_64_ht, &fb->hash_node, fuse_backing_prm); + WARN_ON(err); + } + fb->backing_id = 0; + fuse_backing_put(fb); - WARN_ON_ONCE(refcount_read(&fb->count) != 1); - fuse_backing_free(fb); return 0; } -void fuse_backing_files_free(struct fuse_conn *fc) +static int fuse_dax_notify_failure(struct dax_device *daxdev, u64 offset, u64 len, int mf_flags) { - idr_for_each(&fc->backing_files_map, fuse_backing_id_free, NULL); - idr_destroy(&fc->backing_files_map); + struct fuse_backing *fb; + + guard(rcu)(); + + fb = dax_holder(daxdev); + if (fb) + fb->dax_error = true; + + return 0; +} + +static const struct dax_holder_operations fuse_dax_holder_ops = { + .notify_failure = fuse_dax_notify_failure, +}; + +static int fuse_backing_open_file(struct fuse_conn *fc, struct fuse_backing *fb, struct file *file) +{ + struct inode *inode = file_inode(file); + struct dax_device *daxdev; + int err; + + switch (inode->i_mode & S_IFMT) { + case S_IFREG: + /* TODO: relax CAP_SYS_ADMIN once backing files are visible to lsof */ + if (!fc->passthrough || !capable(CAP_SYS_ADMIN)) + return -EPERM; + + if (inode->i_sb->s_stack_depth >= fc->max_stack_depth) + return -ELOOP; + + fb->type = FUSE_BACKING_PATH; + fb->path = file->f_path; + path_get(&fb->path); + fb->cred = get_current_cred(); + return 0; + + case S_IFCHR: + if (!fc->passthrough || !fc->backing_id_64) + return -EINVAL; + + if (!IS_REACHABLE(CONFIG_DEV_DAX_FSDEV)) + return -EINVAL; + + daxdev = fsdev_dax_from_file(file); + if (!daxdev) + return -EINVAL; + + err = -EPERM; + if (capable(CAP_SYS_RAWIO)) { + err = fs_dax_get(daxdev, fb, &fuse_dax_holder_ops); + if (!err) { + fb->type = FUSE_BACKING_DAXDEV; + fb->dax_dev = daxdev; + } + } + put_dax(daxdev); + return err; + + case S_IFDIR: + return -EISDIR; + + default: + return -EINVAL; + } +} + +static struct fuse_backing *fuse_backing_new(struct fuse_conn *fc, int fd) +{ + struct fuse_backing *fb __free(kfree) = kzalloc_obj(*fb); + int err; + + if (!fb) + return ERR_PTR(-ENOMEM); + + CLASS(fd_raw, f)(fd); + if (fd_empty(f)) + return ERR_PTR(-EBADF); + + err = fuse_backing_open_file(fc, fb, fd_file(f)); + if (err) + return ERR_PTR(err); + + refcount_set(&fb->count, 1); + + return_ptr(fb); +} + +int fuse_backing_open_64(struct fuse_conn *fc, struct fuse_backing_create_in *map) +{ + struct fuse_backing *fb; + int res; + + if (map->padding || map->spare[0] || map->spare[1] || !map->backing_id) + return -EINVAL; + + if (!fc->backing_id_64) + return -EINVAL; + + fb = fuse_backing_new(fc, map->fd); + if (IS_ERR(fb)) + return PTR_ERR(fb); + + fb->backing_id = map->backing_id; + res = fuse_backing_add_64(fc, fb); + if (res < 0) + fuse_backing_free(fb); + + return res; } int fuse_backing_open(struct fuse_conn *fc, struct fuse_backing_map *map) { - struct file *file; - struct super_block *backing_sb; struct fuse_backing *fb = NULL; int res; pr_debug("%s: fd=%d flags=0x%x\n", __func__, map->fd, map->flags); - /* TODO: relax CAP_SYS_ADMIN once backing files are visible to lsof */ - res = -EPERM; - if (!fc->passthrough || !capable(CAP_SYS_ADMIN)) - goto out; - res = -EINVAL; if (map->flags || map->padding) goto out; - file = fget_raw(map->fd); - res = -EBADF; - if (!file) + if (fc->backing_id_64) goto out; - /* read/write/splice/mmap passthrough only relevant for regular files */ - res = d_is_dir(file->f_path.dentry) ? -EISDIR : -EINVAL; - if (!d_is_reg(file->f_path.dentry)) - goto out_fput; - - backing_sb = file_inode(file)->i_sb; - res = -ELOOP; - if (backing_sb->s_stack_depth >= fc->max_stack_depth) - goto out_fput; - - fb = kmalloc_obj(struct fuse_backing); - res = -ENOMEM; - if (!fb) - goto out_fput; - - fb->file = file; - fb->cred = get_current_cred(); - refcount_set(&fb->count, 1); - - res = fuse_backing_id_alloc(fc, fb); - if (res < 0) { - fuse_backing_free(fb); - fb = NULL; + fb = fuse_backing_new(fc, map->fd); + res = PTR_ERR(fb); + if (!IS_ERR(fb)) { + res = fuse_backing_id_alloc(fc, fb); + if (res < 0) { + fuse_backing_free(fb); + fb = NULL; + } } - out: pr_debug("%s: fb=0x%p, ret=%i\n", __func__, fb, res); return res; - -out_fput: - fput(file); - goto out; } int fuse_backing_close(struct fuse_conn *fc, int backing_id) @@ -145,6 +263,9 @@ int fuse_backing_close(struct fuse_conn *fc, int backing_id) pr_debug("%s: backing_id=%d\n", __func__, backing_id); + if (fc->backing_id_64) + return -EINVAL; + /* TODO: relax CAP_SYS_ADMIN once backing files are visible to lsof */ err = -EPERM; if (!fc->passthrough || !capable(CAP_SYS_ADMIN)) @@ -167,14 +288,87 @@ int fuse_backing_close(struct fuse_conn *fc, int backing_id) return err; } -struct fuse_backing *fuse_backing_lookup(struct fuse_conn *fc, int backing_id) +bool fuse_backing_is_dax(struct fuse_backing *fb) +{ + switch (fb->type) { + case FUSE_BACKING_PATH: + return false; + case FUSE_BACKING_DAXDEV: + return true; + case FUSE_BACKING_EXTMAP: + return fuse_ext_map_is_dax(fb); + default: + WARN_ON(1); + return false; + } +} + +struct fuse_backing *fuse_backing_lookup(struct fuse_conn *fc, u64 backing_id) { struct fuse_backing *fb; - rcu_read_lock(); - fb = idr_find(&fc->backing_files_map, backing_id); - fb = fuse_backing_get(fb); - rcu_read_unlock(); + guard(rcu)(); + if (!fc->backing_id_64) + fb = idr_find(&fc->backing_files_map, backing_id); + else + fb = rhashtable_lookup(&fc->backing_64_ht, &backing_id, fuse_backing_prm); - return fb; + return fuse_backing_get(fb); +} + +static void fuse_backing_check_free(struct fuse_backing *fb) +{ + WARN_ON_ONCE(refcount_read(&fb->count) != 1); + fuse_backing_free(fb); +} + +static int fuse_backing_idr_free(int id, void *p, void *data) +{ + fuse_backing_check_free(p); + return 0; +} + +static void fuse_backing_rht_free(void *p, void *data) +{ + fuse_backing_check_free(p); +} + +void fuse_backing_files_free(struct fuse_conn *fc) +{ + struct rhashtable_iter iter; + struct fuse_backing *fb; + + if (!fc->backing_id_64) { + idr_for_each(&fc->backing_files_map, fuse_backing_idr_free, NULL); + idr_destroy(&fc->backing_files_map); + return; + } + + /* + * extents are referencing other backings, put these refs before + * destroying the backings themselves + */ + rhashtable_walk_enter(&fc->backing_64_ht, &iter); + rhashtable_walk_start(&iter); + while ((fb = rhashtable_walk_next(&iter))) { + if (IS_ERR(fb)) { + if (PTR_ERR(fb) == -EAGAIN) + continue; + break; + } + if (fb->type == FUSE_BACKING_EXTMAP && fb->extents) { + rhashtable_walk_stop(&iter); + fuse_ext_map_destroy(fb); + rhashtable_walk_start(&iter); + } + } + rhashtable_walk_stop(&iter); + rhashtable_walk_exit(&iter); + + rhashtable_free_and_destroy(&fc->backing_64_ht, fuse_backing_rht_free, NULL); +} + +void fuse_backing_files_init_64(struct fuse_conn *fc) +{ + rhashtable_init(&fc->backing_64_ht, &fuse_backing_prm); } diff --git a/fs/fuse/cuse.c b/fs/fuse/cuse.c index 4079cf8e5974..2538db153991 100644 --- a/fs/fuse/cuse.c +++ b/fs/fuse/cuse.c @@ -98,7 +98,7 @@ static ssize_t cuse_read_iter(struct kiocb *kiocb, struct iov_iter *to) struct fuse_io_priv io = FUSE_IO_PRIV_SYNC(kiocb); loff_t pos = 0; - return fuse_direct_io(&io, to, &pos, FUSE_DIO_CUSE); + return fuse_do_direct_io(&io, to, &pos, FUSE_DIO_CUSE); } static ssize_t cuse_write_iter(struct kiocb *kiocb, struct iov_iter *from) @@ -109,8 +109,8 @@ static ssize_t cuse_write_iter(struct kiocb *kiocb, struct iov_iter *from) * No locking or generic_write_checks(), the server is * responsible for locking and sanity checks. */ - return fuse_direct_io(&io, from, &pos, - FUSE_DIO_WRITE | FUSE_DIO_CUSE); + return fuse_do_direct_io(&io, from, &pos, + FUSE_DIO_WRITE | FUSE_DIO_CUSE); } static int cuse_open(struct inode *inode, struct file *file) diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index 85cdf0199bc0..b79fa26ce783 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * dax: direct host memory access + * dax: direct host memory access for virtiofs * Copyright (C) 2020 Red Hat, Inc. */ @@ -59,7 +59,7 @@ struct fuse_dax_mapping { }; /* Per-inode dax map */ -struct fuse_inode_dax { +struct fuse_inode_vdax { /* Semaphore to protect modifications to the dmap tree */ struct rw_semaphore sem; @@ -68,7 +68,7 @@ struct fuse_inode_dax { unsigned long nr; }; -struct fuse_conn_dax { +struct fuse_conn_vdax { /* DAX device */ struct dax_device *dev; @@ -102,10 +102,10 @@ node_to_dmap(struct interval_tree_node *node) } static struct fuse_dax_mapping * -alloc_dax_mapping_reclaim(struct fuse_conn_dax *fcd, struct inode *inode); +alloc_dax_mapping_reclaim(struct fuse_conn_vdax *fcd, struct inode *inode); static void -__kick_dmap_free_worker(struct fuse_conn_dax *fcd, unsigned long delay_ms) +__kick_dmap_free_worker(struct fuse_conn_vdax *fcd, unsigned long delay_ms) { unsigned long free_threshold; @@ -117,7 +117,7 @@ __kick_dmap_free_worker(struct fuse_conn_dax *fcd, unsigned long delay_ms) msecs_to_jiffies(delay_ms)); } -static void kick_dmap_free_worker(struct fuse_conn_dax *fcd, +static void kick_dmap_free_worker(struct fuse_conn_vdax *fcd, unsigned long delay_ms) { spin_lock(&fcd->lock); @@ -125,7 +125,7 @@ static void kick_dmap_free_worker(struct fuse_conn_dax *fcd, spin_unlock(&fcd->lock); } -static struct fuse_dax_mapping *alloc_dax_mapping(struct fuse_conn_dax *fcd) +static struct fuse_dax_mapping *alloc_dax_mapping(struct fuse_conn_vdax *fcd) { struct fuse_dax_mapping *dmap; @@ -144,7 +144,7 @@ static struct fuse_dax_mapping *alloc_dax_mapping(struct fuse_conn_dax *fcd) } /* This assumes fcd->lock is held */ -static void __dmap_remove_busy_list(struct fuse_conn_dax *fcd, +static void __dmap_remove_busy_list(struct fuse_conn_vdax *fcd, struct fuse_dax_mapping *dmap) { list_del_init(&dmap->busy_list); @@ -152,7 +152,7 @@ static void __dmap_remove_busy_list(struct fuse_conn_dax *fcd, fcd->nr_busy_ranges--; } -static void dmap_remove_busy_list(struct fuse_conn_dax *fcd, +static void dmap_remove_busy_list(struct fuse_conn_vdax *fcd, struct fuse_dax_mapping *dmap) { spin_lock(&fcd->lock); @@ -161,7 +161,7 @@ static void dmap_remove_busy_list(struct fuse_conn_dax *fcd, } /* This assumes fcd->lock is held */ -static void __dmap_add_to_free_pool(struct fuse_conn_dax *fcd, +static void __dmap_add_to_free_pool(struct fuse_conn_vdax *fcd, struct fuse_dax_mapping *dmap) { list_add_tail(&dmap->list, &fcd->free_ranges); @@ -169,7 +169,7 @@ static void __dmap_add_to_free_pool(struct fuse_conn_dax *fcd, wake_up(&fcd->range_waitq); } -static void dmap_add_to_free_pool(struct fuse_conn_dax *fcd, +static void dmap_add_to_free_pool(struct fuse_conn_vdax *fcd, struct fuse_dax_mapping *dmap) { /* Return fuse_dax_mapping to free list */ @@ -183,7 +183,7 @@ static int fuse_setup_one_mapping(struct inode *inode, unsigned long start_idx, bool upgrade) { struct fuse_mount *fm = get_fuse_mount(inode); - struct fuse_conn_dax *fcd = fm->fc->dax; + struct fuse_conn_vdax *fcd = fm->fc->vdax; struct fuse_inode *fi = get_fuse_inode(inode); struct fuse_setupmapping_in inarg; loff_t offset = start_idx << FUSE_DAX_SHIFT; @@ -218,9 +218,9 @@ static int fuse_setup_one_mapping(struct inode *inode, unsigned long start_idx, */ dmap->inode = inode; dmap->itn.start = dmap->itn.last = start_idx; - /* Protected by fi->dax->sem */ - interval_tree_insert(&dmap->itn, &fi->dax->tree); - fi->dax->nr++; + /* Protected by fi->vdax->sem */ + interval_tree_insert(&dmap->itn, &fi->vdax->tree); + fi->vdax->nr++; spin_lock(&fcd->lock); list_add_tail(&dmap->busy_list, &fcd->busy_ranges); fcd->nr_busy_ranges++; @@ -288,7 +288,7 @@ static int dmap_removemapping_list(struct inode *inode, unsigned int num, * Cleanup dmap entry and add back to free list. This should be called with * fcd->lock held. */ -static void dmap_reinit_add_to_free_pool(struct fuse_conn_dax *fcd, +static void dmap_reinit_add_to_free_pool(struct fuse_conn_vdax *fcd, struct fuse_dax_mapping *dmap) { pr_debug("fuse: freeing memory range start_idx=0x%lx end_idx=0x%lx window_offset=0x%llx length=0x%llx\n", @@ -306,7 +306,7 @@ static void dmap_reinit_add_to_free_pool(struct fuse_conn_dax *fcd, * called from evict_inode() path where we know all dmap entries can be * reclaimed. */ -static void inode_reclaim_dmap_range(struct fuse_conn_dax *fcd, +static void inode_reclaim_dmap_range(struct fuse_conn_vdax *fcd, struct inode *inode, loff_t start, loff_t end) { @@ -319,14 +319,14 @@ static void inode_reclaim_dmap_range(struct fuse_conn_dax *fcd, struct interval_tree_node *node; while (1) { - node = interval_tree_iter_first(&fi->dax->tree, start_idx, + node = interval_tree_iter_first(&fi->vdax->tree, start_idx, end_idx); if (!node) break; dmap = node_to_dmap(node); /* inode is going away. There should not be any users of dmap */ WARN_ON(refcount_read(&dmap->refcnt) > 1); - interval_tree_remove(&dmap->itn, &fi->dax->tree); + interval_tree_remove(&dmap->itn, &fi->vdax->tree); num++; list_add(&dmap->list, &to_remove); } @@ -335,8 +335,8 @@ static void inode_reclaim_dmap_range(struct fuse_conn_dax *fcd, if (list_empty(&to_remove)) return; - WARN_ON(fi->dax->nr < num); - fi->dax->nr -= num; + WARN_ON(fi->vdax->nr < num); + fi->vdax->nr -= num; err = dmap_removemapping_list(inode, num, &to_remove); if (err && err != -ENOTCONN) { pr_warn("Failed to removemappings. start=0x%llx end=0x%llx\n", @@ -367,11 +367,11 @@ static int dmap_removemapping_one(struct inode *inode, /* * It is called from evict_inode() and by that time inode is going away. So - * this function does not take any locks like fi->dax->sem for traversing + * this function does not take any locks like fi->vdax->sem for traversing * that fuse inode interval tree. If that lock is taken then lock validator * complains of deadlock situation w.r.t fs_reclaim lock. */ -void fuse_dax_inode_cleanup(struct inode *inode) +void fuse_vdax_inode_cleanup(struct inode *inode) { struct fuse_conn *fc = get_fuse_conn(inode); struct fuse_inode *fi = get_fuse_inode(inode); @@ -381,8 +381,8 @@ void fuse_dax_inode_cleanup(struct inode *inode) * before we arrive here. So we should not have to worry about any * pages/exception entries still associated with inode. */ - inode_reclaim_dmap_range(fc->dax, inode, 0, -1); - WARN_ON(fi->dax->nr); + inode_reclaim_dmap_range(fc->vdax, inode, 0, -1); + WARN_ON(fi->vdax->nr); } static void fuse_fill_iomap_hole(struct iomap *iomap, loff_t length) @@ -414,7 +414,7 @@ static void fuse_fill_iomap(struct inode *inode, loff_t pos, loff_t length, iomap->type = IOMAP_MAPPED; /* * increace refcnt so that reclaim code knows this dmap is in - * use. This assumes fi->dax->sem mutex is held either + * use. This assumes fi->vdax->sem mutex is held either * shared/exclusive. */ refcount_inc(&dmap->refcnt); @@ -434,7 +434,7 @@ static int fuse_setup_new_dax_mapping(struct inode *inode, loff_t pos, { struct fuse_inode *fi = get_fuse_inode(inode); struct fuse_conn *fc = get_fuse_conn(inode); - struct fuse_conn_dax *fcd = fc->dax; + struct fuse_conn_vdax *fcd = fc->vdax; struct fuse_dax_mapping *dmap, *alloc_dmap = NULL; int ret; bool writable = flags & IOMAP_WRITE; @@ -469,17 +469,17 @@ static int fuse_setup_new_dax_mapping(struct inode *inode, loff_t pos, * Take write lock so that only one caller can try to setup mapping * and other waits. */ - down_write(&fi->dax->sem); + down_write(&fi->vdax->sem); /* * We dropped lock. Check again if somebody else setup * mapping already. */ - node = interval_tree_iter_first(&fi->dax->tree, start_idx, start_idx); + node = interval_tree_iter_first(&fi->vdax->tree, start_idx, start_idx); if (node) { dmap = node_to_dmap(node); fuse_fill_iomap(inode, pos, length, iomap, dmap, flags); dmap_add_to_free_pool(fcd, alloc_dmap); - up_write(&fi->dax->sem); + up_write(&fi->vdax->sem); return 0; } @@ -488,11 +488,11 @@ static int fuse_setup_new_dax_mapping(struct inode *inode, loff_t pos, writable, false); if (ret < 0) { dmap_add_to_free_pool(fcd, alloc_dmap); - up_write(&fi->dax->sem); + up_write(&fi->vdax->sem); return ret; } fuse_fill_iomap(inode, pos, length, iomap, alloc_dmap, flags); - up_write(&fi->dax->sem); + up_write(&fi->vdax->sem); return 0; } @@ -510,14 +510,14 @@ static int fuse_upgrade_dax_mapping(struct inode *inode, loff_t pos, * Take exclusive lock so that only one caller can try to setup * mapping and others wait. */ - down_write(&fi->dax->sem); - node = interval_tree_iter_first(&fi->dax->tree, idx, idx); + down_write(&fi->vdax->sem); + node = interval_tree_iter_first(&fi->vdax->tree, idx, idx); /* We are holding either inode lock or invalidate_lock, and that should * ensure that dmap can't be truncated. We are holding a reference * on dmap and that should make sure it can't be reclaimed. So dmap * should still be there in tree despite the fact we dropped and - * re-acquired the fi->dax->sem lock. + * re-acquired the fi->vdax->sem lock. */ ret = -EIO; if (WARN_ON(!node)) @@ -526,7 +526,7 @@ static int fuse_upgrade_dax_mapping(struct inode *inode, loff_t pos, dmap = node_to_dmap(node); /* We took an extra reference on dmap to make sure its not reclaimd. - * Now we hold fi->dax->sem lock and that reference is not needed + * Now we hold fi->vdax->sem lock and that reference is not needed * anymore. Drop it. */ if (refcount_dec_and_test(&dmap->refcnt)) { @@ -551,7 +551,7 @@ static int fuse_upgrade_dax_mapping(struct inode *inode, loff_t pos, out_fill_iomap: fuse_fill_iomap(inode, pos, length, iomap, dmap, flags); out_err: - up_write(&fi->dax->sem); + up_write(&fi->vdax->sem); return ret; } @@ -576,7 +576,7 @@ static int fuse_iomap_begin(struct inode *inode, loff_t pos, loff_t length, iomap->offset = pos; iomap->flags = 0; iomap->bdev = NULL; - iomap->dax_dev = fc->dax->dev; + iomap->dax_dev = fc->vdax->dev; /* * Both read/write and mmap path can race here. So we need something @@ -585,33 +585,33 @@ static int fuse_iomap_begin(struct inode *inode, loff_t pos, loff_t length, * For now, use a semaphore for this. It probably needs to be * optimized later. */ - down_read(&fi->dax->sem); - node = interval_tree_iter_first(&fi->dax->tree, start_idx, start_idx); + down_read(&fi->vdax->sem); + node = interval_tree_iter_first(&fi->vdax->tree, start_idx, start_idx); if (node) { dmap = node_to_dmap(node); if (writable && !dmap->writable) { /* Upgrade read-only mapping to read-write. This will - * require exclusive fi->dax->sem lock as we don't want + * require exclusive fi->vdax->sem lock as we don't want * two threads to be trying to this simultaneously * for same dmap. So drop shared lock and acquire * exclusive lock. * - * Before dropping fi->dax->sem lock, take reference + * Before dropping fi->vdax->sem lock, take reference * on dmap so that its not freed by range reclaim. */ refcount_inc(&dmap->refcnt); - up_read(&fi->dax->sem); + up_read(&fi->vdax->sem); pr_debug("%s: Upgrading mapping at offset 0x%llx length 0x%llx\n", __func__, pos, length); return fuse_upgrade_dax_mapping(inode, pos, length, flags, iomap); } else { fuse_fill_iomap(inode, pos, length, iomap, dmap, flags); - up_read(&fi->dax->sem); + up_read(&fi->vdax->sem); return 0; } } else { - up_read(&fi->dax->sem); + up_read(&fi->vdax->sem); pr_debug("%s: no mapping at offset 0x%llx length 0x%llx\n", __func__, pos, length); if (pos >= i_size_read(inode)) @@ -668,14 +668,14 @@ static void fuse_wait_dax_page(struct inode *inode) } /* Should be called with mapping->invalidate_lock held exclusively. */ -int fuse_dax_break_layouts(struct inode *inode, u64 dmap_start, +int fuse_vdax_break_layouts(struct inode *inode, u64 dmap_start, u64 dmap_end) { return dax_break_layout(inode, dmap_start, dmap_end, fuse_wait_dax_page); } -ssize_t fuse_dax_read_iter(struct kiocb *iocb, struct iov_iter *to) +ssize_t fuse_vdax_read_iter(struct kiocb *iocb, struct iov_iter *to) { struct inode *inode = file_inode(iocb->ki_filp); ssize_t ret; @@ -709,13 +709,13 @@ static ssize_t fuse_dax_direct_write(struct kiocb *iocb, struct iov_iter *from) struct fuse_io_priv io = FUSE_IO_PRIV_SYNC(iocb); ssize_t ret; - ret = fuse_direct_io(&io, from, &iocb->ki_pos, FUSE_DIO_WRITE); + ret = fuse_do_direct_io(&io, from, &iocb->ki_pos, FUSE_DIO_WRITE); fuse_write_update_attr(inode, iocb->ki_pos, ret); return ret; } -ssize_t fuse_dax_write_iter(struct kiocb *iocb, struct iov_iter *from) +ssize_t fuse_vdax_write_iter(struct kiocb *iocb, struct iov_iter *from) { struct inode *inode = file_inode(iocb->ki_filp); ssize_t ret; @@ -761,7 +761,7 @@ static vm_fault_t __fuse_dax_fault(struct vm_fault *vmf, unsigned int order, unsigned long pfn; int error = 0; struct fuse_conn *fc = get_fuse_conn(inode); - struct fuse_conn_dax *fcd = fc->dax; + struct fuse_conn_vdax *fcd = fc->vdax; bool retry = false; if (write) @@ -822,11 +822,12 @@ static const struct vm_operations_struct fuse_dax_vm_ops = { .pfn_mkwrite = fuse_dax_pfn_mkwrite, }; -int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma) +int fuse_vdax_mmap(struct file *file, struct vm_area_struct *vma) { file_accessed(file); vma->vm_ops = &fuse_dax_vm_ops; vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE); + vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot); return 0; } @@ -869,8 +870,8 @@ static int reclaim_one_dmap_locked(struct inode *inode, return ret; /* Remove dax mapping from inode interval tree now */ - interval_tree_remove(&dmap->itn, &fi->dax->tree); - fi->dax->nr--; + interval_tree_remove(&dmap->itn, &fi->vdax->tree); + fi->vdax->nr--; /* It is possible that umount/shutdown has killed the fuse connection * and worker thread is trying to reclaim memory in parallel. Don't @@ -885,7 +886,7 @@ static int reclaim_one_dmap_locked(struct inode *inode, } /* Find first mapped dmap for an inode and return file offset. Caller needs - * to hold fi->dax->sem lock either shared or exclusive. + * to hold fi->vdax->sem lock either shared or exclusive. */ static struct fuse_dax_mapping *inode_lookup_first_dmap(struct inode *inode) { @@ -893,7 +894,7 @@ static struct fuse_dax_mapping *inode_lookup_first_dmap(struct inode *inode) struct fuse_dax_mapping *dmap; struct interval_tree_node *node; - for (node = interval_tree_iter_first(&fi->dax->tree, 0, -1); node; + for (node = interval_tree_iter_first(&fi->vdax->tree, 0, -1); node; node = interval_tree_iter_next(node, 0, -1)) { dmap = node_to_dmap(node); /* still in use. */ @@ -911,7 +912,7 @@ static struct fuse_dax_mapping *inode_lookup_first_dmap(struct inode *inode) * it back to free pool. */ static struct fuse_dax_mapping * -inode_inline_reclaim_one_dmap(struct fuse_conn_dax *fcd, struct inode *inode, +inode_inline_reclaim_one_dmap(struct fuse_conn_vdax *fcd, struct inode *inode, bool *retry) { struct fuse_inode *fi = get_fuse_inode(inode); @@ -924,14 +925,14 @@ inode_inline_reclaim_one_dmap(struct fuse_conn_dax *fcd, struct inode *inode, filemap_invalidate_lock(inode->i_mapping); /* Lookup a dmap and corresponding file offset to reclaim. */ - down_read(&fi->dax->sem); + down_read(&fi->vdax->sem); dmap = inode_lookup_first_dmap(inode); if (dmap) { start_idx = dmap->itn.start; dmap_start = start_idx << FUSE_DAX_SHIFT; dmap_end = dmap_start + FUSE_DAX_SZ - 1; } - up_read(&fi->dax->sem); + up_read(&fi->vdax->sem); if (!dmap) goto out_mmap_sem; @@ -939,16 +940,16 @@ inode_inline_reclaim_one_dmap(struct fuse_conn_dax *fcd, struct inode *inode, * Make sure there are no references to inode pages using * get_user_pages() */ - ret = fuse_dax_break_layouts(inode, dmap_start, dmap_end); + ret = fuse_vdax_break_layouts(inode, dmap_start, dmap_end); if (ret) { - pr_debug("fuse: fuse_dax_break_layouts() failed. err=%d\n", + pr_debug("fuse: fuse_vdax_break_layouts() failed. err=%d\n", ret); dmap = ERR_PTR(ret); goto out_mmap_sem; } - down_write(&fi->dax->sem); - node = interval_tree_iter_first(&fi->dax->tree, start_idx, start_idx); + down_write(&fi->vdax->sem); + node = interval_tree_iter_first(&fi->vdax->tree, start_idx, start_idx); /* Range already got reclaimed by somebody else */ if (!node) { if (retry) @@ -980,14 +981,14 @@ inode_inline_reclaim_one_dmap(struct fuse_conn_dax *fcd, struct inode *inode, __func__, inode, dmap->window_offset, dmap->length); out_write_dmap_sem: - up_write(&fi->dax->sem); + up_write(&fi->vdax->sem); out_mmap_sem: filemap_invalidate_unlock(inode->i_mapping); return dmap; } static struct fuse_dax_mapping * -alloc_dax_mapping_reclaim(struct fuse_conn_dax *fcd, struct inode *inode) +alloc_dax_mapping_reclaim(struct fuse_conn_vdax *fcd, struct inode *inode) { struct fuse_dax_mapping *dmap; struct fuse_inode *fi = get_fuse_inode(inode); @@ -1014,18 +1015,18 @@ alloc_dax_mapping_reclaim(struct fuse_conn_dax *fcd, struct inode *inode) * if a deadlock is possible if we sleep with * mapping->invalidate_lock held and worker to free memory * can't make progress due to unavailability of - * mapping->invalidate_lock. So sleep only if fi->dax->nr=0 + * mapping->invalidate_lock. So sleep only if fi->vdax->nr=0 */ if (retry) continue; /* * There are no mappings which can be reclaimed. Wait for one. - * We are not holding fi->dax->sem. So it is possible + * We are not holding fi->vdax->sem. So it is possible * that range gets added now. But as we are not holding * mapping->invalidate_lock, worker should still be able to * free up a range and wake us up. */ - if (!fi->dax->nr && !(fcd->nr_free_ranges > 0)) { + if (!fi->vdax->nr && !(fcd->nr_free_ranges > 0)) { if (wait_event_killable_exclusive(fcd->range_waitq, (fcd->nr_free_ranges > 0))) { return ERR_PTR(-EINTR); @@ -1034,7 +1035,7 @@ alloc_dax_mapping_reclaim(struct fuse_conn_dax *fcd, struct inode *inode) } } -static int lookup_and_reclaim_dmap_locked(struct fuse_conn_dax *fcd, +static int lookup_and_reclaim_dmap_locked(struct fuse_conn_vdax *fcd, struct inode *inode, unsigned long start_idx) { @@ -1044,7 +1045,7 @@ static int lookup_and_reclaim_dmap_locked(struct fuse_conn_dax *fcd, struct interval_tree_node *node; /* Find fuse dax mapping at file offset inode. */ - node = interval_tree_iter_first(&fi->dax->tree, start_idx, start_idx); + node = interval_tree_iter_first(&fi->vdax->tree, start_idx, start_idx); /* Range already got cleaned up by somebody else */ if (!node) @@ -1070,10 +1071,10 @@ static int lookup_and_reclaim_dmap_locked(struct fuse_conn_dax *fcd, * Free a range of memory. * Locking: * 1. Take mapping->invalidate_lock to block dax faults. - * 2. Take fi->dax->sem to protect interval tree and also to make sure + * 2. Take fi->vdax->sem to protect interval tree and also to make sure * read/write can not reuse a dmap which we might be freeing. */ -static int lookup_and_reclaim_dmap(struct fuse_conn_dax *fcd, +static int lookup_and_reclaim_dmap(struct fuse_conn_vdax *fcd, struct inode *inode, unsigned long start_idx, unsigned long end_idx) @@ -1084,22 +1085,22 @@ static int lookup_and_reclaim_dmap(struct fuse_conn_dax *fcd, loff_t dmap_end = (dmap_start + FUSE_DAX_SZ) - 1; filemap_invalidate_lock(inode->i_mapping); - ret = fuse_dax_break_layouts(inode, dmap_start, dmap_end); + ret = fuse_vdax_break_layouts(inode, dmap_start, dmap_end); if (ret) { - pr_debug("virtio_fs: fuse_dax_break_layouts() failed. err=%d\n", + pr_debug("virtio_fs: fuse_vdax_break_layouts() failed. err=%d\n", ret); goto out_mmap_sem; } - down_write(&fi->dax->sem); + down_write(&fi->vdax->sem); ret = lookup_and_reclaim_dmap_locked(fcd, inode, start_idx); - up_write(&fi->dax->sem); + up_write(&fi->vdax->sem); out_mmap_sem: filemap_invalidate_unlock(inode->i_mapping); return ret; } -static int try_to_free_dmap_chunks(struct fuse_conn_dax *fcd, +static int try_to_free_dmap_chunks(struct fuse_conn_vdax *fcd, unsigned long nr_to_free) { struct fuse_dax_mapping *dmap, *pos, *temp; @@ -1160,7 +1161,7 @@ static int try_to_free_dmap_chunks(struct fuse_conn_dax *fcd, static void fuse_dax_free_mem_worker(struct work_struct *work) { int ret; - struct fuse_conn_dax *fcd = container_of(work, struct fuse_conn_dax, + struct fuse_conn_vdax *fcd = container_of(work, struct fuse_conn_vdax, free_work.work); ret = try_to_free_dmap_chunks(fcd, FUSE_DAX_RECLAIM_CHUNK); if (ret) { @@ -1185,16 +1186,16 @@ static void fuse_free_dax_mem_ranges(struct list_head *mem_list) } } -void fuse_dax_conn_free(struct fuse_conn *fc) +void fuse_vdax_conn_free(struct fuse_conn *fc) { - if (fc->dax) { - fuse_free_dax_mem_ranges(&fc->dax->free_ranges); - kfree(fc->dax); - fc->dax = NULL; + if (fc->vdax) { + fuse_free_dax_mem_ranges(&fc->vdax->free_ranges); + kfree(fc->vdax); + fc->vdax = NULL; } } -static int fuse_dax_mem_range_init(struct fuse_conn_dax *fcd) +static int fuse_dax_mem_range_init(struct fuse_conn_vdax *fcd) { long nr_pages, nr_ranges; struct fuse_dax_mapping *range; @@ -1246,13 +1247,13 @@ static int fuse_dax_mem_range_init(struct fuse_conn_dax *fcd) return ret; } -int fuse_dax_conn_alloc(struct fuse_conn *fc, enum fuse_dax_mode dax_mode, +int fuse_vdax_conn_alloc(struct fuse_conn *fc, enum fuse_vdax_mode dax_mode, struct dax_device *dax_dev) { - struct fuse_conn_dax *fcd; + struct fuse_conn_vdax *fcd; int err; - fc->dax_mode = dax_mode; + fc->vdax_mode = dax_mode; if (!dax_dev) return 0; @@ -1269,22 +1270,22 @@ int fuse_dax_conn_alloc(struct fuse_conn *fc, enum fuse_dax_mode dax_mode, return err; } - fc->dax = fcd; + fc->vdax = fcd; return 0; } -bool fuse_dax_inode_alloc(struct super_block *sb, struct fuse_inode *fi) +bool fuse_vdax_inode_alloc(struct super_block *sb, struct fuse_inode *fi) { struct fuse_conn *fc = get_fuse_conn_super(sb); - fi->dax = NULL; - if (fc->dax) { - fi->dax = kzalloc_obj(*fi->dax, GFP_KERNEL_ACCOUNT); - if (!fi->dax) + fi->vdax = NULL; + if (fc->vdax) { + fi->vdax = kzalloc_obj(*fi->vdax, GFP_KERNEL_ACCOUNT); + if (!fi->vdax) return false; - init_rwsem(&fi->dax->sem); - fi->dax->tree = RB_ROOT_CACHED; + init_rwsem(&fi->vdax->sem); + fi->vdax->tree = RB_ROOT_CACHED; } return true; @@ -1298,26 +1299,26 @@ static const struct address_space_operations fuse_dax_file_aops = { static bool fuse_should_enable_dax(struct inode *inode, unsigned int flags) { struct fuse_conn *fc = get_fuse_conn(inode); - enum fuse_dax_mode dax_mode = fc->dax_mode; + enum fuse_vdax_mode dax_mode = fc->vdax_mode; - if (dax_mode == FUSE_DAX_NEVER) + if (dax_mode == FUSE_VDAX_NEVER) return false; /* - * fc->dax may be NULL in 'inode' mode when filesystem device doesn't + * fc->vdax may be NULL in 'inode' mode when filesystem device doesn't * support DAX, in which case it will silently fallback to 'never' mode. */ - if (!fc->dax) + if (!fc->vdax) return false; - if (dax_mode == FUSE_DAX_ALWAYS) + if (dax_mode == FUSE_VDAX_ALWAYS) return true; /* dax_mode is FUSE_DAX_INODE* */ - return fc->inode_dax && (flags & FUSE_ATTR_DAX); + return fc->inode_vdax && (flags & FUSE_ATTR_DAX); } -void fuse_dax_inode_init(struct inode *inode, unsigned int flags) +void fuse_vdax_inode_init(struct inode *inode, unsigned int flags) { if (!fuse_should_enable_dax(inode, flags)) return; @@ -1326,18 +1327,18 @@ void fuse_dax_inode_init(struct inode *inode, unsigned int flags) inode->i_data.a_ops = &fuse_dax_file_aops; } -void fuse_dax_dontcache(struct inode *inode, unsigned int flags) +void fuse_vdax_dontcache(struct inode *inode, unsigned int flags) { struct fuse_conn *fc = get_fuse_conn(inode); - if (fuse_is_inode_dax_mode(fc->dax_mode) && + if (fuse_is_inode_vdax_mode(fc->vdax_mode) && ((bool) IS_DAX(inode) != (bool) (flags & FUSE_ATTR_DAX))) d_mark_dontcache(inode); } -bool fuse_dax_check_alignment(struct fuse_conn *fc, unsigned int map_alignment) +bool fuse_vdax_check_alignment(struct fuse_conn *fc, unsigned int map_alignment) { - if (fc->dax && (map_alignment > FUSE_DAX_SHIFT)) { + if (fc->vdax && (map_alignment > FUSE_DAX_SHIFT)) { pr_warn("FUSE: map_alignment %u incompatible with dax mem range size %u\n", map_alignment, FUSE_DAX_SZ); return false; @@ -1345,12 +1346,12 @@ bool fuse_dax_check_alignment(struct fuse_conn *fc, unsigned int map_alignment) return true; } -void fuse_dax_cancel_work(struct fuse_conn *fc) +void fuse_vdax_cancel_work(struct fuse_conn *fc) { - struct fuse_conn_dax *fcd = fc->dax; + struct fuse_conn_vdax *fcd = fc->vdax; if (fcd) cancel_delayed_work_sync(&fcd->free_work); } -EXPORT_SYMBOL_GPL(fuse_dax_cancel_work); +EXPORT_SYMBOL_GPL(fuse_vdax_cancel_work); diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index 4fec31fc0b84..1a40b60afb98 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -466,24 +466,27 @@ static bool fuse_dev_install_with_pq(struct fuse_dev *fud, struct fuse_chan *fch struct fuse_chan *old_fch; guard(spinlock)(&fch->lock); - /* - * Pairs with: - * - xchg() in fuse_dev_release() - * - smp_load_acquire() in fuse_dev_fc_get() - */ - old_fch = cmpxchg(&fud->chan, NULL, fch); - if (old_fch) { + /* protect fud->pq.processing assignment */ + scoped_guard(spinlock, &fud->pq.lock) { /* - * failed to set fud->chan because - * - it was already set to a different fc - * - it was set to disconneted + * Pairs with: + * - xchg() in fuse_dev_release() + * - smp_load_acquire() in fuse_dev_fc_get() */ - kfree(pq); - return false; - } - if (pq) { - WARN_ON(fud->pq.processing); - fud->pq.processing = pq; + old_fch = cmpxchg(&fud->chan, NULL, fch); + if (old_fch) { + /* + * failed to set fud->chan because + * - it was already set to a different fc + * - it was set to disconneted + */ + kfree(pq); + return false; + } + if (pq) { + WARN_ON(fud->pq.processing); + fud->pq.processing = pq; + } } list_add_tail(&fud->entry, &fch->devices); fuse_conn_get(fch->conn); @@ -1061,7 +1064,8 @@ static int fuse_copy_fill(struct fuse_copy_state *cs) err = iov_iter_get_pages2(cs->iter, &page, PAGE_SIZE, 1, &off); if (err < 0) return err; - BUG_ON(!err); + if (!err) + return -EIO; cs->len = err; cs->offset = off; cs->pg = page; @@ -1147,32 +1151,32 @@ static int fuse_try_move_folio(struct fuse_copy_state *cs, struct folio **foliop if (cs->len != folio_size(oldfolio)) goto out_fallback; - if (!pipe_buf_try_steal(cs->pipe, buf)) - goto out_fallback; - newfolio = page_folio(buf->page); - folio_clear_uptodate(newfolio); - folio_clear_mappedtodisk(newfolio); - if (folio_test_large(newfolio)) - goto out_fallback_unlock; - - if (fuse_check_folio(newfolio) != 0) - goto out_fallback_unlock; + goto out_fallback; /* * This is a new and locked page, it shouldn't be mapped or * have any special flags on it */ if (WARN_ON(folio_mapped(oldfolio))) - goto out_fallback_unlock; + goto out_fallback; if (WARN_ON(folio_has_private(oldfolio))) - goto out_fallback_unlock; + goto out_fallback; if (WARN_ON(folio_test_dirty(oldfolio) || folio_test_writeback(oldfolio))) - goto out_fallback_unlock; + goto out_fallback; if (WARN_ON(folio_test_mlocked(oldfolio))) + goto out_fallback; + + if (!pipe_buf_try_steal(cs->pipe, buf)) + goto out_fallback; + + folio_clear_uptodate(newfolio); + folio_clear_mappedtodisk(newfolio); + + if (fuse_check_folio(newfolio) != 0) goto out_fallback_unlock; err = lock_request(cs->req); @@ -1252,31 +1256,10 @@ static int fuse_ref_folio(struct fuse_copy_state *cs, struct folio *folio, * done atomically */ int fuse_copy_folio(struct fuse_copy_state *cs, struct folio **foliop, - unsigned offset, unsigned count, int zeroing) + unsigned offset, unsigned count) { int err; struct folio *folio = *foliop; - size_t size; - - if (folio) { - size = folio_size(folio); - if (zeroing && count < size) { - /* - * When the copy is skipped the folio already holds the - * payload, so only the bytes outside [offset, offset + - * count) may be zeroed. - * - * Otherwise, the whole folio is cleared first so that a - * failed copy leaves zeros rather than stale folio - * contents. - */ - if (cs->skip_folio_copy) - folio_zero_segments(folio, 0, offset, - offset + count, size); - else - folio_zero_range(folio, 0, size); - } - } while (!cs->skip_folio_copy && count) { if (cs->write && cs->pipebufs && folio) { @@ -1293,7 +1276,7 @@ int fuse_copy_folio(struct fuse_copy_state *cs, struct folio **foliop, } } else if (!cs->len) { if (cs->move_folios && folio && - offset == 0 && count == size) { + offset == 0 && count == folio_size(folio)) { err = fuse_try_move_folio(cs, foliop); if (err <= 0) return err; @@ -1309,7 +1292,8 @@ int fuse_copy_folio(struct fuse_copy_state *cs, struct folio **foliop, unsigned int copy = count; unsigned int bytes_copied; - if (folio_test_highmem(folio) && count > PAGE_SIZE - offset_in_page(offset)) + if (folio_test_partial_kmap(folio) && + count > PAGE_SIZE - offset_in_page(offset)) copy = PAGE_SIZE - offset_in_page(offset); bytes_copied = fuse_copy_do(cs, &buf, ©); @@ -1334,10 +1318,25 @@ static int fuse_copy_folios(struct fuse_copy_state *cs, unsigned nbytes, for (i = 0; i < ap->num_folios && (nbytes || zeroing); i++) { int err; + struct folio *folio = ap->folios[i]; unsigned int offset = ap->descs[i].offset; - unsigned int count = min(nbytes, ap->descs[i].length); + unsigned int length = ap->descs[i].length; + unsigned int count = min(nbytes, length); - err = fuse_copy_folio(cs, &ap->folios[i], offset, count, zeroing); + /* + * The reply may be shorter than what was asked for. The full + * descs[i].length is reported as read, so the tail bytes the + * server did not send are about to be marked uptodate and need + * to be zeroed. + * + * Only [offset, offset + length) can be touched since the + * rest of the folio can hold blocks that are already uptodate + * or dirty, and clearing those would lose data. + */ + if (folio && zeroing && count < length) + folio_zero_range(folio, offset + count, length - count); + + err = fuse_copy_folio(cs, &ap->folios[i], offset, count); if (err) return err; @@ -2325,39 +2324,64 @@ static long fuse_dev_ioctl_clone(struct file *file, __u32 __user *argp) return 0; } +static struct fuse_conn *fuse_get_conn_for_passthrough(struct file *file) +{ + struct fuse_dev *fud = fuse_get_dev(file); + + if (IS_ERR(fud)) + return ERR_CAST(fud); + + if (!smp_load_acquire(&fud->chan->initialized)) + return ERR_PTR(-ENOTCONN); + + if (!IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) + return ERR_PTR(-EOPNOTSUPP); + + return fud->chan->conn; +} + static long fuse_dev_ioctl_backing_open(struct file *file, struct fuse_backing_map __user *argp) { - struct fuse_dev *fud = fuse_get_dev(file); + struct fuse_conn *fc = fuse_get_conn_for_passthrough(file); struct fuse_backing_map map; - if (IS_ERR(fud)) - return PTR_ERR(fud); - - if (!IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) - return -EOPNOTSUPP; + if (IS_ERR(fc)) + return PTR_ERR(fc); if (copy_from_user(&map, argp, sizeof(map))) return -EFAULT; - return fuse_backing_open(fud->chan->conn, &map); + return fuse_backing_open(fc, &map); +} + +static long fuse_dev_ioctl_backing_create(struct file *file, + struct fuse_backing_create_in __user *argp) +{ + struct fuse_conn *fc = fuse_get_conn_for_passthrough(file); + struct fuse_backing_create_in map; + + if (IS_ERR(fc)) + return PTR_ERR(fc); + + if (copy_from_user(&map, argp, sizeof(map))) + return -EFAULT; + + return fuse_backing_open_64(fc, &map); } static long fuse_dev_ioctl_backing_close(struct file *file, __u32 __user *argp) { - struct fuse_dev *fud = fuse_get_dev(file); + struct fuse_conn *fc = fuse_get_conn_for_passthrough(file); int backing_id; - if (IS_ERR(fud)) - return PTR_ERR(fud); - - if (!IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) - return -EOPNOTSUPP; + if (IS_ERR(fc)) + return PTR_ERR(fc); if (get_user(backing_id, argp)) return -EFAULT; - return fuse_backing_close(fud->chan->conn, backing_id); + return fuse_backing_close(fc, backing_id); } static long fuse_dev_ioctl_sync_init(struct file *file) @@ -2383,6 +2407,9 @@ static long fuse_dev_ioctl(struct file *file, unsigned int cmd, case FUSE_DEV_IOC_BACKING_OPEN: return fuse_dev_ioctl_backing_open(file, argp); + case FUSE_DEV_IOC_BACKING_CREATE: + return fuse_dev_ioctl_backing_create(file, argp); + case FUSE_DEV_IOC_BACKING_CLOSE: return fuse_dev_ioctl_backing_close(file, argp); diff --git a/fs/fuse/dev.h b/fs/fuse/dev.h index 8d25378c0918..dbffd5bed2f5 100644 --- a/fs/fuse/dev.h +++ b/fs/fuse/dev.h @@ -14,6 +14,7 @@ struct fuse_dev; struct fuse_args; struct fuse_copy_state; struct fuse_backing_map; +struct fuse_backing_create_in; struct file; struct folio; enum fuse_notify_code; @@ -87,10 +88,12 @@ int fuse_notify(struct fuse_conn *fc, enum fuse_notify_code code, int fuse_backing_open(struct fuse_conn *fc, struct fuse_backing_map *map); int fuse_backing_close(struct fuse_conn *fc, int backing_id); +int fuse_backing_open_64(struct fuse_conn *fc, struct fuse_backing_create_in *map); +int fuse_backing_close_64(struct fuse_conn *fc, u64 backing_id); int fuse_copy_one(struct fuse_copy_state *cs, void *val, unsigned size); int fuse_copy_folio(struct fuse_copy_state *cs, struct folio **foliop, - unsigned offset, unsigned count, int zeroing); + unsigned offset, unsigned count); void fuse_copy_finish(struct fuse_copy_state *cs); #ifdef CONFIG_FUSE_IO_URING diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c index c6dd420c4034..460776cd69cd 100644 --- a/fs/fuse/dev_uring.c +++ b/fs/fuse/dev_uring.c @@ -893,6 +893,12 @@ static int fuse_uring_args_to_ring(struct fuse_req *req, fuse_copy_finish(&cs); if (err) { pr_info_ratelimited("%s fuse_copy_args failed\n", __func__); + /* + * fuse-uring does not use pipe buffers, so -EIO here can only + * mean the in-args did not fit ent->payload + */ + if (err == -EIO && args->opcode == FUSE_SETXATTR) + err = -E2BIG; return err; } diff --git a/fs/fuse/dir.c b/fs/fuse/dir.c index 1759f7fd72b0..da23ccd8685c 100644 --- a/fs/fuse/dir.c +++ b/fs/fuse/dir.c @@ -876,6 +876,7 @@ static int fuse_create_open(const struct mnt_idmap *idmap, struct inode *dir, args.out_args[0].value = &outentry; /* Store outarg for fuse_finish_open() */ outopenp = &ff->args->open_outarg; + args.out_argvar = true; /* compat */ args.out_args[1].size = sizeof(*outopenp); args.out_args[1].value = outopenp; @@ -885,7 +886,7 @@ static int fuse_create_open(const struct mnt_idmap *idmap, struct inode *dir, err = fuse_simple_idmap_request(idmap, fm, &args); free_ext_value(&args); - if (err) + if (err < 0) goto out_free_ff; err = -EIO; @@ -1031,7 +1032,6 @@ static struct dentry *create_new_entry(const struct mnt_idmap *idmap, struct fus } kfree(forget); - d_drop(entry); d = d_splice_alias(inode, entry); if (IS_ERR(d)) return d; @@ -2175,12 +2175,15 @@ int fuse_do_setattr(const struct mnt_idmap *idmap, struct dentry *dentry, is_truncate = true; } - if (FUSE_IS_DAX(inode) && is_truncate) { + if (FUSE_IS_VDAX(inode) && is_truncate) { filemap_invalidate_lock(mapping); fault_blocked = true; - err = fuse_dax_break_layouts(inode, 0, -1); + err = fuse_vdax_break_layouts(inode, 0, -1); if (err) goto unlock; + } else if (IS_DAX(inode) && is_truncate) { + /* No truncate for DAX yet */ + return -EINVAL; } if (attr->ia_valid & ATTR_OPEN) { diff --git a/fs/fuse/ext_map.c b/fs/fuse/ext_map.c new file mode 100644 index 000000000000..7f04edc33fe3 --- /dev/null +++ b/fs/fuse/ext_map.c @@ -0,0 +1,376 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include "fuse_i.h" +#include +#include +#include +#include +#include +#include + +struct fuse_iext { + u64 start; + u64 end; /* exclusive */ + struct fuse_backing *backing; + u64 backing_offset; +}; + +void fuse_ext_map_destroy(struct fuse_backing *fb) +{ + unsigned int i; + + for (i = 0; i < fb->nr_extents; i++) + fuse_backing_put(fb->extents[i].backing); + + kvfree(fb->extents); + fb->extents = NULL; + fb->nr_extents = 0; +} + +/* + * Resolve @offset to an extent. + * + * A uniform map (all extents the same length, contiguous from offset zero) is + * resolved with a single array index. A cyclic map is uniform by construction; + * the index wraps and *ncycle is the number of complete cycles before @offset. + * Only an irregular map needs a search: a binary search of the sorted array. + */ +static struct fuse_iext *fuse_find_extent(struct fuse_backing *fb, u64 offset, + u64 *ncycle) +{ + struct fuse_iext *ext = fb->extents; + unsigned int lo, hi; + u64 idx; + + *ncycle = 0; + + /* If the extents are uniform... */ + if (fb->ext_len) { + idx = div64_u64(offset, fb->ext_len); + + if (fb->cycle_length) { + u32 rem; + + *ncycle = div_u64_rem(idx, fb->nr_extents, &rem); + idx = rem; + } else if (idx >= fb->nr_extents) { + return NULL; + } + return &ext[idx]; + } + + /* Extents are not uniform: binary search the extents array */ + lo = 0; + hi = fb->nr_extents; + while (lo < hi) { + unsigned int mid = lo + (hi - lo) / 2; + + if (offset < ext[mid].start) + hi = mid; + else if (offset >= ext[mid].end) + lo = mid + 1; + else + return &ext[mid]; + } + + return NULL; +} + +static int fuse_add_extent(struct fuse_conn *fc, struct fuse_iext *new_fie, + struct fuse_extent *ext) +{ + loff_t end; + u64 tmp; + + if (ext->reserved[0] || ext->reserved[1]) + return fuse_EIO("reserved fields set"); + + if (!PAGE_ALIGNED(ext->offset) || !PAGE_ALIGNED(ext->length) || !PAGE_ALIGNED(ext->addr)) + return fuse_EIO("not page aligned"); + + if (!ext->length) + return fuse_EIO("zero sized extent"); + + if (overflows_type(ext->offset, loff_t) || + check_add_overflow(ext->offset, ext->length, &end) || + check_add_overflow(ext->addr, ext->length, &tmp)) + return fuse_EIO("offset overflow"); + + new_fie->backing = fuse_backing_lookup(fc, ext->backing_id); + if (!new_fie->backing) + return fuse_EIO("backing not found"); + + if (new_fie->backing->type != FUSE_BACKING_DAXDEV) { + fuse_backing_put(new_fie->backing); + new_fie->backing = NULL; + return fuse_EIO("backing is not dax device"); + } + + new_fie->start = ext->offset; + new_fie->end = end; + new_fie->backing_offset = ext->addr; + + return 0; +} + +static int fuse_iext_cmp(const void *a, const void *b) +{ + const struct fuse_iext *x = a, *y = b; + + if (x->start < y->start) + return -1; + return x->start > y->start; +} + +/* + * fb has an array of extents; + * - Sort it to guarantee they are in file offset order + * - Check whether it is uniform (starts at ofs=0, all same size, no gaps) + * - Validate if cyclic + * + * Array-based extent lists have better cache locality than a tree + * (packed into cache lines). Uniform simple extent lists and all currently + * valid cyclic extent lists can be indexed in O(1) at fault time. And + * when search is needed, arrays can be bisected for balanced O(log2 n). + */ +static int fuse_ext_map_finish(struct fuse_backing *fb, bool cyclic) +{ + struct fuse_iext *ext = fb->extents; + unsigned int n = fb->nr_extents; + unsigned int i; + bool uniform; + u64 len; + + sort(ext, n, sizeof(*ext), fuse_iext_cmp, NULL); + + len = ext[0].end - ext[0].start; + uniform = ext[0].start == 0; + + /* Check for uniformity */ + for (i = 1; i < n; i++) { + if (ext[i].start < ext[i - 1].end) + return fuse_EIO("overlap"); + + if (ext[i].start != ext[i - 1].end || + ext[i].end - ext[i].start != len) + uniform = false; + } + + if (cyclic) { + if (!uniform || + check_mul_overflow(len, (u64)n, &fb->cycle_length)) + return -EINVAL; + } + + if (uniform) + fb->ext_len = len; + + return 0; +} + +static int fuse_ext_map_iomap_begin(struct inode *inode, loff_t offset, loff_t length, + unsigned int flags, struct iomap *iomap, struct iomap *srcmap) +{ + struct fuse_backing *fb = fuse_inode_backing(get_fuse_inode(inode)); + struct fuse_iext *fie; + u64 ncycle, seq_off, addr, tmp; + loff_t ext_len; + + if (!fb || fb->type != FUSE_BACKING_EXTMAP) + return fuse_EIO("missing or wrong type backing"); + + fie = fuse_find_extent(fb, offset, &ncycle); + if (!fie) + return fuse_EIO("missing mapping"); + + if (WARN_ON(fie->backing->type != FUSE_BACKING_DAXDEV)) + return fuse_EIO("wrong type backing for extent"); + + if (fie->backing->dax_error) { + fuse_make_bad(inode); + return fuse_EIO("dax error"); + } + + ext_len = fie->end - fie->start; + seq_off = ncycle * fb->cycle_length; + addr = fie->backing_offset; + if (check_mul_overflow(ncycle, ext_len, &tmp) || check_add_overflow(addr, tmp, &addr)) + return fuse_EIO("extent address overflow"); + + if (check_add_overflow(addr, ext_len, &tmp)) + return fuse_EIO("extent end overflow"); + + iomap->offset = fie->start + seq_off; + iomap->addr = addr; + iomap->length = ext_len; + iomap->dax_dev = fie->backing->dax_dev; + iomap->type = IOMAP_MAPPED; + iomap->flags = 0; + + return 0; +} + +static const struct iomap_ops fuse_ext_map_iomap_ops = { + .iomap_begin = fuse_ext_map_iomap_begin, +}; + +static vm_fault_t fuse_ext_map_huge_fault(struct vm_fault *vmf, unsigned int order) +{ + struct inode *inode = file_inode(vmf->vma->vm_file); + bool write_fault = (vmf->flags & FAULT_FLAG_WRITE) && (vmf->vma->vm_flags & VM_SHARED); + vm_fault_t ret; + unsigned long pfn; + + if (!IS_ENABLED(CONFIG_FS_DAX)) + return VM_FAULT_SIGBUS; + + if (WARN_ON_ONCE(!IS_DAX(inode))) + return VM_FAULT_SIGBUS; + + if (write_fault) { + sb_start_pagefault(inode->i_sb); + file_update_time(vmf->vma->vm_file); + } + + filemap_invalidate_lock_shared(inode->i_mapping); + + ret = dax_iomap_fault(vmf, order, &pfn, NULL, &fuse_ext_map_iomap_ops); + if (ret & VM_FAULT_NEEDDSYNC) + ret = dax_finish_sync_fault(vmf, order, pfn); + + filemap_invalidate_unlock_shared(inode->i_mapping); + + if (write_fault) + sb_end_pagefault(inode->i_sb); + + return ret; +} + +static vm_fault_t fuse_ext_map_fault(struct vm_fault *vmf) +{ + return fuse_ext_map_huge_fault(vmf, 0); +} + +static const struct vm_operations_struct fuse_ext_map_vm_ops = { + .fault = fuse_ext_map_fault, + .huge_fault = fuse_ext_map_huge_fault, + .page_mkwrite = fuse_ext_map_fault, + .pfn_mkwrite = fuse_ext_map_fault, +}; + +static void fuse_rw_clamp(struct kiocb *iocb, struct iov_iter *ubuf) +{ + struct inode *inode = iocb->ki_filp->f_mapping->host; + loff_t i_size = i_size_read(inode); + loff_t max_count = iocb->ki_pos >= i_size ? 0 : i_size - iocb->ki_pos; + + if (iov_iter_count(ubuf) > max_count) + iov_iter_truncate(ubuf, max_count); +} + +ssize_t fuse_ext_map_read_iter(struct kiocb *iocb, struct iov_iter *to) +{ + struct inode *inode = iocb->ki_filp->f_mapping->host; + ssize_t res; + + guard(rwsem_read)(&inode->i_rwsem); + + fuse_rw_clamp(iocb, to); + + if (!iov_iter_count(to)) + return 0; + + if (!IS_ENABLED(CONFIG_FS_DAX)) + return -EIO; + + res = dax_iomap_rw(iocb, to, &fuse_ext_map_iomap_ops); + + file_accessed(iocb->ki_filp); + return res; +} + +ssize_t fuse_ext_map_write_iter(struct kiocb *iocb, struct iov_iter *from) +{ + ssize_t res; + + res = generic_write_checks(iocb, from); + if (res <= 0) + return res; + + fuse_rw_clamp(iocb, from); + + if (!iov_iter_count(from)) + return -ENOSPC; + + res = kiocb_modified(iocb); + if (res) + return res; + + if (!IS_ENABLED(CONFIG_FS_DAX)) + return -EIO; + + return dax_iomap_rw(iocb, from, &fuse_ext_map_iomap_ops); +} + +int fuse_ext_map_mmap(struct file *file, struct vm_area_struct *vma) +{ + file_accessed(file); + vma->vm_ops = &fuse_ext_map_vm_ops; + vm_flags_set(vma, VM_HUGEPAGE); + return 0; +} + +bool fuse_ext_map_is_dax(struct fuse_backing *fb) +{ + if (WARN_ON(fb->type != FUSE_BACKING_EXTMAP || !fb->nr_extents)) + return false; + + return fb->extents[0].backing->type == FUSE_BACKING_DAXDEV; +} + +int fuse_ext_map_populate(struct fuse_conn *fc, struct fuse_notify_backing_map_out *arg, + struct fuse_extent *ext) +{ + struct fuse_backing *fb; + unsigned int i; + int err; + + if (!arg->num_extents) + return fuse_EIO("no extents"); + + /* Adding extents to an existing extmap backing is not yet supported */ + if (!(arg->flags & FUSE_BACKING_MAP_CREATE)) + return -EINVAL; + + fb = kzalloc_obj(*fb); + if (!fb) + return -ENOMEM; + + refcount_set(&fb->count, 1); + fb->type = FUSE_BACKING_EXTMAP; + fb->backing_id = arg->backing_id; + + err = -ENOMEM; + fb->extents = kvcalloc(arg->num_extents, sizeof(*fb->extents), GFP_KERNEL); + if (!fb->extents) + goto err_put; + + for (i = 0; i < arg->num_extents; i++) { + err = fuse_add_extent(fc, &fb->extents[i], &ext[i]); + if (err) + goto err_put; + fb->nr_extents++; + } + + err = fuse_ext_map_finish(fb, arg->flags & FUSE_BACKING_MAP_CYCLIC); + if (err) + goto err_put; + + err = fuse_backing_add_64(fc, fb); + if (!err) + return 0; + +err_put: + fuse_backing_put(fb); + return err; +} diff --git a/fs/fuse/file.c b/fs/fuse/file.c index da6e1efab8a3..d5e1cb53d86a 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -28,6 +28,7 @@ static int fuse_send_open(struct fuse_mount *fm, u64 nodeid, { struct fuse_open_in inarg; FUSE_ARGS(args); + int res; memset(&inarg, 0, sizeof(inarg)); inarg.flags = open_flags & ~(O_CREAT | O_EXCL | O_NOCTTY); @@ -45,10 +46,13 @@ static int fuse_send_open(struct fuse_mount *fm, u64 nodeid, args.in_args[0].size = sizeof(inarg); args.in_args[0].value = &inarg; args.out_numargs = 1; + args.out_argvar = true; /* compat */ args.out_args[0].size = sizeof(*outargp); args.out_args[0].value = outargp; - return fuse_simple_request(fm, &args); + res = fuse_simple_request(fm, &args); + + return res < 0 ? res : 0; } struct fuse_file *fuse_file_alloc(struct fuse_mount *fm, bool release) @@ -119,7 +123,7 @@ static void fuse_file_put(struct fuse_file *ff, bool sync) * DAX inodes may need to issue a number of synchronous * request for clearing the mappings. */ - if (ra && ra->inode && FUSE_IS_DAX(ra->inode)) + if (ra && ra->inode && FUSE_IS_VDAX(ra->inode)) args->may_block = true; args->end = fuse_release_end; if (fuse_simple_background(ff->fm, args, @@ -256,26 +260,30 @@ static int fuse_open(struct inode *inode, struct file *file) int err; bool is_truncate = (file->f_flags & O_TRUNC) && fc->atomic_o_trunc; bool is_wb_truncate = is_truncate && fc->writeback_cache; - bool dax_truncate = is_truncate && FUSE_IS_DAX(inode); + bool vdax_truncate = is_truncate && FUSE_IS_VDAX(inode); if (fuse_is_bad(inode)) return -EIO; + /* No truncate for DAX yet */ + if ((file->f_flags & O_TRUNC) && IS_DAX(inode) && !FUSE_IS_VDAX(inode)) + return -EINVAL; + err = generic_file_open(inode, file); if (err) return err; - if (is_wb_truncate || dax_truncate) + if (is_wb_truncate || vdax_truncate) inode_lock(inode); - if (dax_truncate) { + if (vdax_truncate) { filemap_invalidate_lock(inode->i_mapping); - err = fuse_dax_break_layouts(inode, 0, -1); + err = fuse_vdax_break_layouts(inode, 0, -1); if (err) goto out_unlock; } - if (is_wb_truncate || dax_truncate) + if (is_wb_truncate || vdax_truncate) fuse_set_nowrite(inode); err = fuse_do_open(fm, get_node_id(inode), file, false); @@ -288,18 +296,18 @@ static int fuse_open(struct inode *inode, struct file *file) fuse_truncate_update_attr(inode, file); } - if (is_wb_truncate || dax_truncate) + if (is_wb_truncate || vdax_truncate) fuse_release_nowrite(inode); if (!err) { if (is_truncate) truncate_pagecache(inode, 0); - else if (!(ff->open_flags & FOPEN_KEEP_CACHE)) + else if (!(ff->open_flags & FOPEN_KEEP_CACHE) && !IS_DAX(inode)) invalidate_inode_pages2(inode->i_mapping); } out_unlock: - if (dax_truncate) + if (vdax_truncate) filemap_invalidate_unlock(inode->i_mapping); - if (is_wb_truncate || dax_truncate) + if (is_wb_truncate || vdax_truncate) inode_unlock(inode); return err; @@ -311,7 +319,7 @@ static void fuse_prepare_release(struct fuse_inode *fi, struct fuse_file *ff, struct fuse_conn *fc = ff->fm->fc; struct fuse_release_args *ra = &ff->args->release_args; - if (fuse_file_passthrough(ff)) + if (fuse_is_passthrough(ff)) fuse_passthrough_release(ff, fuse_inode_backing(fi)); /* Inode is NULL on error path of fuse_create_open() */ @@ -547,6 +555,17 @@ static int fuse_fsync(struct file *file, loff_t start, loff_t end, if (fuse_is_bad(inode)) return -EIO; + if (IS_DAX(inode) && !fuse_inode_vdax(inode)) { + /* + * Note: the DAX fault path calls this via dax_finish_sync_fault() + * and taking inode lock in that context is prohibited. + * + * FIXME: need to flush CPU caches for the DAX memory range + * FIXME: need to sync metadata to the server + */ + return 0; + } + inode_lock(inode); /* @@ -689,7 +708,7 @@ static void fuse_aio_complete(struct fuse_io_priv *io, int err, ssize_t pos) struct address_space *mapping = io->iocb->ki_filp->f_mapping; ssize_t res = fuse_get_res_by_io(io); - if (res >= 0) { + if (res >= 0 && io->write) { struct fuse_conn *fc = get_fuse_conn(inode); struct fuse_inode *fi = get_fuse_inode(inode); @@ -864,18 +883,29 @@ static int fuse_do_readfolio(struct file *file, struct folio *folio, attr_ver = fuse_get_attr_version(fm->fc); - /* Don't overflow end offset */ - if (pos + (desc.length - 1) == LLONG_MAX) - desc.length--; + /* + * Don't overflow end offset. + * + * Ask the server for len - 1 bytes. desc.length still holds the full + * length. When the reply comes back, it will be one byte shorter than + * desc.length and fuse_copy_folios() will zero that last byte. + * + * For this reason, desc.length must not be decremented too. The caller + * reports the full length to iomap_finish_folio_read(), which marks + * every block it covers uptodate. Shortening the descriptor would + * suppress zeroing and leave the last byte holding stale data. + */ + if (pos + (len - 1) == LLONG_MAX) + len--; - fuse_read_args_fill(&ia, file, pos, desc.length, FUSE_READ); + fuse_read_args_fill(&ia, file, pos, len, FUSE_READ); res = fuse_simple_request(fm, &ia.ap.args); if (res < 0) return res; /* * Short read means EOF. If file size is larger, truncate it */ - if (res < desc.length) + if (res < len) fuse_short_read(inode, attr_ver, res, &ia.ap); return 0; @@ -1068,11 +1098,20 @@ static void fuse_send_readpages(struct fuse_io_args *ia, struct file *file, ap->args.page_zeroing = true; ap->args.page_replace = true; - /* Don't overflow end offset */ - if (pos + (count - 1) == LLONG_MAX) { + /* + * Don't overflow end offset. + * + * Ask the server for count - 1 bytes. The reply is then one byte + * shorter than what the descriptor lengths add up to, so + * fuse_copy_folios() zeroes the last byte when it walks the folios. + * + * ap->descs[] must not be decremented here. It is what + * fuse_readpages_end() reports back to iomap_finish_folio_read(), and + * iomap has already accounted the full descriptor length, so shortening + * it would leave ifs->read_bytes_pending nonzero and the folio locked. + */ + if (pos + (count - 1) == LLONG_MAX) count--; - ap->descs[ap->num_folios - 1].length--; - } WARN_ON((loff_t) (pos + count) < 0); fuse_read_args_fill(ia, file, pos, count, FUSE_READ); @@ -1653,8 +1692,8 @@ static int fuse_get_user_pages(struct fuse_args_pages *ap, struct iov_iter *ii, return ret < 0 ? ret : 0; } -ssize_t fuse_direct_io(struct fuse_io_priv *io, struct iov_iter *iter, - loff_t *ppos, int flags) +ssize_t fuse_do_direct_io(struct fuse_io_priv *io, struct iov_iter *iter, + loff_t *ppos, int flags) { int write = flags & FUSE_DIO_WRITE; int cuse = flags & FUSE_DIO_CUSE; @@ -1755,7 +1794,7 @@ ssize_t fuse_direct_io(struct fuse_io_priv *io, struct iov_iter *iter, return res > 0 ? res : err; } -EXPORT_SYMBOL_GPL(fuse_direct_io); +EXPORT_SYMBOL_GPL(fuse_do_direct_io); static ssize_t __fuse_direct_read(struct fuse_io_priv *io, struct iov_iter *iter, @@ -1764,7 +1803,7 @@ static ssize_t __fuse_direct_read(struct fuse_io_priv *io, ssize_t res; struct inode *inode = file_inode(io->iocb->ki_filp); - res = fuse_direct_io(io, iter, ppos, 0); + res = fuse_do_direct_io(io, iter, ppos, 0); fuse_invalidate_atime(inode); @@ -1806,8 +1845,8 @@ static ssize_t fuse_direct_write_iter(struct kiocb *iocb, struct iov_iter *from) } else { struct fuse_io_priv io = FUSE_IO_PRIV_SYNC(iocb); - res = fuse_direct_io(&io, from, &iocb->ki_pos, - FUSE_DIO_WRITE); + res = fuse_do_direct_io(&io, from, &iocb->ki_pos, + FUSE_DIO_WRITE); fuse_write_update_attr(inode, iocb->ki_pos, res); } if (res > 0 && mapping->nrpages) { @@ -1835,13 +1874,13 @@ static ssize_t fuse_file_read_iter(struct kiocb *iocb, struct iov_iter *to) if (fuse_is_bad(inode)) return -EIO; - if (FUSE_IS_DAX(inode)) - return fuse_dax_read_iter(iocb, to); + if (FUSE_IS_VDAX(inode)) + return fuse_vdax_read_iter(iocb, to); /* FOPEN_DIRECT_IO overrides FOPEN_PASSTHROUGH */ if (ff->open_flags & FOPEN_DIRECT_IO) return fuse_direct_read_iter(iocb, to); - else if (fuse_file_passthrough(ff)) + else if (fuse_is_passthrough(ff)) return fuse_passthrough_read_iter(iocb, to); else return fuse_cache_read_iter(iocb, to); @@ -1856,13 +1895,13 @@ static ssize_t fuse_file_write_iter(struct kiocb *iocb, struct iov_iter *from) if (fuse_is_bad(inode)) return -EIO; - if (FUSE_IS_DAX(inode)) - return fuse_dax_write_iter(iocb, from); + if (FUSE_IS_VDAX(inode)) + return fuse_vdax_write_iter(iocb, from); /* FOPEN_DIRECT_IO overrides FOPEN_PASSTHROUGH */ if (ff->open_flags & FOPEN_DIRECT_IO) return fuse_direct_write_iter(iocb, from); - else if (fuse_file_passthrough(ff)) + else if (fuse_is_passthrough(ff)) return fuse_passthrough_write_iter(iocb, from); else return fuse_cache_write_iter(iocb, from); @@ -1875,7 +1914,10 @@ static ssize_t fuse_splice_read(struct file *in, loff_t *ppos, struct fuse_file *ff = in->private_data; /* FOPEN_DIRECT_IO overrides FOPEN_PASSTHROUGH */ - if (fuse_file_passthrough(ff) && !(ff->open_flags & FOPEN_DIRECT_IO)) + + if (ff->open_flags & FOPEN_DIRECT_IO) + return copy_splice_read(in, ppos, pipe, len, flags); + else if (fuse_is_passthrough(ff)) return fuse_passthrough_splice_read(in, ppos, pipe, len, flags); else return filemap_splice_read(in, ppos, pipe, len, flags); @@ -1887,7 +1929,7 @@ static ssize_t fuse_splice_write(struct pipe_inode_info *pipe, struct file *out, struct fuse_file *ff = out->private_data; /* FOPEN_DIRECT_IO overrides FOPEN_PASSTHROUGH */ - if (fuse_file_passthrough(ff) && !(ff->open_flags & FOPEN_DIRECT_IO)) + if (fuse_is_passthrough(ff) && !(ff->open_flags & FOPEN_DIRECT_IO)) return fuse_passthrough_splice_write(pipe, out, ppos, len, flags); else return iter_file_splice_write(pipe, out, ppos, len, flags); @@ -2327,6 +2369,9 @@ static int fuse_launder_folio(struct folio *folio) .wb_ctx = &data, }; + if (fuse_is_bad(wpc.inode)) + return -EIO; + if (folio_clear_dirty_for_io(folio)) { err = iomap_writeback_folio(&wpc, folio); err = fuse_iomap_writeback_submit(&wpc, err); @@ -2391,18 +2436,17 @@ static int fuse_file_mmap(struct file *file, struct vm_area_struct *vma) struct fuse_file *ff = file->private_data; struct fuse_conn *fc = ff->fm->fc; struct inode *inode = file_inode(file); - int rc; /* DAX mmap is superior to direct_io mmap */ - if (FUSE_IS_DAX(inode)) - return fuse_dax_mmap(file, vma); + if (FUSE_IS_VDAX(inode)) + return fuse_vdax_mmap(file, vma); /* * If inode is in passthrough io mode, because it has some file open * in passthrough mode, either mmap to backing file or fail mmap, * because mixing cached mmap and passthrough io mode is not allowed. */ - if (fuse_file_passthrough(ff)) + if (fuse_is_passthrough(ff)) return fuse_passthrough_mmap(file, vma); else if (fuse_inode_backing(get_fuse_inode(inode))) return -ENODEV; @@ -2433,9 +2477,8 @@ static int fuse_file_mmap(struct file *file, struct vm_area_struct *vma) * After first mmap, the inode stays in caching io mode until * the direct_io file release. */ - rc = fuse_file_cached_io_open(inode, ff); - if (rc) - return rc; + if (!fuse_file_cached_io_open(inode, ff)) + return -ENODEV; } if ((vma->vm_flags & VM_SHARED) && (vma->vm_flags & VM_MAYWRITE)) @@ -2787,7 +2830,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter) } if (iov_iter_rw(iter) == WRITE) { - ret = fuse_direct_io(io, iter, &pos, FUSE_DIO_WRITE); + ret = fuse_do_direct_io(io, iter, &pos, FUSE_DIO_WRITE); fuse_invalidate_attr_mask(inode, FUSE_STATX_MODSIZE); } else { ret = __fuse_direct_read(io, iter, &pos); @@ -2844,7 +2887,7 @@ static long fuse_file_fallocate(struct file *file, int mode, loff_t offset, .mode = mode }; int err; - bool block_faults = FUSE_IS_DAX(inode) && + bool block_faults = FUSE_IS_VDAX(inode) && (!(mode & FALLOC_FL_KEEP_SIZE) || (mode & (FALLOC_FL_PUNCH_HOLE | FALLOC_FL_ZERO_RANGE))); @@ -2855,10 +2898,14 @@ static long fuse_file_fallocate(struct file *file, int mode, loff_t offset, if (fm->fc->no_fallocate) return -EOPNOTSUPP; + /* No size extend/hole punch for DAX yet */ + if (IS_DAX(inode) && !FUSE_IS_VDAX(inode)) + return -EOPNOTSUPP; + inode_lock(inode); if (block_faults) { filemap_invalidate_lock(inode->i_mapping); - err = fuse_dax_break_layouts(inode, 0, -1); + err = fuse_vdax_break_layouts(inode, 0, -1); if (err) goto out; } @@ -2978,14 +3025,16 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in, /* * Write out dirty pages in the destination file before sending the COPY - * request to userspace. After the request is completed, truncate off - * pages (including partial ones) from the cache that have been copied, - * since these contain stale data at that point. + * request to userspace. After the request is completed, drop the + * folios covering the copied range from the cache, since these contain + * stale data at that point. * - * This should be mostly correct, but if the COPY writes to partial - * pages (at the start or end) and the parts not covered by the COPY are + * This should be mostly correct, but if the COPY writes to a partial + * folio (at the start or end) and the parts not covered by the COPY are * written through a memory map after calling fuse_writeback_range(), - * then these partial page modifications will be lost on truncation. + * then that folio is laundered before it is dropped, so the memory map + * modifications are written back over the range the server just copied + * into and the copied data is lost. * * It is unlikely that someone would rely on such mixed style * modifications. Yet this does give less guarantees than if the @@ -3019,7 +3068,7 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in, } err = fuse_simple_request(fm, &args); if (err == -ENOSYS) { - if (fc->no_copy_file_range_64) { + if (args.opcode == FUSE_COPY_FILE_RANGE) { fc->no_copy_file_range = 1; err = -EOPNOTSUPP; } else { @@ -3030,7 +3079,7 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in, if (err) goto out; - bytes_copied = fc->no_copy_file_range_64 ? + bytes_copied = args.opcode == FUSE_COPY_FILE_RANGE ? outarg.size : outarg_64.bytes_copied; if (bytes_copied > len) { @@ -3038,9 +3087,10 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in, goto out; } - truncate_inode_pages_range(inode_out->i_mapping, - ALIGN_DOWN(pos_out, PAGE_SIZE), - ALIGN(pos_out + bytes_copied, PAGE_SIZE) - 1); + if (bytes_copied) + invalidate_inode_pages2_range(inode_out->i_mapping, + pos_out >> PAGE_SHIFT, + (pos_out + bytes_copied - 1) >> PAGE_SHIFT); file_update_time(file_out); fuse_write_update_attr(inode_out, pos_out + bytes_copied, bytes_copied); @@ -3073,6 +3123,16 @@ static ssize_t fuse_copy_file_range(struct file *src_file, loff_t src_off, return ret; } +static int fuse_file_fadvise(struct file *file, loff_t offset, loff_t len, int advice) +{ + struct fuse_file *ff = file->private_data; + + if (fuse_is_passthrough(ff)) + return fuse_passthrough_fadvise(file, offset, len, advice); + + return generic_fadvise(file, offset, len, advice); +} + static const struct file_operations fuse_file_operations = { .llseek = fuse_file_llseek, .read_iter = fuse_file_read_iter, @@ -3093,6 +3153,7 @@ static const struct file_operations fuse_file_operations = { .fallocate = fuse_file_fallocate, .copy_file_range = fuse_copy_file_range, .setlease = generic_setlease, + .fadvise = fuse_file_fadvise, }; static const struct address_space_operations fuse_file_aops = { @@ -3113,6 +3174,7 @@ void fuse_init_file_inode(struct inode *inode, unsigned int flags) { struct fuse_inode *fi = get_fuse_inode(inode); struct fuse_conn *fc = get_fuse_conn(inode); + unsigned int max_folio_pages; inode->i_fop = &fuse_file_operations; inode->i_data.a_ops = &fuse_file_aops; @@ -3126,6 +3188,20 @@ void fuse_init_file_inode(struct inode *inode, unsigned int flags) init_waitqueue_head(&fi->page_waitq); init_waitqueue_head(&fi->direct_io_waitq); - if (IS_ENABLED(CONFIG_FUSE_DAX)) - fuse_dax_inode_init(inode, flags); + if (IS_ENABLED(CONFIG_FUSE_VDAX)) + fuse_vdax_inode_init(inode, flags); + + if (FUSE_IS_VDAX(inode)) + return; + + /* + * A folio is never split across requests so cap the order so that one + * always fits in a single request. + */ + max_folio_pages = min3(fc->max_write >> PAGE_SHIFT, + fc->max_read >> PAGE_SHIFT, fc->max_pages); + + if (max_folio_pages) + mapping_set_folio_order_range(inode->i_mapping, 0, + ilog2(max_folio_pages)); } diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h index 9bf4c738bd74..ec5ccdf18a25 100644 --- a/fs/fuse/fuse_i.h +++ b/fs/fuse/fuse_i.h @@ -11,6 +11,8 @@ # define pr_fmt(fmt) "fuse: " fmt #endif +#define fuse_EIO(fmt, ...) (pr_notice_once("%s: " fmt "\n", __func__, ##__VA_ARGS__), -EIO) + #include "args.h" #include #include @@ -22,7 +24,7 @@ #include #include #include -#include +#include #include #include #include @@ -30,6 +32,7 @@ #include #include #include +#include /** Default max number of pages that can be used in a single read request */ #define FUSE_DEFAULT_MAX_PAGES_PER_REQ 32 @@ -86,11 +89,36 @@ struct fuse_submount_lookup { struct fuse_forget_link *forget; }; +enum fuse_backing_type { + FUSE_BACKING_PATH, + FUSE_BACKING_DAXDEV, + FUSE_BACKING_EXTMAP, +}; + +struct fuse_iext; + /* Container for data related to mapping to backing file */ struct fuse_backing { - struct file *file; - const struct cred *cred; + enum fuse_backing_type type; + union { + struct { + struct path path; + const struct cred *cred; + }; + struct { + struct dax_device *dax_dev; + bool dax_error; + }; + struct { + struct fuse_iext *extents; + unsigned int nr_extents; + u64 ext_len; + u64 cycle_length; + }; + }; + u64 backing_id; + struct rhash_head hash_node; /* refcount */ refcount_t count; struct rcu_head rcu; @@ -218,11 +246,11 @@ struct fuse_inode { /** @lock: Lock to protect write-related fields */ spinlock_t lock; -#ifdef CONFIG_FUSE_DAX +#ifdef CONFIG_FUSE_VDAX /** - * @dax: Dax specific inode data + * @vdax: Virtiofs DAX specific inode data */ - struct fuse_inode_dax *dax; + struct fuse_inode_vdax *vdax; #endif /** @submount_lookup: Submount specific lookup tracking */ struct fuse_submount_lookup *submount_lookup; @@ -364,16 +392,16 @@ struct fuse_io_priv { .iocb = i, \ } -enum fuse_dax_mode { - FUSE_DAX_INODE_DEFAULT, /* default */ - FUSE_DAX_ALWAYS, /* "-o dax=always" */ - FUSE_DAX_NEVER, /* "-o dax=never" */ - FUSE_DAX_INODE_USER, /* "-o dax=inode" */ +enum fuse_vdax_mode { + FUSE_VDAX_INODE_DEFAULT, /* default */ + FUSE_VDAX_ALWAYS, /* "-o dax=always" */ + FUSE_VDAX_NEVER, /* "-o dax=never" */ + FUSE_VDAX_INODE_USER, /* "-o dax=inode" */ }; -static inline bool fuse_is_inode_dax_mode(enum fuse_dax_mode mode) +static inline bool fuse_is_inode_vdax_mode(enum fuse_vdax_mode mode) { - return mode == FUSE_DAX_INODE_DEFAULT || mode == FUSE_DAX_INODE_USER; + return mode == FUSE_VDAX_INODE_DEFAULT || mode == FUSE_VDAX_INODE_USER; } struct fuse_fs_context { @@ -391,13 +419,14 @@ struct fuse_fs_context { bool no_control:1; bool no_force_umount:1; bool legacy_opts_show:1; - enum fuse_dax_mode dax_mode; + bool syncfs_capable:1; + enum fuse_vdax_mode vdax_mode; unsigned int max_read; unsigned int blksize; const char *subtype; - /* DAX device, may be NULL */ - struct dax_device *dax_dev; + /* Virtiofs DAX device, may be NULL */ + struct dax_device *vdax_dev; }; struct fuse_sync_bucket { @@ -675,6 +704,14 @@ struct fuse_conn { /** @sync_fs: Propagate syncfs() to server */ unsigned int sync_fs:1; + /** + * @syncfs_capable: the privilege required to honor FUSE_HAS_SYNCFS was + * present when /dev/fuse was opened (CAP_SYS_ADMIN in the initial user + * namespace), i.e. the same privilege that mounting virtiofs/fuseblk + * requires. + */ + unsigned int syncfs_capable:1; + /** @init_security: Initialize security xattrs when creating a new inode */ unsigned int init_security:1; @@ -684,8 +721,8 @@ struct fuse_conn { */ unsigned int create_supp_group:1; - /** @inode_dax: Does the filesystem support per inode DAX? */ - unsigned int inode_dax:1; + /** @inode_vdax: Does the filesystem support per inode virtiofs DAX? */ + unsigned int inode_vdax:1; /** @no_tmpfile: Is tmpfile not implemented by fs? */ unsigned int no_tmpfile:1; @@ -702,6 +739,9 @@ struct fuse_conn { /** @passthrough: Passthrough support for read/write IO */ unsigned int passthrough:1; + /** @backing_id_64: Backing ID is 64 bit and allocated by the server */ + bool backing_id_64:1; + /** @use_pages_for_kvec_io: Use pages instead of pointer for kernel I/O */ unsigned int use_pages_for_kvec_io:1; @@ -744,12 +784,12 @@ struct fuse_conn { */ struct rw_semaphore killsb; -#ifdef CONFIG_FUSE_DAX - /** @dax_mode: Dax mode */ - enum fuse_dax_mode dax_mode; +#ifdef CONFIG_FUSE_VDAX + /** @vdax_mode: Virtiofs DAX mode */ + enum fuse_vdax_mode vdax_mode; - /** @dax: Dax specific conn data, non-NULL if DAX is enabled */ - struct fuse_conn_dax *dax; + /** @dax: Dax specific conn data, non-NULL if virtiofs DAX is enabled */ + struct fuse_conn_vdax *vdax; #endif /** @mounts: List of filesystems using this connection */ @@ -759,8 +799,14 @@ struct fuse_conn { struct fuse_sync_bucket __rcu *curr_bucket; #ifdef CONFIG_FUSE_PASSTHROUGH - /** @backing_files_map: IDR for backing files ids */ - struct idr backing_files_map; + /* Selected by backing_id_64 */ + union { + /** @backing_files_map: IDR for backing files ids */ + struct idr backing_files_map; + + /** @backing_64_ht: 64 bit ID lookup hash table */ + struct rhashtable backing_64_ht; + }; #endif }; @@ -1167,17 +1213,17 @@ int fuse_do_open(struct fuse_mount *fm, u64 nodeid, struct file *file, bool isdir); /* - * fuse_direct_io() flags + * fuse_do_direct_io() flags */ /** If set, it is WRITE; otherwise - READ */ #define FUSE_DIO_WRITE (1 << 0) -/** CUSE pass fuse_direct_io() a file which f_mapping->host is not from FUSE */ +/** CUSE pass fuse_do_direct_io() a file which f_mapping->host is not from FUSE */ #define FUSE_DIO_CUSE (1 << 1) -ssize_t fuse_direct_io(struct fuse_io_priv *io, struct iov_iter *iter, - loff_t *ppos, int flags); +ssize_t fuse_do_direct_io(struct fuse_io_priv *io, struct iov_iter *iter, + loff_t *ppos, int flags); long fuse_do_ioctl(struct file *file, unsigned int cmd, unsigned long arg, unsigned int flags); long fuse_ioctl_common(struct file *file, unsigned int cmd, @@ -1217,21 +1263,33 @@ void fuse_free_conn(struct fuse_conn *fc); /* dax.c */ -#define FUSE_IS_DAX(inode) (IS_ENABLED(CONFIG_FUSE_DAX) && IS_DAX(inode)) +static inline bool fuse_inode_vdax(struct inode *inode) +{ +#ifdef CONFIG_FUSE_VDAX + return get_fuse_inode(inode)->vdax; +#else + return false; +#endif +} -ssize_t fuse_dax_read_iter(struct kiocb *iocb, struct iov_iter *to); -ssize_t fuse_dax_write_iter(struct kiocb *iocb, struct iov_iter *from); -int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma); -int fuse_dax_break_layouts(struct inode *inode, u64 dmap_start, u64 dmap_end); -int fuse_dax_conn_alloc(struct fuse_conn *fc, enum fuse_dax_mode mode, - struct dax_device *dax_dev); -void fuse_dax_conn_free(struct fuse_conn *fc); -bool fuse_dax_inode_alloc(struct super_block *sb, struct fuse_inode *fi); -void fuse_dax_inode_init(struct inode *inode, unsigned int flags); -void fuse_dax_inode_cleanup(struct inode *inode); -void fuse_dax_dontcache(struct inode *inode, unsigned int flags); -bool fuse_dax_check_alignment(struct fuse_conn *fc, unsigned int map_alignment); -void fuse_dax_cancel_work(struct fuse_conn *fc); +static inline bool FUSE_IS_VDAX(struct inode *inode) +{ + return fuse_inode_vdax(inode) && IS_DAX(inode); +} + +ssize_t fuse_vdax_read_iter(struct kiocb *iocb, struct iov_iter *to); +ssize_t fuse_vdax_write_iter(struct kiocb *iocb, struct iov_iter *from); +int fuse_vdax_mmap(struct file *file, struct vm_area_struct *vma); +int fuse_vdax_break_layouts(struct inode *inode, u64 dmap_start, u64 dmap_end); +int fuse_vdax_conn_alloc(struct fuse_conn *fc, enum fuse_vdax_mode mode, + struct dax_device *vdax_dev); +void fuse_vdax_conn_free(struct fuse_conn *fc); +bool fuse_vdax_inode_alloc(struct super_block *sb, struct fuse_inode *fi); +void fuse_vdax_inode_init(struct inode *inode, unsigned int flags); +void fuse_vdax_inode_cleanup(struct inode *inode); +void fuse_vdax_dontcache(struct inode *inode, unsigned int flags); +bool fuse_vdax_check_alignment(struct fuse_conn *fc, unsigned int map_alignment); +void fuse_vdax_cancel_work(struct fuse_conn *fc); /* ioctl.c */ long fuse_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg); @@ -1242,7 +1300,7 @@ int fuse_fileattr_set(const struct mnt_idmap *idmap, struct dentry *dentry, struct file_kattr *fa); /* iomode.c */ -int fuse_file_cached_io_open(struct inode *inode, struct fuse_file *ff); +bool fuse_file_cached_io_open(struct inode *inode, struct fuse_file *ff); int fuse_inode_uncached_io_start(struct fuse_inode *fi, struct fuse_backing *fb); void fuse_inode_uncached_io_end(struct fuse_inode *fi); @@ -1258,27 +1316,19 @@ void fuse_file_release(struct inode *inode, struct fuse_file *ff, /* backing.c */ #ifdef CONFIG_FUSE_PASSTHROUGH -struct fuse_backing *fuse_backing_get(struct fuse_backing *fb); void fuse_backing_put(struct fuse_backing *fb); -struct fuse_backing *fuse_backing_lookup(struct fuse_conn *fc, int backing_id); #else -static inline struct fuse_backing *fuse_backing_get(struct fuse_backing *fb) -{ - return NULL; -} - static inline void fuse_backing_put(struct fuse_backing *fb) { } -static inline struct fuse_backing *fuse_backing_lookup(struct fuse_conn *fc, - int backing_id) -{ - return NULL; -} #endif +struct fuse_backing *fuse_backing_lookup(struct fuse_conn *fc, u64 backing_id); +int fuse_backing_add_64(struct fuse_conn *fc, struct fuse_backing *fb); +bool fuse_backing_is_dax(struct fuse_backing *fb); void fuse_backing_files_init(struct fuse_conn *fc); +void fuse_backing_files_init_64(struct fuse_conn *fc); void fuse_backing_files_free(struct fuse_conn *fc); /* passthrough.c */ @@ -1301,16 +1351,12 @@ static inline struct fuse_backing *fuse_inode_backing_set(struct fuse_inode *fi, #endif } -struct fuse_backing *fuse_passthrough_open(struct file *file, int backing_id); +int fuse_passthrough_open(struct file *file, struct fuse_backing *fb); void fuse_passthrough_release(struct fuse_file *ff, struct fuse_backing *fb); -static inline struct file *fuse_file_passthrough(struct fuse_file *ff) +static inline bool fuse_is_passthrough(struct fuse_file *ff) { -#ifdef CONFIG_FUSE_PASSTHROUGH - return ff->passthrough; -#else - return NULL; -#endif + return IS_ENABLED(CONFIG_FUSE_PASSTHROUGH) && (ff->open_flags & FOPEN_PASSTHROUGH); } ssize_t fuse_passthrough_read_iter(struct kiocb *iocb, struct iov_iter *iter); @@ -1322,6 +1368,7 @@ ssize_t fuse_passthrough_splice_write(struct pipe_inode_info *pipe, struct file *out, loff_t *ppos, size_t len, unsigned int flags); ssize_t fuse_passthrough_mmap(struct file *file, struct vm_area_struct *vma); +int fuse_passthrough_fadvise(struct file *file, loff_t offset, loff_t len, int advice); #ifdef CONFIG_SYSCTL extern int fuse_sysctl_register(void); @@ -1331,4 +1378,13 @@ extern void fuse_sysctl_unregister(void); #define fuse_sysctl_unregister() do { } while (0) #endif /* CONFIG_SYSCTL */ +/* ext_map.c */ + +void fuse_ext_map_destroy(struct fuse_backing *fb); +ssize_t fuse_ext_map_write_iter(struct kiocb *iocb, struct iov_iter *from); +ssize_t fuse_ext_map_read_iter(struct kiocb *iocb, struct iov_iter *to); +int fuse_ext_map_mmap(struct file *file, struct vm_area_struct *vma); +bool fuse_ext_map_is_dax(struct fuse_backing *fb); +int fuse_ext_map_populate(struct fuse_conn *fc, struct fuse_notify_backing_map_out *arg, + struct fuse_extent *ext); #endif /* _FS_FUSE_I_H */ diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c index e9552be3637b..1341b681d0ed 100644 --- a/fs/fuse/inode.c +++ b/fs/fuse/inode.c @@ -100,7 +100,7 @@ static struct inode *fuse_alloc_inode(struct super_block *sb) if (!fi->forget) goto out_free; - if (IS_ENABLED(CONFIG_FUSE_DAX) && !fuse_dax_inode_alloc(sb, fi)) + if (IS_ENABLED(CONFIG_FUSE_VDAX) && !fuse_vdax_inode_alloc(sb, fi)) goto out_free_forget; if (IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) @@ -121,12 +121,9 @@ static void fuse_free_inode(struct inode *inode) mutex_destroy(&fi->mutex); kfree(fi->forget); -#ifdef CONFIG_FUSE_DAX - kfree(fi->dax); +#ifdef CONFIG_FUSE_VDAX + kfree(fi->vdax); #endif - if (IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) - fuse_backing_put(fuse_inode_backing(fi)); - kmem_cache_free(fuse_inode_cachep, fi); } @@ -148,7 +145,7 @@ static void fuse_evict_inode(struct inode *inode) /* Will write inode on close/munmap and in all other dirtiers */ WARN_ON(inode_state_read_once(inode) & I_DIRTY_INODE); - if (FUSE_IS_DAX(inode)) + if (IS_DAX(inode)) dax_break_layout_final(inode); truncate_inode_pages_final(&inode->i_data); @@ -156,8 +153,8 @@ static void fuse_evict_inode(struct inode *inode) if (inode->i_sb->s_flags & SB_ACTIVE) { struct fuse_conn *fc = get_fuse_conn(inode); - if (FUSE_IS_DAX(inode)) - fuse_dax_inode_cleanup(inode); + if (FUSE_IS_VDAX(inode)) + fuse_vdax_inode_cleanup(inode); if (fi->nlookup) { fuse_chan_queue_forget(fc->chan, fi->forget, fi->nodeid, fi->nlookup); @@ -177,6 +174,9 @@ static void fuse_evict_inode(struct inode *inode) if (inode->i_nlink > 0) atomic64_inc(&fc->evict_ctr); } + if (IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) + fuse_backing_put(fuse_inode_backing(fi)); + if (S_ISREG(inode->i_mode) && !fuse_is_bad(inode)) { WARN_ON(fi->iocachectr != 0); WARN_ON(!list_empty(&fi->write_files)); @@ -385,8 +385,8 @@ static void fuse_change_attributes_i(struct inode *inode, struct fuse_attr *attr invalidate_inode_pages2(inode->i_mapping); } - if (IS_ENABLED(CONFIG_FUSE_DAX)) - fuse_dax_dontcache(inode, attr->flags); + if (IS_ENABLED(CONFIG_FUSE_VDAX)) + fuse_vdax_dontcache(inode, attr->flags); } void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr, @@ -403,6 +403,10 @@ static void fuse_init_submount_lookup(struct fuse_submount_lookup *sl, refcount_set(&sl->count, 1); } +static const struct address_space_operations fuse_dax_aops = { + .dirty_folio = noop_dirty_folio, +}; + static void fuse_init_inode(struct inode *inode, struct fuse_attr *attr, struct fuse_conn *fc) { @@ -413,6 +417,11 @@ static void fuse_init_inode(struct inode *inode, struct fuse_attr *attr, if (S_ISREG(inode->i_mode)) { fuse_init_common(inode); fuse_init_file_inode(inode, attr->flags); + + if ((attr->flags & FUSE_ATTR_DAX) && !fuse_inode_vdax(inode)) { + inode->i_flags |= S_DAX; + inode->i_data.a_ops = &fuse_dax_aops; + } } else if (S_ISDIR(inode->i_mode)) fuse_init_dir(inode); else if (S_ISLNK(inode->i_mode)) @@ -803,6 +812,15 @@ static int fuse_opt_fd(struct fs_context *fsc, struct file *file) if (file->f_cred->user_ns != fsc->user_ns) return invalfc(fsc, "wrong user namespace for fuse device"); + /* + * Record whether the server opened /dev/fuse with CAP_SYS_ADMIN in the + * initial user namespace -- the same privilege that mounting virtiofs + * or fuseblk requires. Only such servers are trusted to receive + * FUSE_SYNCFS (see fuse_syncfs_enable()). + */ + ctx->syncfs_capable = file_ns_capable(file, &init_user_ns, + CAP_SYS_ADMIN); + ctx->fud = fuse_dev_grab(file); return 0; @@ -944,12 +962,12 @@ static int fuse_show_options(struct seq_file *m, struct dentry *root) if (sb->s_bdev && sb->s_blocksize != FUSE_DEFAULT_BLKSIZE) seq_printf(m, ",blksize=%lu", sb->s_blocksize); } -#ifdef CONFIG_FUSE_DAX - if (fc->dax_mode == FUSE_DAX_ALWAYS) +#ifdef CONFIG_FUSE_VDAX + if (fc->vdax_mode == FUSE_VDAX_ALWAYS) seq_puts(m, ",dax=always"); - else if (fc->dax_mode == FUSE_DAX_NEVER) + else if (fc->vdax_mode == FUSE_VDAX_NEVER) seq_puts(m, ",dax=never"); - else if (fc->dax_mode == FUSE_DAX_INODE_USER) + else if (fc->vdax_mode == FUSE_VDAX_INODE_USER) seq_puts(m, ",dax=inode"); #endif @@ -1007,8 +1025,8 @@ void fuse_conn_put(struct fuse_conn *fc) if (!refcount_dec_and_test(&fc->count)) return; - if (IS_ENABLED(CONFIG_FUSE_DAX)) - fuse_dax_conn_free(fc); + if (IS_ENABLED(CONFIG_FUSE_VDAX)) + fuse_vdax_conn_free(fc); cancel_work_sync(&fc->epoch_work); fuse_chan_release(fc->chan); put_pid_ns(fc->pid_ns); @@ -1269,6 +1287,16 @@ struct fuse_init_args { struct fuse_mount *fm; }; +/* + * A server can stall syncfs()/sync(), so only honor FUSE_HAS_SYNCFS for + * servers that opened /dev/fuse with CAP_SYS_ADMIN in the initial user + * namespace -- the same privilege required to mount virtiofs or fuseblk. + */ +static bool fuse_syncfs_enable(struct fuse_conn *fc, u64 flags) +{ + return (flags & FUSE_HAS_SYNCFS) && fc->syncfs_capable; +} + static void process_init_reply(struct fuse_args *args, int error) { struct fuse_init_args *ia = container_of(args, typeof(*ia), args); @@ -1354,13 +1382,13 @@ static void process_init_reply(struct fuse_args *args, int error) if (fc->max_pages > 1) fc->name_max = FUSE_NAME_MAX; } - if (IS_ENABLED(CONFIG_FUSE_DAX)) { + if (IS_ENABLED(CONFIG_FUSE_VDAX)) { if (flags & FUSE_MAP_ALIGNMENT && - !fuse_dax_check_alignment(fc, arg->map_alignment)) { + !fuse_vdax_check_alignment(fc, arg->map_alignment)) { ok = false; } if (flags & FUSE_HAS_INODE_DAX) - fc->inode_dax = 1; + fc->inode_vdax = 1; } if (flags & FUSE_HANDLE_KILLPRIV_V2) { fc->handle_killpriv_v2 = 1; @@ -1389,13 +1417,17 @@ static void process_init_reply(struct fuse_args *args, int error) * them together. */ if (IS_ENABLED(CONFIG_FUSE_PASSTHROUGH) && - (flags & FUSE_PASSTHROUGH) && + (flags & (FUSE_PASSTHROUGH | FUSE_PASSTHROUGH_V2)) && arg->max_stack_depth > 0 && arg->max_stack_depth <= FILESYSTEM_MAX_STACK_DEPTH && !(flags & FUSE_WRITEBACK_CACHE)) { fc->passthrough = 1; fc->max_stack_depth = arg->max_stack_depth; fm->sb->s_stack_depth = arg->max_stack_depth; + if (flags & FUSE_PASSTHROUGH_V2) { + fc->backing_id_64 = true; + fuse_backing_files_init_64(fc); + } } if (flags & FUSE_NO_EXPORT_SUPPORT) fm->sb->s_export_op = &fuse_export_fid_operations; @@ -1410,6 +1442,9 @@ static void process_init_reply(struct fuse_args *args, int error) if (flags & FUSE_REQUEST_TIMEOUT) timeout = arg->request_timeout; + + if (fuse_syncfs_enable(fc, flags)) + fc->sync_fs = 1; } else { ra_pages = fc->max_read / PAGE_SIZE; fc->no_lock = 1; @@ -1469,16 +1504,21 @@ static struct fuse_init_args *fuse_new_init(struct fuse_mount *fm) FUSE_HAS_EXPIRE_ONLY | FUSE_DIRECT_IO_ALLOW_MMAP | FUSE_NO_EXPORT_SUPPORT | FUSE_HAS_RESEND | FUSE_ALLOW_IDMAP | FUSE_REQUEST_TIMEOUT; -#ifdef CONFIG_FUSE_DAX - if (fm->fc->dax) +#ifdef CONFIG_FUSE_VDAX + if (fm->fc->vdax) flags |= FUSE_MAP_ALIGNMENT; - if (fuse_is_inode_dax_mode(fm->fc->dax_mode)) + if (fuse_is_inode_vdax_mode(fm->fc->vdax_mode)) flags |= FUSE_HAS_INODE_DAX; #endif if (fm->fc->auto_submounts) flags |= FUSE_SUBMOUNTS; if (IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) - flags |= FUSE_PASSTHROUGH; + flags |= FUSE_PASSTHROUGH | FUSE_PASSTHROUGH_V2; + /* Only offered to sufficiently privileged servers; see + * fuse_syncfs_enable(). + */ + if (fm->fc->syncfs_capable) + flags |= FUSE_HAS_SYNCFS; if (fuse_uring_enabled()) flags |= FUSE_OVER_IO_URING | FUSE_HAS_IO_URING_BUFPOOL; @@ -1629,6 +1669,8 @@ static int fuse_fill_super_submount(struct super_block *sb, WARN_ON(sb->s_bdi != &noop_backing_dev_info); sb->s_bdi = bdi_get(parent_sb->s_bdi); + sb->s_flags |= parent_sb->s_flags & SB_POSIXACL; + sb->s_xattr = parent_sb->s_xattr; sb->s_export_op = parent_sb->s_export_op; sb->s_time_gran = parent_sb->s_time_gran; @@ -1751,8 +1793,8 @@ int fuse_fill_super_common(struct super_block *sb, struct fuse_fs_context *ctx) sb->s_subtype = ctx->subtype; ctx->subtype = NULL; - if (IS_ENABLED(CONFIG_FUSE_DAX)) { - err = fuse_dax_conn_alloc(fc, ctx->dax_mode, ctx->dax_dev); + if (IS_ENABLED(CONFIG_FUSE_VDAX)) { + err = fuse_vdax_conn_alloc(fc, ctx->vdax_mode, ctx->vdax_dev); if (err) goto err; } @@ -1761,7 +1803,7 @@ int fuse_fill_super_common(struct super_block *sb, struct fuse_fs_context *ctx) fm->sb = sb; err = fuse_bdi_init(fc, sb); if (err) - goto err_free_dax; + goto err_free_vdax; /* Handle umasking inside the fuse code */ if (sb->s_flags & SB_POSIXACL) @@ -1770,6 +1812,7 @@ int fuse_fill_super_common(struct super_block *sb, struct fuse_fs_context *ctx) fc->default_permissions = ctx->default_permissions; fc->allow_other = ctx->allow_other; + fc->syncfs_capable = ctx->syncfs_capable; fc->user_id = ctx->user_id; fc->group_id = ctx->group_id; fc->legacy_opts_show = ctx->legacy_opts_show; @@ -1783,7 +1826,7 @@ int fuse_fill_super_common(struct super_block *sb, struct fuse_fs_context *ctx) set_default_d_op(sb, &fuse_dentry_operations); root_dentry = d_make_root(root); if (!root_dentry) - goto err_free_dax; + goto err_free_vdax; mutex_lock(&fuse_mutex); err = -EINVAL; @@ -1809,9 +1852,9 @@ int fuse_fill_super_common(struct super_block *sb, struct fuse_fs_context *ctx) err_unlock: mutex_unlock(&fuse_mutex); dput(root_dentry); - err_free_dax: - if (IS_ENABLED(CONFIG_FUSE_DAX)) - fuse_dax_conn_free(fc); + err_free_vdax: + if (IS_ENABLED(CONFIG_FUSE_VDAX)) + fuse_vdax_conn_free(fc); err: return err; } diff --git a/fs/fuse/ioctl.c b/fs/fuse/ioctl.c index 3ee45b2e148f..ce1807704da6 100644 --- a/fs/fuse/ioctl.c +++ b/fs/fuse/ioctl.c @@ -130,9 +130,6 @@ static int fuse_setup_measure_verity(unsigned long arg, struct iovec *iov) if (copy_from_user(&digest_size, &uarg->digest_size, sizeof(digest_size))) return -EFAULT; - if (digest_size > SIZE_MAX - sizeof(struct fsverity_digest)) - return -EINVAL; - iov->iov_len = sizeof(struct fsverity_digest) + digest_size; return 0; diff --git a/fs/fuse/iomode.c b/fs/fuse/iomode.c index 3728933188f3..38afe1f238ef 100644 --- a/fs/fuse/iomode.c +++ b/fs/fuse/iomode.c @@ -27,15 +27,15 @@ static inline bool fuse_is_io_cache_wait(struct fuse_inode *fi) * Blocks new parallel dio writes and waits for the in-progress parallel dio * writes to complete. */ -int fuse_file_cached_io_open(struct inode *inode, struct fuse_file *ff) +bool fuse_file_cached_io_open(struct inode *inode, struct fuse_file *ff) { struct fuse_inode *fi = get_fuse_inode(inode); /* There are no io modes if server does not implement open */ if (!ff->args) - return 0; + return true; - spin_lock(&fi->lock); + guard(spinlock)(&fi->lock); /* * Setting the bit advises new direct-io writes to use an exclusive * lock - without it the wait below might be forever. @@ -53,8 +53,7 @@ int fuse_file_cached_io_open(struct inode *inode, struct fuse_file *ff) */ if (fuse_inode_backing(fi)) { clear_bit(FUSE_I_CACHE_IO_MODE, &fi->state); - spin_unlock(&fi->lock); - return -ETXTBSY; + return false; } WARN_ON(ff->iomode == IOM_UNCACHED); @@ -64,8 +63,7 @@ int fuse_file_cached_io_open(struct inode *inode, struct fuse_file *ff) set_bit(FUSE_I_CACHE_IO_MODE, &fi->state); fi->iocachectr++; } - spin_unlock(&fi->lock); - return 0; + return true; } static void fuse_file_cached_io_release(struct fuse_file *ff, @@ -85,19 +83,16 @@ static void fuse_file_cached_io_release(struct fuse_file *ff, int fuse_inode_uncached_io_start(struct fuse_inode *fi, struct fuse_backing *fb) { struct fuse_backing *oldfb; - int err = 0; - spin_lock(&fi->lock); + guard(spinlock)(&fi->lock); /* deny conflicting backing files on same fuse inode */ oldfb = fuse_inode_backing(fi); - if (fb && oldfb && oldfb != fb) { - err = -EBUSY; - goto unlock; - } - if (fi->iocachectr > 0) { - err = -ETXTBSY; - goto unlock; - } + if (fb && oldfb && oldfb != fb) + return -EBUSY; + + if (fi->iocachectr > 0) + return -ETXTBSY; + fi->iocachectr--; /* fuse inode holds a single refcount of backing file */ @@ -107,9 +102,7 @@ int fuse_inode_uncached_io_start(struct fuse_inode *fi, struct fuse_backing *fb) } else { fuse_backing_put(fb); } -unlock: - spin_unlock(&fi->lock); - return err; + return 0; } /* Takes uncached_io inode mode reference to be dropped on file release */ @@ -121,8 +114,11 @@ static int fuse_file_uncached_io_open(struct inode *inode, int err; err = fuse_inode_uncached_io_start(fi, fb); - if (err) - return err; + if (err) { + if (err == -EBUSY) + return fuse_EIO("mismatched backing"); + return fuse_EIO("conflicting caching mode"); + } WARN_ON(ff->iomode != IOM_NONE); ff->iomode = IOM_UNCACHED; @@ -169,26 +165,51 @@ static int fuse_file_passthrough_open(struct inode *inode, struct file *file) { struct fuse_file *ff = file->private_data; struct fuse_conn *fc = get_fuse_conn(inode); + struct fuse_open_out *outarg = &ff->args->open_outarg; struct fuse_backing *fb; + u64 backing_id; int err; /* Check allowed conditions for file open in passthrough mode */ - if (!IS_ENABLED(CONFIG_FUSE_PASSTHROUGH) || !fc->passthrough || - (ff->open_flags & ~FOPEN_PASSTHROUGH_MASK)) - return -EINVAL; + if (!IS_ENABLED(CONFIG_FUSE_PASSTHROUGH) || !fc->passthrough) + return fuse_EIO("passthrough not enabled"); - fb = fuse_passthrough_open(file, ff->args->open_outarg.backing_id); - if (IS_ERR(fb)) - return PTR_ERR(fb); + if (ff->open_flags & ~FOPEN_PASSTHROUGH_MASK) + return fuse_EIO("conflicting open flags"); + + if (!fc->backing_id_64) { + if (outarg->backing_id_64 != 0) + return fuse_EIO("64 bit backing ID set"); + + if (outarg->backing_id <= 0) + return fuse_EIO("invalid backing ID"); + + backing_id = outarg->backing_id; + } else { + if (outarg->backing_id != 0) + return fuse_EIO("32 bit backing ID set"); + + backing_id = outarg->backing_id_64; + } + fb = fuse_backing_lookup(fc, backing_id); + if (!fb) + return fuse_EIO("backing not found"); + + err = fuse_passthrough_open(file, fb); + if (err) + goto backing_put; /* First passthrough file open denies caching inode io mode */ err = fuse_file_uncached_io_open(inode, ff, fb); - if (!err) - return 0; + if (err) + goto passthrough_release; + return 0; + +passthrough_release: fuse_passthrough_release(ff, fb); +backing_put: fuse_backing_put(fb); - return err; } @@ -197,22 +218,26 @@ int fuse_file_io_open(struct file *file, struct inode *inode) { struct fuse_file *ff = file->private_data; struct fuse_inode *fi = get_fuse_inode(inode); - int err; /* - * io modes are not relevant with DAX and with server that does not + * io modes are not relevant with virtiofs DAX and with server that does not * implement open. */ - if (FUSE_IS_DAX(inode) || !ff->args) + if (FUSE_IS_VDAX(inode) || !ff->args) return 0; /* * Server is expected to use FOPEN_PASSTHROUGH for all opens of an inode - * which is already open for passthrough. + * which is already open for passthrough. Using incorrect open mode is + * a server mistake, which results in user visible failure of open() + * with EIO error. Same with DAX inodes. */ - err = -EINVAL; - if (fuse_inode_backing(fi) && !(ff->open_flags & FOPEN_PASSTHROUGH)) - goto fail; + if (!(ff->open_flags & FOPEN_PASSTHROUGH)) { + if (fuse_inode_backing(fi)) + return fuse_EIO("FOPEN_PASSTHROUGH expected"); + if (IS_DAX(inode)) + return fuse_EIO("DAX inode without FOPEN_PASSTHROUGH"); + } /* * FOPEN_PARALLEL_DIRECT_WRITES requires FOPEN_DIRECT_IO. @@ -233,23 +258,12 @@ int fuse_file_io_open(struct file *file, struct inode *inode) return 0; if (ff->open_flags & FOPEN_PASSTHROUGH) - err = fuse_file_passthrough_open(inode, file); - else - err = fuse_file_cached_io_open(inode, ff); - if (err) - goto fail; + return fuse_file_passthrough_open(inode, file); + + if (!fuse_file_cached_io_open(inode, ff)) + return fuse_EIO("conflicting passthrough open"); return 0; - -fail: - pr_debug("failed to open file in requested io mode (open_flags=0x%x, err=%i).\n", - ff->open_flags, err); - /* - * The file open mode determines the inode io mode. - * Using incorrect open mode is a server mistake, which results in - * user visible failure of open() with EIO error. - */ - return -EIO; } /* No more pending io and no new io possible to inode via open/mmapped file */ diff --git a/fs/fuse/notify.c b/fs/fuse/notify.c index 1ba763705d91..984e4945e3e3 100644 --- a/fs/fuse/notify.c +++ b/fs/fuse/notify.c @@ -155,7 +155,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, nodeid = outarg.nodeid; pos = outarg.offset; - num = min(outarg.size, MAX_LFS_FILESIZE - pos); + num = umin(outarg.size, MAX_LFS_FILESIZE - pos); down_read(&fc->killsb); @@ -190,7 +190,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, folio_offset = offset_in_folio(folio, pos); nr_bytes = min(num, folio_size(folio) - folio_offset); - err = fuse_copy_folio(cs, &folio, folio_offset, nr_bytes, 0); + err = fuse_copy_folio(cs, &folio, folio_offset, nr_bytes); if (!folio_test_uptodate(folio) && !err && folio_offset == 0 && (nr_bytes == folio_size(folio) || file_size == end)) { folio_zero_segment(folio, nr_bytes, folio_size(folio)); @@ -409,6 +409,75 @@ static int fuse_notify_prune(struct fuse_conn *fc, unsigned int size, return 0; } +static int fuse_notify_backing_remove(struct fuse_conn *fc, unsigned int size, + struct fuse_copy_state *cs) +{ + struct fuse_notify_backing_remove_out outarg; + int err; + + if (!IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) + return -EOPNOTSUPP; + + if (size != sizeof(outarg)) + return -EINVAL; + + err = fuse_copy_one(cs, &outarg, sizeof(outarg)); + if (err) + return err; + + if (outarg.reserved) + return -EINVAL; + + if (!fc->backing_id_64) + return -EINVAL; + + return fuse_backing_close_64(fc, outarg.backing_id); +} + +static int fuse_notify_map(struct fuse_conn *fc, unsigned int size, + struct fuse_copy_state *cs) +{ + struct fuse_notify_backing_map_out outarg; + struct fuse_extent *ext __free(kvfree) = NULL; + int err; + + if (size < sizeof(outarg)) + return -EINVAL; + + err = fuse_copy_one(cs, &outarg, sizeof(outarg)); + if (err) + return err; + + if (!fc->backing_id_64) + return -EINVAL; + + if (outarg.num_extents > FUSE_MAX_EXTENTS) + return -EINVAL; + + size -= sizeof(outarg); + if (size != outarg.num_extents * sizeof(*ext)) + return -EINVAL; + + if (outarg.reserved[0] != 0 || outarg.reserved[1] != 0) + return -EINVAL; + + if (outarg.flags & ~(FUSE_BACKING_MAP_CREATE | FUSE_BACKING_MAP_CYCLIC)) + return -EINVAL; + + if (!IS_ENABLED(CONFIG_FUSE_PASSTHROUGH)) + return -EOPNOTSUPP; + + ext = kvmalloc_objs(*ext, outarg.num_extents); + if (!ext) + return -ENOMEM; + + err = fuse_copy_one(cs, ext, size); + if (err) + return err; + + return fuse_ext_map_populate(fc, &outarg, ext); +} + int fuse_notify(struct fuse_conn *fc, enum fuse_notify_code code, unsigned int size, struct fuse_copy_state *cs) { @@ -440,6 +509,12 @@ int fuse_notify(struct fuse_conn *fc, enum fuse_notify_code code, case FUSE_NOTIFY_PRUNE: return fuse_notify_prune(fc, size, cs); + case FUSE_NOTIFY_BACKING_REMOVE: + return fuse_notify_backing_remove(fc, size, cs); + + case FUSE_NOTIFY_BACKING_MAP: + return fuse_notify_map(fc, size, cs); + default: return -EINVAL; } diff --git a/fs/fuse/passthrough.c b/fs/fuse/passthrough.c index f2d08ac2459b..0ed964c479d9 100644 --- a/fs/fuse/passthrough.c +++ b/fs/fuse/passthrough.c @@ -11,6 +11,10 @@ #include #include +static inline struct file *fuse_file_passthrough(struct fuse_file *ff) +{ + return ff->passthrough; +} static void fuse_file_accessed(struct file *file) { struct inode *inode = file_inode(file); @@ -31,7 +35,6 @@ ssize_t fuse_passthrough_read_iter(struct kiocb *iocb, struct iov_iter *iter) struct fuse_file *ff = file->private_data; struct file *backing_file = fuse_file_passthrough(ff); size_t count = iov_iter_count(iter); - ssize_t ret; struct backing_file_ctx ctx = { .cred = ff->cred, .accessed = fuse_file_accessed, @@ -44,10 +47,10 @@ ssize_t fuse_passthrough_read_iter(struct kiocb *iocb, struct iov_iter *iter) if (!count) return 0; - ret = backing_file_read_iter(backing_file, iter, iocb, iocb->ki_flags, - &ctx); + if (!backing_file) + return fuse_ext_map_read_iter(iocb, iter); - return ret; + return backing_file_read_iter(backing_file, iter, iocb, iocb->ki_flags, &ctx); } ssize_t fuse_passthrough_write_iter(struct kiocb *iocb, @@ -70,10 +73,13 @@ ssize_t fuse_passthrough_write_iter(struct kiocb *iocb, if (!count) return 0; - inode_lock(inode); + guard(rwsem_write)(&inode->i_rwsem); + + if (!backing_file) + return fuse_ext_map_write_iter(iocb, iter); + ret = backing_file_write_iter(backing_file, iter, iocb, iocb->ki_flags, &ctx); - inode_unlock(inode); return ret; } @@ -94,6 +100,9 @@ ssize_t fuse_passthrough_splice_read(struct file *in, loff_t *ppos, pr_debug("%s: backing_file=0x%p, pos=%lld, len=%zu, flags=0x%x\n", __func__, backing_file, *ppos, len, flags); + if (!backing_file) + return copy_splice_read(in, ppos, pipe, len, flags); + init_sync_kiocb(&iocb, in); iocb.ki_pos = *ppos; ret = backing_file_splice_read(backing_file, &iocb, pipe, len, flags, &ctx); @@ -119,6 +128,9 @@ ssize_t fuse_passthrough_splice_write(struct pipe_inode_info *pipe, pr_debug("%s: backing_file=0x%p, pos=%lld, len=%zu, flags=0x%x\n", __func__, backing_file, *ppos, len, flags); + if (!backing_file) + return iter_file_splice_write(pipe, out, ppos, len, flags); + inode_lock(inode); init_sync_kiocb(&iocb, out); iocb.ki_pos = *ppos; @@ -141,56 +153,74 @@ ssize_t fuse_passthrough_mmap(struct file *file, struct vm_area_struct *vma) pr_debug("%s: backing_file=0x%p, start=%lu, end=%lu\n", __func__, backing_file, vma->vm_start, vma->vm_end); + if (!backing_file) + return fuse_ext_map_mmap(file, vma); + return backing_file_mmap(backing_file, vma, &ctx); } +int fuse_passthrough_fadvise(struct file *file, loff_t offset, loff_t len, int advice) +{ + struct fuse_file *ff = file->private_data; + struct file *backing_file = fuse_file_passthrough(ff); + const struct cred *old_cred; + int ret; + + if (!backing_file) + return generic_fadvise(file, offset, len, advice); + + old_cred = override_creds(ff->cred); + ret = vfs_fadvise(backing_file, offset, len, advice); + revert_creds(old_cred); + + return ret; +} + /* * Setup passthrough to a backing file. - * - * Returns an fb object with elevated refcount to be stored in fuse inode. */ -struct fuse_backing *fuse_passthrough_open(struct file *file, int backing_id) +int fuse_passthrough_open(struct file *file, struct fuse_backing *fb) { struct fuse_file *ff = file->private_data; - struct fuse_conn *fc = ff->fm->fc; - struct fuse_backing *fb = NULL; struct file *backing_file; - int err; - err = -EINVAL; - if (backing_id <= 0) - goto out; - - err = -ENOENT; - fb = fuse_backing_lookup(fc, backing_id); - if (!fb) - goto out; - - /* Allocate backing file per fuse file to store fuse path */ - backing_file = backing_file_open(file, file->f_flags, - &fb->file->f_path, fb->cred); - err = PTR_ERR(backing_file); - if (IS_ERR(backing_file)) { - fuse_backing_put(fb); - goto out; + if (fb->type == FUSE_BACKING_EXTMAP) { + if (fuse_backing_is_dax(fb) != !!IS_DAX(file_inode(file))) + return fuse_EIO("dax mode mismatch"); + return 0; } - err = 0; + if (fb->type != FUSE_BACKING_PATH) + return fuse_EIO("invalid backing type"); + + if (IS_DAX(file_inode(file))) + return fuse_EIO("dax mode on backing file not supported"); + + /* Allocate backing file per fuse file to store fuse path */ + backing_file = backing_file_open(file, file->f_flags, &fb->path, fb->cred); + if (IS_ERR(backing_file)) + return fuse_EIO("failed to open backing file (%ld)", PTR_ERR(backing_file)); + ff->passthrough = backing_file; ff->cred = get_cred(fb->cred); -out: - pr_debug("%s: backing_id=%d, fb=0x%p, backing_file=0x%p, err=%i\n", __func__, - backing_id, fb, ff->passthrough, err); - return err ? ERR_PTR(err) : fb; + pr_debug("%s: backing_id=%llu, fb=0x%p, backing_file=0x%p\n", __func__, + fb->backing_id, fb, ff->passthrough); + + return 0; } void fuse_passthrough_release(struct fuse_file *ff, struct fuse_backing *fb) { - pr_debug("%s: fb=0x%p, backing_file=0x%p\n", __func__, - fb, ff->passthrough); + struct file *backing_file = fuse_file_passthrough(ff); - fput(ff->passthrough); + pr_debug("%s: fb=0x%p, backing_file=0x%p\n", __func__, + fb, backing_file); + + if (!backing_file) + return; + + fput(backing_file); ff->passthrough = NULL; put_cred(ff->cred); ff->cred = NULL; diff --git a/fs/fuse/readdir.c b/fs/fuse/readdir.c index d2599043f7ec..1f6cf565477f 100644 --- a/fs/fuse/readdir.c +++ b/fs/fuse/readdir.c @@ -34,11 +34,11 @@ static void fuse_add_dirent_to_cache(struct file *file, struct fuse_inode *fi = get_fuse_inode(file_inode(file)); size_t reclen = FUSE_DIRENT_SIZE(dirent); pgoff_t index; - struct page *page; + struct folio *folio; loff_t size; u64 version; unsigned int offset; - void *addr; + size_t folio_offset; /* Dirent doesn't fit in readdir cache page? Skip caching. */ if (reclen > PAGE_SIZE) @@ -64,13 +64,11 @@ static void fuse_add_dirent_to_cache(struct file *file, } spin_unlock(&fi->rdc.lock); - if (offset) { - page = find_lock_page(file->f_mapping, index); - } else { - page = find_or_create_page(file->f_mapping, index, - mapping_gfp_mask(file->f_mapping)); - } - if (!page) + if (offset) + folio = filemap_lock_folio(file->f_mapping, index); + else + folio = filemap_grab_folio(file->f_mapping, index); + if (IS_ERR(folio)) return; spin_lock(&fi->rdc.lock); @@ -79,19 +77,21 @@ static void fuse_add_dirent_to_cache(struct file *file, WARN_ON(fi->rdc.pos != pos)) goto unlock; - addr = kmap_local_page(page); + folio_offset = offset_in_folio(folio, (index << PAGE_SHIFT) + offset); if (!offset) { - clear_page(addr); - SetPageUptodate(page); + folio_fill_tail(folio, folio_offset, (const char *)dirent, + reclen); + folio_mark_uptodate(folio); + } else { + memcpy_to_folio(folio, folio_offset, (const char *)dirent, + reclen); } - memcpy(addr + offset, dirent, reclen); - kunmap_local(addr); fi->rdc.size = (index << PAGE_SHIFT) + offset + reclen; fi->rdc.pos = dirent->off; unlock: spin_unlock(&fi->rdc.lock); - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); } static void fuse_readdir_cache_end(struct file *file, loff_t pos) @@ -502,7 +502,7 @@ static int fuse_readdir_cached(struct file *file, struct dir_context *ctx) enum fuse_parse_result res; pgoff_t index; unsigned int size; - struct page *page; + struct folio *folio; void *addr; /* Seeked? If so, reset the cache stream */ @@ -580,42 +580,44 @@ static int fuse_readdir_cached(struct file *file, struct dir_context *ctx) if (offset_in_page(ff->readdir.cache_off) == size) return 0; - page = find_get_page_flags(file->f_mapping, index, - FGP_ACCESSED | FGP_LOCK); - /* Page gone missing, then re-added to cache, but not initialized? */ - if (page && !PageUptodate(page)) { - unlock_page(page); - put_page(page); - page = NULL; + folio = __filemap_get_folio(file->f_mapping, index, + FGP_ACCESSED | FGP_LOCK, 0); + if (IS_ERR(folio)) + folio = NULL; + /* Folio gone missing, then re-added to cache, but not initialized? */ + if (folio && !folio_test_uptodate(folio)) { + folio_unlock(folio); + folio_put(folio); + folio = NULL; } spin_lock(&fi->rdc.lock); - if (!page) { + if (!folio) { /* - * Uh-oh: page gone missing, cache is useless + * Uh-oh: folio gone missing, cache is useless */ if (fi->rdc.version == ff->readdir.version) fuse_rdc_reset(inode); goto retry_locked; } - /* Make sure it's still the same version after getting the page. */ + /* Make sure it's still the same version after getting the folio. */ if (ff->readdir.version != fi->rdc.version) { spin_unlock(&fi->rdc.lock); - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); goto retry; } spin_unlock(&fi->rdc.lock); /* - * Contents of the page are now protected against changing by holding - * the page lock. + * Contents of the folio are now protected against changing by holding + * the folio lock. */ - addr = kmap_local_page(page); + addr = kmap_local_folio(folio, 0); res = fuse_parse_cache(ff, addr, size, ctx); kunmap_local(addr); - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); if (res == FOUND_ERR) return -EIO; diff --git a/fs/fuse/req.c b/fs/fuse/req.c index a133b012d041..6274a23af445 100644 --- a/fs/fuse/req.c +++ b/fs/fuse/req.c @@ -11,12 +11,12 @@ static int fuse_fill_creds(struct fuse_mount *fm, struct fuse_args *args, kuid_t fsuid = mapped_fsuid(idmap, fc->user_ns); kgid_t fsgid = mapped_fsgid(idmap, fc->user_ns); + if (args->nocreds) + return 0; + args->pid = pid_nr_ns(task_pid(current), fc->pid_ns); if (args->force) { - if (args->nocreds) - return 0; - if (no_idmap) { args->uid = from_kuid_munged(fc->user_ns, current_fsuid()); args->gid = from_kgid_munged(fc->user_ns, current_fsgid()); @@ -27,7 +27,6 @@ static int fuse_fill_creds(struct fuse_mount *fm, struct fuse_args *args, return 0; } - WARN_ON(args->nocreds); /* * Keep the old behavior when idmappings support was not * declared by a FUSE server. diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index f15e516ebcb5..ba8d79c4bcac 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -102,9 +102,9 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, gfp_t gfp); static const struct constant_table dax_param_enums[] = { - {"always", FUSE_DAX_ALWAYS }, - {"never", FUSE_DAX_NEVER }, - {"inode", FUSE_DAX_INODE_USER }, + {"always", FUSE_VDAX_ALWAYS }, + {"never", FUSE_VDAX_NEVER }, + {"inode", FUSE_VDAX_INODE_USER }, {} }; @@ -132,10 +132,10 @@ static int virtio_fs_parse_param(struct fs_context *fsc, switch (opt) { case OPT_DAX: - ctx->dax_mode = FUSE_DAX_ALWAYS; + ctx->vdax_mode = FUSE_VDAX_ALWAYS; break; case OPT_DAX_ENUM: - ctx->dax_mode = result.uint_32; + ctx->vdax_mode = result.uint_32; break; default: return -EINVAL; @@ -455,26 +455,23 @@ static int virtio_fs_add_instance(struct virtio_device *vdev, return ret; } -/* Return the virtio_fs with a given tag, or NULL */ +/* Return the virtio_fs with a given tag, or NULL. + * Callers hold virtio_fs_mutex, which also keeps the virtqueues alive. + */ static struct virtio_fs *virtio_fs_find_instance(const char *tag) { struct virtio_fs *fs; - mutex_lock(&virtio_fs_mutex); + lockdep_assert_held(&virtio_fs_mutex); list_for_each_entry(fs, &virtio_fs_instances, list) { if (strcmp(fs->tag, tag) == 0) { kobject_get(&fs->kobj); - goto found; + return fs; } } - fs = NULL; /* not found */ - -found: - mutex_unlock(&virtio_fs_mutex); - - return fs; + return NULL; /* not found */ } static void virtio_fs_free_devs(struct virtio_fs *fs) @@ -730,6 +727,10 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req) unsigned int num_out; unsigned int i; + /* fuse_request_end() returns this error; there are no args to copy. */ + if (req->out.h.error) + goto out; + remaining = req->out.h.len - sizeof(req->out.h); num_in = args->in_numargs - args->in_pages; num_out = args->out_numargs - args->out_pages; @@ -755,6 +756,7 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req) if (args->out_argvar) args->out_args[args->out_numargs - 1].size = remaining; +out: kfree(req->argbuf); req->argbuf = NULL; } @@ -762,7 +764,9 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req) /* Verify that the server properly follows the FUSE protocol */ static bool virtio_fs_verify_response(struct fuse_req *req, unsigned int len) { + struct fuse_args *args = req->args; struct fuse_out_header *oh = &req->out.h; + unsigned int expected; if (len < sizeof(*oh)) { pr_warn("virtio-fs: response too short (%u)\n", len); @@ -777,6 +781,29 @@ static bool virtio_fs_verify_response(struct fuse_req *req, unsigned int len) oh->unique, req->in.h.unique); return false; } + if (oh->error <= -ERESTARTSYS || oh->error > 0) { + pr_warn("virtio-fs: invalid error value (%d)\n", oh->error); + return false; + } + + if (oh->error) { + if (len != sizeof(*oh)) { + pr_warn("virtio-fs: error response too long (%u)\n", len); + return false; + } + return true; + } + + expected = sizeof(*oh) + + fuse_len_args(args->out_numargs, args->out_args); + if (len > expected || + (len < expected && + (!args->out_argvar || + expected - len > args->out_args[args->out_numargs - 1].size))) { + pr_warn("virtio-fs: invalid response length (%u, expected %u)\n", + len, expected); + return false; + } return true; } @@ -798,9 +825,11 @@ static void virtio_fs_request_complete(struct fuse_req *req, for (i = 0; i < ap->num_folios; i++) { thislen = ap->descs[i].length; if (len < thislen) { - WARN_ON(ap->descs[i].offset); + unsigned int offset = ap->descs[i].offset; + folio = ap->folios[i]; - folio_zero_segment(folio, len, thislen); + folio_zero_segment(folio, offset + len, + offset + thislen); len = 0; } else { len -= thislen; @@ -1079,7 +1108,7 @@ static int virtio_fs_setup_dax(struct virtio_device *vdev, struct virtio_fs *fs) struct dev_pagemap *pgmap; bool have_cache; - if (!IS_ENABLED(CONFIG_FUSE_DAX)) + if (!IS_ENABLED(CONFIG_FUSE_VDAX)) return 0; dax_dev = alloc_dax(fs, &virtio_fs_dax_ops); @@ -1592,14 +1621,14 @@ static int virtio_fs_fill_super(struct super_block *sb, struct fs_context *fsc) goto err_free_fuse_devs; } - if (ctx->dax_mode != FUSE_DAX_NEVER) { - if (ctx->dax_mode == FUSE_DAX_ALWAYS && !fs->dax_dev) { + if (ctx->vdax_mode != FUSE_VDAX_NEVER) { + if (ctx->vdax_mode == FUSE_VDAX_ALWAYS && !fs->dax_dev) { err = -EINVAL; pr_err("virtio-fs: dax can't be enabled as filesystem" " device does not support it.\n"); goto err_free_fuse_devs; } - ctx->dax_dev = fs->dax_dev; + ctx->vdax_dev = fs->dax_dev; } err = fuse_fill_super_common(sb, ctx); if (err < 0) @@ -1633,8 +1662,8 @@ static void virtio_fs_conn_destroy(struct fuse_mount *fm) /* Stop dax worker. Soon evict_inodes() will be called which * will free all memory ranges belonging to all inodes. */ - if (IS_ENABLED(CONFIG_FUSE_DAX)) - fuse_dax_cancel_work(fc); + if (IS_ENABLED(CONFIG_FUSE_VDAX)) + fuse_vdax_cancel_work(fc); /* Stop forget queue. Soon destroy will be sent */ spin_lock(&fsvq->lock); @@ -1698,13 +1727,17 @@ static int virtio_fs_get_tree(struct fs_context *fsc) * in chan->iq->priv. Once fuse_conn is going away, it calls ->put() * to drop the reference to this object. */ + mutex_lock(&virtio_fs_mutex); fs = virtio_fs_find_instance(fsc->source); + if (fs) + virtqueue_size = virtqueue_get_vring_size(fs->vqs[VQ_REQUEST].vq); + mutex_unlock(&virtio_fs_mutex); + if (!fs) { pr_info("virtio-fs: tag <%s> not found\n", fsc->source); return -EINVAL; } - virtqueue_size = virtqueue_get_vring_size(fs->vqs[VQ_REQUEST].vq); if (WARN_ON(virtqueue_size <= FUSE_HEADER_OVERHEAD)) goto out_err; diff --git a/include/linux/dax.h b/include/linux/dax.h index f2d47975d905..a3af0c63c61b 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -54,7 +54,6 @@ struct dax_device *alloc_dax(void *private, const struct dax_operations *ops); void *dax_holder(struct dax_device *dax_dev); void put_dax(struct dax_device *dax_dev); void kill_dax(struct dax_device *dax_dev); -struct dax_device *dax_dev_get(dev_t devt); void dax_write_cache(struct dax_device *dax_dev, bool wc); bool dax_write_cache_enabled(struct dax_device *dax_dev); bool dax_synchronous(struct dax_device *dax_dev); @@ -311,3 +310,5 @@ typedef int (*walk_hmem_fn)(struct device *dev, int target_nid, const struct resource *res); int walk_hmem_resources(struct device *dev, walk_hmem_fn fn); #endif + +struct dax_device *fsdev_dax_from_file(struct file *file); diff --git a/include/uapi/linux/fuse.h b/include/uapi/linux/fuse.h index 7435e09c87fe..616430a2973c 100644 --- a/include/uapi/linux/fuse.h +++ b/include/uapi/linux/fuse.h @@ -248,6 +248,15 @@ * - add bufpool offset field to fuse_uring_ent_in_out struct * - add FUSE_URING_ZERO_COPY, FUSE_URING_ENT_ZERO_COPY, and * FOPEN_IO_URING_ZERO_COPY flag + * + * 7.47 + * - add FUSE_HAS_SYNCFS opt-in flag for privileged userspace servers + * - add FUSE_PASSTHROUGH_V2 + * - add FUSE_DEV_IOC_BACKING_CREATE, struct fuse_backing_create_in + * - add FUSE_NOTIFY_BACKING_REMOVE, struct fuse_notify_backing_remove_out + * - add backing_id_64 to fuse_open_out + * - add FUSE_NOTIFY_BACKING_MAP, fuse_notify_backing_map_out, fuse_extent + * - add FUSE_BACKING_MAP_CREATE, FUSE_BACKING_MAP_CYCLIC */ #ifndef _LINUX_FUSE_H @@ -283,7 +292,7 @@ #define FUSE_KERNEL_VERSION 7 /** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 46 +#define FUSE_KERNEL_MINOR_VERSION 47 /** The node ID of the root inode */ #define FUSE_ROOT_ID 1 @@ -364,7 +373,7 @@ struct fuse_file_lock { uint32_t pid; /* tgid */ }; -/** +/* * Bitmasks for fuse_setattr_in.valid */ #define FATTR_MODE (1 << 0) @@ -380,7 +389,7 @@ struct fuse_file_lock { #define FATTR_CTIME (1 << 10) #define FATTR_KILL_SUIDGID (1 << 11) -/** +/* * Flags returned by the OPEN request * * FOPEN_DIRECT_IO: bypass page cache for this open file @@ -408,7 +417,7 @@ struct fuse_file_lock { #define FOPEN_PASSTHROUGH (1 << 7) #define FOPEN_IO_URING_ZERO_COPY (1 << 8) -/** +/* * INIT request/reply flags * * FUSE_ASYNC_READ: asynchronous read requests @@ -464,6 +473,13 @@ struct fuse_file_lock { * FUSE_REQUEST_TIMEOUT: kernel supports timing out requests. * init_out.request_timeout contains the timeout (in secs) * FUSE_HAS_IO_URING_BUFPOOL: kernel supports io-uring buffer pools + * FUSE_HAS_SYNCFS: server requests that syncfs()/sync() be propagated as + * FUSE_SYNCFS requests. Since an untrusted server can use this + * to stall sync(), it is only honored when /dev/fuse was opened + * with CAP_SYS_ADMIN in the initial user namespace (the same + * privilege that mounting virtiofs or fuseblk requires). + * Insufficiently privileged servers ignore it. + * FUSE_PASSTHROUGH_V2: use 64 bit server allocated backing ID */ #define FUSE_ASYNC_READ (1 << 0) #define FUSE_POSIX_LOCKS (1 << 1) @@ -512,31 +528,33 @@ struct fuse_file_lock { #define FUSE_OVER_IO_URING (1ULL << 41) #define FUSE_REQUEST_TIMEOUT (1ULL << 42) #define FUSE_HAS_IO_URING_BUFPOOL (1ULL << 43) +#define FUSE_HAS_SYNCFS (1ULL << 44) +#define FUSE_PASSTHROUGH_V2 (1ULL << 45) -/** +/* * CUSE INIT request/reply flags * * CUSE_UNRESTRICTED_IOCTL: use unrestricted ioctl */ #define CUSE_UNRESTRICTED_IOCTL (1 << 0) -/** +/* * Release flags */ #define FUSE_RELEASE_FLUSH (1 << 0) #define FUSE_RELEASE_FLOCK_UNLOCK (1 << 1) -/** +/* * Getattr flags */ #define FUSE_GETATTR_FH (1 << 0) -/** +/* * Lock flags */ #define FUSE_LK_FLOCK (1 << 0) -/** +/* * WRITE flags * * FUSE_WRITE_CACHE: delayed write from page cache, file handle is guessed @@ -550,12 +568,12 @@ struct fuse_file_lock { /* Obsolete alias; this flag implies killing suid/sgid only. */ #define FUSE_WRITE_KILL_PRIV FUSE_WRITE_KILL_SUIDGID -/** +/* * Read flags */ #define FUSE_READ_LOCKOWNER (1 << 1) -/** +/* * Ioctl flags * * FUSE_IOCTL_COMPAT: 32bit compat ioctl on 64bit machine @@ -576,21 +594,21 @@ struct fuse_file_lock { #define FUSE_IOCTL_MAX_IOV 256 -/** +/* * Poll flags * * FUSE_POLL_SCHEDULE_NOTIFY: request poll notify */ #define FUSE_POLL_SCHEDULE_NOTIFY (1 << 0) -/** +/* * Fsync flags * * FUSE_FSYNC_FDATASYNC: Sync data only, not metadata */ #define FUSE_FSYNC_FDATASYNC (1 << 0) -/** +/* * fuse_attr flags * * FUSE_ATTR_SUBMOUNT: Object is a submount root @@ -599,28 +617,28 @@ struct fuse_file_lock { #define FUSE_ATTR_SUBMOUNT (1 << 0) #define FUSE_ATTR_DAX (1 << 1) -/** +/* * Open flags * FUSE_OPEN_KILL_SUIDGID: Kill suid and sgid if executable */ #define FUSE_OPEN_KILL_SUIDGID (1 << 0) -/** +/* * setxattr flags * FUSE_SETXATTR_ACL_KILL_SGID: Clear SGID when system.posix_acl_access is set */ #define FUSE_SETXATTR_ACL_KILL_SGID (1 << 0) -/** +/* * notify_inval_entry flags * FUSE_EXPIRE_ONLY */ #define FUSE_EXPIRE_ONLY (1 << 0) /** - * extension type - * FUSE_MAX_NR_SECCTX: maximum value of &fuse_secctx_header.nr_secctx - * FUSE_EXT_GROUPS: &fuse_supp_groups extension + * enum fuse_ext_type - extension type + * @FUSE_MAX_NR_SECCTX: maximum value of &fuse_secctx_header.nr_secctx + * @FUSE_EXT_GROUPS: &fuse_supp_groups extension */ enum fuse_ext_type { /* Types 0..31 are reserved for fuse_secctx_header */ @@ -699,6 +717,8 @@ enum fuse_notify_code { FUSE_NOTIFY_RESEND = 7, FUSE_NOTIFY_INC_EPOCH = 8, FUSE_NOTIFY_PRUNE = 9, + FUSE_NOTIFY_BACKING_REMOVE = 10, + FUSE_NOTIFY_BACKING_MAP = 11, }; /* The read buffer is required to be at least 8k, but may be much larger */ @@ -825,6 +845,7 @@ struct fuse_open_out { uint64_t fh; uint32_t open_flags; int32_t backing_id; + uint64_t backing_id_64; }; struct fuse_release_in { @@ -1026,14 +1047,14 @@ struct fuse_fallocate_in { }; /** - * FUSE request unique ID flag + * define FUSE_UNIQUE_RESEND - FUSE request unique ID flag * * Indicates whether this is a resend request. The receiver should handle this * request accordingly. */ #define FUSE_UNIQUE_RESEND (1ULL << 63) -/** +/* * This value will be set by the kernel to * (struct fuse_in_header).{uid,gid} fields in * case when: @@ -1149,13 +1170,20 @@ struct fuse_backing_map { uint64_t padding; }; +struct fuse_backing_create_in { + int32_t fd; + uint32_t padding; + uint64_t backing_id; /* Zero value is reserved */ + uint64_t spare[2]; +}; + /* Device ioctls: */ #define FUSE_DEV_IOC_MAGIC 229 #define FUSE_DEV_IOC_CLONE _IOR(FUSE_DEV_IOC_MAGIC, 0, uint32_t) -#define FUSE_DEV_IOC_BACKING_OPEN _IOW(FUSE_DEV_IOC_MAGIC, 1, \ - struct fuse_backing_map) +#define FUSE_DEV_IOC_BACKING_OPEN _IOW(FUSE_DEV_IOC_MAGIC, 1, struct fuse_backing_map) #define FUSE_DEV_IOC_BACKING_CLOSE _IOW(FUSE_DEV_IOC_MAGIC, 2, uint32_t) #define FUSE_DEV_IOC_SYNC_INIT _IO(FUSE_DEV_IOC_MAGIC, 3) +#define FUSE_DEV_IOC_BACKING_CREATE _IOW(FUSE_DEV_IOC_MAGIC, 4, struct fuse_backing_create_in) struct fuse_lseek_in { uint64_t fh; @@ -1183,6 +1211,37 @@ struct fuse_copy_file_range_out { uint64_t bytes_copied; }; +struct fuse_notify_backing_remove_out { + uint64_t backing_id; + uint64_t reserved; +}; + +/** + * notify_map flags + * + * FUSE_BACKING_MAP_CREATE: create backing with the supplied ID + * FUSE_BACKING_MAP_CYCLIC: map repeats after last extent + */ +#define FUSE_BACKING_MAP_CREATE (1 << 0) +#define FUSE_BACKING_MAP_CYCLIC (1 << 1) + +struct fuse_notify_backing_map_out { + uint64_t backing_id; + uint32_t num_extents; + uint32_t flags; + uint64_t reserved[2]; +}; + +#define FUSE_MAX_EXTENTS 1365 /* (1 << 16) / sizeof(struct fuse_extent) */ + +struct fuse_extent { + uint64_t offset; /* offset of extent into parent backing */ + uint64_t length; /* extent length */ + uint64_t backing_id; /* target backing */ + uint64_t addr; /* target offset within backing file/device */ + uint64_t reserved[2]; +}; + #define FUSE_SETUPMAPPING_FLAG_WRITE (1ull << 0) #define FUSE_SETUPMAPPING_FLAG_READ (1ull << 1) struct fuse_setupmapping_in { @@ -1263,12 +1322,12 @@ struct fuse_supp_groups { }; /** - * Size of the ring buffer header + * define FUSE_URING_IN_OUT_HEADER_SZ - Size of the ring buffer header */ #define FUSE_URING_IN_OUT_HEADER_SZ 128 #define FUSE_URING_OP_IN_OUT_SZ 128 -/** +/* * fuse_uring_ent_in_out flags * * FUSE_URING_ENT_ZERO_COPY: Set if the ent's payload is zero-copied @@ -1295,34 +1354,52 @@ struct fuse_uring_ent_in_out { }; /** - * Header for all fuse-io-uring requests + * struct fuse_uring_req_header - Header for all fuse-io-uring requests */ struct fuse_uring_req_header { - /* struct fuse_in_header / struct fuse_out_header */ + /** + * @in_out: struct fuse_in_header / struct fuse_out_header + */ char in_out[FUSE_URING_IN_OUT_HEADER_SZ]; - /* per op code header */ + /** + * @op_in: per op code header + */ char op_in[FUSE_URING_OP_IN_OUT_SZ]; + /** + * @ring_ent_in_out: TBD + */ struct fuse_uring_ent_in_out ring_ent_in_out; }; /** - * sqe commands to the kernel + * enum fuse_uring_cmd - sqe commands to the kernel */ enum fuse_uring_cmd { + /** @FUSE_IO_URING_CMD_INVALID: invalid command value */ FUSE_IO_URING_CMD_INVALID = 0, - /* register the request buffer and fetch a fuse request */ + /** + * @FUSE_IO_URING_CMD_REGISTER: register the request buffer and + * fetch a fuse request + */ FUSE_IO_URING_CMD_REGISTER = 1, - /* commit fuse request result and fetch next request */ + /** + * @FUSE_IO_URING_CMD_COMMIT_AND_FETCH: commit fuse request result + * and fetch next request + */ FUSE_IO_URING_CMD_COMMIT_AND_FETCH = 2, - /* add a queue */ + /** + * @FUSE_IO_URING_CMD_ADD_QUEUE: add a queue + */ FUSE_IO_URING_CMD_ADD_QUEUE = 3, - /* add a bufpool to a queue */ + /** + * @FUSE_IO_URING_CMD_ADD_BUFPOOL: add a bufpool to a queue + */ FUSE_IO_URING_CMD_ADD_BUFPOOL = 4, }; @@ -1334,7 +1411,7 @@ enum fuse_uring_cmd { */ #define FUSE_URING_ZERO_COPY (1 << 0) -/** +/* * In the 80B command area of the SQE. */ struct fuse_uring_cmd_req { diff --git a/tools/testing/selftests/filesystems/fuse/.gitignore b/tools/testing/selftests/filesystems/fuse/.gitignore index fb51603fe419..7c787313e69b 100644 --- a/tools/testing/selftests/filesystems/fuse/.gitignore +++ b/tools/testing/selftests/filesystems/fuse/.gitignore @@ -1,4 +1,8 @@ # SPDX-License-Identifier: GPL-2.0-only fuse_mnt fusectl_test +test_syncfs write_extend_eof_test +fuse_acl_cache_test +fuse_symlink_cache_test +fuse_readdir_cache_test diff --git a/tools/testing/selftests/filesystems/fuse/Makefile b/tools/testing/selftests/filesystems/fuse/Makefile index 95a1ee947ca7..e3ae3acff113 100644 --- a/tools/testing/selftests/filesystems/fuse/Makefile +++ b/tools/testing/selftests/filesystems/fuse/Makefile @@ -2,33 +2,35 @@ CFLAGS += -Wall -O2 -g $(KHDR_INCLUDES) -TEST_GEN_PROGS := fusectl_test +TEST_GEN_PROGS := fusectl_test test_syncfs TEST_GEN_PROGS += write_extend_eof_test -TEST_GEN_FILES := fuse_mnt - -# fuse_acl_cache_test requires libfuse3; add it only when the library is present. -ACL_CFLAGS := $(shell pkg-config fuse3 --cflags 2>/dev/null) -ACL_LDLIBS := $(shell pkg-config fuse3 --libs 2>/dev/null) -ifneq ($(ACL_CFLAGS),) TEST_GEN_PROGS += fuse_acl_cache_test -endif +TEST_GEN_PROGS += fuse_symlink_cache_test +TEST_GEN_PROGS += fuse_readdir_cache_test +TEST_GEN_FILES := fuse_mnt include ../../lib.mk $(OUTPUT)/write_extend_eof_test: LDLIBS += -lpthread -VAR_CFLAGS := $(shell pkg-config fuse --cflags 2>/dev/null) +VAR_CFLAGS := $(shell pkg-config fuse3 --cflags 2>/dev/null) ifeq ($(VAR_CFLAGS),) -VAR_CFLAGS := -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse +VAR_CFLAGS := -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse3 endif -VAR_LDLIBS := $(shell pkg-config fuse --libs 2>/dev/null) +VAR_LDLIBS := $(shell pkg-config fuse3 --libs 2>/dev/null) ifeq ($(VAR_LDLIBS),) -VAR_LDLIBS := -lfuse -pthread +VAR_LDLIBS := -lfuse3 -pthread endif +CFLAGS += $(VAR_CFLAGS) +LDLIBS += $(VAR_LDLIBS) + $(OUTPUT)/fuse_mnt: CFLAGS += $(VAR_CFLAGS) $(OUTPUT)/fuse_mnt: LDLIBS += $(VAR_LDLIBS) -$(OUTPUT)/fuse_acl_cache_test: CFLAGS += $(ACL_CFLAGS) -$(OUTPUT)/fuse_acl_cache_test: LDLIBS += $(ACL_LDLIBS) +$(OUTPUT)/fuse_acl_cache_test: fuse_common.c fuse_acl_cache_test.c +$(OUTPUT)/fuse_symlink_cache_test: fuse_common.c fuse_symlink_cache_test.c +$(OUTPUT)/fuse_readdir_cache_test: fuse_common.c fuse_readdir_cache_test.c + +EXTRA_CLEAN := fuse_common.o diff --git a/tools/testing/selftests/filesystems/fuse/fuse_acl_cache_test.c b/tools/testing/selftests/filesystems/fuse/fuse_acl_cache_test.c index 2411a6e285f1..9c8840fe816b 100644 --- a/tools/testing/selftests/filesystems/fuse/fuse_acl_cache_test.c +++ b/tools/testing/selftests/filesystems/fuse/fuse_acl_cache_test.c @@ -33,23 +33,15 @@ */ #define _GNU_SOURCE -#include #include #include -#include #include -#include -#include -#include -#include #include -#include - -#define FUSE_USE_VERSION 31 -#include #include "kselftest_harness.h" +#include "fuse_common.h" + /* ---- ACL binary encoding ------------------------------------------------ */ /* * POSIX ACL v2 xattr format (little-endian): @@ -88,9 +80,12 @@ static const uint8_t acl_b[] = { struct daemon_state { pthread_mutex_t lock; - const uint8_t *acl; + uint8_t *acl; size_t acl_size; int getxattr_count; + uid_t uid; + uid_t gid; + bool cache; }; /* @@ -99,9 +94,17 @@ struct daemon_state { */ static struct daemon_state g_ds = { .lock = PTHREAD_MUTEX_INITIALIZER, + .cache = false, }; /* ---- FUSE lowlevel callbacks -------------------------------------------- */ +static void fs_init(void *userdata, struct fuse_conn_info *conn) +{ + pthread_mutex_lock(&g_ds.lock); + if (g_ds.cache) + fuse_set_feature_flag(conn, FUSE_CAP_POSIX_ACL); + pthread_mutex_unlock(&g_ds.lock); +} static void fs_lookup(fuse_req_t req, fuse_ino_t parent, const char *name) { @@ -123,6 +126,10 @@ static void fs_lookup(fuse_req_t req, fuse_ino_t parent, const char *name) e.attr.st_ino = FILE_INO; e.attr.st_mode = S_IFREG | 0644; e.attr.st_nlink = 1; + pthread_mutex_lock(&g_ds.lock); + e.attr.st_uid = g_ds.uid; + e.attr.st_gid = g_ds.gid; + pthread_mutex_unlock(&g_ds.lock); fuse_reply_entry(req, &e); } @@ -140,6 +147,10 @@ static void fs_getattr(fuse_req_t req, fuse_ino_t ino, st.st_ino = FILE_INO; st.st_mode = S_IFREG | 0644; st.st_nlink = 1; + pthread_mutex_lock(&g_ds.lock); + st.st_uid = g_ds.uid; + st.st_gid = g_ds.gid; + pthread_mutex_unlock(&g_ds.lock); } else { fuse_reply_err(req, ENOENT); return; @@ -150,95 +161,112 @@ static void fs_getattr(fuse_req_t req, fuse_ino_t ino, static void fs_getxattr(fuse_req_t req, fuse_ino_t ino, const char *name, size_t size) { + uint8_t *acl = NULL; + size_t acl_size; + if (ino != FILE_INO || strcmp(name, "system.posix_acl_access") != 0) { fuse_reply_err(req, ENODATA); return; } + if (size) { + acl = malloc(size); + if (!acl) { + fuse_reply_err(req, ENOMEM); + return; + } + } pthread_mutex_lock(&g_ds.lock); - const uint8_t *acl = g_ds.acl; - size_t acl_size = g_ds.acl_size; + acl_size = g_ds.acl_size; + if (acl && acl_size && (size >= acl_size)) + memcpy(acl, g_ds.acl, acl_size); g_ds.getxattr_count++; pthread_mutex_unlock(&g_ds.lock); - if (size == 0) + if (acl_size == 0) + fuse_reply_err(req, ENODATA); + else if (size == 0) fuse_reply_xattr(req, acl_size); else if (size < acl_size) fuse_reply_err(req, ERANGE); else fuse_reply_buf(req, (const char *)acl, acl_size); + + free(acl); +} + +static void fs_setxattr(fuse_req_t req, fuse_ino_t ino, const char *name, + const char *value, size_t size, int flags) +{ + int ret = 0; + uint8_t *acl; + + if (ino != FILE_INO) { + ret = ENOENT; + goto out; + } else if (strcmp(name, "system.posix_acl_access") != 0) { + ret = ENOTSUP; + goto out; + } + acl = malloc(size); + if (!acl) { + ret = ENOMEM; + goto out; + } + memcpy(acl, value, size); + pthread_mutex_lock(&g_ds.lock); + if (g_ds.acl) + free(g_ds.acl); + g_ds.acl = acl; + g_ds.acl_size = size; + pthread_mutex_unlock(&g_ds.lock); + +out: + fuse_reply_err(req, ret); } static const struct fuse_lowlevel_ops fs_ops = { + .init = fs_init, .lookup = fs_lookup, .getattr = fs_getattr, .getxattr = fs_getxattr, + .setxattr = fs_setxattr, }; -/* ---- Daemon thread ------------------------------------------------------- */ - -static void *run_daemon(void *arg) -{ - fuse_session_loop((struct fuse_session *)arg); - return NULL; -} - /* ---- kselftest harness --------------------------------------------------- */ FIXTURE(acl_cache) { - struct fuse_session *se; - char mountpoint[PATH_MAX]; - char file_path[PATH_MAX]; - pthread_t thread; + struct fuse_common_ctx ctx; + char file_path[PATH_MAX]; }; FIXTURE_SETUP(acl_cache) { - char *fuse_argv[] = { "fuse_acl_cache_test", NULL }; - struct fuse_args args = FUSE_ARGS_INIT(1, fuse_argv); + char err[MAX_ERR_MSG]; - g_ds.acl = acl_a; - g_ds.acl_size = sizeof(acl_a); + pthread_mutex_lock(&g_ds.lock); + g_ds.acl_size = sizeof(acl_a); + g_ds.acl = malloc(g_ds.acl_size); + ASSERT_NE(g_ds.acl, NULL); + memcpy(g_ds.acl, acl_a, g_ds.acl_size); g_ds.getxattr_count = 0; + g_ds.uid = getuid(); + g_ds.gid = getgid(); + g_ds.cache = false; + pthread_mutex_unlock(&g_ds.lock); - strcpy(self->mountpoint, "/tmp/acl_cache_test_XXXXXX"); - if (!mkdtemp(self->mountpoint)) - SKIP(return, "mkdtemp: %s", strerror(errno)); + if (fs_setup(&fs_ops, &self->ctx, err)) + SKIP(return, err); snprintf(self->file_path, sizeof(self->file_path), - "%s/" FILE_NAME, self->mountpoint); - - self->se = fuse_session_new(&args, &fs_ops, sizeof(fs_ops), NULL); - if (!self->se) { - rmdir(self->mountpoint); - SKIP(return, "fuse_session_new failed"); - } - - if (fuse_session_mount(self->se, self->mountpoint)) { - fuse_session_destroy(self->se); - rmdir(self->mountpoint); - SKIP(return, "fuse_session_mount failed " - "(missing fusermount3 or insufficient privileges)"); - } - - if (pthread_create(&self->thread, NULL, run_daemon, self->se)) { - fuse_session_unmount(self->se); - fuse_session_destroy(self->se); - rmdir(self->mountpoint); - SKIP(return, "pthread_create: %s", strerror(errno)); - } - - fuse_opt_free_args(&args); + "%s/" FILE_NAME, self->ctx.mountpoint); } FIXTURE_TEARDOWN(acl_cache) { - fuse_session_exit(self->se); - fuse_session_unmount(self->se); - pthread_join(self->thread, NULL); - fuse_session_destroy(self->se); - rmdir(self->mountpoint); + fs_teardown(&self->ctx); + free(g_ds.acl); } static int do_force_statx(const char *path) @@ -254,6 +282,7 @@ TEST_F(acl_cache, stale_after_force_sync) char buf[512]; ssize_t sz; int count; + uint8_t *acl; /* * Step 1: two getxattr calls before any statx(FORCE_SYNC). @@ -314,8 +343,13 @@ TEST_F(acl_cache, stale_after_force_sync) * !fc->posix_acl mounts (it skips forget_all_cached_acls in that case). * On a fixed kernel the ACL was never cached, so this is moot. */ + acl = malloc(sizeof(acl_b)); + ASSERT_NE(acl, NULL); + memcpy(acl, acl_b, sizeof(acl_b)); + pthread_mutex_lock(&g_ds.lock); - g_ds.acl = acl_b; + free(g_ds.acl); + g_ds.acl = acl; g_ds.acl_size = sizeof(acl_b); pthread_mutex_unlock(&g_ds.lock); TH_LOG("step 4: daemon switched to ACL_B (%zu bytes)", sizeof(acl_b)); @@ -344,4 +378,145 @@ TEST_F(acl_cache, stale_after_force_sync) EXPECT_EQ(count, 4); } +FIXTURE(acl_cache_onoff) +{ + struct fuse_common_ctx ctx; + char pathname[PATH_MAX]; +}; + +FIXTURE_VARIANT(acl_cache_onoff) { bool cache; }; +FIXTURE_VARIANT_ADD(acl_cache_onoff, nocache) { .cache = false, }; +FIXTURE_VARIANT_ADD(acl_cache_onoff, docache) { .cache = true, }; + +FIXTURE_SETUP(acl_cache_onoff) +{ + char err[MAX_ERR_MSG]; + + pthread_mutex_lock(&g_ds.lock); + g_ds.acl = NULL; + g_ds.acl_size = 0; + g_ds.getxattr_count = 0; + g_ds.cache = variant->cache; + g_ds.uid = getuid(); + g_ds.gid = getgid(); + pthread_mutex_unlock(&g_ds.lock); + + if (fs_setup(&fs_ops, &self->ctx, err)) + SKIP(return, err); + + snprintf(self->pathname, sizeof(self->pathname), + "%s/" FILE_NAME, self->ctx.mountpoint); +} + +FIXTURE_TEARDOWN(acl_cache_onoff) +{ + fs_teardown(&self->ctx); + free(g_ds.acl); +} + +/* + * This is the most basic ACL caching test: verify that, when reading ACLs for + * an inode, user-space is called: + * - Only once if ACLs cache is enabled, or + * - Once per access if cache is disabled. + */ +TEST_F(acl_cache_onoff, test_acl_cache_enable_disable) +{ + char buf[512]; + ssize_t sz; + bool cache; + int counter; + int i; + + ASSERT_EQ(lsetxattr(self->pathname, "system.posix_acl_access", + acl_a, sizeof(acl_a), 0), 0); + + for (i = 0; i < 100; i++) { + sz = lgetxattr(self->pathname, "system.posix_acl_access", + buf, sizeof(buf)); + ASSERT_EQ(sz, sizeof(acl_a)); + ASSERT_EQ(memcmp(buf, acl_a, sz), 0); + } + + pthread_mutex_lock(&g_ds.lock); + counter = g_ds.getxattr_count; + cache = g_ds.cache; + pthread_mutex_unlock(&g_ds.lock); + + if (cache) { + ASSERT_EQ(counter, 1); + } else { + ASSERT_EQ(counter, 100); + } + + TH_LOG("User-space called %d time(s) with ACL caching %s", + counter, cache ? "enabled" : "disabled"); +} + +/* + * Test caching invalidation for several scenarios: + * 1. When a new ACL is set + * 2. When invalidating an inode (NOTIFY_INVAL_INODE) + */ +TEST_F(acl_cache_onoff, test_acl_cache_invalidation) +{ + char buf[512]; + ssize_t sz; + int counter; + bool cache; + int i; + + /* Set an ACL */ + ASSERT_EQ(lsetxattr(self->pathname, "system.posix_acl_access", + acl_a, sizeof(acl_a), 0), 0); + + for (i = 0; i < 100; i++) { + sz = lgetxattr(self->pathname, "system.posix_acl_access", + buf, sizeof(buf)); + ASSERT_EQ(sz, sizeof(acl_a)); + ASSERT_EQ(memcmp(buf, acl_a, sz), 0); + } + + /* 1. force cache invalidation by setting a new ACL */ + ASSERT_EQ(lsetxattr(self->pathname, "system.posix_acl_access", + acl_b, sizeof(acl_b), 0), 0); + + sz = lgetxattr(self->pathname, "system.posix_acl_access", + buf, sizeof(buf)); + ASSERT_EQ(sz, sizeof(acl_b)); + ASSERT_EQ(memcmp(buf, acl_b, sz), 0); + + pthread_mutex_lock(&g_ds.lock); + counter = g_ds.getxattr_count; + cache = g_ds.cache; + pthread_mutex_unlock(&g_ds.lock); + + if (cache) { + ASSERT_EQ(counter, 2); + } else { + ASSERT_EQ(counter, 101); + } + TH_LOG("Invalidation by setting new ACL: OK"); + + /* 2. send FUSE_NOTIFY_INVAL_INODE */ + fuse_lowlevel_notify_inval_inode(self->ctx.se, FILE_INO, 0, 0); + + sz = lgetxattr(self->pathname, "system.posix_acl_access", + buf, sizeof(buf)); + ASSERT_EQ(sz, sizeof(acl_b)); + ASSERT_EQ(memcmp(buf, acl_b, sz), 0); + + pthread_mutex_lock(&g_ds.lock); + counter = g_ds.getxattr_count; + cache = g_ds.cache; + pthread_mutex_unlock(&g_ds.lock); + + if (cache) { + ASSERT_EQ(counter, 3); + } else { + ASSERT_EQ(counter, 102); + } + TH_LOG("Invalidation through FUSE_NOTIFY_INVAL_INODE: OK"); +} + TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/filesystems/fuse/fuse_common.c b/tools/testing/selftests/filesystems/fuse/fuse_common.c new file mode 100644 index 000000000000..d1d3dfd2ea67 --- /dev/null +++ b/tools/testing/selftests/filesystems/fuse/fuse_common.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "fuse_common.h" + +static void *run_daemon(void *arg) +{ + fuse_session_loop((struct fuse_session *)arg); + return NULL; +} + +int fs_setup(const struct fuse_lowlevel_ops *fs_ops, + struct fuse_common_ctx *ctx, char *err) +{ + char *fuse_argv[] = { "fuse_test", NULL }; + struct fuse_args args = FUSE_ARGS_INIT(1, fuse_argv); + + strcpy(ctx->mountpoint, MOUNTPOINT_TEMPLATE); + if (!mkdtemp(ctx->mountpoint)) { + snprintf(err, MAX_ERR_MSG, "mkdtemp: %s", strerror(errno)); + return -1; + } + + ctx->se = fuse_session_new(&args, fs_ops, sizeof(*fs_ops), NULL); + if (!ctx->se) { + rmdir(ctx->mountpoint); + snprintf(err, MAX_ERR_MSG, "fuse_session_new failed"); + return -1; + } + + if (fuse_session_mount(ctx->se, ctx->mountpoint)) { + fuse_session_destroy(ctx->se); + rmdir(ctx->mountpoint); + snprintf(err, MAX_ERR_MSG, "fuse_session_mount failed " + "(missing fusermount3 or insufficient privileges)"); + return -1; + } + + if (pthread_create(&ctx->thread, NULL, run_daemon, ctx->se)) { + fuse_session_unmount(ctx->se); + fuse_session_destroy(ctx->se); + rmdir(ctx->mountpoint); + snprintf(err, MAX_ERR_MSG, "pthread_create: %s", strerror(errno)); + return -1; + } + + fuse_opt_free_args(&args); + + return 0; +} + +void fs_teardown(struct fuse_common_ctx *ctx) +{ + fuse_session_exit(ctx->se); + fuse_session_unmount(ctx->se); + pthread_join(ctx->thread, NULL); + fuse_session_destroy(ctx->se); + rmdir(ctx->mountpoint); +} diff --git a/tools/testing/selftests/filesystems/fuse/fuse_common.h b/tools/testing/selftests/filesystems/fuse/fuse_common.h new file mode 100644 index 000000000000..5beb6bf89425 --- /dev/null +++ b/tools/testing/selftests/filesystems/fuse/fuse_common.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __SELFTEST_FUSE_COMMON_H__ +#define __SELFTEST_FUSE_COMMON_H__ + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +#define FUSE_USE_VERSION 31 +#include + +#define MAX_ERR_MSG 256 + +#define MOUNTPOINT_TEMPLATE "/tmp/fuse_test_XXXXXX" +#define MOUNTPOINT_SZ 64 + +struct fuse_common_ctx { + struct fuse_session *se; + char mountpoint[MOUNTPOINT_SZ]; + pthread_t thread; +}; + +int fs_setup(const struct fuse_lowlevel_ops *fs_ops, + struct fuse_common_ctx *ctx, char *err); +void fs_teardown(struct fuse_common_ctx *ctx); + +#endif /* __SELFTEST_FUSE_COMMON_H__ */ diff --git a/tools/testing/selftests/filesystems/fuse/fuse_mnt.c b/tools/testing/selftests/filesystems/fuse/fuse_mnt.c index d12b17f30fad..5d335fa5cf05 100644 --- a/tools/testing/selftests/filesystems/fuse/fuse_mnt.c +++ b/tools/testing/selftests/filesystems/fuse/fuse_mnt.c @@ -4,7 +4,7 @@ * Creates a simple FUSE filesystem with a single read-write file (/test) */ -#define FUSE_USE_VERSION 26 +#define FUSE_USE_VERSION 31 #include #include @@ -20,7 +20,8 @@ static char *content; static size_t content_size = 0; static const char test_path[] = "/test"; -static int test_getattr(const char *path, struct stat *st) +static int test_getattr(const char *path, struct stat *st, + struct fuse_file_info *fi) { memset(st, 0, sizeof(*st)); @@ -41,14 +42,15 @@ static int test_getattr(const char *path, struct stat *st) } static int test_readdir(const char *path, void *buf, fuse_fill_dir_t filler, - off_t offset, struct fuse_file_info *fi) + off_t offset, struct fuse_file_info *fi, + enum fuse_readdir_flags flags) { if (strcmp(path, "/")) return -ENOENT; - filler(buf, ".", NULL, 0); - filler(buf, "..", NULL, 0); - filler(buf, test_path + 1, NULL, 0); + filler(buf, ".", NULL, 0, FUSE_FILL_DIR_DEFAULTS); + filler(buf, "..", NULL, 0, FUSE_FILL_DIR_DEFAULTS); + filler(buf, test_path + 1, NULL, 0, FUSE_FILL_DIR_DEFAULTS); return 0; } @@ -107,7 +109,8 @@ static int test_write(const char *path, const char *buf, size_t size, return size; } -static int test_truncate(const char *path, off_t size) +static int test_truncate(const char *path, off_t size, + struct fuse_file_info *fi) { if (strcmp(path, test_path) != 0) return -ENOENT; diff --git a/tools/testing/selftests/filesystems/fuse/fuse_readdir_cache_test.c b/tools/testing/selftests/filesystems/fuse/fuse_readdir_cache_test.c new file mode 100644 index 000000000000..59314f996449 --- /dev/null +++ b/tools/testing/selftests/filesystems/fuse/fuse_readdir_cache_test.c @@ -0,0 +1,273 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Simple filesystem to test FUSE readdir cache + * + * It will simply perform readdir operations on a directory checking how many + * times a request is sent to user-space using all the possible caching + * combination setting (FOPEN_KEEP_CACHE and FOPEN_CACHE_DIR flags). + */ + +#include +#include + +#include "kselftest_harness.h" + +#include "fuse_common.h" + +#define DIRNAME "mydir" +#define FILENAME "myfile" + +#define DIR_INO 42 +#define FILE_INO 43 +#define DOT_INO 40 +#define DOTDOT_INO 41 + +#define TIMEOUT 86400.0f + +struct test_state { + pthread_mutex_t lock; + time_t mtime; + int readdir_counter; + bool cache_readdir; + bool keep_cache; +} test_state = { + .lock = PTHREAD_MUTEX_INITIALIZER, +}; + +static void fs_lookup(fuse_req_t req, fuse_ino_t parent, const char *name) +{ + struct fuse_entry_param e = {}; + + if (parent != FUSE_ROOT_ID || strcmp(name, DIRNAME) != 0) { + fuse_reply_err(req, ENOENT); + return; + } + + e.ino = DIR_INO; + e.attr.st_mode = S_IFDIR | 0755; + e.attr.st_nlink = 1; + pthread_mutex_lock(&test_state.lock); + e.attr.st_mtime = test_state.mtime; + pthread_mutex_unlock(&test_state.lock); + e.attr_timeout = TIMEOUT; + e.entry_timeout = TIMEOUT; + + fuse_reply_entry(req, &e); +} + +static int fill_stat(fuse_ino_t ino, struct stat *st) +{ + int ret = 0; + + st->st_ino = ino; + pthread_mutex_lock(&test_state.lock); + st->st_mtime = test_state.mtime; + pthread_mutex_unlock(&test_state.lock); + + switch (ino) { + case FUSE_ROOT_ID: + st->st_mode = S_IFDIR | 0755; + st->st_nlink = 2; + break; + case DOT_INO: + case DOTDOT_INO: + case DIR_INO: + st->st_mode = S_IFDIR | 0755; + st->st_nlink = 1; + break; + case FILE_INO: + st->st_mode = S_IFREG | 0444; + st->st_nlink = 1; + break; + default: + ret = -1; + break; + } + + return ret; +} + +static void fs_getattr(fuse_req_t req, fuse_ino_t ino, + struct fuse_file_info *fi) +{ + struct stat st = {}; + + if (fill_stat(ino, &st) < 0) + fuse_reply_err(req, ENOENT); + else + fuse_reply_attr(req, &st, TIMEOUT); +} + +static void fs_opendir(fuse_req_t req, fuse_ino_t ino, + struct fuse_file_info *fi) +{ + pthread_mutex_lock(&test_state.lock); + fi->keep_cache = test_state.keep_cache; + fi->cache_readdir = test_state.cache_readdir; + pthread_mutex_unlock(&test_state.lock); + fuse_reply_open(req, fi); +} + +static void fs_readdir(fuse_req_t req, fuse_ino_t ino, size_t size, + off_t offset, struct fuse_file_info *fi) +{ + struct stat st = {}; + char buf[1024]; + char *pbuf; + size_t rem = size; + size_t sz; + int nextoff = 0; + + if (ino != DIR_INO) { + fuse_reply_err(req, ENOTDIR); + return; + } + if (offset) { + fuse_reply_buf(req, NULL, 0); + return; + } + pbuf = buf; + fill_stat(DOT_INO, &st); + sz = fuse_add_direntry(req, pbuf, rem, ".", &st, nextoff++); + rem -= sz; + pbuf += sz; + fill_stat(DOTDOT_INO, &st); + sz = fuse_add_direntry(req, pbuf, rem, "..", &st, nextoff++); + rem -= sz; + pbuf += sz; + fill_stat(FILE_INO, &st); + sz = fuse_add_direntry(req, pbuf, rem, FILENAME, &st, nextoff++); + rem -= sz; + + fuse_reply_buf(req, buf, size - rem); + + pthread_mutex_lock(&test_state.lock); + test_state.readdir_counter++; + pthread_mutex_unlock(&test_state.lock); +} + +static const struct fuse_lowlevel_ops fs_ops = { + .lookup = fs_lookup, + .getattr = fs_getattr, + .opendir = fs_opendir, + .readdir = fs_readdir, +}; + +FIXTURE(readdir_cache) +{ + struct fuse_common_ctx ctx; +}; + +FIXTURE_VARIANT(readdir_cache) +{ + bool cache_readdir; + bool keep_cache; +}; +FIXTURE_VARIANT_ADD(readdir_cache, nocache) +{ + .cache_readdir = false, + .keep_cache = false, +}; +FIXTURE_VARIANT_ADD(readdir_cache, cache_readdir) +{ + .cache_readdir = true, + .keep_cache = false, +}; +FIXTURE_VARIANT_ADD(readdir_cache, keep_cache) +{ + .cache_readdir = false, + .keep_cache = true, +}; +FIXTURE_VARIANT_ADD(readdir_cache, cache) +{ + .cache_readdir = true, + .keep_cache = true, +}; + +FIXTURE_SETUP(readdir_cache) +{ + char err[MAX_ERR_MSG]; + + pthread_mutex_lock(&test_state.lock); + test_state.mtime = time(NULL); + test_state.readdir_counter = 0; + test_state.cache_readdir = variant->cache_readdir; + test_state.keep_cache = variant->keep_cache; + pthread_mutex_unlock(&test_state.lock); + + if (fs_setup(&fs_ops, &self->ctx, err)) + SKIP(return, err); +} + +FIXTURE_TEARDOWN(readdir_cache) +{ + fs_teardown(&self->ctx); +} + +TEST_F(readdir_cache, test_readdir_cache) +{ + struct dirent *dentry; + DIR *dir; + char pathname[PATH_MAX]; + int total_counter, rewind_counter; + int dentrycount; + + sprintf(pathname, "%s/%s", self->ctx.mountpoint, DIRNAME); + + dir = opendir(pathname); + if (dir == NULL) + TH_LOG("opendir(): %s", strerror(errno)); + ASSERT_NE(dir, NULL); + + errno = 0; + dentrycount = 0; + while ((dentry = readdir(dir))) + dentrycount++; + ASSERT_EQ(errno, 0); + ASSERT_EQ(dentrycount, 3); + + rewinddir(dir); + errno = 0; + dentrycount = 0; + while ((dentry = readdir(dir))) + dentrycount++; + ASSERT_EQ(errno, 0); + ASSERT_EQ(dentrycount, 3); + + ASSERT_EQ(closedir(dir), 0); + + pthread_mutex_lock(&test_state.lock); + rewind_counter = test_state.readdir_counter; + pthread_mutex_unlock(&test_state.lock); + + dir = opendir(pathname); + if (dir == NULL) + TH_LOG("opendir(): %s", strerror(errno)); + ASSERT_NE(dir, NULL); + + errno = 0; + dentrycount = 0; + while ((dentry = readdir(dir))) + dentrycount++; + ASSERT_EQ(errno, 0); + ASSERT_EQ(dentrycount, 3); + + ASSERT_EQ(closedir(dir), 0); + + pthread_mutex_lock(&test_state.lock); + total_counter = test_state.readdir_counter; + pthread_mutex_unlock(&test_state.lock); + + if (!variant->cache_readdir) { + ASSERT_EQ(rewind_counter, 2); + ASSERT_EQ(total_counter, 3); + } else if (!variant->keep_cache) { + ASSERT_EQ(rewind_counter, 1); + ASSERT_EQ(total_counter, 2); + } else { + ASSERT_EQ(rewind_counter, 1); + ASSERT_EQ(total_counter, 1); + } +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/filesystems/fuse/fusectl_test.c b/tools/testing/selftests/filesystems/fuse/fusectl_test.c index 0d1d012c35ed..b828173c01bd 100644 --- a/tools/testing/selftests/filesystems/fuse/fusectl_test.c +++ b/tools/testing/selftests/filesystems/fuse/fusectl_test.c @@ -106,8 +106,15 @@ TEST_F(fusectl, abort) char path_buf[PATH_MAX]; int abort_fd, test_fd, ret; - sprintf(path_buf, "/sys/fs/fuse/connections/%d/abort", self->connection); + snprintf(path_buf, PATH_MAX, "%s/%d", FUSECTL_MOUNTPOINT, + self->connection); + if (access(path_buf, F_OK) != 0) + SKIP(return, + "fusectl doesn't seem to be mounted: %s", + strerror(errno)); + snprintf(path_buf, PATH_MAX, "%s/%d/abort", FUSECTL_MOUNTPOINT, + self->connection); ASSERT_EQ(0, access(path_buf, F_OK)); abort_fd = open(path_buf, O_WRONLY); diff --git a/tools/testing/selftests/filesystems/fuse/test_syncfs.c b/tools/testing/selftests/filesystems/fuse/test_syncfs.c new file mode 100644 index 000000000000..d3ca2613b45c --- /dev/null +++ b/tools/testing/selftests/filesystems/fuse/test_syncfs.c @@ -0,0 +1,309 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test that FUSE_SYNCFS is propagated to a userspace server only when the + * server opts in with FUSE_HAS_SYNCFS *and* opened /dev/fuse with + * CAP_SYS_ADMIN in the initial user namespace. + * + * Unlike the libfuse-based selftests, this talks the raw FUSE wire protocol + * over /dev/fuse so it can (a) choose whether to advertise FUSE_HAS_SYNCFS in + * the INIT reply and (b) observe directly whether a FUSE_SYNCFS opcode arrives. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../kselftest.h" + +#define FUSE_ROOT_ID 1 + +/* + * Add or drop CAP_SYS_ADMIN in the effective set via raw capget/capset + * (avoids a libcap dependency). Used to construct a server that opens + * /dev/fuse without the privilege FUSE_HAS_SYNCFS requires. + */ +static int set_sysadmin(int on) +{ + struct __user_cap_header_struct hdr = { + .version = _LINUX_CAPABILITY_VERSION_3, + .pid = 0, + }; + struct __user_cap_data_struct data[_LINUX_CAPABILITY_U32S_3] = {}; + unsigned int idx = CAP_SYS_ADMIN >> 5; + __u32 bit = 1U << (CAP_SYS_ADMIN & 31); + + if (syscall(SYS_capget, &hdr, data)) + return -1; + if (on) + data[idx].effective |= bit; + else + data[idx].effective &= ~bit; + return syscall(SYS_capset, &hdr, data); +} + +/* + * eventfd the server child writes once when it receives FUSE_SYNCFS; the + * parent poll()s it to observe (or rule out) propagation. + */ +static int syncfs_evfd; + +static void reply(int fd, uint64_t unique, int error, void *data, size_t len) +{ + struct fuse_out_header oh = { + .len = sizeof(oh) + len, + .error = error, + .unique = unique, + }; + struct iovec iov[2] = { + { &oh, sizeof(oh) }, + { data, len }, + }; + + if (writev(fd, iov, data ? 2 : 1) < 0) + ksft_print_msg("server writev failed: %s\n", strerror(errno)); +} + +static void fill_attr(struct fuse_attr *a, uint64_t ino, uint32_t mode, + uint32_t nlink) +{ + memset(a, 0, sizeof(*a)); + a->ino = ino; + a->mode = mode; + a->nlink = nlink; + a->blksize = 4096; +} + +/* + * Minimal FUSE server. Advertises FUSE_HAS_SYNCFS in its INIT reply iff + * @advertise is set. Signals syncfs_evfd when a FUSE_SYNCFS opcode arrives. + */ +#define SERVER_MAX_WRITE 65536 +static void run_server(int fd, int advertise) +{ + /* + * The kernel rejects reads (EINVAL) whose buffer is smaller than + * max_write + header, so size generously for the max_write we + * advertise in the INIT reply below. + */ + static char buf[SERVER_MAX_WRITE + 4096]; + + for (;;) { + ssize_t n = read(fd, buf, sizeof(buf)); + struct fuse_in_header *ih = (void *)buf; + + if (n < 0) { + if (errno == EINTR || errno == EAGAIN) + continue; + return; /* device closed on unmount/abort */ + } + if (n < (ssize_t)sizeof(*ih)) + continue; + + switch (ih->opcode) { + case FUSE_INIT: { + struct fuse_init_in *in = (void *)(ih + 1); + struct fuse_init_out out = {0}; + uint64_t flags = FUSE_INIT_EXT; + + out.major = FUSE_KERNEL_VERSION; + out.minor = FUSE_KERNEL_MINOR_VERSION; + out.max_readahead = in->max_readahead; + out.max_write = SERVER_MAX_WRITE; + out.max_background = 16; + out.congestion_threshold = 12; + if (advertise) + flags |= FUSE_HAS_SYNCFS; + out.flags = flags; + out.flags2 = flags >> 32; + reply(fd, ih->unique, 0, &out, sizeof(out)); + break; + } + case FUSE_GETATTR: { + struct fuse_attr_out out = {0}; + + fill_attr(&out.attr, FUSE_ROOT_ID, S_IFDIR | 0755, 2); + reply(fd, ih->unique, 0, &out, sizeof(out)); + break; + } + case FUSE_SYNCFS: { + uint64_t one = 1; + + if (write(syncfs_evfd, &one, sizeof(one)) < 0) + ksft_print_msg("server eventfd write failed: %s\n", + strerror(errno)); + reply(fd, ih->unique, 0, NULL, 0); + break; + } + default: + /* + * Anything else (e.g. OPENDIR from opening the mount + * root) is not needed to drive this test; -ENOSYS lets + * the kernel proceed. + */ + reply(fd, ih->unique, -ENOSYS, NULL, 0); + break; + } + } +} + +/* + * Mount a fuse fs backed by a forked server, issue syncfs(), and report + * whether the server observed FUSE_SYNCFS. Returns 0 on success, -1 if the + * environment could not support the test (caller should skip). + * + * If @unpriv_open is set, /dev/fuse is opened with CAP_SYS_ADMIN dropped + * (regained only for the mount() syscall), so the device opener -- the + * server principal FUSE_HAS_SYNCFS is gated on -- lacks the privilege even + * though it remains in the initial user namespace. + */ +static int do_mount_and_syncfs(const char *mnt, int advertise, int unpriv_open, + int *seen) +{ + struct pollfd pfd = { .events = POLLIN }; + char opts[256]; + int fd, mfd = -1, i; + pid_t pid; + + syncfs_evfd = eventfd(0, EFD_CLOEXEC); + if (syncfs_evfd < 0) + return -1; + + if (unpriv_open && set_sysadmin(0)) + goto out_evfd; + fd = open("/dev/fuse", O_RDWR); + if (unpriv_open && set_sysadmin(1)) { + if (fd >= 0) + close(fd); + goto out_evfd; + } + if (fd < 0) + goto out_evfd; + + mkdir(mnt, 0755); + snprintf(opts, sizeof(opts), + "fd=%d,rootmode=40000,user_id=%d,group_id=%d", + fd, getuid(), getgid()); + + if (mount("fuse", mnt, "fuse", 0, opts) < 0) + goto out_fd; + + pid = fork(); + if (pid < 0) + goto out_umount; + if (pid == 0) { + run_server(fd, advertise); + _exit(0); + } + + /* + * The parent does not service the fuse fd; the child does. Close our + * copy so the kernel sees a single server, and so that if the child + * dies the connection aborts instead of hanging us forever. + */ + close(fd); + + /* + * mount() returns before the server has answered FUSE_INIT, so the + * first open() can race and fail with ENOTCONN; retry until the + * handshake settles. + */ + for (i = 0; i < 1000; i++) { + mfd = open(mnt, O_RDONLY | O_DIRECTORY); + if (mfd >= 0) + break; + usleep(1000); + } + if (mfd >= 0) { + syncfs(mfd); + close(mfd); + } + + /* + * No waiting is needed: the server writes syncfs_evfd before it replies + * to FUSE_SYNCFS, and that reply is what unblocks the synchronous + * syncfs() above. So once syncfs() has returned, the eventfd is already + * signalled if the opcode was propagated, and will never be otherwise. + * poll() with a zero timeout therefore decides both cases immediately. + */ + pfd.fd = syncfs_evfd; + *seen = poll(&pfd, 1, 0) > 0 && (pfd.revents & POLLIN); + + kill(pid, SIGKILL); + waitpid(pid, NULL, 0); + umount2(mnt, MNT_DETACH); + close(syncfs_evfd); + return 0; + +out_umount: + umount2(mnt, MNT_DETACH); +out_fd: + close(fd); +out_evfd: + close(syncfs_evfd); + return -1; +} + +int main(void) +{ + char mnt[] = "/tmp/fuse_syncfs_XXXXXX"; + int seen, ret; + + ksft_print_header(); + ksft_set_plan(3); + + /* Hard watchdog: never let a stuck syncfs hang the test runner. */ + signal(SIGALRM, SIG_DFL); + alarm(60); + + if (geteuid() != 0) + ksft_exit_skip("test requires root to mount fuse\n"); + + if (!mkdtemp(mnt)) + ksft_exit_fail_msg("mkdtemp failed\n"); + + /* T1: host-root mount, server opts in -> syncfs must reach server. */ + ret = do_mount_and_syncfs(mnt, 1, 0, &seen); + if (ret < 0) + ksft_test_result_skip("T1: could not mount fuse\n"); + else + ksft_test_result(seen, + "T1 host-root + FUSE_HAS_SYNCFS: server receives FUSE_SYNCFS\n"); + + /* T2: host-root mount, server does NOT opt in -> no FUSE_SYNCFS. */ + ret = do_mount_and_syncfs(mnt, 0, 0, &seen); + if (ret < 0) + ksft_test_result_skip("T2: could not mount fuse\n"); + else + ksft_test_result(!seen, + "T2 host-root, no opt-in: server does NOT receive FUSE_SYNCFS\n"); + + /* + * T3: server opts in but opened /dev/fuse without CAP_SYS_ADMIN while + * still in the initial user namespace -> kernel must withhold + * FUSE_SYNCFS. This is the case that distinguishes gating on the + * server's privilege from gating on the mount's user namespace. + */ + ret = do_mount_and_syncfs(mnt, 1, 1, &seen); + if (ret < 0) + ksft_test_result_skip("T3: could not mount fuse unprivileged\n"); + else + ksft_test_result(!seen, + "T3 init_userns, opener lacks CAP_SYS_ADMIN: FUSE_SYNCFS withheld\n"); + + rmdir(mnt); + ksft_finished(); +}