github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-08-01 03:59:40 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Merge fea3043314 ("Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4") into android-mainline

Browse Source 
Steps on the way to 5.19-rc1

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I1eac31f4d4c79c1276b4d03b7488a8a4592a0d69
This commit is contained in:
Greg Kroah-Hartman 2022-06-07 18:23:29 +02:00
commit 79bd68ad25
119 changed files with 7208 additions and 4960 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

178
Documentation/filesystems/caching/cachefiles.rst
View File

@ -28,6 +28,7 @@ Cache on Already Mounted Filesystem
(*) Debugging.
(*) On-demand Read.
Overview
@ -482,3 +483,180 @@ the control file. For example::
echo $((1|4|8)) >/sys/module/cachefiles/parameters/debug
will turn on all function entry debugging.
On-demand Read
==============
When working in its original mode, CacheFiles serves as a local cache for a
remote networking fs - while in on-demand read mode, CacheFiles can boost the
scenario where on-demand read semantics are needed, e.g. container image
distribution.
The essential difference between these two modes is seen when a cache miss
occurs: In the original mode, the netfs will fetch the data from the remote
server and then write it to the cache file; in on-demand read mode, fetching
the data and writing it into the cache is delegated to a user daemon.
``CONFIG_CACHEFILES_ONDEMAND`` should be enabled to support on-demand read mode.
Protocol Communication
----------------------
The on-demand read mode uses a simple protocol for communication between kernel
and user daemon. The protocol can be modeled as::
kernel --[request]--> user daemon --[reply]--> kernel
CacheFiles will send requests to the user daemon when needed. The user daemon
should poll the devnode ('/dev/cachefiles') to check if there's a pending
request to be processed. A POLLIN event will be returned when there's a pending
request.
The user daemon then reads the devnode to fetch a request to process. It should
be noted that each read only gets one request. When it has finished processing
the request, the user daemon should write the reply to the devnode.
Each request starts with a message header of the form::
struct cachefiles_msg {
__u32 msg_id;
__u32 opcode;
__u32 len;
__u32 object_id;
__u8 data[];
};
where:
* ``msg_id`` is a unique ID identifying this request among all pending
requests.
* ``opcode`` indicates the type of this request.
* ``object_id`` is a unique ID identifying the cache file operated on.
* ``data`` indicates the payload of this request.
* ``len`` indicates the whole length of this request, including the
header and following type-specific payload.
Turning on On-demand Mode
-------------------------
An optional parameter becomes available to the "bind" command::
bind [ondemand]
When the "bind" command is given no argument, it defaults to the original mode.
When it is given the "ondemand" argument, i.e. "bind ondemand", on-demand read
mode will be enabled.
The OPEN Request
----------------
When the netfs opens a cache file for the first time, a request with the
CACHEFILES_OP_OPEN opcode, a.k.a an OPEN request will be sent to the user
daemon. The payload format is of the form::
struct cachefiles_open {
__u32 volume_key_size;
__u32 cookie_key_size;
__u32 fd;
__u32 flags;
__u8 data[];
};
where:
* ``data`` contains the volume_key followed directly by the cookie_key.
The volume key is a NUL-terminated string; the cookie key is binary
data.
* ``volume_key_size`` indicates the size of the volume key in bytes.
* ``cookie_key_size`` indicates the size of the cookie key in bytes.
* ``fd`` indicates an anonymous fd referring to the cache file, through
which the user daemon can perform write/llseek file operations on the
cache file.
The user daemon can use the given (volume_key, cookie_key) pair to distinguish
the requested cache file. With the given anonymous fd, the user daemon can
fetch the data and write it to the cache file in the background, even when
kernel has not triggered a cache miss yet.
Be noted that each cache file has a unique object_id, while it may have multiple
anonymous fds. The user daemon may duplicate anonymous fds from the initial
anonymous fd indicated by the @fd field through dup(). Thus each object_id can
be mapped to multiple anonymous fds, while the usr daemon itself needs to
maintain the mapping.
When implementing a user daemon, please be careful of RLIMIT_NOFILE,
``/proc/sys/fs/nr_open`` and ``/proc/sys/fs/file-max``. Typically these needn't
be huge since they're related to the number of open device blobs rather than
open files of each individual filesystem.
The user daemon should reply the OPEN request by issuing a "copen" (complete
open) command on the devnode::
copen <msg_id>,<cache_size>
where:
* ``msg_id`` must match the msg_id field of the OPEN request.
* When >= 0, ``cache_size`` indicates the size of the cache file;
when < 0, ``cache_size`` indicates any error code encountered by the
user daemon.
The CLOSE Request
-----------------
When a cookie withdrawn, a CLOSE request (opcode CACHEFILES_OP_CLOSE) will be
sent to the user daemon. This tells the user daemon to close all anonymous fds
associated with the given object_id. The CLOSE request has no extra payload,
and shouldn't be replied.
The READ Request
----------------
When a cache miss is encountered in on-demand read mode, CacheFiles will send a
READ request (opcode CACHEFILES_OP_READ) to the user daemon. This tells the user
daemon to fetch the contents of the requested file range. The payload is of the
form::
struct cachefiles_read {
__u64 off;
__u64 len;
};
where:
* ``off`` indicates the starting offset of the requested file range.
* ``len`` indicates the length of the requested file range.
When it receives a READ request, the user daemon should fetch the requested data
and write it to the cache file identified by object_id.
When it has finished processing the READ request, the user daemon should reply
by using the CACHEFILES_IOC_READ_COMPLETE ioctl on one of the anonymous fds
associated with the object_id given in the READ request. The ioctl is of the
form::
ioctl(fd, CACHEFILES_IOC_READ_COMPLETE, msg_id);
where:
* ``fd`` is one of the anonymous fds associated with the object_id
given.
* ``msg_id`` must match the msg_id field of the READ request.

7
block/bdev.c
View File

@ -200,6 +200,13 @@ int sync_blockdev(struct block_device *bdev)
}
EXPORT_SYMBOL(sync_blockdev);
int sync_blockdev_range(struct block_device *bdev, loff_t lstart, loff_t lend)
{
return filemap_write_and_wait_range(bdev->bd_inode->i_mapping,
lstart, lend);
}
EXPORT_SYMBOL(sync_blockdev_range);
/*
* Write out and wait upon all dirty data associated with this
* device. Filesystem data as well as the underlying block

39
fs/btrfs/acl.c
View File

@ -55,9 +55,8 @@ struct posix_acl *btrfs_get_acl(struct inode *inode, int type, bool rcu)
return acl;
}
static int __btrfs_set_acl(struct btrfs_trans_handle *trans,
struct user_namespace *mnt_userns,
struct inode *inode, struct posix_acl *acl, int type)
int __btrfs_set_acl(struct btrfs_trans_handle *trans, struct inode *inode,
struct posix_acl *acl, int type)
{
int ret, size = 0;
const char *name;
@ -123,40 +122,8 @@ int btrfs_set_acl(struct user_namespace *mnt_userns, struct inode *inode,
if (ret)
return ret;
}
ret = __btrfs_set_acl(NULL, mnt_userns, inode, acl, type);
ret = __btrfs_set_acl(NULL, inode, acl, type);
if (ret)
inode->i_mode = old_mode;
return ret;
}
int btrfs_init_acl(struct btrfs_trans_handle *trans,
struct inode *inode, struct inode *dir)
{
struct posix_acl *default_acl, *acl;
int ret = 0;
/* this happens with subvols */
if (!dir)
return 0;
ret = posix_acl_create(dir, &inode->i_mode, &default_acl, &acl);
if (ret)
return ret;
if (default_acl) {
ret = __btrfs_set_acl(trans, &init_user_ns, inode, default_acl,
ACL_TYPE_DEFAULT);
posix_acl_release(default_acl);
}
if (acl) {
if (!ret)
ret = __btrfs_set_acl(trans, &init_user_ns, inode, acl,
ACL_TYPE_ACCESS);
posix_acl_release(acl);
}
if (!default_acl && !acl)
cache_no_acl(inode);
return ret;
}

124
fs/btrfs/async-thread.c
View File

@ -15,13 +15,12 @@
enum {
WORK_DONE_BIT,
WORK_ORDER_DONE_BIT,
WORK_HIGH_PRIO_BIT,
};
#define NO_THRESHOLD (-1)
#define DFT_THRESHOLD (32)
struct __btrfs_workqueue {
struct btrfs_workqueue {
struct workqueue_struct *normal_wq;
/* File system this workqueue services */
@ -48,12 +47,7 @@ struct __btrfs_workqueue {
spinlock_t thres_lock;
};
struct btrfs_workqueue {
struct __btrfs_workqueue *normal;
struct __btrfs_workqueue *high;
};
struct btrfs_fs_info * __pure btrfs_workqueue_owner(const struct __btrfs_workqueue *wq)
struct btrfs_fs_info * __pure btrfs_workqueue_owner(const struct btrfs_workqueue *wq)
{
return wq->fs_info;
}
@ -66,22 +60,22 @@ struct btrfs_fs_info * __pure btrfs_work_owner(const struct btrfs_work *work)
bool btrfs_workqueue_normal_congested(const struct btrfs_workqueue *wq)
{
/*
* We could compare wq->normal->pending with num_online_cpus()
* We could compare wq->pending with num_online_cpus()
* to support "thresh == NO_THRESHOLD" case, but it requires
* moving up atomic_inc/dec in thresh_queue/exec_hook. Let's
* postpone it until someone needs the support of that case.
*/
if (wq->normal->thresh == NO_THRESHOLD)
if (wq->thresh == NO_THRESHOLD)
return false;
return atomic_read(&wq->normal->pending) > wq->normal->thresh * 2;
return atomic_read(&wq->pending) > wq->thresh * 2;
}
static struct __btrfs_workqueue *
__btrfs_alloc_workqueue(struct btrfs_fs_info *fs_info, const char *name,
unsigned int flags, int limit_active, int thresh)
struct btrfs_workqueue *btrfs_alloc_workqueue(struct btrfs_fs_info *fs_info,
const char *name, unsigned int flags,
int limit_active, int thresh)
{
struct __btrfs_workqueue *ret = kzalloc(sizeof(*ret), GFP_KERNEL);
struct btrfs_workqueue *ret = kzalloc(sizeof(*ret), GFP_KERNEL);
if (!ret)
return NULL;
@ -105,12 +99,8 @@ __btrfs_alloc_workqueue(struct btrfs_fs_info *fs_info, const char *name,
ret->thresh = thresh;
}
if (flags & WQ_HIGHPRI)
ret->normal_wq = alloc_workqueue("btrfs-%s-high", flags,
ret->current_active, name);
else
ret->normal_wq = alloc_workqueue("btrfs-%s", flags,
ret->current_active, name);
ret->normal_wq = alloc_workqueue("btrfs-%s", flags, ret->current_active,
name);
if (!ret->normal_wq) {
kfree(ret);
return NULL;
@ -119,41 +109,7 @@ __btrfs_alloc_workqueue(struct btrfs_fs_info *fs_info, const char *name,
INIT_LIST_HEAD(&ret->ordered_list);
spin_lock_init(&ret->list_lock);
spin_lock_init(&ret->thres_lock);
trace_btrfs_workqueue_alloc(ret, name, flags & WQ_HIGHPRI);
return ret;
}
static inline void
__btrfs_destroy_workqueue(struct __btrfs_workqueue *wq);
struct btrfs_workqueue *btrfs_alloc_workqueue(struct btrfs_fs_info *fs_info,
const char *name,
unsigned int flags,
int limit_active,
int thresh)
{
struct btrfs_workqueue *ret = kzalloc(sizeof(*ret), GFP_KERNEL);
if (!ret)
return NULL;
ret->normal = __btrfs_alloc_workqueue(fs_info, name,
flags & ~WQ_HIGHPRI,
limit_active, thresh);
if (!ret->normal) {
kfree(ret);
return NULL;
}
if (flags & WQ_HIGHPRI) {
ret->high = __btrfs_alloc_workqueue(fs_info, name, flags,
limit_active, thresh);
if (!ret->high) {
__btrfs_destroy_workqueue(ret->normal);
kfree(ret);
return NULL;
}
}
trace_btrfs_workqueue_alloc(ret, name);
return ret;
}
@ -162,7 +118,7 @@ struct btrfs_workqueue *btrfs_alloc_workqueue(struct btrfs_fs_info *fs_info,
* This hook WILL be called in IRQ handler context,
* so workqueue_set_max_active MUST NOT be called in this hook
*/
static inline void thresh_queue_hook(struct __btrfs_workqueue *wq)
static inline void thresh_queue_hook(struct btrfs_workqueue *wq)
{
if (wq->thresh == NO_THRESHOLD)
return;
@ -174,7 +130,7 @@ static inline void thresh_queue_hook(struct __btrfs_workqueue *wq)
* This hook is called in kthread content.
* So workqueue_set_max_active is called here.
*/
static inline void thresh_exec_hook(struct __btrfs_workqueue *wq)
static inline void thresh_exec_hook(struct btrfs_workqueue *wq)
{
int new_current_active;
long pending;
@ -217,7 +173,7 @@ static inline void thresh_exec_hook(struct __btrfs_workqueue *wq)
}
}
static void run_ordered_work(struct __btrfs_workqueue *wq,
static void run_ordered_work(struct btrfs_workqueue *wq,
struct btrfs_work *self)
{
struct list_head *list = &wq->ordered_list;
@ -305,7 +261,7 @@ static void btrfs_work_helper(struct work_struct *normal_work)
{
struct btrfs_work *work = container_of(normal_work, struct btrfs_work,
normal_work);
struct __btrfs_workqueue *wq;
struct btrfs_workqueue *wq = work->wq;
int need_order = 0;
/*
@ -318,7 +274,6 @@ static void btrfs_work_helper(struct work_struct *normal_work)
*/
if (work->ordered_func)
need_order = 1;
wq = work->wq;
trace_btrfs_work_sched(work);
thresh_exec_hook(wq);
@ -350,8 +305,7 @@ void btrfs_init_work(struct btrfs_work *work, btrfs_func_t func,
work->flags = 0;
}
static inline void __btrfs_queue_work(struct __btrfs_workqueue *wq,
struct btrfs_work *work)
void btrfs_queue_work(struct btrfs_workqueue *wq, struct btrfs_work *work)
{
unsigned long flags;
@ -366,54 +320,22 @@ static inline void __btrfs_queue_work(struct __btrfs_workqueue *wq,
queue_work(wq->normal_wq, &work->normal_work);
}
void btrfs_queue_work(struct btrfs_workqueue *wq,
struct btrfs_work *work)
{
struct __btrfs_workqueue *dest_wq;
if (test_bit(WORK_HIGH_PRIO_BIT, &work->flags) && wq->high)
dest_wq = wq->high;
else
dest_wq = wq->normal;
__btrfs_queue_work(dest_wq, work);
}
static inline void
__btrfs_destroy_workqueue(struct __btrfs_workqueue *wq)
void btrfs_destroy_workqueue(struct btrfs_workqueue *wq)
{
if (!wq)
return;
destroy_workqueue(wq->normal_wq);
trace_btrfs_workqueue_destroy(wq);
kfree(wq);
}
void btrfs_destroy_workqueue(struct btrfs_workqueue *wq)
{
if (!wq)
return;
if (wq->high)
__btrfs_destroy_workqueue(wq->high);
__btrfs_destroy_workqueue(wq->normal);
kfree(wq);
}
void btrfs_workqueue_set_max(struct btrfs_workqueue *wq, int limit_active)
{
if (!wq)
return;
wq->normal->limit_active = limit_active;
if (wq->high)
wq->high->limit_active = limit_active;
}
void btrfs_set_work_high_priority(struct btrfs_work *work)
{
set_bit(WORK_HIGH_PRIO_BIT, &work->flags);
if (wq)
wq->limit_active = limit_active;
}
void btrfs_flush_workqueue(struct btrfs_workqueue *wq)
{
if (wq->high)
flush_workqueue(wq->high->normal_wq);
flush_workqueue(wq->normal->normal_wq);
flush_workqueue(wq->normal_wq);
}

7
fs/btrfs/async-thread.h
View File

@ -11,8 +11,6 @@
struct btrfs_fs_info;
struct btrfs_workqueue;
/* Internal use only */
struct __btrfs_workqueue;
struct btrfs_work;
typedef void (*btrfs_func_t)(struct btrfs_work *arg);
typedef void (*btrfs_work_func_t)(struct work_struct *arg);
@ -25,7 +23,7 @@ struct btrfs_work {
/* Don't touch things below */
struct work_struct normal_work;
struct list_head ordered_list;
struct __btrfs_workqueue *wq;
struct btrfs_workqueue *wq;
unsigned long flags;
};
@ -40,9 +38,8 @@ void btrfs_queue_work(struct btrfs_workqueue *wq,
struct btrfs_work *work);
void btrfs_destroy_workqueue(struct btrfs_workqueue *wq);
void btrfs_workqueue_set_max(struct btrfs_workqueue *wq, int max);
void btrfs_set_work_high_priority(struct btrfs_work *work);
struct btrfs_fs_info * __pure btrfs_work_owner(const struct btrfs_work *work);
struct btrfs_fs_info * __pure btrfs_workqueue_owner(const struct __btrfs_workqueue *wq);
struct btrfs_fs_info * __pure btrfs_workqueue_owner(const struct btrfs_workqueue *wq);
bool btrfs_workqueue_normal_congested(const struct btrfs_workqueue *wq);
void btrfs_flush_workqueue(struct btrfs_workqueue *wq);

205
fs/btrfs/block-group.c
View File

@ -168,11 +168,12 @@ static int btrfs_add_block_group_cache(struct btrfs_fs_info *info,
struct rb_node **p;
struct rb_node *parent = NULL;
struct btrfs_block_group *cache;
bool leftmost = true;
ASSERT(block_group->length != 0);
spin_lock(&info->block_group_cache_lock);
p = &info->block_group_cache_tree.rb_node;
write_lock(&info->block_group_cache_lock);
p = &info->block_group_cache_tree.rb_root.rb_node;
while (*p) {
parent = *p;
@ -181,20 +182,18 @@ static int btrfs_add_block_group_cache(struct btrfs_fs_info *info,
p = &(*p)->rb_left;
} else if (block_group->start > cache->start) {
p = &(*p)->rb_right;
leftmost = false;
} else {
spin_unlock(&info->block_group_cache_lock);
write_unlock(&info->block_group_cache_lock);
return -EEXIST;
}
}
rb_link_node(&block_group->cache_node, parent, p);
rb_insert_color(&block_group->cache_node,
&info->block_group_cache_tree);
rb_insert_color_cached(&block_group->cache_node,
&info->block_group_cache_tree, leftmost);
if (info->first_logical_byte > block_group->start)
info->first_logical_byte = block_group->start;
spin_unlock(&info->block_group_cache_lock);
write_unlock(&info->block_group_cache_lock);
return 0;
}
@ -210,8 +209,8 @@ static struct btrfs_block_group *block_group_cache_tree_search(
struct rb_node *n;
u64 end, start;
spin_lock(&info->block_group_cache_lock);
n = info->block_group_cache_tree.rb_node;
read_lock(&info->block_group_cache_lock);
n = info->block_group_cache_tree.rb_root.rb_node;
while (n) {
cache = rb_entry(n, struct btrfs_block_group, cache_node);
@ -233,12 +232,9 @@ static struct btrfs_block_group *block_group_cache_tree_search(
break;
}
}
if (ret) {
if (ret)
btrfs_get_block_group(ret);
if (bytenr == 0 && info->first_logical_byte > ret->start)
info->first_logical_byte = ret->start;
}
spin_unlock(&info->block_group_cache_lock);
read_unlock(&info->block_group_cache_lock);
return ret;
}
@ -267,15 +263,15 @@ struct btrfs_block_group *btrfs_next_block_group(
struct btrfs_fs_info *fs_info = cache->fs_info;
struct rb_node *node;
spin_lock(&fs_info->block_group_cache_lock);
read_lock(&fs_info->block_group_cache_lock);
/* If our block group was removed, we need a full search. */
if (RB_EMPTY_NODE(&cache->cache_node)) {
const u64 next_bytenr = cache->start + cache->length;
spin_unlock(&fs_info->block_group_cache_lock);
read_unlock(&fs_info->block_group_cache_lock);
btrfs_put_block_group(cache);
cache = btrfs_lookup_first_block_group(fs_info, next_bytenr); return cache;
return btrfs_lookup_first_block_group(fs_info, next_bytenr);
}
node = rb_next(&cache->cache_node);
btrfs_put_block_group(cache);
@ -284,46 +280,70 @@ struct btrfs_block_group *btrfs_next_block_group(
btrfs_get_block_group(cache);
} else
cache = NULL;
spin_unlock(&fs_info->block_group_cache_lock);
read_unlock(&fs_info->block_group_cache_lock);
return cache;
}
bool btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
/**
* Check if we can do a NOCOW write for a given extent.
*
* @fs_info: The filesystem information object.
* @bytenr: Logical start address of the extent.
*
* Check if we can do a NOCOW write for the given extent, and increments the
* number of NOCOW writers in the block group that contains the extent, as long
* as the block group exists and it's currently not in read-only mode.
*
* Returns: A non-NULL block group pointer if we can do a NOCOW write, the caller
* is responsible for calling btrfs_dec_nocow_writers() later.
*
* Or NULL if we can not do a NOCOW write
*/
struct btrfs_block_group *btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info,
u64 bytenr)
{
struct btrfs_block_group *bg;
bool ret = true;
bool can_nocow = true;
bg = btrfs_lookup_block_group(fs_info, bytenr);
if (!bg)
return false;
return NULL;
spin_lock(&bg->lock);
if (bg->ro)
ret = false;
can_nocow = false;
else
atomic_inc(&bg->nocow_writers);
spin_unlock(&bg->lock);
/* No put on block group, done by btrfs_dec_nocow_writers */
if (!ret)
if (!can_nocow) {
btrfs_put_block_group(bg);
return NULL;
}
return ret;
/* No put on block group, done by btrfs_dec_nocow_writers(). */
return bg;
}
void btrfs_dec_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
/**
* Decrement the number of NOCOW writers in a block group.
*
* @bg: The block group.
*
* This is meant to be called after a previous call to btrfs_inc_nocow_writers(),
* and on the block group returned by that call. Typically this is called after
* creating an ordered extent for a NOCOW write, to prevent races with scrub and
* relocation.
*
* After this call, the caller should not use the block group anymore. It it wants
* to use it, then it should get a reference on it before calling this function.
*/
void btrfs_dec_nocow_writers(struct btrfs_block_group *bg)
{
struct btrfs_block_group *bg;
bg = btrfs_lookup_block_group(fs_info, bytenr);
ASSERT(bg);
if (atomic_dec_and_test(&bg->nocow_writers))
wake_up_var(&bg->nocow_writers);
/*
* Once for our lookup and once for the lookup done by a previous call
* to btrfs_inc_nocow_writers()
*/
btrfs_put_block_group(bg);
/* For the lookup done by a previous call to btrfs_inc_nocow_writers(). */
btrfs_put_block_group(bg);
}
@ -772,10 +792,10 @@ int btrfs_cache_block_group(struct btrfs_block_group *cache, int load_cache_only
cache->has_caching_ctl = 1;
spin_unlock(&cache->lock);
spin_lock(&fs_info->block_group_cache_lock);
write_lock(&fs_info->block_group_cache_lock);
refcount_inc(&caching_ctl->count);
list_add_tail(&caching_ctl->list, &fs_info->caching_block_groups);
spin_unlock(&fs_info->block_group_cache_lock);
write_unlock(&fs_info->block_group_cache_lock);
btrfs_get_block_group(cache);
@ -957,17 +977,15 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
if (ret)
goto out;
spin_lock(&fs_info->block_group_cache_lock);
rb_erase(&block_group->cache_node,
&fs_info->block_group_cache_tree);
write_lock(&fs_info->block_group_cache_lock);
rb_erase_cached(&block_group->cache_node,
&fs_info->block_group_cache_tree);
RB_CLEAR_NODE(&block_group->cache_node);
/* Once for the block groups rbtree */
btrfs_put_block_group(block_group);
if (fs_info->first_logical_byte == block_group->start)
fs_info->first_logical_byte = (u64)-1;
spin_unlock(&fs_info->block_group_cache_lock);
write_unlock(&fs_info->block_group_cache_lock);
down_write(&block_group->space_info->groups_sem);
/*
@ -992,7 +1010,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
if (block_group->cached == BTRFS_CACHE_STARTED)
btrfs_wait_block_group_cache_done(block_group);
if (block_group->has_caching_ctl) {
spin_lock(&fs_info->block_group_cache_lock);
write_lock(&fs_info->block_group_cache_lock);
if (!caching_ctl) {
struct btrfs_caching_control *ctl;
@ -1006,7 +1024,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
}
if (caching_ctl)
list_del_init(&caching_ctl->list);
spin_unlock(&fs_info->block_group_cache_lock);
write_unlock(&fs_info->block_group_cache_lock);
if (caching_ctl) {
/* Once for the caching bgs list and once for us. */
btrfs_put_caching_control(caching_ctl);
@ -1367,6 +1385,14 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)
goto next;
}
ret = btrfs_zone_finish(block_group);
if (ret < 0) {
btrfs_dec_block_group_ro(block_group);
if (ret == -EAGAIN)
ret = 0;
goto next;
}
/*
* Want to do this before we do anything else so we can recover
* properly if we fail to join the transaction.
@ -1512,6 +1538,13 @@ static int reclaim_bgs_cmp(void *unused, const struct list_head *a,
return bg1->used > bg2->used;
}
static inline bool btrfs_should_reclaim(struct btrfs_fs_info *fs_info)
{
if (btrfs_is_zoned(fs_info))
return btrfs_zoned_should_reclaim(fs_info);
return true;
}
void btrfs_reclaim_bgs_work(struct work_struct *work)
{
struct btrfs_fs_info *fs_info =
@ -1522,6 +1555,9 @@ void btrfs_reclaim_bgs_work(struct work_struct *work)
if (!test_bit(BTRFS_FS_OPEN, &fs_info->flags))
return;
if (!btrfs_should_reclaim(fs_info))
return;
sb_start_write(fs_info->sb);
if (!btrfs_exclop_start(fs_info, BTRFS_EXCLOP_BALANCE)) {
@ -1692,35 +1728,13 @@ static int find_first_block_group(struct btrfs_fs_info *fs_info,
struct btrfs_root *root = btrfs_block_group_root(fs_info);
int ret;
struct btrfs_key found_key;
struct extent_buffer *leaf;
int slot;
ret = btrfs_search_slot(NULL, root, key, path, 0, 0);
if (ret < 0)
return ret;
while (1) {
slot = path->slots[0];
leaf = path->nodes[0];
if (slot >= btrfs_header_nritems(leaf)) {
ret = btrfs_next_leaf(root, path);
if (ret == 0)
continue;
if (ret < 0)
goto out;
break;
}
btrfs_item_key_to_cpu(leaf, &found_key, slot);
btrfs_for_each_slot(root, key, &found_key, path, ret) {
if (found_key.objectid >= key->objectid &&
found_key.type == BTRFS_BLOCK_GROUP_ITEM_KEY) {
ret = read_bg_from_eb(fs_info, &found_key, path);
break;
return read_bg_from_eb(fs_info, &found_key, path);
}
path->slots[0]++;
}
out:
return ret;
}
@ -3220,6 +3234,31 @@ int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans)
return ret;
}
static inline bool should_reclaim_block_group(struct btrfs_block_group *bg,
u64 bytes_freed)
{
const struct btrfs_space_info *space_info = bg->space_info;
const int reclaim_thresh = READ_ONCE(space_info->bg_reclaim_threshold);
const u64 new_val = bg->used;
const u64 old_val = new_val + bytes_freed;
u64 thresh;
if (reclaim_thresh == 0)
return false;
thresh = div_factor_fine(bg->length, reclaim_thresh);
/*
* If we were below the threshold before don't reclaim, we are likely a
* brand new block group and we don't want to relocate new block groups.
*/
if (old_val < thresh)
return false;
if (new_val >= thresh)
return false;
return true;
}
int btrfs_update_block_group(struct btrfs_trans_handle *trans,
u64 bytenr, u64 num_bytes, bool alloc)
{
@ -3242,6 +3281,8 @@ int btrfs_update_block_group(struct btrfs_trans_handle *trans,
spin_unlock(&info->delalloc_root_lock);
while (total) {
bool reclaim;
cache = btrfs_lookup_block_group(info, bytenr);
if (!cache) {
ret = -ENOENT;
@ -3287,6 +3328,8 @@ int btrfs_update_block_group(struct btrfs_trans_handle *trans,
cache->space_info, num_bytes);
cache->space_info->bytes_used -= num_bytes;
cache->space_info->disk_used -= num_bytes * factor;
reclaim = should_reclaim_block_group(cache, num_bytes);
spin_unlock(&cache->lock);
spin_unlock(&cache->space_info->lock);
@ -3313,6 +3356,8 @@ int btrfs_update_block_group(struct btrfs_trans_handle *trans,
if (!alloc && old_val == 0) {
if (!btrfs_test_opt(info, DISCARD_ASYNC))
btrfs_mark_bg_unused(cache);
} else if (!alloc && reclaim) {
btrfs_mark_bg_to_reclaim(cache);
}
btrfs_put_block_group(cache);
@ -3957,14 +4002,14 @@ int btrfs_free_block_groups(struct btrfs_fs_info *info)
struct btrfs_caching_control *caching_ctl;
struct rb_node *n;
spin_lock(&info->block_group_cache_lock);
write_lock(&info->block_group_cache_lock);
while (!list_empty(&info->caching_block_groups)) {
caching_ctl = list_entry(info->caching_block_groups.next,
struct btrfs_caching_control, list);
list_del(&caching_ctl->list);
btrfs_put_caching_control(caching_ctl);
}
spin_unlock(&info->block_group_cache_lock);
write_unlock(&info->block_group_cache_lock);
spin_lock(&info->unused_bgs_lock);
while (!list_empty(&info->unused_bgs)) {
@ -3994,14 +4039,14 @@ int btrfs_free_block_groups(struct btrfs_fs_info *info)
}
spin_unlock(&info->zone_active_bgs_lock);
spin_lock(&info->block_group_cache_lock);
while ((n = rb_last(&info->block_group_cache_tree)) != NULL) {
write_lock(&info->block_group_cache_lock);
while ((n = rb_last(&info->block_group_cache_tree.rb_root)) != NULL) {
block_group = rb_entry(n, struct btrfs_block_group,
cache_node);
rb_erase(&block_group->cache_node,
&info->block_group_cache_tree);
rb_erase_cached(&block_group->cache_node,
&info->block_group_cache_tree);
RB_CLEAR_NODE(&block_group->cache_node);
spin_unlock(&info->block_group_cache_lock);
write_unlock(&info->block_group_cache_lock);
down_write(&block_group->space_info->groups_sem);
list_del(&block_group->list);
@ -4024,9 +4069,9 @@ int btrfs_free_block_groups(struct btrfs_fs_info *info)
ASSERT(block_group->swap_extents == 0);
btrfs_put_block_group(block_group);
spin_lock(&info->block_group_cache_lock);
write_lock(&info->block_group_cache_lock);
}
spin_unlock(&info->block_group_cache_lock);
write_unlock(&info->block_group_cache_lock);
btrfs_release_global_block_rsv(info);

7
fs/btrfs/block-group.h
View File

@ -212,6 +212,8 @@ struct btrfs_block_group {
u64 meta_write_pointer;
struct map_lookup *physical_map;
struct list_head active_bg_list;
struct work_struct zone_finish_work;
struct extent_buffer *last_eb;
};
static inline u64 btrfs_block_group_end(struct btrfs_block_group *block_group)
@ -254,8 +256,9 @@ void btrfs_put_block_group(struct btrfs_block_group *cache);
void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info,
const u64 start);
void btrfs_wait_block_group_reservations(struct btrfs_block_group *bg);
bool btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr);
void btrfs_dec_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr);
struct btrfs_block_group *btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info,
u64 bytenr);
void btrfs_dec_nocow_writers(struct btrfs_block_group *bg);
void btrfs_wait_nocow_writers(struct btrfs_block_group *bg);
void btrfs_wait_block_group_cache_progress(struct btrfs_block_group *cache,
u64 num_bytes);

25
fs/btrfs/btrfs_inode.h
View File

@ -395,31 +395,6 @@ static inline bool btrfs_inode_can_compress(const struct btrfs_inode *inode)
return true;
}
struct btrfs_dio_private {
struct inode *inode;
/*
* Since DIO can use anonymous page, we cannot use page_offset() to
* grab the file offset, thus need a dedicated member for file offset.
*/
u64 file_offset;
u64 disk_bytenr;
/* Used for bio::bi_size */
u32 bytes;
/*
* References to this structure. There is one reference per in-flight
* bio plus one while we're still setting up.
*/
refcount_t refs;
/* dio_bio came from fs/direct-io.c */
struct bio *dio_bio;
/* Array of checksums */
u8 csums[];
};
/*
* btrfs_inode_item stores flags in a u64, btrfs_inode stores them in two
* separate u32s. These two functions convert between the two representations.

178
fs/btrfs/check-integrity.c
View File

@ -1552,21 +1552,18 @@ static int btrfsic_read_block(struct btrfsic_state *state,
return -ENOMEM;
block_ctx->datav = block_ctx->mem_to_free;
block_ctx->pagev = (struct page **)(block_ctx->datav + num_pages);
for (i = 0; i < num_pages; i++) {
block_ctx->pagev[i] = alloc_page(GFP_NOFS);
if (!block_ctx->pagev[i])
return -1;
}
ret = btrfs_alloc_page_array(num_pages, block_ctx->pagev);
if (ret)
return ret;
dev_bytenr = block_ctx->dev_bytenr;
for (i = 0; i < num_pages;) {
struct bio *bio;
unsigned int j;
bio = btrfs_bio_alloc(num_pages - i);
bio_set_dev(bio, block_ctx->dev->bdev);
bio = bio_alloc(block_ctx->dev->bdev, num_pages - i,
REQ_OP_READ, GFP_NOFS);
bio->bi_iter.bi_sector = dev_bytenr >> 9;
bio->bi_opf = REQ_OP_READ;
for (j = i; j < num_pages; j++) {
ret = bio_add_page(bio, block_ctx->pagev[j],
@ -2033,7 +2030,7 @@ static void btrfsic_process_written_block(struct btrfsic_dev_state *dev_state,
static void btrfsic_bio_end_io(struct bio *bp)
{
struct btrfsic_block *block = (struct btrfsic_block *)bp->bi_private;
struct btrfsic_block *block = bp->bi_private;
int iodone_w_error;
/* mutex is not held! This is not save if IO is not yet completed
@ -2635,102 +2632,95 @@ static struct btrfsic_dev_state *btrfsic_dev_state_lookup(dev_t dev)
&btrfsic_dev_state_hashtable);
}
static void __btrfsic_submit_bio(struct bio *bio)
static void btrfsic_check_write_bio(struct bio *bio, struct btrfsic_dev_state *dev_state)
{
unsigned int segs = bio_segments(bio);
u64 dev_bytenr = 512 * bio->bi_iter.bi_sector;
u64 cur_bytenr = dev_bytenr;
struct bvec_iter iter;
struct bio_vec bvec;
char **mapped_datav;
int bio_is_patched = 0;
int i = 0;
if (dev_state->state->print_mask & BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH)
pr_info(
"submit_bio(rw=%d,0x%x, bi_vcnt=%u, bi_sector=%llu (bytenr %llu), bi_bdev=%p)\n",
bio_op(bio), bio->bi_opf, segs,
bio->bi_iter.bi_sector, dev_bytenr, bio->bi_bdev);
mapped_datav = kmalloc_array(segs, sizeof(*mapped_datav), GFP_NOFS);
if (!mapped_datav)
return;
bio_for_each_segment(bvec, bio, iter) {
BUG_ON(bvec.bv_len != PAGE_SIZE);
mapped_datav[i] = page_address(bvec.bv_page);
i++;
if (dev_state->state->print_mask &
BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH_VERBOSE)
pr_info("#%u: bytenr=%llu, len=%u, offset=%u\n",
i, cur_bytenr, bvec.bv_len, bvec.bv_offset);
cur_bytenr += bvec.bv_len;
}
btrfsic_process_written_block(dev_state, dev_bytenr, mapped_datav, segs,
bio, &bio_is_patched, bio->bi_opf);
kfree(mapped_datav);
}
static void btrfsic_check_flush_bio(struct bio *bio, struct btrfsic_dev_state *dev_state)
{
if (dev_state->state->print_mask & BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH)
pr_info("submit_bio(rw=%d,0x%x FLUSH, bdev=%p)\n",
bio_op(bio), bio->bi_opf, bio->bi_bdev);
if (dev_state->dummy_block_for_bio_bh_flush.is_iodone) {
struct btrfsic_block *const block =
&dev_state->dummy_block_for_bio_bh_flush;
block->is_iodone = 0;
block->never_written = 0;
block->iodone_w_error = 0;
block->flush_gen = dev_state->last_flush_gen + 1;
block->submit_bio_bh_rw = bio->bi_opf;
block->orig_bio_private = bio->bi_private;
block->orig_bio_end_io = bio->bi_end_io;
block->next_in_same_bio = NULL;
bio->bi_private = block;
bio->bi_end_io = btrfsic_bio_end_io;
} else if ((dev_state->state->print_mask &
(BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH |
BTRFSIC_PRINT_MASK_VERBOSE))) {
pr_info(
"btrfsic_submit_bio(%pg) with FLUSH but dummy block already in use (ignored)!\n",
dev_state->bdev);
}
}
void btrfsic_check_bio(struct bio *bio)
{
struct btrfsic_dev_state *dev_state;
if (!btrfsic_is_initialized)
return;
mutex_lock(&btrfsic_mutex);
/* since btrfsic_submit_bio() is also called before
* btrfsic_mount(), this might return NULL */
/*
* We can be called before btrfsic_mount, so there might not be a
* dev_state.
*/
dev_state = btrfsic_dev_state_lookup(bio->bi_bdev->bd_dev);
if (NULL != dev_state &&
(bio_op(bio) == REQ_OP_WRITE) && bio_has_data(bio)) {
int i = 0;
u64 dev_bytenr;
u64 cur_bytenr;
struct bio_vec bvec;
struct bvec_iter iter;
int bio_is_patched;
char **mapped_datav;
unsigned int segs = bio_segments(bio);
dev_bytenr = 512 * bio->bi_iter.bi_sector;
bio_is_patched = 0;
if (dev_state->state->print_mask &
BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH)
pr_info("submit_bio(rw=%d,0x%x, bi_vcnt=%u, bi_sector=%llu (bytenr %llu), bi_bdev=%p)\n",
bio_op(bio), bio->bi_opf, segs,
bio->bi_iter.bi_sector, dev_bytenr, bio->bi_bdev);
mapped_datav = kmalloc_array(segs,
sizeof(*mapped_datav), GFP_NOFS);
if (!mapped_datav)
goto leave;
cur_bytenr = dev_bytenr;
bio_for_each_segment(bvec, bio, iter) {
BUG_ON(bvec.bv_len != PAGE_SIZE);
mapped_datav[i] = page_address(bvec.bv_page);
i++;
if (dev_state->state->print_mask &
BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH_VERBOSE)
pr_info("#%u: bytenr=%llu, len=%u, offset=%u\n",
i, cur_bytenr, bvec.bv_len, bvec.bv_offset);
cur_bytenr += bvec.bv_len;
}
btrfsic_process_written_block(dev_state, dev_bytenr,
mapped_datav, segs,
bio, &bio_is_patched,
bio->bi_opf);
kfree(mapped_datav);
} else if (NULL != dev_state && (bio->bi_opf & REQ_PREFLUSH)) {
if (dev_state->state->print_mask &
BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH)
pr_info("submit_bio(rw=%d,0x%x FLUSH, bdev=%p)\n",
bio_op(bio), bio->bi_opf, bio->bi_bdev);
if (!dev_state->dummy_block_for_bio_bh_flush.is_iodone) {
if ((dev_state->state->print_mask &
(BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH |
BTRFSIC_PRINT_MASK_VERBOSE)))
pr_info(
"btrfsic_submit_bio(%pg) with FLUSH but dummy block already in use (ignored)!\n",
dev_state->bdev);
} else {
struct btrfsic_block *const block =
&dev_state->dummy_block_for_bio_bh_flush;
block->is_iodone = 0;
block->never_written = 0;
block->iodone_w_error = 0;
block->flush_gen = dev_state->last_flush_gen + 1;
block->submit_bio_bh_rw = bio->bi_opf;
block->orig_bio_private = bio->bi_private;
block->orig_bio_end_io = bio->bi_end_io;
block->next_in_same_bio = NULL;
bio->bi_private = block;
bio->bi_end_io = btrfsic_bio_end_io;
}
mutex_lock(&btrfsic_mutex);
if (dev_state) {
if (bio_op(bio) == REQ_OP_WRITE && bio_has_data(bio))
btrfsic_check_write_bio(bio, dev_state);
else if (bio->bi_opf & REQ_PREFLUSH)
btrfsic_check_flush_bio(bio, dev_state);
}
leave:
mutex_unlock(&btrfsic_mutex);
}
void btrfsic_submit_bio(struct bio *bio)
{
__btrfsic_submit_bio(bio);
submit_bio(bio);
}
int btrfsic_submit_bio_wait(struct bio *bio)
{
__btrfsic_submit_bio(bio);
return submit_bio_wait(bio);
}
int btrfsic_mount(struct btrfs_fs_info *fs_info,
struct btrfs_fs_devices *fs_devices,
int including_extent_data, u32 print_mask)

6
fs/btrfs/check-integrity.h
View File

@ -7,11 +7,9 @@
#define BTRFS_CHECK_INTEGRITY_H
#ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY
void btrfsic_submit_bio(struct bio *bio);
int btrfsic_submit_bio_wait(struct bio *bio);
void btrfsic_check_bio(struct bio *bio);
#else
#define btrfsic_submit_bio submit_bio
#define btrfsic_submit_bio_wait submit_bio_wait
static inline void btrfsic_check_bio(struct bio *bio) { }
#endif
int btrfsic_mount(struct btrfs_fs_info *fs_info,

62
fs/btrfs/compression.c
View File

@ -425,7 +425,6 @@ static void end_compressed_bio_write(struct bio *bio)
}
static blk_status_t submit_compressed_bio(struct btrfs_fs_info *fs_info,
struct compressed_bio *cb,
struct bio *bio, int mirror_num)
{
blk_status_t ret;
@ -604,7 +603,7 @@ blk_status_t btrfs_submit_compressed_write(struct btrfs_inode *inode, u64 start,
goto finish_cb;
}
ret = submit_compressed_bio(fs_info, cb, bio, 0);
ret = submit_compressed_bio(fs_info, bio, 0);
if (ret)
goto finish_cb;
bio = NULL;
@ -802,15 +801,13 @@ static noinline int add_ra_bio_pages(struct inode *inode,
* After the compressed pages are read, we copy the bytes into the
* bio we were passed and then call the bio end_io calls
*/
blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
int mirror_num, unsigned long bio_flags)
void btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
int mirror_num)
{
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
struct extent_map_tree *em_tree;
struct compressed_bio *cb;
unsigned int compressed_len;
unsigned int nr_pages;
unsigned int pg_index;
struct bio *comp_bio = NULL;
const u64 disk_bytenr = bio->bi_iter.bi_sector << SECTOR_SHIFT;
u64 cur_disk_byte = disk_bytenr;
@ -820,7 +817,8 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
u64 em_start;
struct extent_map *em;
blk_status_t ret;
int faili = 0;
int ret2;
int i;
u8 *sums;
em_tree = &BTRFS_I(inode)->extent_tree;
@ -855,32 +853,26 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
em_len = em->len;
em_start = em->start;
cb->len = bio->bi_iter.bi_size;
cb->compressed_len = compressed_len;
cb->compress_type = em->compress_type;
cb->orig_bio = bio;
free_extent_map(em);
em = NULL;
cb->len = bio->bi_iter.bi_size;
cb->compressed_len = compressed_len;
cb->compress_type = extent_compress_type(bio_flags);
cb->orig_bio = bio;
nr_pages = DIV_ROUND_UP(compressed_len, PAGE_SIZE);
cb->compressed_pages = kcalloc(nr_pages, sizeof(struct page *),
GFP_NOFS);
cb->nr_pages = DIV_ROUND_UP(compressed_len, PAGE_SIZE);
cb->compressed_pages = kcalloc(cb->nr_pages, sizeof(struct page *), GFP_NOFS);
if (!cb->compressed_pages) {
ret = BLK_STS_RESOURCE;
goto fail1;
goto fail;
}
for (pg_index = 0; pg_index < nr_pages; pg_index++) {
cb->compressed_pages[pg_index] = alloc_page(GFP_NOFS);
if (!cb->compressed_pages[pg_index]) {
faili = pg_index - 1;
ret = BLK_STS_RESOURCE;
goto fail2;
}
ret2 = btrfs_alloc_page_array(cb->nr_pages, cb->compressed_pages);
if (ret2) {
ret = BLK_STS_RESOURCE;
goto fail;
}
faili = nr_pages - 1;
cb->nr_pages = nr_pages;
add_ra_bio_pages(inode, em_start + em_len, cb);
@ -949,28 +941,29 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
fs_info->sectorsize);
sums += fs_info->csum_size * nr_sectors;
ret = submit_compressed_bio(fs_info, cb, comp_bio, mirror_num);
ret = submit_compressed_bio(fs_info, comp_bio, mirror_num);
if (ret)
goto finish_cb;
comp_bio = NULL;
}
}
return BLK_STS_OK;
return;
fail2:
while (faili >= 0) {
__free_page(cb->compressed_pages[faili]);
faili--;
fail:
if (cb->compressed_pages) {
for (i = 0; i < cb->nr_pages; i++) {
if (cb->compressed_pages[i])
__free_page(cb->compressed_pages[i]);
}
}
kfree(cb->compressed_pages);
fail1:
kfree(cb);
out:
free_extent_map(em);
bio->bi_status = ret;
bio_endio(bio);
return ret;
return;
finish_cb:
if (comp_bio) {
comp_bio->bi_status = ret;
@ -978,7 +971,7 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
}
/* All bytes of @cb is submitted, endio will free @cb */
if (cur_disk_byte == disk_bytenr + compressed_len)
return ret;
return;
wait_var_event(cb, refcount_read(&cb->pending_sectors) ==
(disk_bytenr + compressed_len - cur_disk_byte) >>
@ -990,7 +983,6 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
ASSERT(refcount_read(&cb->pending_sectors));
/* Now we are the only one referring @cb, can finish it safely. */
finish_compressed_bio_read(cb);
return ret;
}
/*

4
fs/btrfs/compression.h
View File

@ -102,8 +102,8 @@ blk_status_t btrfs_submit_compressed_write(struct btrfs_inode *inode, u64 start,
unsigned int write_flags,
struct cgroup_subsys_state *blkcg_css,
bool writeback);
blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
int mirror_num, unsigned long bio_flags);
void btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
int mirror_num);
unsigned int btrfs_compress_str2level(unsigned int type, const char *str);

102
fs/btrfs/ctree.c
View File

@ -16,6 +16,7 @@
#include "volumes.h"
#include "qgroup.h"
#include "tree-mod-log.h"
#include "tree-checker.h"
static int split_node(struct btrfs_trans_handle *trans, struct btrfs_root
*root, struct btrfs_path *path, int level);
@ -342,7 +343,7 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,
int level = btrfs_header_level(buf);
ret = btrfs_set_disk_extent_flags(trans, buf,
new_flags, level, 0);
new_flags, level);
if (ret)
return ret;
}
@ -1390,12 +1391,13 @@ static noinline void unlock_up(struct btrfs_path *path, int level,
}
/*
* helper function for btrfs_search_slot. The goal is to find a block
* in cache without setting the path to blocking. If we find the block
* we return zero and the path is unchanged.
* Helper function for btrfs_search_slot() and other functions that do a search
* on a btree. The goal is to find a tree block in the cache (the radix tree at
* fs_info->buffer_radix), but if we can't find it, or it's not up to date, read
* its pages from disk.
*
* If we can't find the block, we set the path blocking and do some
* reada. -EAGAIN is returned and the search must be repeated.
* Returns -EAGAIN, with the path unlocked, if the caller needs to repeat the
* whole btree search, starting again from the current root node.
*/
static int
read_block_for_search(struct btrfs_root *root, struct btrfs_path *p,
@ -1409,12 +1411,21 @@ read_block_for_search(struct btrfs_root *root, struct btrfs_path *p,
struct btrfs_key first_key;
int ret;
int parent_level;
bool unlock_up;
unlock_up = ((level + 1 < BTRFS_MAX_LEVEL) && p->locks[level + 1]);
blocknr = btrfs_node_blockptr(*eb_ret, slot);
gen = btrfs_node_ptr_generation(*eb_ret, slot);
parent_level = btrfs_header_level(*eb_ret);
btrfs_node_key_to_cpu(*eb_ret, &first_key, slot);
/*
* If we need to read an extent buffer from disk and we are holding locks
* on upper level nodes, we unlock all the upper nodes before reading the
* extent buffer, and then return -EAGAIN to the caller as it needs to
* restart the search. We don't release the lock on the current level
* because we need to walk this node to figure out which blocks to read.
*/
tmp = find_extent_buffer(fs_info, blocknr);
if (tmp) {
if (p->reada == READA_FORWARD_ALWAYS)
@ -1436,30 +1447,38 @@ read_block_for_search(struct btrfs_root *root, struct btrfs_path *p,
return 0;
}
if (unlock_up)
btrfs_unlock_up_safe(p, level + 1);
/* now we're allowed to do a blocking uptodate check */
ret = btrfs_read_buffer(tmp, gen, parent_level - 1, &first_key);
ret = btrfs_read_extent_buffer(tmp, gen, parent_level - 1, &first_key);
if (ret) {
free_extent_buffer(tmp);
btrfs_release_path(p);
return -EIO;
}
*eb_ret = tmp;
return 0;
if (btrfs_check_eb_owner(tmp, root->root_key.objectid)) {
free_extent_buffer(tmp);
btrfs_release_path(p);
return -EUCLEAN;
}
if (unlock_up)
ret = -EAGAIN;
goto out;
}
/*
* reduce lock contention at high levels
* of the btree by dropping locks before
* we read. Don't release the lock on the current
* level because we need to walk this node to figure
* out which blocks to read.
*/
btrfs_unlock_up_safe(p, level + 1);
if (unlock_up) {
btrfs_unlock_up_safe(p, level + 1);
ret = -EAGAIN;
} else {
ret = 0;
}
if (p->reada != READA_NONE)
reada_for_search(fs_info, p, level, slot, key->objectid);
ret = -EAGAIN;
tmp = read_tree_block(fs_info, blocknr, root->root_key.objectid,
gen, parent_level - 1, &first_key);
if (IS_ERR(tmp)) {
@ -1474,9 +1493,15 @@ read_block_for_search(struct btrfs_root *root, struct btrfs_path *p,
*/
if (!extent_buffer_uptodate(tmp))
ret = -EIO;
free_extent_buffer(tmp);
btrfs_release_path(p);
out:
if (ret == 0) {
*eb_ret = tmp;
} else {
free_extent_buffer(tmp);
btrfs_release_path(p);
}
return ret;
}
@ -2279,6 +2304,43 @@ int btrfs_search_backwards(struct btrfs_root *root, struct btrfs_key *key,
return ret;
}
/**
* Search for a valid slot for the given path.
*
* @root: The root node of the tree.
* @key: Will contain a valid item if found.
* @path: The starting point to validate the slot.
*
* Return: 0 if the item is valid
* 1 if not found
* <0 if error.
*/
int btrfs_get_next_valid_item(struct btrfs_root *root, struct btrfs_key *key,
struct btrfs_path *path)
{
while (1) {
int ret;
const int slot = path->slots[0];
const struct extent_buffer *leaf = path->nodes[0];
/* This is where we start walking the path. */
if (slot >= btrfs_header_nritems(leaf)) {
/*
* If we've reached the last slot in this leaf we need
* to go to the next leaf and reset the path.
*/
ret = btrfs_next_leaf(root, path);
if (ret)
return ret;
continue;
}
/* Store the found, valid item in @key. */
btrfs_item_key_to_cpu(leaf, key, slot);
break;
}
return 0;
}
/*
* adjust the pointers going up the tree, starting at level
* making sure the right key of each node is points to 'key'.

165
fs/btrfs/ctree.h
View File

@ -675,13 +675,13 @@ struct btrfs_fs_info {
rwlock_t global_root_lock;
struct rb_root global_root_tree;
spinlock_t fs_roots_radix_lock;
struct radix_tree_root fs_roots_radix;
/* The xarray that holds all the FS roots */
spinlock_t fs_roots_lock;
struct xarray fs_roots;
/* block group cache stuff */
spinlock_t block_group_cache_lock;
u64 first_logical_byte;
struct rb_root block_group_cache_tree;
rwlock_t block_group_cache_lock;
struct rb_root_cached block_group_cache_tree;
/* keep track of unallocated space */
atomic64_t free_chunk_space;
@ -848,12 +848,13 @@ struct btrfs_fs_info {
* two
*/
struct btrfs_workqueue *workers;
struct btrfs_workqueue *hipri_workers;
struct btrfs_workqueue *delalloc_workers;
struct btrfs_workqueue *flush_workers;
struct btrfs_workqueue *endio_workers;
struct btrfs_workqueue *endio_meta_workers;
struct btrfs_workqueue *endio_raid56_workers;
struct btrfs_workqueue *rmw_workers;
struct workqueue_struct *rmw_workers;
struct btrfs_workqueue *endio_meta_write_workers;
struct btrfs_workqueue *endio_write_workers;
struct btrfs_workqueue *endio_freespace_worker;
@ -946,9 +947,9 @@ struct btrfs_fs_info {
* running.
*/
refcount_t scrub_workers_refcnt;
struct btrfs_workqueue *scrub_workers;
struct btrfs_workqueue *scrub_wr_completion_workers;
struct btrfs_workqueue *scrub_parity_workers;
struct workqueue_struct *scrub_workers;
struct workqueue_struct *scrub_wr_completion_workers;
struct workqueue_struct *scrub_parity_workers;
struct btrfs_subpage_info *subpage_info;
struct btrfs_discard_ctl discard_ctl;
@ -994,10 +995,10 @@ struct btrfs_fs_info {
struct btrfs_delayed_root *delayed_root;
/* Extent buffer radix tree */
/* Extent buffer xarray */
spinlock_t buffer_lock;
/* Entries are eb->start / sectorsize */
struct radix_tree_root buffer_radix;
struct xarray extent_buffers;
/* next backup root to be overwritten */
int backup_root_index;
@ -1045,10 +1046,7 @@ struct btrfs_fs_info {
* Zone size > 0 when in ZONED mode, otherwise it's used for a check
* if the mode is enabled
*/
union {
u64 zone_size;
u64 zoned;
};
u64 zone_size;
struct mutex zoned_meta_io_lock;
spinlock_t treelog_bg_lock;
@ -1121,7 +1119,8 @@ enum {
*/
BTRFS_ROOT_SHAREABLE,
BTRFS_ROOT_TRACK_DIRTY,
BTRFS_ROOT_IN_RADIX,
/* The root is tracked in fs_info::fs_roots */
BTRFS_ROOT_REGISTERED,
BTRFS_ROOT_ORPHAN_ITEM_INSERTED,
BTRFS_ROOT_DEFRAG_RUNNING,
BTRFS_ROOT_FORCE_COW,
@ -1225,10 +1224,10 @@ struct btrfs_root {
struct rb_root inode_tree;
/*
* radix tree that keeps track of delayed nodes of every inode,
* protected by inode_lock
* Xarray that keeps track of delayed nodes of every inode, protected
* by inode_lock
*/
struct radix_tree_root delayed_nodes_tree;
struct xarray delayed_nodes;
/*
* right now this just gets used so that a root has its own devid
* for stat. It may be used for more later
@ -2784,7 +2783,8 @@ int btrfs_pin_extent_for_log_replay(struct btrfs_trans_handle *trans,
u64 bytenr, u64 num_bytes);
int btrfs_exclude_logged_extents(struct extent_buffer *eb);
int btrfs_cross_ref_exist(struct btrfs_root *root,
u64 objectid, u64 offset, u64 bytenr, bool strict);
u64 objectid, u64 offset, u64 bytenr, bool strict,
struct btrfs_path *path);
struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
u64 parent, u64 root_objectid,
@ -2811,8 +2811,7 @@ int btrfs_inc_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root,
int btrfs_dec_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root,
struct extent_buffer *buf, int full_backref);
int btrfs_set_disk_extent_flags(struct btrfs_trans_handle *trans,
struct extent_buffer *eb, u64 flags,
int level, int is_data);
struct extent_buffer *eb, u64 flags, int level);
int btrfs_free_extent(struct btrfs_trans_handle *trans, struct btrfs_ref *ref);
int btrfs_free_reserved_extent(struct btrfs_fs_info *fs_info,
@ -2892,7 +2891,7 @@ void btrfs_subvolume_release_metadata(struct btrfs_root *root,
void btrfs_delalloc_release_extents(struct btrfs_inode *inode, u64 num_bytes);
int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes,
u64 disk_num_bytes);
u64 disk_num_bytes, bool noflush);
u64 btrfs_account_ro_block_groups_free_space(struct btrfs_space_info *sinfo);
int btrfs_error_unpin_extent_range(struct btrfs_fs_info *fs_info,
u64 start, u64 end);
@ -3039,6 +3038,35 @@ int btrfs_next_old_leaf(struct btrfs_root *root, struct btrfs_path *path,
int btrfs_search_backwards(struct btrfs_root *root, struct btrfs_key *key,
struct btrfs_path *path);
int btrfs_get_next_valid_item(struct btrfs_root *root, struct btrfs_key *key,
struct btrfs_path *path);
/*
* Search in @root for a given @key, and store the slot found in @found_key.
*
* @root: The root node of the tree.
* @key: The key we are looking for.
* @found_key: Will hold the found item.
* @path: Holds the current slot/leaf.
* @iter_ret: Contains the value returned from btrfs_search_slot or
* btrfs_get_next_valid_item, whichever was executed last.
*
* The @iter_ret is an output variable that will contain the return value of
* btrfs_search_slot, if it encountered an error, or the value returned from
* btrfs_get_next_valid_item otherwise. That return value can be 0, if a valid
* slot was found, 1 if there were no more leaves, and <0 if there was an error.
*
* It's recommended to use a separate variable for iter_ret and then use it to
* set the function return value so there's no confusion of the 0/1/errno
* values stemming from btrfs_search_slot.
*/
#define btrfs_for_each_slot(root, key, found_key, path, iter_ret) \
for (iter_ret = btrfs_search_slot(NULL, (root), (key), (path), 0, 0); \
(iter_ret) >= 0 && \
(iter_ret = btrfs_get_next_valid_item((root), (found_key), (path))) == 0; \
(path)->slots[0]++ \
)
static inline int btrfs_next_old_item(struct btrfs_root *root,
struct btrfs_path *p, u64 time_seq)
{
@ -3190,7 +3218,6 @@ int btrfs_del_orphan_item(struct btrfs_trans_handle *trans,
int btrfs_find_orphan_item(struct btrfs_root *root, u64 offset);
/* file-item.c */
struct btrfs_dio_private;
int btrfs_del_csums(struct btrfs_trans_handle *trans,
struct btrfs_root *root, u64 bytenr, u64 len);
blk_status_t btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio, u8 *dst);
@ -3224,8 +3251,8 @@ void btrfs_inode_safe_disk_i_size_write(struct btrfs_inode *inode, u64 new_i_siz
u64 btrfs_file_extent_end(const struct btrfs_path *path);
/* inode.c */
blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio,
int mirror_num, unsigned long bio_flags);
void btrfs_submit_data_bio(struct inode *inode, struct bio *bio,
int mirror_num, enum btrfs_compression_type compress_type);
unsigned int btrfs_verify_data_csum(struct btrfs_bio *bbio,
u32 bio_offset, struct page *page,
u64 start, u64 end);
@ -3255,10 +3282,28 @@ int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, long nr,
int btrfs_set_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
unsigned int extra_bits,
struct extent_state **cached_state);
int btrfs_create_subvol_root(struct btrfs_trans_handle *trans,
struct btrfs_root *new_root,
struct btrfs_root *parent_root,
struct user_namespace *mnt_userns);
struct btrfs_new_inode_args {
/* Input */
struct inode *dir;
struct dentry *dentry;
struct inode *inode;
bool orphan;
bool subvol;
/*
* Output from btrfs_new_inode_prepare(), input to
* btrfs_create_new_inode().
*/
struct posix_acl *default_acl;
struct posix_acl *acl;
};
int btrfs_new_inode_prepare(struct btrfs_new_inode_args *args,
unsigned int *trans_num_items);
int btrfs_create_new_inode(struct btrfs_trans_handle *trans,
struct btrfs_new_inode_args *args);
void btrfs_new_inode_args_destroy(struct btrfs_new_inode_args *args);
struct inode *btrfs_new_subvol_inode(struct user_namespace *mnt_userns,
struct inode *dir);
void btrfs_set_delalloc_extent(struct inode *inode, struct extent_state *state,
unsigned *bits);
void btrfs_clear_delalloc_extent(struct inode *inode,
@ -3269,7 +3314,6 @@ void btrfs_split_delalloc_extent(struct inode *inode,
struct extent_state *orig, u64 split);
void btrfs_set_range_writeback(struct btrfs_inode *inode, u64 start, u64 end);
vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf);
int btrfs_readpage(struct file *file, struct page *page);
void btrfs_evict_inode(struct inode *inode);
int btrfs_write_inode(struct inode *inode, struct writeback_control *wbc);
struct inode *btrfs_alloc_inode(struct super_block *sb);
@ -3314,9 +3358,9 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
const struct btrfs_ioctl_encoded_io_args *encoded);
ssize_t btrfs_dio_rw(struct kiocb *iocb, struct iov_iter *iter, size_t done_before);
extern const struct dentry_operations btrfs_dentry_operations;
extern const struct iomap_ops btrfs_dio_iomap_ops;
extern const struct iomap_dio_ops btrfs_dio_ops;
/* Inode locking type flags, by default the exclusive lock is taken */
#define BTRFS_ILOCK_SHARED (1U << 0)
@ -3328,6 +3372,7 @@ void btrfs_inode_unlock(struct inode *inode, unsigned int ilock_flags);
void btrfs_update_inode_bytes(struct btrfs_inode *inode,
const u64 add_bytes,
const u64 del_bytes);
void btrfs_assert_inode_range_clean(struct btrfs_inode *inode, u64 start, u64 end);
/* ioctl.c */
long btrfs_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
@ -3403,11 +3448,29 @@ void btrfs_no_printk(const struct btrfs_fs_info *fs_info, const char *fmt, ...)
{
}
#ifdef CONFIG_PRINTK
#ifdef CONFIG_PRINTK_INDEX
#define btrfs_printk(fs_info, fmt, args...) \
do { \
printk_index_subsys_emit("%sBTRFS %s (device %s): ", NULL, fmt); \
_btrfs_printk(fs_info, fmt, ##args); \
} while (0)
__printf(2, 3)
__cold
void btrfs_printk(const struct btrfs_fs_info *fs_info, const char *fmt, ...);
void _btrfs_printk(const struct btrfs_fs_info *fs_info, const char *fmt, ...);
#elif defined(CONFIG_PRINTK)
#define btrfs_printk(fs_info, fmt, args...) \
_btrfs_printk(fs_info, fmt, ##args)
__printf(2, 3)
__cold
void _btrfs_printk(const struct btrfs_fs_info *fs_info, const char *fmt, ...);
#else
#define btrfs_printk(fs_info, fmt, args...) \
btrfs_no_printk(fs_info, fmt, ##args)
#endif
@ -3658,12 +3721,25 @@ do { \
__LINE__, (errno)); \
} while (0)
#ifdef CONFIG_PRINTK_INDEX
#define btrfs_handle_fs_error(fs_info, errno, fmt, args...) \
do { \
__btrfs_handle_fs_error((fs_info), __func__, __LINE__, \
(errno), fmt, ##args); \
do { \
printk_index_subsys_emit( \
"BTRFS: error (device %s%s) in %s:%d: errno=%d %s", \
KERN_CRIT, fmt); \
__btrfs_handle_fs_error((fs_info), __func__, __LINE__, \
(errno), fmt, ##args); \
} while (0)
#else
#define btrfs_handle_fs_error(fs_info, errno, fmt, args...) \
__btrfs_handle_fs_error((fs_info), __func__, __LINE__, \
(errno), fmt, ##args)
#endif
#define BTRFS_FS_ERROR(fs_info) (unlikely(test_bit(BTRFS_FS_STATE_ERROR, \
&(fs_info)->fs_state)))
#define BTRFS_FS_LOG_CLEANUP_ERROR(fs_info) \
@ -3816,15 +3892,16 @@ static inline int __btrfs_fs_compat_ro(struct btrfs_fs_info *fs_info, u64 flag)
struct posix_acl *btrfs_get_acl(struct inode *inode, int type, bool rcu);
int btrfs_set_acl(struct user_namespace *mnt_userns, struct inode *inode,
struct posix_acl *acl, int type);
int btrfs_init_acl(struct btrfs_trans_handle *trans,
struct inode *inode, struct inode *dir);
int __btrfs_set_acl(struct btrfs_trans_handle *trans, struct inode *inode,
struct posix_acl *acl, int type);
#else
#define btrfs_get_acl NULL
#define btrfs_set_acl NULL
static inline int btrfs_init_acl(struct btrfs_trans_handle *trans,
struct inode *inode, struct inode *dir)
static inline int __btrfs_set_acl(struct btrfs_trans_handle *trans,
struct inode *inode, struct posix_acl *acl,
int type)
{
return 0;
return -EOPNOTSUPP;
}
#endif
@ -3929,7 +4006,7 @@ static inline int btrfs_is_testing(struct btrfs_fs_info *fs_info)
static inline bool btrfs_is_zoned(const struct btrfs_fs_info *fs_info)
{
return fs_info->zoned != 0;
return fs_info->zone_size > 0;
}
static inline bool btrfs_is_data_reloc_root(const struct btrfs_root *root)

9
fs/btrfs/delalloc-space.c
View File

@ -289,7 +289,7 @@ static void calc_inode_reservations(struct btrfs_fs_info *fs_info,
}
int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes,
u64 disk_num_bytes)
u64 disk_num_bytes, bool noflush)
{
struct btrfs_root *root = inode->root;
struct btrfs_fs_info *fs_info = root->fs_info;
@ -308,7 +308,7 @@ int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes,
* If we have a transaction open (can happen if we call truncate_block
* from truncate), then we need FLUSH_LIMIT so we don't deadlock.
*/
if (btrfs_is_free_space_inode(inode)) {
if (noflush || btrfs_is_free_space_inode(inode)) {
flush = BTRFS_RESERVE_NO_FLUSH;
} else {
if (current->journal_info)
@ -333,7 +333,8 @@ int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes,
*/
calc_inode_reservations(fs_info, num_bytes, disk_num_bytes,
&meta_reserve, &qgroup_reserve);
ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_reserve, true);
ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_reserve, true,
noflush);
if (ret)
return ret;
ret = btrfs_reserve_metadata_bytes(fs_info, block_rsv, meta_reserve, flush);
@ -456,7 +457,7 @@ int btrfs_delalloc_reserve_space(struct btrfs_inode *inode,
ret = btrfs_check_data_free_space(inode, reserved, start, len);
if (ret < 0)
return ret;
ret = btrfs_delalloc_reserve_metadata(inode, len, len);
ret = btrfs_delalloc_reserve_metadata(inode, len, len, false);
if (ret < 0) {
btrfs_free_reserved_data_space(inode, *reserved, start, len);
extent_changeset_free(*reserved);

84
fs/btrfs/delayed-inode.c
View File

@ -78,7 +78,7 @@ static struct btrfs_delayed_node *btrfs_get_delayed_node(
}
spin_lock(&root->inode_lock);
node = radix_tree_lookup(&root->delayed_nodes_tree, ino);
node = xa_load(&root->delayed_nodes, ino);
if (node) {
if (btrfs_inode->delayed_node) {
@ -90,9 +90,9 @@ static struct btrfs_delayed_node *btrfs_get_delayed_node(
/*
* It's possible that we're racing into the middle of removing
* this node from the radix tree. In this case, the refcount
* this node from the xarray. In this case, the refcount
* was zero and it should never go back to one. Just return
* NULL like it was never in the radix at all; our release
* NULL like it was never in the xarray at all; our release
* function is in the process of removing it.
*
* Some implementations of refcount_inc refuse to bump the
@ -100,7 +100,7 @@ static struct btrfs_delayed_node *btrfs_get_delayed_node(
* here, refcount_inc() may decide to just WARN_ONCE() instead
* of actually bumping the refcount.
*
* If this node is properly in the radix, we want to bump the
* If this node is properly in the xarray, we want to bump the
* refcount twice, once for the inode and once for this get
* operation.
*/
@ -128,36 +128,30 @@ static struct btrfs_delayed_node *btrfs_get_or_create_delayed_node(
u64 ino = btrfs_ino(btrfs_inode);
int ret;
again:
node = btrfs_get_delayed_node(btrfs_inode);
if (node)
return node;
do {
node = btrfs_get_delayed_node(btrfs_inode);
if (node)
return node;
node = kmem_cache_zalloc(delayed_node_cache, GFP_NOFS);
if (!node)
return ERR_PTR(-ENOMEM);
btrfs_init_delayed_node(node, root, ino);
node = kmem_cache_zalloc(delayed_node_cache, GFP_NOFS);
if (!node)
return ERR_PTR(-ENOMEM);
btrfs_init_delayed_node(node, root, ino);
/* cached in the btrfs inode and can be accessed */
refcount_set(&node->refs, 2);
/* Cached in the inode and can be accessed */
refcount_set(&node->refs, 2);
ret = radix_tree_preload(GFP_NOFS);
if (ret) {
kmem_cache_free(delayed_node_cache, node);
return ERR_PTR(ret);
}
spin_lock(&root->inode_lock);
ret = radix_tree_insert(&root->delayed_nodes_tree, ino, node);
if (ret == -EEXIST) {
spin_unlock(&root->inode_lock);
kmem_cache_free(delayed_node_cache, node);
radix_tree_preload_end();
goto again;
}
spin_lock(&root->inode_lock);
ret = xa_insert(&root->delayed_nodes, ino, node, GFP_NOFS);
if (ret) {
spin_unlock(&root->inode_lock);
kmem_cache_free(delayed_node_cache, node);
if (ret != -EBUSY)
return ERR_PTR(ret);
}
} while (ret);
btrfs_inode->delayed_node = node;
spin_unlock(&root->inode_lock);
radix_tree_preload_end();
return node;
}
@ -276,8 +270,7 @@ static void __btrfs_release_delayed_node(
* back up. We can delete it now.
*/
ASSERT(refcount_read(&delayed_node->refs) == 0);
radix_tree_delete(&root->delayed_nodes_tree,
delayed_node->inode_id);
xa_erase(&root->delayed_nodes, delayed_node->inode_id);
spin_unlock(&root->inode_lock);
kmem_cache_free(delayed_node_cache, delayed_node);
}
@ -1870,34 +1863,35 @@ void btrfs_kill_delayed_inode_items(struct btrfs_inode *inode)
void btrfs_kill_all_delayed_nodes(struct btrfs_root *root)
{
u64 inode_id = 0;
unsigned long index = 0;
struct btrfs_delayed_node *delayed_node;
struct btrfs_delayed_node *delayed_nodes[8];
int i, n;
while (1) {
int n = 0;
spin_lock(&root->inode_lock);
n = radix_tree_gang_lookup(&root->delayed_nodes_tree,
(void **)delayed_nodes, inode_id,
ARRAY_SIZE(delayed_nodes));
if (!n) {
if (xa_empty(&root->delayed_nodes)) {
spin_unlock(&root->inode_lock);
break;
return;
}
inode_id = delayed_nodes[n - 1]->inode_id + 1;
for (i = 0; i < n; i++) {
xa_for_each_start(&root->delayed_nodes, index, delayed_node, index) {
/*
* Don't increase refs in case the node is dead and
* about to be removed from the tree in the loop below
*/
if (!refcount_inc_not_zero(&delayed_nodes[i]->refs))
delayed_nodes[i] = NULL;
if (refcount_inc_not_zero(&delayed_node->refs)) {
delayed_nodes[n] = delayed_node;
n++;
}
if (n >= ARRAY_SIZE(delayed_nodes))
break;
}
index++;
spin_unlock(&root->inode_lock);
for (i = 0; i < n; i++) {
if (!delayed_nodes[i])
continue;
for (int i = 0; i < n; i++) {
__btrfs_kill_delayed_node(delayed_nodes[i]);
btrfs_release_delayed_node(delayed_nodes[i]);
}

4
fs/btrfs/delayed-ref.c
View File

@ -930,7 +930,6 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans,
is_system = (generic_ref->tree_ref.owning_root == BTRFS_CHUNK_TREE_OBJECTID);
ASSERT(generic_ref->type == BTRFS_REF_METADATA && generic_ref->action);
BUG_ON(extent_op && extent_op->is_data);
ref = kmem_cache_alloc(btrfs_delayed_tree_ref_cachep, GFP_NOFS);
if (!ref)
return -ENOMEM;
@ -1103,8 +1102,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_trans_handle *trans,
return -ENOMEM;
init_delayed_ref_head(head_ref, NULL, bytenr, num_bytes, 0, 0,
BTRFS_UPDATE_DELAYED_HEAD, extent_op->is_data,
false);
BTRFS_UPDATE_DELAYED_HEAD, false, false);
head_ref->extent_op = extent_op;
delayed_refs = &trans->transaction->delayed_refs;

1
fs/btrfs/delayed-ref.h
View File

@ -58,7 +58,6 @@ struct btrfs_delayed_extent_op {
u8 level;
bool update_key;
bool update_flags;
bool is_data;
u64 flags_to_set;
};

52
fs/btrfs/dev-replace.c
View File

@ -474,6 +474,7 @@ static int mark_block_group_to_copy(struct btrfs_fs_info *fs_info,
struct btrfs_dev_extent *dev_extent = NULL;
struct btrfs_block_group *cache;
struct btrfs_trans_handle *trans;
int iter_ret = 0;
int ret = 0;
u64 chunk_offset;
@ -524,29 +525,8 @@ static int mark_block_group_to_copy(struct btrfs_fs_info *fs_info,
key.type = BTRFS_DEV_EXTENT_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto free_path;
if (ret > 0) {
if (path->slots[0] >=
btrfs_header_nritems(path->nodes[0])) {
ret = btrfs_next_leaf(root, path);
if (ret < 0)
goto free_path;
if (ret > 0) {
ret = 0;
goto free_path;
}
} else {
ret = 0;
}
}
while (1) {
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
struct extent_buffer *leaf = path->nodes[0];
int slot = path->slots[0];
btrfs_item_key_to_cpu(leaf, &found_key, slot);
if (found_key.objectid != src_dev->devid)
break;
@ -557,30 +537,23 @@ static int mark_block_group_to_copy(struct btrfs_fs_info *fs_info,
if (found_key.offset < key.offset)
break;
dev_extent = btrfs_item_ptr(leaf, slot, struct btrfs_dev_extent);
dev_extent = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_dev_extent);
chunk_offset = btrfs_dev_extent_chunk_offset(leaf, dev_extent);
cache = btrfs_lookup_block_group(fs_info, chunk_offset);
if (!cache)
goto skip;
continue;
spin_lock(&cache->lock);
cache->to_copy = 1;
spin_unlock(&cache->lock);
btrfs_put_block_group(cache);
skip:
ret = btrfs_next_item(root, path);
if (ret != 0) {
if (ret > 0)
ret = 0;
break;
}
}
if (iter_ret < 0)
ret = iter_ret;
free_path:
btrfs_free_path(path);
unlock:
mutex_unlock(&fs_info->chunk_mutex);
@ -881,6 +854,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
int scrub_ret)
{
struct btrfs_dev_replace *dev_replace = &fs_info->dev_replace;
struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
struct btrfs_device *tgt_device;
struct btrfs_device *src_device;
struct btrfs_root *root = fs_info->tree_root;
@ -930,12 +904,12 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
WARN_ON(ret);
/* Prevent write_all_supers() during the finishing procedure */
mutex_lock(&fs_info->fs_devices->device_list_mutex);
mutex_lock(&fs_devices->device_list_mutex);
/* Prevent new chunks being allocated on the source device */
mutex_lock(&fs_info->chunk_mutex);
if (!list_empty(&src_device->post_commit_list)) {
mutex_unlock(&fs_info->fs_devices->device_list_mutex);
mutex_unlock(&fs_devices->device_list_mutex);
mutex_unlock(&fs_info->chunk_mutex);
} else {
break;
@ -972,7 +946,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
error:
up_write(&dev_replace->rwsem);
mutex_unlock(&fs_info->chunk_mutex);
mutex_unlock(&fs_info->fs_devices->device_list_mutex);
mutex_unlock(&fs_devices->device_list_mutex);
btrfs_rm_dev_replace_blocked(fs_info);
if (tgt_device)
btrfs_destroy_dev_replace_tgtdev(tgt_device);
@ -1001,8 +975,8 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
btrfs_assign_next_active_device(src_device, tgt_device);
list_add(&tgt_device->dev_alloc_list, &fs_info->fs_devices->alloc_list);
fs_info->fs_devices->rw_devices++;
list_add(&tgt_device->dev_alloc_list, &fs_devices->alloc_list);
fs_devices->rw_devices++;
up_write(&dev_replace->rwsem);
btrfs_rm_dev_replace_blocked(fs_info);
@ -1025,7 +999,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
* belong to this filesystem.
*/
mutex_unlock(&fs_info->chunk_mutex);
mutex_unlock(&fs_info->fs_devices->device_list_mutex);
mutex_unlock(&fs_devices->device_list_mutex);
/* replace the sysfs entry */
btrfs_sysfs_remove_device(src_device);

31
fs/btrfs/dir-item.c
View File

@ -325,36 +325,15 @@ btrfs_search_dir_index_item(struct btrfs_root *root,
struct btrfs_path *path, u64 dirid,
const char *name, int name_len)
{
struct extent_buffer *leaf;
struct btrfs_dir_item *di;
struct btrfs_key key;
u32 nritems;
int ret;
key.objectid = dirid;
key.type = BTRFS_DIR_INDEX_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
return ERR_PTR(ret);
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
while (1) {
if (path->slots[0] >= nritems) {
ret = btrfs_next_leaf(root, path);
if (ret < 0)
return ERR_PTR(ret);
if (ret > 0)
break;
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
continue;
}
btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
btrfs_for_each_slot(root, &key, &key, path, ret) {
if (key.objectid != dirid || key.type != BTRFS_DIR_INDEX_KEY)
break;
@ -362,10 +341,12 @@ btrfs_search_dir_index_item(struct btrfs_root *root,
name, name_len);
if (di)
return di;
path->slots[0]++;
}
return NULL;
/* Adjust return code if the key was not found in the next leaf. */
if (ret > 0)
ret = 0;
return ERR_PTR(ret);
}
struct btrfs_dir_item *btrfs_lookup_xattr(struct btrfs_trans_handle *trans,

314
fs/btrfs/disk-io.c
View File

@ -5,7 +5,6 @@
#include <linux/fs.h>
#include <linux/blkdev.h>
#include <linux/radix-tree.h>
#include <linux/writeback.h>
#include <linux/workqueue.h>
#include <linux/kthread.h>
@ -374,9 +373,9 @@ int btrfs_verify_level_key(struct extent_buffer *eb, int level,
* @level: expected level, mandatory check
* @first_key: expected key of first slot, skip check if NULL
*/
static int btree_read_extent_buffer_pages(struct extent_buffer *eb,
u64 parent_transid, int level,
struct btrfs_key *first_key)
int btrfs_read_extent_buffer(struct extent_buffer *eb,
u64 parent_transid, int level,
struct btrfs_key *first_key)
{
struct btrfs_fs_info *fs_info = eb->fs_info;
struct extent_io_tree *io_tree;
@ -486,7 +485,7 @@ static int csum_dirty_subpage_buffers(struct btrfs_fs_info *fs_info,
uptodate = btrfs_subpage_test_uptodate(fs_info, page, cur,
fs_info->nodesize);
/* A dirty eb shouldn't disappear from buffer_radix */
/* A dirty eb shouldn't disappear from extent_buffers */
if (WARN_ON(!eb))
return -EUCLEAN;
@ -519,7 +518,7 @@ static int csum_dirty_buffer(struct btrfs_fs_info *fs_info, struct bio_vec *bvec
u64 found_start;
struct extent_buffer *eb;
if (fs_info->sectorsize < PAGE_SIZE)
if (fs_info->nodesize < PAGE_SIZE)
return csum_dirty_subpage_buffers(fs_info, bvec);
eb = (struct extent_buffer *)page->private;
@ -704,7 +703,7 @@ int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio,
ASSERT(page->private);
if (btrfs_sb(page->mapping->host->i_sb)->sectorsize < PAGE_SIZE)
if (btrfs_sb(page->mapping->host->i_sb)->nodesize < PAGE_SIZE)
return validate_subpage_buffer(page, start, end, mirror);
eb = (struct extent_buffer *)page->private;
@ -850,8 +849,7 @@ static void run_one_async_free(struct btrfs_work *work)
}
blk_status_t btrfs_wq_submit_bio(struct inode *inode, struct bio *bio,
int mirror_num, unsigned long bio_flags,
u64 dio_file_offset,
int mirror_num, u64 dio_file_offset,
extent_submit_bio_start_t *submit_bio_start)
{
struct btrfs_fs_info *fs_info = BTRFS_I(inode)->root->fs_info;
@ -874,9 +872,9 @@ blk_status_t btrfs_wq_submit_bio(struct inode *inode, struct bio *bio,
async->status = 0;
if (op_is_sync(bio->bi_opf))
btrfs_set_work_high_priority(&async->work);
btrfs_queue_work(fs_info->workers, &async->work);
btrfs_queue_work(fs_info->hipri_workers, &async->work);
else
btrfs_queue_work(fs_info->workers, &async->work);
return 0;
}
@ -920,8 +918,7 @@ static bool should_async_write(struct btrfs_fs_info *fs_info,
return true;
}
blk_status_t btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio,
int mirror_num, unsigned long bio_flags)
void btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio, int mirror_num)
{
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
blk_status_t ret;
@ -933,31 +930,25 @@ blk_status_t btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio,
*/
ret = btrfs_bio_wq_end_io(fs_info, bio,
BTRFS_WQ_ENDIO_METADATA);
if (ret)
goto out_w_error;
ret = btrfs_map_bio(fs_info, bio, mirror_num);
if (!ret)
ret = btrfs_map_bio(fs_info, bio, mirror_num);
} else if (!should_async_write(fs_info, BTRFS_I(inode))) {
ret = btree_csum_one_bio(bio);
if (ret)
goto out_w_error;
ret = btrfs_map_bio(fs_info, bio, mirror_num);
if (!ret)
ret = btrfs_map_bio(fs_info, bio, mirror_num);
} else {
/*
* kthread helpers are used to submit writes so that
* checksumming can happen in parallel across all CPUs
*/
ret = btrfs_wq_submit_bio(inode, bio, mirror_num, 0,
0, btree_submit_bio_start);
btree_submit_bio_start);
}
if (ret)
goto out_w_error;
return 0;
out_w_error:
bio->bi_status = ret;
bio_endio(bio);
return ret;
if (ret) {
bio->bi_status = ret;
bio_endio(bio);
}
}
#ifdef CONFIG_MIGRATION
@ -1118,12 +1109,15 @@ struct extent_buffer *read_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr,
if (IS_ERR(buf))
return buf;
ret = btree_read_extent_buffer_pages(buf, parent_transid,
level, first_key);
ret = btrfs_read_extent_buffer(buf, parent_transid, level, first_key);
if (ret) {
free_extent_buffer_stale(buf);
return ERR_PTR(ret);
}
if (btrfs_check_eb_owner(buf, owner_root)) {
free_extent_buffer_stale(buf);
return ERR_PTR(-EUCLEAN);
}
return buf;
}
@ -1164,7 +1158,7 @@ static void __setup_root(struct btrfs_root *root, struct btrfs_fs_info *fs_info,
root->nr_delalloc_inodes = 0;
root->nr_ordered_extents = 0;
root->inode_tree = RB_ROOT;
INIT_RADIX_TREE(&root->delayed_nodes_tree, GFP_ATOMIC);
xa_init_flags(&root->delayed_nodes, GFP_ATOMIC);
btrfs_init_root_block_rsv(root);
@ -1216,9 +1210,9 @@ static void __setup_root(struct btrfs_root *root, struct btrfs_fs_info *fs_info,
btrfs_qgroup_init_swapped_blocks(&root->swapped_blocks);
#ifdef CONFIG_BTRFS_DEBUG
INIT_LIST_HEAD(&root->leak_list);
spin_lock(&fs_info->fs_roots_radix_lock);
spin_lock(&fs_info->fs_roots_lock);
list_add_tail(&root->leak_list, &fs_info->allocated_roots);
spin_unlock(&fs_info->fs_roots_radix_lock);
spin_unlock(&fs_info->fs_roots_lock);
#endif
}
@ -1563,6 +1557,23 @@ static struct btrfs_root *read_tree_root_path(struct btrfs_root *tree_root,
ret = -EIO;
goto fail;
}
/*
* For real fs, and not log/reloc trees, root owner must
* match its root node owner
*/
if (!test_bit(BTRFS_FS_STATE_DUMMY_FS_INFO, &fs_info->fs_state) &&
root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID &&
root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID &&
root->root_key.objectid != btrfs_header_owner(root->node)) {
btrfs_crit(fs_info,
"root=%llu block=%llu, tree root owner mismatch, have %llu expect %llu",
root->root_key.objectid, root->node->start,
btrfs_header_owner(root->node),
root->root_key.objectid);
ret = -EUCLEAN;
goto fail;
}
root->commit_root = btrfs_root_node(root);
return root;
fail:
@ -1648,12 +1659,11 @@ static struct btrfs_root *btrfs_lookup_fs_root(struct btrfs_fs_info *fs_info,
{
struct btrfs_root *root;
spin_lock(&fs_info->fs_roots_radix_lock);
root = radix_tree_lookup(&fs_info->fs_roots_radix,
(unsigned long)root_id);
spin_lock(&fs_info->fs_roots_lock);
root = xa_load(&fs_info->fs_roots, (unsigned long)root_id);
if (root)
root = btrfs_grab_root(root);
spin_unlock(&fs_info->fs_roots_radix_lock);
spin_unlock(&fs_info->fs_roots_lock);
return root;
}
@ -1695,20 +1705,14 @@ int btrfs_insert_fs_root(struct btrfs_fs_info *fs_info,
{
int ret;
ret = radix_tree_preload(GFP_NOFS);
if (ret)
return ret;
spin_lock(&fs_info->fs_roots_radix_lock);
ret = radix_tree_insert(&fs_info->fs_roots_radix,
(unsigned long)root->root_key.objectid,
root);
spin_lock(&fs_info->fs_roots_lock);
ret = xa_insert(&fs_info->fs_roots, (unsigned long)root->root_key.objectid,
root, GFP_NOFS);
if (ret == 0) {
btrfs_grab_root(root);
set_bit(BTRFS_ROOT_IN_RADIX, &root->state);
set_bit(BTRFS_ROOT_REGISTERED, &root->state);
}
spin_unlock(&fs_info->fs_roots_radix_lock);
radix_tree_preload_end();
spin_unlock(&fs_info->fs_roots_lock);
return ret;
}
@ -1964,7 +1968,7 @@ static void end_workqueue_fn(struct btrfs_work *work)
static int cleaner_kthread(void *arg)
{
struct btrfs_fs_info *fs_info = (struct btrfs_fs_info *)arg;
struct btrfs_fs_info *fs_info = arg;
int again;
while (1) {
@ -2266,10 +2270,12 @@ static void btrfs_stop_all_workers(struct btrfs_fs_info *fs_info)
{
btrfs_destroy_workqueue(fs_info->fixup_workers);
btrfs_destroy_workqueue(fs_info->delalloc_workers);
btrfs_destroy_workqueue(fs_info->hipri_workers);
btrfs_destroy_workqueue(fs_info->workers);
btrfs_destroy_workqueue(fs_info->endio_workers);
btrfs_destroy_workqueue(fs_info->endio_raid56_workers);
btrfs_destroy_workqueue(fs_info->rmw_workers);
if (fs_info->rmw_workers)
destroy_workqueue(fs_info->rmw_workers);
btrfs_destroy_workqueue(fs_info->endio_write_workers);
btrfs_destroy_workqueue(fs_info->endio_freespace_worker);
btrfs_destroy_workqueue(fs_info->delayed_workers);
@ -2336,9 +2342,9 @@ void btrfs_put_root(struct btrfs_root *root)
btrfs_drew_lock_destroy(&root->snapshot_lock);
free_root_extent_buffers(root);
#ifdef CONFIG_BTRFS_DEBUG
spin_lock(&root->fs_info->fs_roots_radix_lock);
spin_lock(&root->fs_info->fs_roots_lock);
list_del_init(&root->leak_list);
spin_unlock(&root->fs_info->fs_roots_radix_lock);
spin_unlock(&root->fs_info->fs_roots_lock);
#endif
kfree(root);
}
@ -2346,28 +2352,21 @@ void btrfs_put_root(struct btrfs_root *root)
void btrfs_free_fs_roots(struct btrfs_fs_info *fs_info)
{
int ret;
struct btrfs_root *gang[8];
int i;
struct btrfs_root *root;
unsigned long index = 0;
while (!list_empty(&fs_info->dead_roots)) {
gang[0] = list_entry(fs_info->dead_roots.next,
struct btrfs_root, root_list);
list_del(&gang[0]->root_list);
root = list_entry(fs_info->dead_roots.next,
struct btrfs_root, root_list);
list_del(&root->root_list);
if (test_bit(BTRFS_ROOT_IN_RADIX, &gang[0]->state))
btrfs_drop_and_free_fs_root(fs_info, gang[0]);
btrfs_put_root(gang[0]);
if (test_bit(BTRFS_ROOT_REGISTERED, &root->state))
btrfs_drop_and_free_fs_root(fs_info, root);
btrfs_put_root(root);
}
while (1) {
ret = radix_tree_gang_lookup(&fs_info->fs_roots_radix,
(void **)gang, 0,
ARRAY_SIZE(gang));
if (!ret)
break;
for (i = 0; i < ret; i++)
btrfs_drop_and_free_fs_root(fs_info, gang[i]);
xa_for_each(&fs_info->fs_roots, index, root) {
btrfs_drop_and_free_fs_root(fs_info, root);
}
}
@ -2444,7 +2443,9 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info)
unsigned int flags = WQ_MEM_RECLAIM | WQ_FREEZABLE | WQ_UNBOUND;
fs_info->workers =
btrfs_alloc_workqueue(fs_info, "worker",
btrfs_alloc_workqueue(fs_info, "worker", flags, max_active, 16);
fs_info->hipri_workers =
btrfs_alloc_workqueue(fs_info, "worker-high",
flags | WQ_HIGHPRI, max_active, 16);
fs_info->delalloc_workers =
@ -2476,8 +2477,7 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info)
fs_info->endio_raid56_workers =
btrfs_alloc_workqueue(fs_info, "endio-raid56", flags,
max_active, 4);
fs_info->rmw_workers =
btrfs_alloc_workqueue(fs_info, "rmw", flags, max_active, 2);
fs_info->rmw_workers = alloc_workqueue("btrfs-rmw", flags, max_active);
fs_info->endio_write_workers =
btrfs_alloc_workqueue(fs_info, "endio-write", flags,
max_active, 2);
@ -2492,8 +2492,8 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info)
fs_info->discard_ctl.discard_workers =
alloc_workqueue("btrfs_discard", WQ_UNBOUND | WQ_FREEZABLE, 1);
if (!(fs_info->workers && fs_info->delalloc_workers &&
fs_info->flush_workers &&
if (!(fs_info->workers && fs_info->hipri_workers &&
fs_info->delalloc_workers && fs_info->flush_workers &&
fs_info->endio_workers && fs_info->endio_meta_workers &&
fs_info->endio_meta_write_workers &&
fs_info->endio_write_workers && fs_info->endio_raid56_workers &&
@ -2815,12 +2815,14 @@ static int validate_super(struct btrfs_fs_info *fs_info,
}
/*
* For 4K page size, we only support 4K sector size.
* For 64K page size, we support 64K and 4K sector sizes.
* We only support at most two sectorsizes: 4K and PAGE_SIZE.
*
* We can support 16K sectorsize with 64K page size without problem,
* but such sectorsize/pagesize combination doesn't make much sense.
* 4K will be our future standard, PAGE_SIZE is supported from the very
* beginning.
*/
if ((PAGE_SIZE == SZ_4K && sectorsize != PAGE_SIZE) ||
(PAGE_SIZE == SZ_64K && (sectorsize != SZ_4K &&
sectorsize != SZ_64K))) {
if (sectorsize > PAGE_SIZE || (sectorsize != SZ_4K && sectorsize != PAGE_SIZE)) {
btrfs_err(fs_info,
"sectorsize %llu not yet supported for page size %lu",
sectorsize, PAGE_SIZE);
@ -3132,8 +3134,8 @@ static int __cold init_tree_roots(struct btrfs_fs_info *fs_info)
void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
{
INIT_RADIX_TREE(&fs_info->fs_roots_radix, GFP_ATOMIC);
INIT_RADIX_TREE(&fs_info->buffer_radix, GFP_ATOMIC);
xa_init_flags(&fs_info->fs_roots, GFP_ATOMIC);
xa_init_flags(&fs_info->extent_buffers, GFP_ATOMIC);
INIT_LIST_HEAD(&fs_info->trans_list);
INIT_LIST_HEAD(&fs_info->dead_roots);
INIT_LIST_HEAD(&fs_info->delayed_iputs);
@ -3141,7 +3143,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
INIT_LIST_HEAD(&fs_info->caching_block_groups);
spin_lock_init(&fs_info->delalloc_root_lock);
spin_lock_init(&fs_info->trans_lock);
spin_lock_init(&fs_info->fs_roots_radix_lock);
spin_lock_init(&fs_info->fs_roots_lock);
spin_lock_init(&fs_info->delayed_iput_lock);
spin_lock_init(&fs_info->defrag_inodes_lock);
spin_lock_init(&fs_info->super_lock);
@ -3209,9 +3211,8 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
btrfs_init_balance(fs_info);
btrfs_init_async_reclaim_work(fs_info);
spin_lock_init(&fs_info->block_group_cache_lock);
fs_info->block_group_cache_tree = RB_ROOT;
fs_info->first_logical_byte = (u64)-1;
rwlock_init(&fs_info->block_group_cache_lock);
fs_info->block_group_cache_tree = RB_ROOT_CACHED;
extent_io_tree_init(fs_info, &fs_info->excluded_extents,
IO_TREE_FS_EXCLUDED_EXTENTS, NULL);
@ -3295,7 +3296,7 @@ static int init_mount_fs_info(struct btrfs_fs_info *fs_info, struct super_block
static int btrfs_uuid_rescan_kthread(void *data)
{
struct btrfs_fs_info *fs_info = (struct btrfs_fs_info *)data;
struct btrfs_fs_info *fs_info = data;
int ret;
/*
@ -3373,7 +3374,7 @@ int btrfs_start_pre_rw_mount(struct btrfs_fs_info *fs_info)
/*
* btrfs_find_orphan_roots() is responsible for finding all the dead
* roots (with 0 refs), flag them with BTRFS_ROOT_DEAD_TREE and load
* them into the fs_info->fs_roots_radix tree. This must be done before
* them into the fs_info->fs_roots. This must be done before
* calling btrfs_orphan_cleanup() on the tree root. If we don't do it
* first, then btrfs_orphan_cleanup() will delete a dead root's orphan
* item before the root's tree is deleted - this means that if we unmount
@ -3611,7 +3612,7 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device
~BTRFS_FEATURE_INCOMPAT_SUPP;
if (features) {
btrfs_err(fs_info,
"cannot mount because of unsupported optional features (%llx)",
"cannot mount because of unsupported optional features (0x%llx)",
features);
err = -EINVAL;
goto fail_alloc;
@ -3649,7 +3650,7 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device
~BTRFS_FEATURE_COMPAT_RO_SUPP;
if (!sb_rdonly(sb) && features) {
btrfs_err(fs_info,
"cannot mount read-write because of unsupported optional features (%llx)",
"cannot mount read-write because of unsupported optional features (0x%llx)",
features);
err = -EINVAL;
goto fail_alloc;
@ -3672,14 +3673,6 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device
btrfs_warn(fs_info,
"read-write for sector size %u with page size %lu is experimental",
sectorsize, PAGE_SIZE);
if (btrfs_super_incompat_flags(fs_info->super_copy) &
BTRFS_FEATURE_INCOMPAT_RAID56) {
btrfs_err(fs_info,
"RAID56 is not yet supported for sector size %u with page size %lu",
sectorsize, PAGE_SIZE);
err = -EINVAL;
goto fail_alloc;
}
subpage_info = kzalloc(sizeof(*subpage_info), GFP_KERNEL);
if (!subpage_info)
goto fail_alloc;
@ -4157,7 +4150,8 @@ static int write_dev_supers(struct btrfs_device *device,
if (i == 0 && !btrfs_test_opt(device->fs_info, NOBARRIER))
bio->bi_opf |= REQ_FUA;
btrfsic_submit_bio(bio);
btrfsic_check_bio(bio);
submit_bio(bio);
if (btrfs_advance_sb_log(device, i))
errors++;
@ -4271,7 +4265,8 @@ static void write_dev_flush(struct btrfs_device *device)
init_completion(&device->flush_wait);
bio->bi_private = &device->flush_wait;
btrfsic_submit_bio(bio);
btrfsic_check_bio(bio);
submit_bio(bio);
set_bit(BTRFS_DEV_STATE_FLUSH_SENT, &device->dev_state);
}
@ -4504,12 +4499,11 @@ void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
{
bool drop_ref = false;
spin_lock(&fs_info->fs_roots_radix_lock);
radix_tree_delete(&fs_info->fs_roots_radix,
(unsigned long)root->root_key.objectid);
if (test_and_clear_bit(BTRFS_ROOT_IN_RADIX, &root->state))
spin_lock(&fs_info->fs_roots_lock);
xa_erase(&fs_info->fs_roots, (unsigned long)root->root_key.objectid);
if (test_and_clear_bit(BTRFS_ROOT_REGISTERED, &root->state))
drop_ref = true;
spin_unlock(&fs_info->fs_roots_radix_lock);
spin_unlock(&fs_info->fs_roots_lock);
if (BTRFS_FS_ERROR(fs_info)) {
ASSERT(root->log_root == NULL);
@ -4525,50 +4519,48 @@ void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
int btrfs_cleanup_fs_roots(struct btrfs_fs_info *fs_info)
{
u64 root_objectid = 0;
struct btrfs_root *gang[8];
int i = 0;
struct btrfs_root *roots[8];
unsigned long index = 0;
int i;
int err = 0;
unsigned int ret = 0;
int grabbed;
while (1) {
spin_lock(&fs_info->fs_roots_radix_lock);
ret = radix_tree_gang_lookup(&fs_info->fs_roots_radix,
(void **)gang, root_objectid,
ARRAY_SIZE(gang));
if (!ret) {
spin_unlock(&fs_info->fs_roots_radix_lock);
break;
}
root_objectid = gang[ret - 1]->root_key.objectid + 1;
struct btrfs_root *root;
for (i = 0; i < ret; i++) {
/* Avoid to grab roots in dead_roots */
if (btrfs_root_refs(&gang[i]->root_item) == 0) {
gang[i] = NULL;
continue;
}
/* grab all the search result for later use */
gang[i] = btrfs_grab_root(gang[i]);
spin_lock(&fs_info->fs_roots_lock);
if (!xa_find(&fs_info->fs_roots, &index, ULONG_MAX, XA_PRESENT)) {
spin_unlock(&fs_info->fs_roots_lock);
return err;
}
spin_unlock(&fs_info->fs_roots_radix_lock);
for (i = 0; i < ret; i++) {
if (!gang[i])
continue;
root_objectid = gang[i]->root_key.objectid;
err = btrfs_orphan_cleanup(gang[i]);
if (err)
grabbed = 0;
xa_for_each_start(&fs_info->fs_roots, index, root, index) {
/* Avoid grabbing roots in dead_roots */
if (btrfs_root_refs(&root->root_item) > 0)
roots[grabbed++] = btrfs_grab_root(root);
if (grabbed >= ARRAY_SIZE(roots))
break;
btrfs_put_root(gang[i]);
}
root_objectid++;
spin_unlock(&fs_info->fs_roots_lock);
for (i = 0; i < grabbed; i++) {
if (!roots[i])
continue;
index = roots[i]->root_key.objectid;
err = btrfs_orphan_cleanup(roots[i]);
if (err)
goto out;
btrfs_put_root(roots[i]);
}
index++;
}
/* release the uncleaned roots due to error */
for (; i < ret; i++) {
if (gang[i])
btrfs_put_root(gang[i]);
out:
/* Release the roots that remain uncleaned due to error */
for (; i < grabbed; i++) {
if (roots[i])
btrfs_put_root(roots[i]);
}
return err;
}
@ -4863,13 +4855,6 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_fs_info *fs_info)
__btrfs_btree_balance_dirty(fs_info, 0);
}
int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid, int level,
struct btrfs_key *first_key)
{
return btree_read_extent_buffer_pages(buf, parent_transid,
level, first_key);
}
static void btrfs_error_commit_super(struct btrfs_fs_info *fs_info)
{
/* cleanup FS via transaction */
@ -4885,31 +4870,28 @@ static void btrfs_error_commit_super(struct btrfs_fs_info *fs_info)
static void btrfs_drop_all_logs(struct btrfs_fs_info *fs_info)
{
struct btrfs_root *gang[8];
u64 root_objectid = 0;
int ret;
unsigned long index = 0;
int grabbed = 0;
struct btrfs_root *roots[8];
spin_lock(&fs_info->fs_roots_radix_lock);
while ((ret = radix_tree_gang_lookup(&fs_info->fs_roots_radix,
(void **)gang, root_objectid,
ARRAY_SIZE(gang))) != 0) {
int i;
spin_lock(&fs_info->fs_roots_lock);
while ((grabbed = xa_extract(&fs_info->fs_roots, (void **)roots, index,
ULONG_MAX, 8, XA_PRESENT))) {
for (int i = 0; i < grabbed; i++)
roots[i] = btrfs_grab_root(roots[i]);
spin_unlock(&fs_info->fs_roots_lock);
for (i = 0; i < ret; i++)
gang[i] = btrfs_grab_root(gang[i]);
spin_unlock(&fs_info->fs_roots_radix_lock);
for (i = 0; i < ret; i++) {
if (!gang[i])
for (int i = 0; i < grabbed; i++) {
if (!roots[i])
continue;
root_objectid = gang[i]->root_key.objectid;
btrfs_free_log(NULL, gang[i]);
btrfs_put_root(gang[i]);
index = roots[i]->root_key.objectid;
btrfs_free_log(NULL, roots[i]);
btrfs_put_root(roots[i]);
}
root_objectid++;
spin_lock(&fs_info->fs_roots_radix_lock);
index++;
spin_lock(&fs_info->fs_roots_lock);
}
spin_unlock(&fs_info->fs_roots_radix_lock);
spin_unlock(&fs_info->fs_roots_lock);
btrfs_free_log_root_tree(NULL, fs_info);
}

10
fs/btrfs/disk-io.h
View File

@ -87,8 +87,7 @@ void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info,
int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio,
struct page *page, u64 start, u64 end,
int mirror);
blk_status_t btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio,
int mirror_num, unsigned long bio_flags);
void btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio, int mirror_num);
#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
struct btrfs_root *btrfs_alloc_dummy_root(struct btrfs_fs_info *fs_info);
#endif
@ -120,13 +119,12 @@ void btrfs_put_root(struct btrfs_root *root);
void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
int atomic);
int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid, int level,
struct btrfs_key *first_key);
int btrfs_read_extent_buffer(struct extent_buffer *buf, u64 parent_transid,
int level, struct btrfs_key *first_key);
blk_status_t btrfs_bio_wq_end_io(struct btrfs_fs_info *info, struct bio *bio,
enum btrfs_wq_endio_type metadata);
blk_status_t btrfs_wq_submit_bio(struct inode *inode, struct bio *bio,
int mirror_num, unsigned long bio_flags,
u64 dio_file_offset,
int mirror_num, u64 dio_file_offset,
extent_submit_bio_start_t *submit_bio_start);
blk_status_t btrfs_submit_bio_done(void *private_data, struct bio *bio,
int mirror_num);

61
fs/btrfs/extent-tree.c
View File

@ -895,7 +895,13 @@ int lookup_inline_extent_backref(struct btrfs_trans_handle *trans,
err = -ENOENT;
while (1) {
if (ptr >= end) {
WARN_ON(ptr > end);
if (ptr > end) {
err = -EUCLEAN;
btrfs_print_leaf(path->nodes[0]);
btrfs_crit(fs_info,
"overrun extent record at slot %d while looking for inline extent for root %llu owner %llu offset %llu parent %llu",
path->slots[0], root_objectid, owner, offset, parent);
}
break;
}
iref = (struct btrfs_extent_inline_ref *)ptr;
@ -1577,12 +1583,12 @@ static int run_delayed_extent_op(struct btrfs_trans_handle *trans,
u32 item_size;
int ret;
int err = 0;
int metadata = !extent_op->is_data;
int metadata = 1;
if (TRANS_ABORTED(trans))
return 0;
if (metadata && !btrfs_fs_incompat(fs_info, SKINNY_METADATA))
if (!btrfs_fs_incompat(fs_info, SKINNY_METADATA))
metadata = 0;
path = btrfs_alloc_path();
@ -2180,7 +2186,7 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,
int btrfs_set_disk_extent_flags(struct btrfs_trans_handle *trans,
struct extent_buffer *eb, u64 flags,
int level, int is_data)
int level)
{
struct btrfs_delayed_extent_op *extent_op;
int ret;
@ -2192,7 +2198,6 @@ int btrfs_set_disk_extent_flags(struct btrfs_trans_handle *trans,
extent_op->flags_to_set = flags;
extent_op->update_flags = true;
extent_op->update_key = false;
extent_op->is_data = is_data ? true : false;
extent_op->level = level;
ret = btrfs_add_delayed_extent_op(trans, eb->start, eb->len, extent_op);
@ -2357,15 +2362,10 @@ static noinline int check_committed_ref(struct btrfs_root *root,
}
int btrfs_cross_ref_exist(struct btrfs_root *root, u64 objectid, u64 offset,
u64 bytenr, bool strict)
u64 bytenr, bool strict, struct btrfs_path *path)
{
struct btrfs_path *path;
int ret;
path = btrfs_alloc_path();
if (!path)
return -ENOMEM;
do {
ret = check_committed_ref(root, path, objectid,
offset, bytenr, strict);
@ -2376,7 +2376,7 @@ int btrfs_cross_ref_exist(struct btrfs_root *root, u64 objectid, u64 offset,
} while (ret == -EAGAIN);
out:
btrfs_free_path(path);
btrfs_release_path(path);
if (btrfs_is_data_reloc_root(root))
WARN_ON(ret > 0);
return ret;
@ -2497,24 +2497,21 @@ static u64 get_alloc_profile_by_root(struct btrfs_root *root, int data)
return ret;
}
static u64 first_logical_byte(struct btrfs_fs_info *fs_info, u64 search_start)
static u64 first_logical_byte(struct btrfs_fs_info *fs_info)
{
struct btrfs_block_group *cache;
u64 bytenr;
struct rb_node *leftmost;
u64 bytenr = 0;
spin_lock(&fs_info->block_group_cache_lock);
bytenr = fs_info->first_logical_byte;
spin_unlock(&fs_info->block_group_cache_lock);
read_lock(&fs_info->block_group_cache_lock);
/* Get the block group with the lowest logical start address. */
leftmost = rb_first_cached(&fs_info->block_group_cache_tree);
if (leftmost) {
struct btrfs_block_group *bg;
if (bytenr < (u64)-1)
return bytenr;
cache = btrfs_lookup_first_block_group(fs_info, search_start);
if (!cache)
return 0;
bytenr = cache->start;
btrfs_put_block_group(cache);
bg = rb_entry(leftmost, struct btrfs_block_group, cache_node);
bytenr = bg->start;
}
read_unlock(&fs_info->block_group_cache_lock);
return bytenr;
}
@ -3803,8 +3800,7 @@ static int do_allocation_zoned(struct btrfs_block_group *block_group,
/* Check RO and no space case before trying to activate it */
spin_lock(&block_group->lock);
if (block_group->ro ||
block_group->alloc_offset == block_group->zone_capacity) {
if (block_group->ro || btrfs_zoned_bg_is_full(block_group)) {
ret = 1;
/*
* May need to clear fs_info->{treelog,data_reloc}_bg.
@ -4272,7 +4268,7 @@ static noinline int find_free_extent(struct btrfs_root *root,
return ret;
ffe_ctl->search_start = max(ffe_ctl->search_start,
first_logical_byte(fs_info, 0));
first_logical_byte(fs_info));
ffe_ctl->search_start = max(ffe_ctl->search_start, ffe_ctl->hint_byte);
if (ffe_ctl->search_start == ffe_ctl->hint_byte) {
block_group = btrfs_lookup_block_group(fs_info,
@ -4959,7 +4955,6 @@ struct extent_buffer *btrfs_alloc_tree_block(struct btrfs_trans_handle *trans,
extent_op->flags_to_set = flags;
extent_op->update_key = skinny_metadata ? false : true;
extent_op->update_flags = true;
extent_op->is_data = false;
extent_op->level = level;
btrfs_init_generic_ref(&generic_ref, BTRFS_ADD_DELAYED_EXTENT,
@ -5144,7 +5139,7 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans,
ret = btrfs_dec_ref(trans, root, eb, 0);
BUG_ON(ret); /* -ENOMEM */
ret = btrfs_set_disk_extent_flags(trans, eb, flag,
btrfs_header_level(eb), 0);
btrfs_header_level(eb));
BUG_ON(ret); /* -ENOMEM */
wc->flags[level] |= flag;
}
@ -5818,7 +5813,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root, int update_ref, int for_reloc)
btrfs_qgroup_convert_reserved_meta(root, INT_MAX);
btrfs_qgroup_free_meta_all_pertrans(root);
if (test_bit(BTRFS_ROOT_IN_RADIX, &root->state))
if (test_bit(BTRFS_ROOT_REGISTERED, &root->state))
btrfs_add_dropped_root(trans, root);
else
btrfs_put_root(root);

621
fs/btrfs/extent_io.c
View File

File diff suppressed because it is too large Load Diff

47
fs/btrfs/extent_io.h
View File

@ -7,15 +7,9 @@
#include <linux/refcount.h>
#include <linux/fiemap.h>
#include <linux/btrfs_tree.h>
#include "compression.h"
#include "ulist.h"
/*
* flags for bio submission. The high bits indicate the compression
* type for this bio
*/
#define EXTENT_BIO_COMPRESSED 1
#define EXTENT_BIO_FLAG_SHIFT 16
enum {
EXTENT_BUFFER_UPTODATE,
EXTENT_BUFFER_DIRTY,
@ -32,7 +26,6 @@ enum {
/* write IO error */
EXTENT_BUFFER_WRITE_ERR,
EXTENT_BUFFER_NO_CHECK,
EXTENT_BUFFER_ZONE_FINISH,
};
/* these are flags for __process_pages_contig */
@ -71,9 +64,9 @@ struct btrfs_fs_info;
struct io_failure_record;
struct extent_io_tree;
typedef blk_status_t (submit_bio_hook_t)(struct inode *inode, struct bio *bio,
typedef void (submit_bio_hook_t)(struct inode *inode, struct bio *bio,
int mirror_num,
unsigned long bio_flags);
enum btrfs_compression_type compress_type);
typedef blk_status_t (extent_submit_bio_start_t)(struct inode *inode,
struct bio *bio, u64 dio_file_offset);
@ -102,17 +95,6 @@ struct extent_buffer {
#endif
};
/*
* Structure to record info about the bio being assembled, and other info like
* how many bytes are there before stripe/ordered extent boundary.
*/
struct btrfs_bio_ctrl {
struct bio *bio;
unsigned long bio_flags;
u32 len_to_stripe_boundary;
u32 len_to_oe_boundary;
};
/*
* Structure to record how many bytes and which ranges are set/cleared
*/
@ -158,17 +140,6 @@ static inline void extent_changeset_free(struct extent_changeset *changeset)
kfree(changeset);
}
static inline void extent_set_compress_type(unsigned long *bio_flags,
int compress_type)
{
*bio_flags |= compress_type << EXTENT_BIO_FLAG_SHIFT;
}
static inline int extent_compress_type(unsigned long bio_flags)
{
return bio_flags >> EXTENT_BIO_FLAG_SHIFT;
}
struct extent_map_tree;
typedef struct extent_map *(get_extent_t)(struct btrfs_inode *inode,
@ -178,11 +149,7 @@ typedef struct extent_map *(get_extent_t)(struct btrfs_inode *inode,
int try_release_extent_mapping(struct page *page, gfp_t mask);
int try_release_extent_buffer(struct page *page);
int __must_check submit_one_bio(struct bio *bio, int mirror_num,
unsigned long bio_flags);
int btrfs_do_readpage(struct page *page, struct extent_map **em_cached,
struct btrfs_bio_ctrl *bio_ctrl,
unsigned int read_flags, u64 *prev_em_start);
int btrfs_readpage(struct file *file, struct page *page);
int extent_write_full_page(struct page *page, struct writeback_control *wbc);
int extent_write_locked_range(struct inode *inode, u64 start, u64 end);
int extent_writepages(struct address_space *mapping,
@ -277,8 +244,10 @@ void extent_range_redirty_for_io(struct inode *inode, u64 start, u64 end);
void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
struct page *locked_page,
u32 bits_to_clear, unsigned long page_ops);
int btrfs_alloc_page_array(unsigned int nr_pages, struct page **page_array);
struct bio *btrfs_bio_alloc(unsigned int nr_iovecs);
struct bio *btrfs_bio_clone(struct bio *bio);
struct bio *btrfs_bio_clone(struct block_device *bdev, struct bio *bio);
struct bio *btrfs_bio_clone_partial(struct bio *orig, u64 offset, u64 size);
void end_extent_writepage(struct page *page, int err, u64 start, u64 end);
@ -297,7 +266,7 @@ struct io_failure_record {
u64 start;
u64 len;
u64 logical;
unsigned long bio_flags;
enum btrfs_compression_type compress_type;
int this_mirror;
int failed_mirror;
};

286
fs/btrfs/file.c
View File

@ -1460,8 +1460,27 @@ lock_and_cleanup_extent_if_need(struct btrfs_inode *inode, struct page **pages,
return ret;
}
static int check_can_nocow(struct btrfs_inode *inode, loff_t pos,
size_t *write_bytes, bool nowait)
/*
* Check if we can do nocow write into the range [@pos, @pos + @write_bytes)
*
* @pos: File offset.
* @write_bytes: The length to write, will be updated to the nocow writeable
* range.
*
* This function will flush ordered extents in the range to ensure proper
* nocow checks.
*
* Return:
* > 0 If we can nocow, and updates @write_bytes.
* 0 If we can't do a nocow write.
* -EAGAIN If we can't do a nocow write because snapshoting of the inode's
* root is in progress.
* < 0 If an error happened.
*
* NOTE: Callers need to call btrfs_check_nocow_unlock() if we return > 0.
*/
int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
size_t *write_bytes)
{
struct btrfs_fs_info *fs_info = inode->root->fs_info;
struct btrfs_root *root = inode->root;
@ -1472,7 +1491,7 @@ static int check_can_nocow(struct btrfs_inode *inode, loff_t pos,
if (!(inode->flags & (BTRFS_INODE_NODATACOW | BTRFS_INODE_PREALLOC)))
return 0;
if (!nowait && !btrfs_drew_try_write_lock(&root->snapshot_lock))
if (!btrfs_drew_try_write_lock(&root->snapshot_lock))
return -EAGAIN;
lockstart = round_down(pos, fs_info->sectorsize);
@ -1480,71 +1499,21 @@ static int check_can_nocow(struct btrfs_inode *inode, loff_t pos,
fs_info->sectorsize) - 1;
num_bytes = lockend - lockstart + 1;
if (nowait) {
struct btrfs_ordered_extent *ordered;
if (!try_lock_extent(&inode->io_tree, lockstart, lockend))
return -EAGAIN;
ordered = btrfs_lookup_ordered_range(inode, lockstart,
num_bytes);
if (ordered) {
btrfs_put_ordered_extent(ordered);
ret = -EAGAIN;
goto out_unlock;
}
} else {
btrfs_lock_and_flush_ordered_range(inode, lockstart,
lockend, NULL);
}
btrfs_lock_and_flush_ordered_range(inode, lockstart, lockend, NULL);
ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
NULL, NULL, NULL, false);
if (ret <= 0) {
ret = 0;
if (!nowait)
btrfs_drew_write_unlock(&root->snapshot_lock);
btrfs_drew_write_unlock(&root->snapshot_lock);
} else {
*write_bytes = min_t(size_t, *write_bytes ,
num_bytes - pos + lockstart);
}
out_unlock:
unlock_extent(&inode->io_tree, lockstart, lockend);
return ret;
}
static int check_nocow_nolock(struct btrfs_inode *inode, loff_t pos,
size_t *write_bytes)
{
return check_can_nocow(inode, pos, write_bytes, true);
}
/*
* Check if we can do nocow write into the range [@pos, @pos + @write_bytes)
*
* @pos: File offset
* @write_bytes: The length to write, will be updated to the nocow writeable
* range
*
* This function will flush ordered extents in the range to ensure proper
* nocow checks.
*
* Return:
* >0 and update @write_bytes if we can do nocow write
* 0 if we can't do nocow write
* -EAGAIN if we can't get the needed lock or there are ordered extents
* for * (nowait == true) case
* <0 if other error happened
*
* NOTE: Callers need to release the lock by btrfs_check_nocow_unlock().
*/
int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
size_t *write_bytes)
{
return check_can_nocow(inode, pos, write_bytes, false);
}
void btrfs_check_nocow_unlock(struct btrfs_inode *inode)
{
btrfs_drew_write_unlock(&inode->root->snapshot_lock);
@ -1579,20 +1548,15 @@ static int btrfs_write_check(struct kiocb *iocb, struct iov_iter *from,
loff_t oldsize;
loff_t start_pos;
if (iocb->ki_flags & IOCB_NOWAIT) {
size_t nocow_bytes = count;
/* We will allocate space in case nodatacow is not set, so bail */
if (check_nocow_nolock(BTRFS_I(inode), pos, &nocow_bytes) <= 0)
return -EAGAIN;
/*
* There are holes in the range or parts of the range that must
* be COWed (shared extents, RO block groups, etc), so just bail
* out.
*/
if (nocow_bytes < count)
return -EAGAIN;
}
/*
* Quickly bail out on NOWAIT writes if we don't have the nodatacow or
* prealloc flags, as without those flags we always have to COW. We will
* later check if we can really COW into the target range (using
* can_nocow_extent() at btrfs_get_blocks_direct_write()).
*/
if ((iocb->ki_flags & IOCB_NOWAIT) &&
!(BTRFS_I(inode)->flags & (BTRFS_INODE_NODATACOW | BTRFS_INODE_PREALLOC)))
return -EAGAIN;
current->backing_dev_info = inode_to_bdi(inode);
ret = file_remove_privs(file);
@ -1720,7 +1684,7 @@ static noinline ssize_t btrfs_buffered_write(struct kiocb *iocb,
WARN_ON(reserve_bytes == 0);
ret = btrfs_delalloc_reserve_metadata(BTRFS_I(inode),
reserve_bytes,
reserve_bytes);
reserve_bytes, false);
if (ret) {
if (!only_release_metadata)
btrfs_free_reserved_data_space(BTRFS_I(inode),
@ -1965,8 +1929,7 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from)
*/
again:
from->nofault = true;
err = iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops,
IOMAP_DIO_PARTIAL, written);
err = btrfs_dio_rw(iocb, from, written);
from->nofault = false;
/* No increment (+=) because iomap returns a cumulative value. */
@ -2570,10 +2533,10 @@ static int find_first_non_hole(struct btrfs_inode *inode, u64 *start, u64 *len)
return ret;
}
static int btrfs_punch_hole_lock_range(struct inode *inode,
const u64 lockstart,
const u64 lockend,
struct extent_state **cached_state)
static void btrfs_punch_hole_lock_range(struct inode *inode,
const u64 lockstart,
const u64 lockend,
struct extent_state **cached_state)
{
/*
* For subpage case, if the range is not at page boundary, we could
@ -2587,40 +2550,29 @@ static int btrfs_punch_hole_lock_range(struct inode *inode,
const u64 page_lockend = round_down(lockend + 1, PAGE_SIZE) - 1;
while (1) {
struct btrfs_ordered_extent *ordered;
int ret;
truncate_pagecache_range(inode, lockstart, lockend);
lock_extent_bits(&BTRFS_I(inode)->io_tree, lockstart, lockend,
cached_state);
ordered = btrfs_lookup_first_ordered_extent(BTRFS_I(inode),
lockend);
/*
* We need to make sure we have no ordered extents in this range
* and nobody raced in and read a page in this range, if we did
* we need to try again.
* We can't have ordered extents in the range, nor dirty/writeback
* pages, because we have locked the inode's VFS lock in exclusive
* mode, we have locked the inode's i_mmap_lock in exclusive mode,
* we have flushed all delalloc in the range and we have waited
* for any ordered extents in the range to complete.
* We can race with anyone reading pages from this range, so after
* locking the range check if we have pages in the range, and if
* we do, unlock the range and retry.
*/
if ((!ordered ||
(ordered->file_offset + ordered->num_bytes <= lockstart ||
ordered->file_offset > lockend)) &&
!filemap_range_has_page(inode->i_mapping,
page_lockstart, page_lockend)) {
if (ordered)
btrfs_put_ordered_extent(ordered);
if (!filemap_range_has_page(inode->i_mapping, page_lockstart,
page_lockend))
break;
}
if (ordered)
btrfs_put_ordered_extent(ordered);
unlock_extent_cached(&BTRFS_I(inode)->io_tree, lockstart,
lockend, cached_state);
ret = btrfs_wait_ordered_range(inode, lockstart,
lockend - lockstart + 1);
if (ret)
return ret;
}
return 0;
btrfs_assert_inode_range_clean(BTRFS_I(inode), lockstart, lockend);
}
static int btrfs_insert_replace_extent(struct btrfs_trans_handle *trans,
@ -2976,11 +2928,12 @@ static int btrfs_punch_hole(struct file *file, loff_t offset, loff_t len)
bool truncated_block = false;
bool updated_inode = false;
btrfs_inode_lock(inode, BTRFS_ILOCK_MMAP);
ret = btrfs_wait_ordered_range(inode, offset, len);
if (ret)
return ret;
goto out_only_mutex;
btrfs_inode_lock(inode, BTRFS_ILOCK_MMAP);
ino_size = round_up(inode->i_size, fs_info->sectorsize);
ret = find_first_non_hole(BTRFS_I(inode), &offset, &len);
if (ret < 0)
@ -3072,10 +3025,7 @@ static int btrfs_punch_hole(struct file *file, loff_t offset, loff_t len)
goto out_only_mutex;
}
ret = btrfs_punch_hole_lock_range(inode, lockstart, lockend,
&cached_state);
if (ret)
goto out_only_mutex;
btrfs_punch_hole_lock_range(inode, lockstart, lockend, &cached_state);
path = btrfs_alloc_path();
if (!path) {
@ -3237,8 +3187,6 @@ static int btrfs_zero_range(struct inode *inode,
u64 bytes_to_reserve = 0;
bool space_reserved = false;
inode_dio_wait(inode);
em = btrfs_get_extent(BTRFS_I(inode), NULL, 0, alloc_start,
alloc_end - alloc_start);
if (IS_ERR(em)) {
@ -3368,10 +3316,8 @@ static int btrfs_zero_range(struct inode *inode,
if (ret < 0)
goto out;
space_reserved = true;
ret = btrfs_punch_hole_lock_range(inode, lockstart, lockend,
&cached_state);
if (ret)
goto out;
btrfs_punch_hole_lock_range(inode, lockstart, lockend,
&cached_state);
ret = btrfs_qgroup_reserve_data(BTRFS_I(inode), &data_reserved,
alloc_start, bytes_to_reserve);
if (ret) {
@ -3417,6 +3363,9 @@ static long btrfs_fallocate(struct file *file, int mode,
u64 alloc_hint = 0;
u64 locked_end;
u64 actual_end = 0;
u64 data_space_needed = 0;
u64 data_space_reserved = 0;
u64 qgroup_reserved = 0;
struct extent_map *em;
int blocksize = btrfs_inode_sectorsize(BTRFS_I(inode));
int ret;
@ -3437,18 +3386,6 @@ static long btrfs_fallocate(struct file *file, int mode,
if (mode & FALLOC_FL_PUNCH_HOLE)
return btrfs_punch_hole(file, offset, len);
/*
* Only trigger disk allocation, don't trigger qgroup reserve
*
* For qgroup space, it will be checked later.
*/
if (!(mode & FALLOC_FL_ZERO_RANGE)) {
ret = btrfs_alloc_data_chunk_ondemand(BTRFS_I(inode),
alloc_end - alloc_start);
if (ret < 0)
return ret;
}
btrfs_inode_lock(inode, BTRFS_ILOCK_MMAP);
if (!(mode & FALLOC_FL_KEEP_SIZE) && offset + len > inode->i_size) {
@ -3485,8 +3422,12 @@ static long btrfs_fallocate(struct file *file, int mode,
}
/*
* wait for ordered IO before we have any locks. We'll loop again
* below with the locks held.
* We have locked the inode at the VFS level (in exclusive mode) and we
* have locked the i_mmap_lock lock (in exclusive mode). Now before
* locking the file range, flush all dealloc in the range and wait for
* all ordered extents in the range to complete. After this we can lock
* the file range and, due to the previous locking we did, we know there
* can't be more delalloc or ordered extents in the range.
*/
ret = btrfs_wait_ordered_range(inode, alloc_start,
alloc_end - alloc_start);
@ -3500,38 +3441,10 @@ static long btrfs_fallocate(struct file *file, int mode,
}
locked_end = alloc_end - 1;
while (1) {
struct btrfs_ordered_extent *ordered;
lock_extent_bits(&BTRFS_I(inode)->io_tree, alloc_start, locked_end,
&cached_state);
/* the extent lock is ordered inside the running
* transaction
*/
lock_extent_bits(&BTRFS_I(inode)->io_tree, alloc_start,
locked_end, &cached_state);
ordered = btrfs_lookup_first_ordered_extent(BTRFS_I(inode),
locked_end);
if (ordered &&
ordered->file_offset + ordered->num_bytes > alloc_start &&
ordered->file_offset < alloc_end) {
btrfs_put_ordered_extent(ordered);
unlock_extent_cached(&BTRFS_I(inode)->io_tree,
alloc_start, locked_end,
&cached_state);
/*
* we can't wait on the range with the transaction
* running or with the extent lock held
*/
ret = btrfs_wait_ordered_range(inode, alloc_start,
alloc_end - alloc_start);
if (ret)
goto out;
} else {
if (ordered)
btrfs_put_ordered_extent(ordered);
break;
}
}
btrfs_assert_inode_range_clean(BTRFS_I(inode), alloc_start, locked_end);
/* First, check if we exceed the qgroup limit */
INIT_LIST_HEAD(&reserve_list);
@ -3548,48 +3461,64 @@ static long btrfs_fallocate(struct file *file, int mode,
if (em->block_start == EXTENT_MAP_HOLE ||
(cur_offset >= inode->i_size &&
!test_bit(EXTENT_FLAG_PREALLOC, &em->flags))) {
ret = add_falloc_range(&reserve_list, cur_offset,
last_byte - cur_offset);
const u64 range_len = last_byte - cur_offset;
ret = add_falloc_range(&reserve_list, cur_offset, range_len);
if (ret < 0) {
free_extent_map(em);
break;
}
ret = btrfs_qgroup_reserve_data(BTRFS_I(inode),
&data_reserved, cur_offset,
last_byte - cur_offset);
&data_reserved, cur_offset, range_len);
if (ret < 0) {
cur_offset = last_byte;
free_extent_map(em);
break;
}
} else {
/*
* Do not need to reserve unwritten extent for this
* range, free reserved data space first, otherwise
* it'll result in false ENOSPC error.
*/
btrfs_free_reserved_data_space(BTRFS_I(inode),
data_reserved, cur_offset,
last_byte - cur_offset);
qgroup_reserved += range_len;
data_space_needed += range_len;
}
free_extent_map(em);
cur_offset = last_byte;
}
if (!ret && data_space_needed > 0) {
/*
* We are safe to reserve space here as we can't have delalloc
* in the range, see above.
*/
ret = btrfs_alloc_data_chunk_ondemand(BTRFS_I(inode),
data_space_needed);
if (!ret)
data_space_reserved = data_space_needed;
}
/*
* If ret is still 0, means we're OK to fallocate.
* Or just cleanup the list and exit.
*/
list_for_each_entry_safe(range, tmp, &reserve_list, list) {
if (!ret)
if (!ret) {
ret = btrfs_prealloc_file_range(inode, mode,
range->start,
range->len, i_blocksize(inode),
offset + len, &alloc_hint);
else
/*
* btrfs_prealloc_file_range() releases space even
* if it returns an error.
*/
data_space_reserved -= range->len;
qgroup_reserved -= range->len;
} else if (data_space_reserved > 0) {
btrfs_free_reserved_data_space(BTRFS_I(inode),
data_reserved, range->start,
range->len);
data_reserved, range->start,
range->len);
data_space_reserved -= range->len;
qgroup_reserved -= range->len;
} else if (qgroup_reserved > 0) {
btrfs_qgroup_free_data(BTRFS_I(inode), data_reserved,
range->start, range->len);
qgroup_reserved -= range->len;
}
list_del(&range->list);
kfree(range);
}
@ -3606,10 +3535,6 @@ static long btrfs_fallocate(struct file *file, int mode,
&cached_state);
out:
btrfs_inode_unlock(inode, BTRFS_ILOCK_MMAP);
/* Let go of our reservation. */
if (ret != 0 && !(mode & FALLOC_FL_ZERO_RANGE))
btrfs_free_reserved_data_space(BTRFS_I(inode), data_reserved,
cur_offset, alloc_end - cur_offset);
extent_changeset_free(data_reserved);
return ret;
}
@ -3767,8 +3692,7 @@ static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to)
*/
pagefault_disable();
to->nofault = true;
ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops,
IOMAP_DIO_PARTIAL, read);
ret = btrfs_dio_rw(iocb, to, read);
to->nofault = false;
pagefault_enable();

9
fs/btrfs/free-space-cache.c
View File

@ -2630,16 +2630,19 @@ int __btrfs_add_free_space(struct btrfs_block_group *block_group,
static int __btrfs_add_free_space_zoned(struct btrfs_block_group *block_group,
u64 bytenr, u64 size, bool used)
{
struct btrfs_fs_info *fs_info = block_group->fs_info;
struct btrfs_space_info *sinfo = block_group->space_info;
struct btrfs_free_space_ctl *ctl = block_group->free_space_ctl;
u64 offset = bytenr - block_group->start;
u64 to_free, to_unusable;
const int bg_reclaim_threshold = READ_ONCE(fs_info->bg_reclaim_threshold);
int bg_reclaim_threshold = 0;
bool initial = (size == block_group->length);
u64 reclaimable_unusable;
WARN_ON(!initial && offset + size > block_group->zone_capacity);
if (!initial)
bg_reclaim_threshold = READ_ONCE(sinfo->bg_reclaim_threshold);
spin_lock(&ctl->tree_lock);
if (!used)
to_free = size;
@ -4069,7 +4072,7 @@ static int cleanup_free_space_cache_v1(struct btrfs_fs_info *fs_info,
btrfs_info(fs_info, "cleaning free space cache v1");
node = rb_first(&fs_info->block_group_cache_tree);
node = rb_first_cached(&fs_info->block_group_cache_tree);
while (node) {
block_group = rb_entry(node, struct btrfs_block_group, cache_node);
ret = btrfs_remove_free_space_inode(trans, NULL, block_group);

2
fs/btrfs/free-space-tree.c
View File

@ -1178,7 +1178,7 @@ int btrfs_create_free_space_tree(struct btrfs_fs_info *fs_info)
goto abort;
}
node = rb_first(&fs_info->block_group_cache_tree);
node = rb_first_cached(&fs_info->block_group_cache_tree);
while (node) {
block_group = rb_entry(node, struct btrfs_block_group,
cache_node);

1872
fs/btrfs/inode.c
View File

File diff suppressed because it is too large Load Diff

280
fs/btrfs/ioctl.c
View File

@ -540,9 +540,35 @@ int __pure btrfs_is_empty_uuid(u8 *uuid)
return 1;
}
/*
* Calculate the number of transaction items to reserve for creating a subvolume
* or snapshot, not including the inode, directory entries, or parent directory.
*/
static unsigned int create_subvol_num_items(struct btrfs_qgroup_inherit *inherit)
{
/*
* 1 to add root block
* 1 to add root item
* 1 to add root ref
* 1 to add root backref
* 1 to add UUID item
* 1 to add qgroup info
* 1 to add qgroup limit
*
* Ideally the last two would only be accounted if qgroups are enabled,
* but that can change between now and the time we would insert them.
*/
unsigned int num_items = 7;
if (inherit) {
/* 2 to add qgroup relations for each inherited qgroup */
num_items += 2 * inherit->num_qgroups;
}
return num_items;
}
static noinline int create_subvol(struct user_namespace *mnt_userns,
struct inode *dir, struct dentry *dentry,
const char *name, int namelen,
struct btrfs_qgroup_inherit *inherit)
{
struct btrfs_fs_info *fs_info = btrfs_sb(dir->i_sb);
@ -555,11 +581,15 @@ static noinline int create_subvol(struct user_namespace *mnt_userns,
struct btrfs_root *new_root;
struct btrfs_block_rsv block_rsv;
struct timespec64 cur_time = current_time(dir);
struct inode *inode;
struct btrfs_new_inode_args new_inode_args = {
.dir = dir,
.dentry = dentry,
.subvol = true,
};
unsigned int trans_num_items;
int ret;
dev_t anon_dev = 0;
dev_t anon_dev;
u64 objectid;
u64 index = 0;
root_item = kzalloc(sizeof(*root_item), GFP_KERNEL);
if (!root_item)
@ -567,11 +597,7 @@ static noinline int create_subvol(struct user_namespace *mnt_userns,
ret = btrfs_get_free_objectid(fs_info->tree_root, &objectid);
if (ret)
goto fail_free;
ret = get_anon_bdev(&anon_dev);
if (ret < 0)
goto fail_free;
goto out_root_item;
/*
* Don't create subvolume whose level is not zero. Or qgroup will be
@ -579,36 +605,47 @@ static noinline int create_subvol(struct user_namespace *mnt_userns,
*/
if (btrfs_qgroup_level(objectid)) {
ret = -ENOSPC;
goto fail_free;
goto out_root_item;
}
btrfs_init_block_rsv(&block_rsv, BTRFS_BLOCK_RSV_TEMP);
/*
* The same as the snapshot creation, please see the comment
* of create_snapshot().
*/
ret = btrfs_subvolume_reserve_metadata(root, &block_rsv, 8, false);
ret = get_anon_bdev(&anon_dev);
if (ret < 0)
goto out_root_item;
new_inode_args.inode = btrfs_new_subvol_inode(mnt_userns, dir);
if (!new_inode_args.inode) {
ret = -ENOMEM;
goto out_anon_dev;
}
ret = btrfs_new_inode_prepare(&new_inode_args, &trans_num_items);
if (ret)
goto fail_free;
goto out_inode;
trans_num_items += create_subvol_num_items(inherit);
btrfs_init_block_rsv(&block_rsv, BTRFS_BLOCK_RSV_TEMP);
ret = btrfs_subvolume_reserve_metadata(root, &block_rsv,
trans_num_items, false);
if (ret)
goto out_new_inode_args;
trans = btrfs_start_transaction(root, 0);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
btrfs_subvolume_release_metadata(root, &block_rsv);
goto fail_free;
goto out_new_inode_args;
}
trans->block_rsv = &block_rsv;
trans->bytes_reserved = block_rsv.size;
ret = btrfs_qgroup_inherit(trans, 0, objectid, inherit);
if (ret)
goto fail;
goto out;
leaf = btrfs_alloc_tree_block(trans, root, 0, objectid, NULL, 0, 0, 0,
BTRFS_NESTING_NORMAL);
if (IS_ERR(leaf)) {
ret = PTR_ERR(leaf);
goto fail;
goto out;
}
btrfs_mark_buffer_dirty(leaf);
@ -663,75 +700,46 @@ static noinline int create_subvol(struct user_namespace *mnt_userns,
btrfs_tree_unlock(leaf);
btrfs_free_tree_block(trans, objectid, leaf, 0, 1);
free_extent_buffer(leaf);
goto fail;
goto out;
}
free_extent_buffer(leaf);
leaf = NULL;
key.offset = (u64)-1;
new_root = btrfs_get_new_fs_root(fs_info, objectid, anon_dev);
if (IS_ERR(new_root)) {
free_anon_bdev(anon_dev);
ret = PTR_ERR(new_root);
btrfs_abort_transaction(trans, ret);
goto fail;
goto out;
}
/* Freeing will be done in btrfs_put_root() of new_root */
/* anon_dev is owned by new_root now. */
anon_dev = 0;
BTRFS_I(new_inode_args.inode)->root = new_root;
/* ... and new_root is owned by new_inode_args.inode now. */
ret = btrfs_record_root_in_trans(trans, new_root);
if (ret) {
btrfs_put_root(new_root);
btrfs_abort_transaction(trans, ret);
goto fail;
}
ret = btrfs_create_subvol_root(trans, new_root, root, mnt_userns);
btrfs_put_root(new_root);
if (ret) {
/* We potentially lose an unused inode item here */
btrfs_abort_transaction(trans, ret);
goto fail;
}
/*
* insert the directory item
*/
ret = btrfs_set_inode_index(BTRFS_I(dir), &index);
if (ret) {
btrfs_abort_transaction(trans, ret);
goto fail;
}
ret = btrfs_insert_dir_item(trans, name, namelen, BTRFS_I(dir), &key,
BTRFS_FT_DIR, index);
if (ret) {
btrfs_abort_transaction(trans, ret);
goto fail;
}
btrfs_i_size_write(BTRFS_I(dir), dir->i_size + namelen * 2);
ret = btrfs_update_inode(trans, root, BTRFS_I(dir));
if (ret) {
btrfs_abort_transaction(trans, ret);
goto fail;
}
ret = btrfs_add_root_ref(trans, objectid, root->root_key.objectid,
btrfs_ino(BTRFS_I(dir)), index, name, namelen);
if (ret) {
btrfs_abort_transaction(trans, ret);
goto fail;
goto out;
}
ret = btrfs_uuid_tree_add(trans, root_item->uuid,
BTRFS_UUID_KEY_SUBVOL, objectid);
if (ret)
if (ret) {
btrfs_abort_transaction(trans, ret);
goto out;
}
fail:
kfree(root_item);
ret = btrfs_create_new_inode(trans, &new_inode_args);
if (ret) {
btrfs_abort_transaction(trans, ret);
goto out;
}
d_instantiate_new(dentry, new_inode_args.inode);
new_inode_args.inode = NULL;
out:
trans->block_rsv = NULL;
trans->bytes_reserved = 0;
btrfs_subvolume_release_metadata(root, &block_rsv);
@ -740,18 +748,14 @@ static noinline int create_subvol(struct user_namespace *mnt_userns,
btrfs_end_transaction(trans);
else
ret = btrfs_commit_transaction(trans);
if (!ret) {
inode = btrfs_lookup_dentry(dir, dentry);
if (IS_ERR(inode))
return PTR_ERR(inode);
d_instantiate(dentry, inode);
}
return ret;
fail_free:
out_new_inode_args:
btrfs_new_inode_args_destroy(&new_inode_args);
out_inode:
iput(new_inode_args.inode);
out_anon_dev:
if (anon_dev)
free_anon_bdev(anon_dev);
out_root_item:
kfree(root_item);
return ret;
}
@ -763,6 +767,7 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir,
struct btrfs_fs_info *fs_info = btrfs_sb(dir->i_sb);
struct inode *inode;
struct btrfs_pending_snapshot *pending_snapshot;
unsigned int trans_num_items;
struct btrfs_trans_handle *trans;
int ret;
@ -800,16 +805,14 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir,
btrfs_init_block_rsv(&pending_snapshot->block_rsv,
BTRFS_BLOCK_RSV_TEMP);
/*
* 1 - parent dir inode
* 2 - dir entries
* 1 - root item
* 2 - root ref/backref
* 1 - root of snapshot
* 1 - UUID item
* 1 to add dir item
* 1 to add dir index
* 1 to update parent inode item
*/
trans_num_items = create_subvol_num_items(inherit) + 3;
ret = btrfs_subvolume_reserve_metadata(BTRFS_I(dir)->root,
&pending_snapshot->block_rsv, 8,
false);
&pending_snapshot->block_rsv,
trans_num_items, false);
if (ret)
goto free_pending;
@ -979,7 +982,7 @@ static noinline int btrfs_mksubvol(const struct path *parent,
if (snap_src)
error = create_snapshot(snap_src, dir, dentry, readonly, inherit);
else
error = create_subvol(mnt_userns, dir, dentry, name, namelen, inherit);
error = create_subvol(mnt_userns, dir, dentry, inherit);
if (!error)
fsnotify_mkdir(dir, dentry);
@ -1413,8 +1416,19 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
if (!em)
break;
/* Skip hole/inline/preallocated extents */
if (em->block_start >= EXTENT_MAP_LAST_BYTE ||
/*
* If the file extent is an inlined one, we may still want to
* defrag it (fallthrough) if it will cause a regular extent.
* This is for users who want to convert inline extents to
* regular ones through max_inline= mount option.
*/
if (em->block_start == EXTENT_MAP_INLINE &&
em->len <= inode->root->fs_info->max_inline)
goto next;
/* Skip hole/delalloc/preallocated extents */
if (em->block_start == EXTENT_MAP_HOLE ||
em->block_start == EXTENT_MAP_DELALLOC ||
test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
goto next;
@ -1473,6 +1487,15 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
if (em->len >= get_extent_max_capacity(em))
goto next;
/*
* Normally there are no more extents after an inline one, thus
* @next_mergeable will normally be false and not defragged.
* So if an inline extent passed all above checks, just add it
* for defrag, and be converted to regular extents.
*/
if (em->block_start == EXTENT_MAP_INLINE)
goto add;
next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
extent_thresh, newer_than, locked);
if (!next_mergeable) {
@ -2594,7 +2617,7 @@ static noinline int search_ioctl(struct inode *inode,
static noinline int btrfs_ioctl_tree_search(struct inode *inode,
void __user *argp)
{
struct btrfs_ioctl_search_args __user *uargs;
struct btrfs_ioctl_search_args __user *uargs = argp;
struct btrfs_ioctl_search_key sk;
int ret;
size_t buf_size;
@ -2602,8 +2625,6 @@ static noinline int btrfs_ioctl_tree_search(struct inode *inode,
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
uargs = (struct btrfs_ioctl_search_args __user *)argp;
if (copy_from_user(&sk, &uargs->key, sizeof(sk)))
return -EFAULT;
@ -2626,7 +2647,7 @@ static noinline int btrfs_ioctl_tree_search(struct inode *inode,
static noinline int btrfs_ioctl_tree_search_v2(struct inode *inode,
void __user *argp)
{
struct btrfs_ioctl_search_args_v2 __user *uarg;
struct btrfs_ioctl_search_args_v2 __user *uarg = argp;
struct btrfs_ioctl_search_args_v2 args;
int ret;
size_t buf_size;
@ -2636,7 +2657,6 @@ static noinline int btrfs_ioctl_tree_search_v2(struct inode *inode,
return -EPERM;
/* copy search header and buffer size */
uarg = (struct btrfs_ioctl_search_args_v2 __user *)argp;
if (copy_from_user(&args, uarg, sizeof(args)))
return -EFAULT;
@ -4344,10 +4364,6 @@ static long btrfs_ioctl_balance(struct file *file, void __user *arg)
bool need_unlock; /* for mut. excl. ops lock */
int ret;
if (!arg)
btrfs_warn(fs_info,
"IOC_BALANCE ioctl (v1) is deprecated and will be removed in kernel 5.18");
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@ -4355,6 +4371,13 @@ static long btrfs_ioctl_balance(struct file *file, void __user *arg)
if (ret)
return ret;
bargs = memdup_user(arg, sizeof(*bargs));
if (IS_ERR(bargs)) {
ret = PTR_ERR(bargs);
bargs = NULL;
goto out;
}
again:
if (btrfs_exclop_start(fs_info, BTRFS_EXCLOP_BALANCE)) {
mutex_lock(&fs_info->balance_mutex);
@ -4402,59 +4425,42 @@ static long btrfs_ioctl_balance(struct file *file, void __user *arg)
}
locked:
if (arg) {
bargs = memdup_user(arg, sizeof(*bargs));
if (IS_ERR(bargs)) {
ret = PTR_ERR(bargs);
if (bargs->flags & BTRFS_BALANCE_RESUME) {
if (!fs_info->balance_ctl) {
ret = -ENOTCONN;
goto out_unlock;
}
if (bargs->flags & BTRFS_BALANCE_RESUME) {
if (!fs_info->balance_ctl) {
ret = -ENOTCONN;
goto out_bargs;
}
bctl = fs_info->balance_ctl;
spin_lock(&fs_info->balance_lock);
bctl->flags |= BTRFS_BALANCE_RESUME;
spin_unlock(&fs_info->balance_lock);
btrfs_exclop_balance(fs_info, BTRFS_EXCLOP_BALANCE);
bctl = fs_info->balance_ctl;
spin_lock(&fs_info->balance_lock);
bctl->flags |= BTRFS_BALANCE_RESUME;
spin_unlock(&fs_info->balance_lock);
btrfs_exclop_balance(fs_info, BTRFS_EXCLOP_BALANCE);
goto do_balance;
}
goto do_balance;
}
} else {
bargs = NULL;
if (bargs->flags & ~(BTRFS_BALANCE_ARGS_MASK | BTRFS_BALANCE_TYPE_MASK)) {
ret = -EINVAL;
goto out_unlock;
}
if (fs_info->balance_ctl) {
ret = -EINPROGRESS;
goto out_bargs;
goto out_unlock;
}
bctl = kzalloc(sizeof(*bctl), GFP_KERNEL);
if (!bctl) {
ret = -ENOMEM;
goto out_bargs;
goto out_unlock;
}
if (arg) {
memcpy(&bctl->data, &bargs->data, sizeof(bctl->data));
memcpy(&bctl->meta, &bargs->meta, sizeof(bctl->meta));
memcpy(&bctl->sys, &bargs->sys, sizeof(bctl->sys));
bctl->flags = bargs->flags;
} else {
/* balance everything - no filters */
bctl->flags |= BTRFS_BALANCE_TYPE_MASK;
}
if (bctl->flags & ~(BTRFS_BALANCE_ARGS_MASK | BTRFS_BALANCE_TYPE_MASK)) {
ret = -EINVAL;
goto out_bctl;
}
memcpy(&bctl->data, &bargs->data, sizeof(bctl->data));
memcpy(&bctl->meta, &bargs->meta, sizeof(bctl->meta));
memcpy(&bctl->sys, &bargs->sys, sizeof(bctl->sys));
bctl->flags = bargs->flags;
do_balance:
/*
* Ownership of bctl and exclusive operation goes to btrfs_balance.
@ -4467,21 +4473,19 @@ static long btrfs_ioctl_balance(struct file *file, void __user *arg)
ret = btrfs_balance(fs_info, bctl, bargs);
bctl = NULL;
if ((ret == 0 || ret == -ECANCELED) && arg) {
if (ret == 0 || ret == -ECANCELED) {
if (copy_to_user(arg, bargs, sizeof(*bargs)))
ret = -EFAULT;
}
out_bctl:
kfree(bctl);
out_bargs:
kfree(bargs);
out_unlock:
mutex_unlock(&fs_info->balance_mutex);
if (need_unlock)
btrfs_exclop_finish(fs_info);
out:
mnt_drop_write_file(file);
kfree(bargs);
return ret;
}

40
fs/btrfs/props.c
View File

@ -380,9 +380,8 @@ static struct prop_handler prop_handlers[] = {
},
};
static int inherit_props(struct btrfs_trans_handle *trans,
struct inode *inode,
struct inode *parent)
int btrfs_inode_inherit_props(struct btrfs_trans_handle *trans,
struct inode *inode, struct inode *parent)
{
struct btrfs_root *root = BTRFS_I(inode)->root;
struct btrfs_fs_info *fs_info = root->fs_info;
@ -457,41 +456,6 @@ static int inherit_props(struct btrfs_trans_handle *trans,
return 0;
}
int btrfs_inode_inherit_props(struct btrfs_trans_handle *trans,
struct inode *inode,
struct inode *dir)
{
if (!dir)
return 0;
return inherit_props(trans, inode, dir);
}
int btrfs_subvol_inherit_props(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
struct btrfs_root *parent_root)
{
struct super_block *sb = root->fs_info->sb;
struct inode *parent_inode, *child_inode;
int ret;
parent_inode = btrfs_iget(sb, BTRFS_FIRST_FREE_OBJECTID, parent_root);
if (IS_ERR(parent_inode))
return PTR_ERR(parent_inode);
child_inode = btrfs_iget(sb, BTRFS_FIRST_FREE_OBJECTID, root);
if (IS_ERR(child_inode)) {
iput(parent_inode);
return PTR_ERR(child_inode);
}
ret = inherit_props(trans, child_inode, parent_inode);
iput(child_inode);
iput(parent_inode);
return ret;
}
void __init btrfs_props_init(void)
{
int i;

4
fs/btrfs/props.h
View File

@ -23,8 +23,4 @@ int btrfs_inode_inherit_props(struct btrfs_trans_handle *trans,
struct inode *inode,
struct inode *dir);
int btrfs_subvol_inherit_props(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
struct btrfs_root *parent_root);
#endif

7
fs/btrfs/qgroup.c
View File

@ -2290,7 +2290,7 @@ int btrfs_qgroup_trace_subtree(struct btrfs_trans_handle *trans,
return 0;
if (!extent_buffer_uptodate(root_eb)) {
ret = btrfs_read_buffer(root_eb, root_gen, root_level, NULL);
ret = btrfs_read_extent_buffer(root_eb, root_gen, root_level, NULL);
if (ret)
goto out;
}
@ -3939,12 +3939,13 @@ int btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes,
}
int __btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes,
enum btrfs_qgroup_rsv_type type, bool enforce)
enum btrfs_qgroup_rsv_type type, bool enforce,
bool noflush)
{
int ret;
ret = btrfs_qgroup_reserve_meta(root, num_bytes, type, enforce);
if (ret <= 0 && ret != -EDQUOT)
if ((ret <= 0 && ret != -EDQUOT) || noflush)
return ret;
ret = try_flush_qgroup(root);

12
fs/btrfs/qgroup.h
View File

@ -364,19 +364,23 @@ int btrfs_qgroup_free_data(struct btrfs_inode *inode,
int btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes,
enum btrfs_qgroup_rsv_type type, bool enforce);
int __btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes,
enum btrfs_qgroup_rsv_type type, bool enforce);
enum btrfs_qgroup_rsv_type type, bool enforce,
bool noflush);
/* Reserve metadata space for pertrans and prealloc type */
static inline int btrfs_qgroup_reserve_meta_pertrans(struct btrfs_root *root,
int num_bytes, bool enforce)
{
return __btrfs_qgroup_reserve_meta(root, num_bytes,
BTRFS_QGROUP_RSV_META_PERTRANS, enforce);
BTRFS_QGROUP_RSV_META_PERTRANS,
enforce, false);
}
static inline int btrfs_qgroup_reserve_meta_prealloc(struct btrfs_root *root,
int num_bytes, bool enforce)
int num_bytes, bool enforce,
bool noflush)
{
return __btrfs_qgroup_reserve_meta(root, num_bytes,
BTRFS_QGROUP_RSV_META_PREALLOC, enforce);
BTRFS_QGROUP_RSV_META_PREALLOC,
enforce, noflush);
}
void __btrfs_qgroup_free_meta(struct btrfs_root *root, int num_bytes,

807
fs/btrfs/raid56.c
View File

File diff suppressed because it is too large Load Diff

9
fs/btrfs/raid56.h
View File

@ -31,15 +31,14 @@ struct btrfs_raid_bio;
struct btrfs_device;
int raid56_parity_recover(struct bio *bio, struct btrfs_io_context *bioc,
u64 stripe_len, int mirror_num, int generic_io);
int raid56_parity_write(struct bio *bio, struct btrfs_io_context *bioc,
u64 stripe_len);
u32 stripe_len, int mirror_num, int generic_io);
int raid56_parity_write(struct bio *bio, struct btrfs_io_context *bioc, u32 stripe_len);
void raid56_add_scrub_pages(struct btrfs_raid_bio *rbio, struct page *page,
u64 logical);
unsigned int pgoff, u64 logical);
struct btrfs_raid_bio *raid56_parity_alloc_scrub_rbio(struct bio *bio,
struct btrfs_io_context *bioc, u64 stripe_len,
struct btrfs_io_context *bioc, u32 stripe_len,
struct btrfs_device *scrub_dev,
unsigned long *dbitmap, int stripe_nsectors);
void raid56_parity_submit_scrub_rbio(struct btrfs_raid_bio *rbio);

23
fs/btrfs/reflink.c
View File

@ -614,14 +614,23 @@ static void btrfs_double_extent_unlock(struct inode *inode1, u64 loff1,
static void btrfs_double_extent_lock(struct inode *inode1, u64 loff1,
struct inode *inode2, u64 loff2, u64 len)
{
u64 range1_end = loff1 + len - 1;
u64 range2_end = loff2 + len - 1;
if (inode1 < inode2) {
swap(inode1, inode2);
swap(loff1, loff2);
swap(range1_end, range2_end);
} else if (inode1 == inode2 && loff2 < loff1) {
swap(loff1, loff2);
swap(range1_end, range2_end);
}
lock_extent(&BTRFS_I(inode1)->io_tree, loff1, loff1 + len - 1);
lock_extent(&BTRFS_I(inode2)->io_tree, loff2, loff2 + len - 1);
lock_extent(&BTRFS_I(inode1)->io_tree, loff1, range1_end);
lock_extent(&BTRFS_I(inode2)->io_tree, loff2, range2_end);
btrfs_assert_inode_range_clean(BTRFS_I(inode1), loff1, range1_end);
btrfs_assert_inode_range_clean(BTRFS_I(inode2), loff2, range2_end);
}
static void btrfs_double_mmap_lock(struct inode *inode1, struct inode *inode2)
@ -771,7 +780,6 @@ static int btrfs_remap_file_range_prep(struct file *file_in, loff_t pos_in,
struct inode *inode_in = file_inode(file_in);
struct inode *inode_out = file_inode(file_out);
u64 bs = BTRFS_I(inode_out)->root->fs_info->sb->s_blocksize;
bool same_inode = inode_out == inode_in;
u64 wb_len;
int ret;
@ -809,15 +817,6 @@ static int btrfs_remap_file_range_prep(struct file *file_in, loff_t pos_in,
else
wb_len = ALIGN(*len, bs);
/*
* Since we don't lock ranges, wait for ongoing lockless dio writes (as
* any in progress could create its ordered extents after we wait for
* existing ordered extents below).
*/
inode_dio_wait(inode_in);
if (!same_inode)
inode_dio_wait(inode_out);
/*
* Workaround to make sure NOCOW buffered write reach disk as NOCOW.
*

19
fs/btrfs/relocation.c
View File

@ -362,7 +362,7 @@ struct btrfs_root *find_reloc_root(struct btrfs_fs_info *fs_info, u64 bytenr)
rb_node = rb_simple_search(&rc->reloc_root_tree.rb_root, bytenr);
if (rb_node) {
node = rb_entry(rb_node, struct mapping_node, rb_node);
root = (struct btrfs_root *)node->data;
root = node->data;
}
spin_unlock(&rc->reloc_root_tree.lock);
return btrfs_grab_root(root);
@ -2997,7 +2997,8 @@ static int relocate_one_page(struct inode *inode, struct file_ra_state *ra,
/* Reserve metadata for this range */
ret = btrfs_delalloc_reserve_metadata(BTRFS_I(inode),
clamped_len, clamped_len);
clamped_len, clamped_len,
false);
if (ret)
goto release_page;
@ -3845,8 +3846,7 @@ struct inode *create_reloc_inode(struct btrfs_fs_info *fs_info,
btrfs_end_transaction(trans);
btrfs_btree_balance_dirty(fs_info);
if (err) {
if (inode)
iput(inode);
iput(inode);
inode = ERR_PTR(err);
}
return inode;
@ -3977,6 +3977,17 @@ int btrfs_relocate_block_group(struct btrfs_fs_info *fs_info, u64 group_start)
if (!bg)
return -ENOENT;
/*
* Relocation of a data block group creates ordered extents. Without
* sb_start_write(), we can freeze the filesystem while unfinished
* ordered extents are left. Such ordered extents can cause a deadlock
* e.g. when syncfs() is waiting for their completion but they can't
* finish because they block when joining a transaction, due to the
* fact that the freeze locks are being held in write mode.
*/
if (bg->flags & BTRFS_BLOCK_GROUP_DATA)
ASSERT(sb_write_started(fs_info->sb));
if (btrfs_pinned_by_swapfile(fs_info, bg)) {
btrfs_put_block_group(bg);
return -ETXTBSY;

3
fs/btrfs/root-tree.c
View File

@ -509,7 +509,8 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root,
/* One for parent inode, two for dir entries */
qgroup_num_bytes = 3 * fs_info->nodesize;
ret = btrfs_qgroup_reserve_meta_prealloc(root,
qgroup_num_bytes, true);
qgroup_num_bytes, true,
false);
if (ret)
return ret;
}

1899
fs/btrfs/scrub.c
View File

File diff suppressed because it is too large Load Diff

400
fs/btrfs/send.c
View File

@ -10,7 +10,6 @@
#include <linux/mount.h>
#include <linux/xattr.h>
#include <linux/posix_acl_xattr.h>
#include <linux/radix-tree.h>
#include <linux/vmalloc.h>
#include <linux/string.h>
#include <linux/compat.h>
@ -128,11 +127,18 @@ struct send_ctx {
struct list_head new_refs;
struct list_head deleted_refs;
struct radix_tree_root name_cache;
struct xarray name_cache;
struct list_head name_cache_list;
int name_cache_size;
/*
* The inode we are currently processing. It's not NULL only when we
* need to issue write commands for data extents from this inode.
*/
struct inode *cur_inode;
struct file_ra_state ra;
u64 page_cache_clear_start;
bool clean_page_cache;
/*
* We process inodes by their increasing order, so if before an
@ -262,14 +268,13 @@ struct orphan_dir_info {
struct name_cache_entry {
struct list_head list;
/*
* radix_tree has only 32bit entries but we need to handle 64bit inums.
* We use the lower 32bit of the 64bit inum to store it in the tree. If
* more then one inum would fall into the same entry, we use radix_list
* to store the additional entries. radix_list is also used to store
* entries where two entries have the same inum but different
* generations.
* On 32bit kernels, xarray has only 32bit indices, but we need to
* handle 64bit inums. We use the lower 32bit of the 64bit inum to store
* it in the tree. If more than one inum would fall into the same entry,
* we use inum_aliases to store the additional entries. inum_aliases is
* also used to store entries with the same inum but different generations.
*/
struct list_head radix_list;
struct list_head inum_aliases;
u64 ino;
u64 gen;
u64 parent_ino;
@ -2019,9 +2024,9 @@ static int did_overwrite_first_ref(struct send_ctx *sctx, u64 ino, u64 gen)
}
/*
* Insert a name cache entry. On 32bit kernels the radix tree index is 32bit,
* Insert a name cache entry. On 32bit kernels the xarray index is 32bit,
* so we need to do some special handling in case we have clashes. This function
* takes care of this with the help of name_cache_entry::radix_list.
* takes care of this with the help of name_cache_entry::inum_aliases.
* In case of error, nce is kfreed.
*/
static int name_cache_insert(struct send_ctx *sctx,
@ -2030,8 +2035,7 @@ static int name_cache_insert(struct send_ctx *sctx,
int ret = 0;
struct list_head *nce_head;
nce_head = radix_tree_lookup(&sctx->name_cache,
(unsigned long)nce->ino);
nce_head = xa_load(&sctx->name_cache, (unsigned long)nce->ino);
if (!nce_head) {
nce_head = kmalloc(sizeof(*nce_head), GFP_KERNEL);
if (!nce_head) {
@ -2040,14 +2044,14 @@ static int name_cache_insert(struct send_ctx *sctx,
}
INIT_LIST_HEAD(nce_head);
ret = radix_tree_insert(&sctx->name_cache, nce->ino, nce_head);
ret = xa_insert(&sctx->name_cache, nce->ino, nce_head, GFP_KERNEL);
if (ret < 0) {
kfree(nce_head);
kfree(nce);
return ret;
}
}
list_add_tail(&nce->radix_list, nce_head);
list_add_tail(&nce->inum_aliases, nce_head);
list_add_tail(&nce->list, &sctx->name_cache_list);
sctx->name_cache_size++;
@ -2059,15 +2063,14 @@ static void name_cache_delete(struct send_ctx *sctx,
{
struct list_head *nce_head;
nce_head = radix_tree_lookup(&sctx->name_cache,
(unsigned long)nce->ino);
nce_head = xa_load(&sctx->name_cache, (unsigned long)nce->ino);
if (!nce_head) {
btrfs_err(sctx->send_root->fs_info,
"name_cache_delete lookup failed ino %llu cache size %d, leaking memory",
nce->ino, sctx->name_cache_size);
}
list_del(&nce->radix_list);
list_del(&nce->inum_aliases);
list_del(&nce->list);
sctx->name_cache_size--;
@ -2075,7 +2078,7 @@ static void name_cache_delete(struct send_ctx *sctx,
* We may not get to the final release of nce_head if the lookup fails
*/
if (nce_head && list_empty(nce_head)) {
radix_tree_delete(&sctx->name_cache, (unsigned long)nce->ino);
xa_erase(&sctx->name_cache, (unsigned long)nce->ino);
kfree(nce_head);
}
}
@ -2086,11 +2089,11 @@ static struct name_cache_entry *name_cache_search(struct send_ctx *sctx,
struct list_head *nce_head;
struct name_cache_entry *cur;
nce_head = radix_tree_lookup(&sctx->name_cache, (unsigned long)ino);
nce_head = xa_load(&sctx->name_cache, (unsigned long)ino);
if (!nce_head)
return NULL;
list_for_each_entry(cur, nce_head, radix_list) {
list_for_each_entry(cur, nce_head, inum_aliases) {
if (cur->ino == ino && cur->gen == gen)
return cur;
}
@ -2675,61 +2678,43 @@ static int send_create_inode(struct send_ctx *sctx, u64 ino)
static int did_create_dir(struct send_ctx *sctx, u64 dir)
{
int ret = 0;
int iter_ret = 0;
struct btrfs_path *path = NULL;
struct btrfs_key key;
struct btrfs_key found_key;
struct btrfs_key di_key;
struct extent_buffer *eb;
struct btrfs_dir_item *di;
int slot;
path = alloc_path_for_send();
if (!path) {
ret = -ENOMEM;
goto out;
}
if (!path)
return -ENOMEM;
key.objectid = dir;
key.type = BTRFS_DIR_INDEX_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, sctx->send_root, &key, path, 0, 0);
if (ret < 0)
goto out;
while (1) {
eb = path->nodes[0];
slot = path->slots[0];
if (slot >= btrfs_header_nritems(eb)) {
ret = btrfs_next_leaf(sctx->send_root, path);
if (ret < 0) {
goto out;
} else if (ret > 0) {
ret = 0;
break;
}
continue;
}
btrfs_for_each_slot(sctx->send_root, &key, &found_key, path, iter_ret) {
struct extent_buffer *eb = path->nodes[0];
btrfs_item_key_to_cpu(eb, &found_key, slot);
if (found_key.objectid != key.objectid ||
found_key.type != key.type) {
ret = 0;
goto out;
break;
}
di = btrfs_item_ptr(eb, slot, struct btrfs_dir_item);
di = btrfs_item_ptr(eb, path->slots[0], struct btrfs_dir_item);
btrfs_dir_item_key_to_cpu(eb, di, &di_key);
if (di_key.type != BTRFS_ROOT_ITEM_KEY &&
di_key.objectid < sctx->send_progress) {
ret = 1;
goto out;
break;
}
path->slots[0]++;
}
/* Catch error found during iteration */
if (iter_ret < 0)
ret = iter_ret;
out:
btrfs_free_path(path);
return ret;
}
@ -2933,6 +2918,7 @@ static int can_rmdir(struct send_ctx *sctx, u64 dir, u64 dir_gen,
u64 send_progress)
{
int ret = 0;
int iter_ret = 0;
struct btrfs_root *root = sctx->parent_root;
struct btrfs_path *path;
struct btrfs_key key;
@ -2959,23 +2945,9 @@ static int can_rmdir(struct send_ctx *sctx, u64 dir, u64 dir_gen,
if (odi)
key.offset = odi->last_dir_index_offset;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto out;
while (1) {
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
struct waiting_dir_move *dm;
if (path->slots[0] >= btrfs_header_nritems(path->nodes[0])) {
ret = btrfs_next_leaf(root, path);
if (ret < 0)
goto out;
else if (ret > 0)
break;
continue;
}
btrfs_item_key_to_cpu(path->nodes[0], &found_key,
path->slots[0]);
if (found_key.objectid != key.objectid ||
found_key.type != key.type)
break;
@ -3010,8 +2982,10 @@ static int can_rmdir(struct send_ctx *sctx, u64 dir, u64 dir_gen,
ret = 0;
goto out;
}
path->slots[0]++;
}
if (iter_ret < 0) {
ret = iter_ret;
goto out;
}
free_orphan_dir_info(sctx, odi);
@ -3579,7 +3553,7 @@ static int check_ino_in_path(struct btrfs_root *root,
}
/*
* Check if ino ino1 is an ancestor of inode ino2 in the given root for any
* Check if inode ino1 is an ancestor of inode ino2 in the given root for any
* possible path (in case ino2 is not a directory and has multiple hard links).
* Return 1 if true, 0 if false and < 0 on error.
*/
@ -3591,6 +3565,7 @@ static int is_ancestor(struct btrfs_root *root,
{
bool free_fs_path = false;
int ret = 0;
int iter_ret = 0;
struct btrfs_path *path = NULL;
struct btrfs_key key;
@ -3611,26 +3586,12 @@ static int is_ancestor(struct btrfs_root *root,
key.type = BTRFS_INODE_REF_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto out;
while (true) {
btrfs_for_each_slot(root, &key, &key, path, iter_ret) {
struct extent_buffer *leaf = path->nodes[0];
int slot = path->slots[0];
u32 cur_offset = 0;
u32 item_size;
if (slot >= btrfs_header_nritems(leaf)) {
ret = btrfs_next_leaf(root, path);
if (ret < 0)
goto out;
if (ret > 0)
break;
continue;
}
btrfs_item_key_to_cpu(leaf, &key, slot);
if (key.objectid != ino2)
break;
if (key.type != BTRFS_INODE_REF_KEY &&
@ -3668,10 +3629,12 @@ static int is_ancestor(struct btrfs_root *root,
if (ret)
goto out;
}
path->slots[0]++;
}
ret = 0;
out:
if (iter_ret < 0)
ret = iter_ret;
out:
btrfs_free_path(path);
if (free_fs_path)
fs_path_free(fs_path);
@ -4551,13 +4514,12 @@ static int record_changed_ref(struct send_ctx *sctx)
static int process_all_refs(struct send_ctx *sctx,
enum btrfs_compare_tree_result cmd)
{
int ret;
int ret = 0;
int iter_ret = 0;
struct btrfs_root *root;
struct btrfs_path *path;
struct btrfs_key key;
struct btrfs_key found_key;
struct extent_buffer *eb;
int slot;
iterate_inode_ref_t cb;
int pending_move = 0;
@ -4581,24 +4543,7 @@ static int process_all_refs(struct send_ctx *sctx,
key.objectid = sctx->cmp_key->objectid;
key.type = BTRFS_INODE_REF_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto out;
while (1) {
eb = path->nodes[0];
slot = path->slots[0];
if (slot >= btrfs_header_nritems(eb)) {
ret = btrfs_next_leaf(root, path);
if (ret < 0)
goto out;
else if (ret > 0)
break;
continue;
}
btrfs_item_key_to_cpu(eb, &found_key, slot);
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
if (found_key.objectid != key.objectid ||
(found_key.type != BTRFS_INODE_REF_KEY &&
found_key.type != BTRFS_INODE_EXTREF_KEY))
@ -4607,8 +4552,11 @@ static int process_all_refs(struct send_ctx *sctx,
ret = iterate_inode_ref(root, path, &found_key, 0, cb, sctx);
if (ret < 0)
goto out;
path->slots[0]++;
}
/* Catch error found during iteration */
if (iter_ret < 0) {
ret = iter_ret;
goto out;
}
btrfs_release_path(path);
@ -4870,13 +4818,12 @@ static int process_changed_xattr(struct send_ctx *sctx)
static int process_all_new_xattrs(struct send_ctx *sctx)
{
int ret;
int ret = 0;
int iter_ret = 0;
struct btrfs_root *root;
struct btrfs_path *path;
struct btrfs_key key;
struct btrfs_key found_key;
struct extent_buffer *eb;
int slot;
path = alloc_path_for_send();
if (!path)
@ -4887,39 +4834,21 @@ static int process_all_new_xattrs(struct send_ctx *sctx)
key.objectid = sctx->cmp_key->objectid;
key.type = BTRFS_XATTR_ITEM_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto out;
while (1) {
eb = path->nodes[0];
slot = path->slots[0];
if (slot >= btrfs_header_nritems(eb)) {
ret = btrfs_next_leaf(root, path);
if (ret < 0) {
goto out;
} else if (ret > 0) {
ret = 0;
break;
}
continue;
}
btrfs_item_key_to_cpu(eb, &found_key, slot);
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
if (found_key.objectid != key.objectid ||
found_key.type != key.type) {
ret = 0;
goto out;
break;
}
ret = iterate_dir_item(root, path, __process_new_xattr, sctx);
if (ret < 0)
goto out;
path->slots[0]++;
break;
}
/* Catch error found during iteration */
if (iter_ret < 0)
ret = iter_ret;
out:
btrfs_free_path(path);
return ret;
}
@ -4946,7 +4875,6 @@ static int put_file_data(struct send_ctx *sctx, u64 offset, u32 len)
{
struct btrfs_root *root = sctx->send_root;
struct btrfs_fs_info *fs_info = root->fs_info;
struct inode *inode;
struct page *page;
pgoff_t index = offset >> PAGE_SHIFT;
pgoff_t last_index;
@ -4957,37 +4885,30 @@ static int put_file_data(struct send_ctx *sctx, u64 offset, u32 len)
if (ret)
return ret;
inode = btrfs_iget(fs_info->sb, sctx->cur_ino, root);
if (IS_ERR(inode))
return PTR_ERR(inode);
last_index = (offset + len - 1) >> PAGE_SHIFT;
/* initial readahead */
memset(&sctx->ra, 0, sizeof(struct file_ra_state));
file_ra_state_init(&sctx->ra, inode->i_mapping);
while (index <= last_index) {
unsigned cur_len = min_t(unsigned, len,
PAGE_SIZE - pg_offset);
page = find_lock_page(inode->i_mapping, index);
page = find_lock_page(sctx->cur_inode->i_mapping, index);
if (!page) {
page_cache_sync_readahead(inode->i_mapping, &sctx->ra,
NULL, index, last_index + 1 - index);
page_cache_sync_readahead(sctx->cur_inode->i_mapping,
&sctx->ra, NULL, index,
last_index + 1 - index);
page = find_or_create_page(inode->i_mapping, index,
GFP_KERNEL);
page = find_or_create_page(sctx->cur_inode->i_mapping,
index, GFP_KERNEL);
if (!page) {
ret = -ENOMEM;
break;
}
}
if (PageReadahead(page)) {
page_cache_async_readahead(inode->i_mapping, &sctx->ra,
NULL, page, index, last_index + 1 - index);
}
if (PageReadahead(page))
page_cache_async_readahead(sctx->cur_inode->i_mapping,
&sctx->ra, NULL, page, index,
last_index + 1 - index);
if (!PageUptodate(page)) {
btrfs_readpage(NULL, page);
@ -5013,7 +4934,7 @@ static int put_file_data(struct send_ctx *sctx, u64 offset, u32 len)
len -= cur_len;
sctx->send_size += cur_len;
}
iput(inode);
return ret;
}
@ -5220,12 +5141,49 @@ static int send_extent_data(struct send_ctx *sctx,
const u64 offset,
const u64 len)
{
const u64 end = offset + len;
u64 read_size = max_send_read_size(sctx);
u64 sent = 0;
if (sctx->flags & BTRFS_SEND_FLAG_NO_FILE_DATA)
return send_update_extent(sctx, offset, len);
if (sctx->cur_inode == NULL) {
struct btrfs_root *root = sctx->send_root;
sctx->cur_inode = btrfs_iget(root->fs_info->sb, sctx->cur_ino, root);
if (IS_ERR(sctx->cur_inode)) {
int err = PTR_ERR(sctx->cur_inode);
sctx->cur_inode = NULL;
return err;
}
memset(&sctx->ra, 0, sizeof(struct file_ra_state));
file_ra_state_init(&sctx->ra, sctx->cur_inode->i_mapping);
/*
* It's very likely there are no pages from this inode in the page
* cache, so after reading extents and sending their data, we clean
* the page cache to avoid trashing the page cache (adding pressure
* to the page cache and forcing eviction of other data more useful
* for applications).
*
* We decide if we should clean the page cache simply by checking
* if the inode's mapping nrpages is 0 when we first open it, and
* not by using something like filemap_range_has_page() before
* reading an extent because when we ask the readahead code to
* read a given file range, it may (and almost always does) read
* pages from beyond that range (see the documentation for
* page_cache_sync_readahead()), so it would not be reliable,
* because after reading the first extent future calls to
* filemap_range_has_page() would return true because the readahead
* on the previous extent resulted in reading pages of the current
* extent as well.
*/
sctx->clean_page_cache = (sctx->cur_inode->i_mapping->nrpages == 0);
sctx->page_cache_clear_start = round_down(offset, PAGE_SIZE);
}
while (sent < len) {
u64 size = min(len - sent, read_size);
int ret;
@ -5235,6 +5193,37 @@ static int send_extent_data(struct send_ctx *sctx,
return ret;
sent += size;
}
if (sctx->clean_page_cache && IS_ALIGNED(end, PAGE_SIZE)) {
/*
* Always operate only on ranges that are a multiple of the page
* size. This is not only to prevent zeroing parts of a page in
* the case of subpage sector size, but also to guarantee we evict
* pages, as passing a range that is smaller than page size does
* not evict the respective page (only zeroes part of its content).
*
* Always start from the end offset of the last range cleared.
* This is because the readahead code may (and very often does)
* reads pages beyond the range we request for readahead. So if
* we have an extent layout like this:
*
* [ extent A ] [ extent B ] [ extent C ]
*
* When we ask page_cache_sync_readahead() to read extent A, it
* may also trigger reads for pages of extent B. If we are doing
* an incremental send and extent B has not changed between the
* parent and send snapshots, some or all of its pages may end
* up being read and placed in the page cache. So when truncating
* the page cache we always start from the end offset of the
* previously processed extent up to the end of the current
* extent.
*/
truncate_inode_pages_range(&sctx->cur_inode->i_data,
sctx->page_cache_clear_start,
end - 1);
sctx->page_cache_clear_start = end;
}
return 0;
}
@ -5965,13 +5954,12 @@ static int process_extent(struct send_ctx *sctx,
static int process_all_extents(struct send_ctx *sctx)
{
int ret;
int ret = 0;
int iter_ret = 0;
struct btrfs_root *root;
struct btrfs_path *path;
struct btrfs_key key;
struct btrfs_key found_key;
struct extent_buffer *eb;
int slot;
root = sctx->send_root;
path = alloc_path_for_send();
@ -5981,41 +5969,21 @@ static int process_all_extents(struct send_ctx *sctx)
key.objectid = sctx->cmp_key->objectid;
key.type = BTRFS_EXTENT_DATA_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto out;
while (1) {
eb = path->nodes[0];
slot = path->slots[0];
if (slot >= btrfs_header_nritems(eb)) {
ret = btrfs_next_leaf(root, path);
if (ret < 0) {
goto out;
} else if (ret > 0) {
ret = 0;
break;
}
continue;
}
btrfs_item_key_to_cpu(eb, &found_key, slot);
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
if (found_key.objectid != key.objectid ||
found_key.type != key.type) {
ret = 0;
goto out;
break;
}
ret = process_extent(sctx, path, &found_key);
if (ret < 0)
goto out;
path->slots[0]++;
break;
}
/* Catch error found during iteration */
if (iter_ret < 0)
ret = iter_ret;
out:
btrfs_free_path(path);
return ret;
}
@ -6205,8 +6173,11 @@ static int btrfs_unlink_all_paths(struct send_ctx *sctx)
{
LIST_HEAD(deleted_refs);
struct btrfs_path *path;
struct btrfs_root *root = sctx->parent_root;
struct btrfs_key key;
struct btrfs_key found_key;
struct parent_paths_ctx ctx;
int iter_ret = 0;
int ret;
path = alloc_path_for_send();
@ -6216,39 +6187,26 @@ static int btrfs_unlink_all_paths(struct send_ctx *sctx)
key.objectid = sctx->cur_ino;
key.type = BTRFS_INODE_REF_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, sctx->parent_root, &key, path, 0, 0);
if (ret < 0)
goto out;
ctx.refs = &deleted_refs;
ctx.sctx = sctx;
while (true) {
struct extent_buffer *eb = path->nodes[0];
int slot = path->slots[0];
if (slot >= btrfs_header_nritems(eb)) {
ret = btrfs_next_leaf(sctx->parent_root, path);
if (ret < 0)
goto out;
else if (ret > 0)
break;
continue;
}
btrfs_item_key_to_cpu(eb, &key, slot);
if (key.objectid != sctx->cur_ino)
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
if (found_key.objectid != key.objectid)
break;
if (key.type != BTRFS_INODE_REF_KEY &&
key.type != BTRFS_INODE_EXTREF_KEY)
if (found_key.type != key.type &&
found_key.type != BTRFS_INODE_EXTREF_KEY)
break;
ret = iterate_inode_ref(sctx->parent_root, path, &key, 1,
ret = iterate_inode_ref(root, path, &found_key, 1,
record_parent_ref, &ctx);
if (ret < 0)
goto out;
path->slots[0]++;
}
/* Catch error found during iteration */
if (iter_ret < 0) {
ret = iter_ret;
goto out;
}
while (!list_empty(&deleted_refs)) {
@ -6270,6 +6228,30 @@ static int btrfs_unlink_all_paths(struct send_ctx *sctx)
return ret;
}
static void close_current_inode(struct send_ctx *sctx)
{
u64 i_size;
if (sctx->cur_inode == NULL)
return;
i_size = i_size_read(sctx->cur_inode);
/*
* If we are doing an incremental send, we may have extents between the
* last processed extent and the i_size that have not been processed
* because they haven't changed but we may have read some of their pages
* through readahead, see the comments at send_extent_data().
*/
if (sctx->clean_page_cache && sctx->page_cache_clear_start < i_size)
truncate_inode_pages_range(&sctx->cur_inode->i_data,
sctx->page_cache_clear_start,
round_up(i_size, PAGE_SIZE) - 1);
iput(sctx->cur_inode);
sctx->cur_inode = NULL;
}
static int changed_inode(struct send_ctx *sctx,
enum btrfs_compare_tree_result result)
{
@ -6280,6 +6262,8 @@ static int changed_inode(struct send_ctx *sctx,
u64 left_gen = 0;
u64 right_gen = 0;
close_current_inode(sctx);
sctx->cur_ino = key->objectid;
sctx->cur_inode_new_gen = 0;
sctx->cur_inode_last_extent = (u64)-1;
@ -7534,7 +7518,7 @@ long btrfs_ioctl_send(struct inode *inode, struct btrfs_ioctl_send_args *arg)
INIT_LIST_HEAD(&sctx->new_refs);
INIT_LIST_HEAD(&sctx->deleted_refs);
INIT_RADIX_TREE(&sctx->name_cache, GFP_KERNEL);
xa_init_flags(&sctx->name_cache, GFP_KERNEL);
INIT_LIST_HEAD(&sctx->name_cache_list);
sctx->flags = arg->flags;
@ -7766,6 +7750,8 @@ long btrfs_ioctl_send(struct inode *inode, struct btrfs_ioctl_send_args *arg)
name_cache_free(sctx);
close_current_inode(sctx);
kfree(sctx);
}

11
fs/btrfs/space-info.c
View File

@ -181,6 +181,12 @@ void btrfs_clear_space_info_full(struct btrfs_fs_info *info)
found->full = 0;
}
/*
* Block groups with more than this value (percents) of unusable space will be
* scheduled for background reclaim.
*/
#define BTRFS_DEFAULT_ZONED_RECLAIM_THRESH (75)
static int create_space_info(struct btrfs_fs_info *info, u64 flags)
{
@ -203,6 +209,9 @@ static int create_space_info(struct btrfs_fs_info *info, u64 flags)
INIT_LIST_HEAD(&space_info->priority_tickets);
space_info->clamp = 1;
if (btrfs_is_zoned(info))
space_info->bg_reclaim_threshold = BTRFS_DEFAULT_ZONED_RECLAIM_THRESH;
ret = btrfs_sysfs_add_space_info_type(info, space_info);
if (ret)
return ret;
@ -519,7 +528,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info,
items = calc_reclaim_items_nr(fs_info, to_reclaim) * 2;
}
trans = (struct btrfs_trans_handle *)current->journal_info;
trans = current->journal_info;
/*
* If we are doing more ordered than delalloc we need to just wait on

8
fs/btrfs/space-info.h
View File

@ -3,6 +3,8 @@
#ifndef BTRFS_SPACE_INFO_H
#define BTRFS_SPACE_INFO_H
#include "volumes.h"
struct btrfs_space_info {
spinlock_t lock;
@ -24,6 +26,12 @@ struct btrfs_space_info {
the space info if we had an ENOSPC in the
allocator. */
/*
* Once a block group drops below this threshold (percents) we'll
* schedule it for reclaim.
*/
int bg_reclaim_threshold;
int clamp; /* Used to scale our threshold for preemptive
flushing. The value is >> clamp, so turns
out to be a 2^clamp divisor. */

55
fs/btrfs/subpage.c
View File

@ -63,6 +63,29 @@
* This means a slightly higher tree locking latency.
*/
bool btrfs_is_subpage(const struct btrfs_fs_info *fs_info, struct page *page)
{
if (fs_info->sectorsize >= PAGE_SIZE)
return false;
/*
* Only data pages (either through DIO or compression) can have no
* mapping. And if page->mapping->host is data inode, it's subpage.
* As we have ruled our sectorsize >= PAGE_SIZE case already.
*/
if (!page->mapping || !page->mapping->host ||
is_data_inode(page->mapping->host))
return true;
/*
* Now the only remaining case is metadata, which we only go subpage
* routine if nodesize < PAGE_SIZE.
*/
if (fs_info->nodesize < PAGE_SIZE)
return true;
return false;
}
void btrfs_init_subpage_info(struct btrfs_subpage_info *subpage_info, u32 sectorsize)
{
unsigned int cur = 0;
@ -107,7 +130,7 @@ int btrfs_attach_subpage(const struct btrfs_fs_info *fs_info,
ASSERT(PageLocked(page));
/* Either not subpage, or the page already has private attached */
if (fs_info->sectorsize == PAGE_SIZE || PagePrivate(page))
if (!btrfs_is_subpage(fs_info, page) || PagePrivate(page))
return 0;
subpage = btrfs_alloc_subpage(fs_info, type);
@ -124,10 +147,10 @@ void btrfs_detach_subpage(const struct btrfs_fs_info *fs_info,
struct btrfs_subpage *subpage;
/* Either not subpage, or already detached */
if (fs_info->sectorsize == PAGE_SIZE || !PagePrivate(page))
if (!btrfs_is_subpage(fs_info, page) || !PagePrivate(page))
return;
subpage = (struct btrfs_subpage *)detach_page_private(page);
subpage = detach_page_private(page);
ASSERT(subpage);
btrfs_free_subpage(subpage);
}
@ -175,7 +198,7 @@ void btrfs_page_inc_eb_refs(const struct btrfs_fs_info *fs_info,
{
struct btrfs_subpage *subpage;
if (fs_info->sectorsize == PAGE_SIZE)
if (!btrfs_is_subpage(fs_info, page))
return;
ASSERT(PagePrivate(page) && page->mapping);
@ -190,7 +213,7 @@ void btrfs_page_dec_eb_refs(const struct btrfs_fs_info *fs_info,
{
struct btrfs_subpage *subpage;
if (fs_info->sectorsize == PAGE_SIZE)
if (!btrfs_is_subpage(fs_info, page))
return;
ASSERT(PagePrivate(page) && page->mapping);
@ -319,7 +342,7 @@ bool btrfs_subpage_end_and_test_writer(const struct btrfs_fs_info *fs_info,
int btrfs_page_start_writer_lock(const struct btrfs_fs_info *fs_info,
struct page *page, u64 start, u32 len)
{
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE) {
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page)) {
lock_page(page);
return 0;
}
@ -336,7 +359,7 @@ int btrfs_page_start_writer_lock(const struct btrfs_fs_info *fs_info,
void btrfs_page_end_writer_lock(const struct btrfs_fs_info *fs_info,
struct page *page, u64 start, u32 len)
{
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE)
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page))
return unlock_page(page);
btrfs_subpage_clamp_range(page, &start, &len);
if (btrfs_subpage_end_and_test_writer(fs_info, page, start, len))
@ -620,7 +643,7 @@ IMPLEMENT_BTRFS_SUBPAGE_TEST_OP(checked);
void btrfs_page_set_##name(const struct btrfs_fs_info *fs_info, \
struct page *page, u64 start, u32 len) \
{ \
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE) { \
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page)) { \
set_page_func(page); \
return; \
} \
@ -629,7 +652,7 @@ void btrfs_page_set_##name(const struct btrfs_fs_info *fs_info, \
void btrfs_page_clear_##name(const struct btrfs_fs_info *fs_info, \
struct page *page, u64 start, u32 len) \
{ \
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE) { \
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page)) { \
clear_page_func(page); \
return; \
} \
@ -638,14 +661,14 @@ void btrfs_page_clear_##name(const struct btrfs_fs_info *fs_info, \
bool btrfs_page_test_##name(const struct btrfs_fs_info *fs_info, \
struct page *page, u64 start, u32 len) \
{ \
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE) \
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page)) \
return test_page_func(page); \
return btrfs_subpage_test_##name(fs_info, page, start, len); \
} \
void btrfs_page_clamp_set_##name(const struct btrfs_fs_info *fs_info, \
struct page *page, u64 start, u32 len) \
{ \
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE) { \
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page)) { \
set_page_func(page); \
return; \
} \
@ -655,7 +678,7 @@ void btrfs_page_clamp_set_##name(const struct btrfs_fs_info *fs_info, \
void btrfs_page_clamp_clear_##name(const struct btrfs_fs_info *fs_info, \
struct page *page, u64 start, u32 len) \
{ \
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE) { \
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page)) { \
clear_page_func(page); \
return; \
} \
@ -665,7 +688,7 @@ void btrfs_page_clamp_clear_##name(const struct btrfs_fs_info *fs_info, \
bool btrfs_page_clamp_test_##name(const struct btrfs_fs_info *fs_info, \
struct page *page, u64 start, u32 len) \
{ \
if (unlikely(!fs_info) || fs_info->sectorsize == PAGE_SIZE) \
if (unlikely(!fs_info) || !btrfs_is_subpage(fs_info, page)) \
return test_page_func(page); \
btrfs_subpage_clamp_range(page, &start, &len); \
return btrfs_subpage_test_##name(fs_info, page, start, len); \
@ -694,7 +717,7 @@ void btrfs_page_assert_not_dirty(const struct btrfs_fs_info *fs_info,
return;
ASSERT(!PageDirty(page));
if (fs_info->sectorsize == PAGE_SIZE)
if (!btrfs_is_subpage(fs_info, page))
return;
ASSERT(PagePrivate(page) && page->private);
@ -722,8 +745,8 @@ void btrfs_page_unlock_writer(struct btrfs_fs_info *fs_info, struct page *page,
struct btrfs_subpage *subpage;
ASSERT(PageLocked(page));
/* For regular page size case, we just unlock the page */
if (fs_info->sectorsize == PAGE_SIZE)
/* For non-subpage case, we just unlock the page */
if (!btrfs_is_subpage(fs_info, page))
return unlock_page(page);
ASSERT(PagePrivate(page) && page->private);

2
fs/btrfs/subpage.h
View File

@ -74,6 +74,8 @@ enum btrfs_subpage_type {
BTRFS_SUBPAGE_DATA,
};
bool btrfs_is_subpage(const struct btrfs_fs_info *fs_info, struct page *page);
void btrfs_init_subpage_info(struct btrfs_subpage_info *subpage_info, u32 sectorsize);
int btrfs_attach_subpage(const struct btrfs_fs_info *fs_info,
struct page *page, enum btrfs_subpage_type type);

9
fs/btrfs/super.c
View File

@ -261,7 +261,7 @@ static struct ratelimit_state printk_limits[] = {
RATELIMIT_STATE_INIT(printk_limits[7], DEFAULT_RATELIMIT_INTERVAL, 100),
};
void __cold btrfs_printk(const struct btrfs_fs_info *fs_info, const char *fmt, ...)
void __cold _btrfs_printk(const struct btrfs_fs_info *fs_info, const char *fmt, ...)
{
char lvl[PRINTK_MAX_SINGLE_HEADER_LEN + 1] = "\0";
struct va_format vaf;
@ -292,10 +292,10 @@ void __cold btrfs_printk(const struct btrfs_fs_info *fs_info, const char *fmt, .
char statestr[STATE_STRING_BUF_LEN];
btrfs_state_to_string(fs_info, statestr);
printk("%sBTRFS %s (device %s%s): %pV\n", lvl, type,
_printk("%sBTRFS %s (device %s%s): %pV\n", lvl, type,
fs_info->sb->s_id, statestr, &vaf);
} else {
printk("%sBTRFS %s: %pV\n", lvl, type, &vaf);
_printk("%sBTRFS %s: %pV\n", lvl, type, &vaf);
}
}
@ -1903,6 +1903,7 @@ static void btrfs_resize_thread_pool(struct btrfs_fs_info *fs_info,
old_pool_size, new_pool_size);
btrfs_workqueue_set_max(fs_info->workers, new_pool_size);
btrfs_workqueue_set_max(fs_info->hipri_workers, new_pool_size);
btrfs_workqueue_set_max(fs_info->delalloc_workers, new_pool_size);
btrfs_workqueue_set_max(fs_info->caching_workers, new_pool_size);
btrfs_workqueue_set_max(fs_info->endio_workers, new_pool_size);
@ -1912,8 +1913,6 @@ static void btrfs_resize_thread_pool(struct btrfs_fs_info *fs_info,
btrfs_workqueue_set_max(fs_info->endio_write_workers, new_pool_size);
btrfs_workqueue_set_max(fs_info->endio_freespace_worker, new_pool_size);
btrfs_workqueue_set_max(fs_info->delayed_workers, new_pool_size);
btrfs_workqueue_set_max(fs_info->scrub_wr_completion_workers,
new_pool_size);
}
static inline void btrfs_remount_begin(struct btrfs_fs_info *fs_info,

43
fs/btrfs/sysfs.c
View File

@ -394,11 +394,9 @@ static ssize_t supported_sectorsizes_show(struct kobject *kobj,
{
ssize_t ret = 0;
/* 4K sector size is also supported with 64K page size */
if (PAGE_SIZE == SZ_64K)
/* An artificial limit to only support 4K and PAGE_SIZE */
if (PAGE_SIZE > SZ_4K)
ret += sysfs_emit_at(buf, ret, "%u ", SZ_4K);
/* Only sectorsize == PAGE_SIZE is now supported */
ret += sysfs_emit_at(buf, ret, "%lu\n", PAGE_SIZE);
return ret;
@ -722,6 +720,42 @@ SPACE_INFO_ATTR(bytes_zone_unusable);
SPACE_INFO_ATTR(disk_used);
SPACE_INFO_ATTR(disk_total);
static ssize_t btrfs_sinfo_bg_reclaim_threshold_show(struct kobject *kobj,
struct kobj_attribute *a,
char *buf)
{
struct btrfs_space_info *space_info = to_space_info(kobj);
ssize_t ret;
ret = sysfs_emit(buf, "%d\n", READ_ONCE(space_info->bg_reclaim_threshold));
return ret;
}
static ssize_t btrfs_sinfo_bg_reclaim_threshold_store(struct kobject *kobj,
struct kobj_attribute *a,
const char *buf, size_t len)
{
struct btrfs_space_info *space_info = to_space_info(kobj);
int thresh;
int ret;
ret = kstrtoint(buf, 10, &thresh);
if (ret)
return ret;
if (thresh < 0 || thresh > 100)
return -EINVAL;
WRITE_ONCE(space_info->bg_reclaim_threshold, thresh);
return len;
}
BTRFS_ATTR_RW(space_info, bg_reclaim_threshold,
btrfs_sinfo_bg_reclaim_threshold_show,
btrfs_sinfo_bg_reclaim_threshold_store);
/*
* Allocation information about block group types.
*
@ -738,6 +772,7 @@ static struct attribute *space_info_attrs[] = {
BTRFS_ATTR_PTR(space_info, bytes_zone_unusable),
BTRFS_ATTR_PTR(space_info, disk_used),
BTRFS_ATTR_PTR(space_info, disk_total),
BTRFS_ATTR_PTR(space_info, bg_reclaim_threshold),
NULL,
};
ATTRIBUTE_GROUPS(space_info);

24
fs/btrfs/tests/btrfs-tests.c
View File

@ -150,8 +150,8 @@ struct btrfs_fs_info *btrfs_alloc_dummy_fs_info(u32 nodesize, u32 sectorsize)
void btrfs_free_dummy_fs_info(struct btrfs_fs_info *fs_info)
{
struct radix_tree_iter iter;
void **slot;
unsigned long index;
struct extent_buffer *eb;
struct btrfs_device *dev, *tmp;
if (!fs_info)
@ -163,25 +163,9 @@ void btrfs_free_dummy_fs_info(struct btrfs_fs_info *fs_info)
test_mnt->mnt_sb->s_fs_info = NULL;
spin_lock(&fs_info->buffer_lock);
radix_tree_for_each_slot(slot, &fs_info->buffer_radix, &iter, 0) {
struct extent_buffer *eb;
eb = radix_tree_deref_slot_protected(slot, &fs_info->buffer_lock);
if (!eb)
continue;
/* Shouldn't happen but that kind of thinking creates CVE's */
if (radix_tree_exception(eb)) {
if (radix_tree_deref_retry(eb))
slot = radix_tree_iter_retry(&iter);
continue;
}
slot = radix_tree_iter_resume(slot, &iter);
spin_unlock(&fs_info->buffer_lock);
xa_for_each(&fs_info->extent_buffers, index, eb) {
free_extent_buffer_stale(eb);
spin_lock(&fs_info->buffer_lock);
}
spin_unlock(&fs_info->buffer_lock);
btrfs_mapping_tree_free(&fs_info->mapping_tree);
list_for_each_entry_safe(dev, tmp, &fs_info->fs_devices->devices,
@ -202,7 +186,7 @@ void btrfs_free_dummy_root(struct btrfs_root *root)
if (!root)
return;
/* Will be freed by btrfs_free_fs_roots */
if (WARN_ON(test_bit(BTRFS_ROOT_IN_RADIX, &root->state)))
if (WARN_ON(test_bit(BTRFS_ROOT_REGISTERED, &root->state)))
return;
btrfs_global_root_delete(root);
btrfs_put_root(root);

108
fs/btrfs/transaction.c
View File

@ -23,7 +23,7 @@
#include "space-info.h"
#include "zoned.h"
#define BTRFS_ROOT_TRANS_TAG 0
#define BTRFS_ROOT_TRANS_TAG XA_MARK_0
/*
* Transaction states and transitions
@ -221,7 +221,7 @@ static noinline void switch_commit_roots(struct btrfs_trans_handle *trans)
* the caching thread will re-start it's search from 3, and thus find
* the hole from [4,6) to add to the free space cache.
*/
spin_lock(&fs_info->block_group_cache_lock);
write_lock(&fs_info->block_group_cache_lock);
list_for_each_entry_safe(caching_ctl, next,
&fs_info->caching_block_groups, list) {
struct btrfs_block_group *cache = caching_ctl->block_group;
@ -234,7 +234,7 @@ static noinline void switch_commit_roots(struct btrfs_trans_handle *trans)
cache->last_byte_to_unpin = caching_ctl->progress;
}
}
spin_unlock(&fs_info->block_group_cache_lock);
write_unlock(&fs_info->block_group_cache_lock);
up_write(&fs_info->commit_root_sem);
}
@ -437,15 +437,15 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
*/
smp_wmb();
spin_lock(&fs_info->fs_roots_radix_lock);
spin_lock(&fs_info->fs_roots_lock);
if (root->last_trans == trans->transid && !force) {
spin_unlock(&fs_info->fs_roots_radix_lock);
spin_unlock(&fs_info->fs_roots_lock);
return 0;
}
radix_tree_tag_set(&fs_info->fs_roots_radix,
(unsigned long)root->root_key.objectid,
BTRFS_ROOT_TRANS_TAG);
spin_unlock(&fs_info->fs_roots_radix_lock);
xa_set_mark(&fs_info->fs_roots,
(unsigned long)root->root_key.objectid,
BTRFS_ROOT_TRANS_TAG);
spin_unlock(&fs_info->fs_roots_lock);
root->last_trans = trans->transid;
/* this is pretty tricky. We don't want to
@ -487,11 +487,9 @@ void btrfs_add_dropped_root(struct btrfs_trans_handle *trans,
spin_unlock(&cur_trans->dropped_roots_lock);
/* Make sure we don't try to update the root at commit time */
spin_lock(&fs_info->fs_roots_radix_lock);
radix_tree_tag_clear(&fs_info->fs_roots_radix,
(unsigned long)root->root_key.objectid,
BTRFS_ROOT_TRANS_TAG);
spin_unlock(&fs_info->fs_roots_radix_lock);
xa_clear_mark(&fs_info->fs_roots,
(unsigned long)root->root_key.objectid,
BTRFS_ROOT_TRANS_TAG);
}
int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
@ -1404,9 +1402,8 @@ void btrfs_add_dead_root(struct btrfs_root *root)
static noinline int commit_fs_roots(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_root *gang[8];
int i;
int ret;
struct btrfs_root *root;
unsigned long index;
/*
* At this point no one can be using this transaction to modify any tree
@ -1414,57 +1411,46 @@ static noinline int commit_fs_roots(struct btrfs_trans_handle *trans)
*/
ASSERT(trans->transaction->state == TRANS_STATE_COMMIT_DOING);
spin_lock(&fs_info->fs_roots_radix_lock);
while (1) {
ret = radix_tree_gang_lookup_tag(&fs_info->fs_roots_radix,
(void **)gang, 0,
ARRAY_SIZE(gang),
BTRFS_ROOT_TRANS_TAG);
if (ret == 0)
break;
for (i = 0; i < ret; i++) {
struct btrfs_root *root = gang[i];
int ret2;
spin_lock(&fs_info->fs_roots_lock);
xa_for_each_marked(&fs_info->fs_roots, index, root, BTRFS_ROOT_TRANS_TAG) {
int ret;
/*
* At this point we can neither have tasks logging inodes
* from a root nor trying to commit a log tree.
*/
ASSERT(atomic_read(&root->log_writers) == 0);
ASSERT(atomic_read(&root->log_commit[0]) == 0);
ASSERT(atomic_read(&root->log_commit[1]) == 0);
/*
* At this point we can neither have tasks logging inodes
* from a root nor trying to commit a log tree.
*/
ASSERT(atomic_read(&root->log_writers) == 0);
ASSERT(atomic_read(&root->log_commit[0]) == 0);
ASSERT(atomic_read(&root->log_commit[1]) == 0);
radix_tree_tag_clear(&fs_info->fs_roots_radix,
(unsigned long)root->root_key.objectid,
BTRFS_ROOT_TRANS_TAG);
spin_unlock(&fs_info->fs_roots_radix_lock);
xa_clear_mark(&fs_info->fs_roots,
(unsigned long)root->root_key.objectid,
BTRFS_ROOT_TRANS_TAG);
spin_unlock(&fs_info->fs_roots_lock);
btrfs_free_log(trans, root);
ret2 = btrfs_update_reloc_root(trans, root);
if (ret2)
return ret2;
btrfs_free_log(trans, root);
ret = btrfs_update_reloc_root(trans, root);
if (ret)
return ret;
/* see comments in should_cow_block() */
clear_bit(BTRFS_ROOT_FORCE_COW, &root->state);
smp_mb__after_atomic();
/* See comments in should_cow_block() */
clear_bit(BTRFS_ROOT_FORCE_COW, &root->state);
smp_mb__after_atomic();
if (root->commit_root != root->node) {
list_add_tail(&root->dirty_list,
&trans->transaction->switch_commits);
btrfs_set_root_node(&root->root_item,
root->node);
}
ret2 = btrfs_update_root(trans, fs_info->tree_root,
&root->root_key,
&root->root_item);
if (ret2)
return ret2;
spin_lock(&fs_info->fs_roots_radix_lock);
btrfs_qgroup_free_meta_all_pertrans(root);
if (root->commit_root != root->node) {
list_add_tail(&root->dirty_list,
&trans->transaction->switch_commits);
btrfs_set_root_node(&root->root_item, root->node);
}
ret = btrfs_update_root(trans, fs_info->tree_root,
&root->root_key, &root->root_item);
if (ret)
return ret;
spin_lock(&fs_info->fs_roots_lock);
btrfs_qgroup_free_meta_all_pertrans(root);
}
spin_unlock(&fs_info->fs_roots_radix_lock);
spin_unlock(&fs_info->fs_roots_lock);
return 0;
}

55
fs/btrfs/tree-checker.c
View File

@ -1855,3 +1855,58 @@ int btrfs_check_node(struct extent_buffer *node)
return ret;
}
ALLOW_ERROR_INJECTION(btrfs_check_node, ERRNO);
int btrfs_check_eb_owner(const struct extent_buffer *eb, u64 root_owner)
{
const bool is_subvol = is_fstree(root_owner);
const u64 eb_owner = btrfs_header_owner(eb);
/*
* Skip dummy fs, as selftests don't create unique ebs for each dummy
* root.
*/
if (test_bit(BTRFS_FS_STATE_DUMMY_FS_INFO, &eb->fs_info->fs_state))
return 0;
/*
* There are several call sites (backref walking, qgroup, and data
* reloc) passing 0 as @root_owner, as they are not holding the
* tree root. In that case, we can not do a reliable ownership check,
* so just exit.
*/
if (root_owner == 0)
return 0;
/*
* These trees use key.offset as their owner, our callers don't have
* the extra capacity to pass key.offset here. So we just skip them.
*/
if (root_owner == BTRFS_TREE_LOG_OBJECTID ||
root_owner == BTRFS_TREE_RELOC_OBJECTID)
return 0;
if (!is_subvol) {
/* For non-subvolume trees, the eb owner should match root owner */
if (unlikely(root_owner != eb_owner)) {
btrfs_crit(eb->fs_info,
"corrupted %s, root=%llu block=%llu owner mismatch, have %llu expect %llu",
btrfs_header_level(eb) == 0 ? "leaf" : "node",
root_owner, btrfs_header_bytenr(eb), eb_owner,
root_owner);
return -EUCLEAN;
}
return 0;
}
/*
* For subvolume trees, owners can mismatch, but they should all belong
* to subvolume trees.
*/
if (unlikely(is_subvol != is_fstree(eb_owner))) {
btrfs_crit(eb->fs_info,
"corrupted %s, root=%llu block=%llu owner mismatch, have %llu expect [%llu, %llu]",
btrfs_header_level(eb) == 0 ? "leaf" : "node",
root_owner, btrfs_header_bytenr(eb), eb_owner,
BTRFS_FIRST_FREE_OBJECTID, BTRFS_LAST_FREE_OBJECTID);
return -EUCLEAN;
}
return 0;
}

1
fs/btrfs/tree-checker.h
View File

@ -25,5 +25,6 @@ int btrfs_check_node(struct extent_buffer *node);
int btrfs_check_chunk_valid(struct extent_buffer *leaf,
struct btrfs_chunk *chunk, u64 logical);
int btrfs_check_eb_owner(const struct extent_buffer *eb, u64 root_owner);
#endif

11
fs/btrfs/tree-log.c
View File

@ -333,7 +333,7 @@ static int process_one_buffer(struct btrfs_root *log,
* pin down any logged extents, so we have to read the block.
*/
if (btrfs_fs_incompat(fs_info, MIXED_GROUPS)) {
ret = btrfs_read_buffer(eb, gen, level, NULL);
ret = btrfs_read_extent_buffer(eb, gen, level, NULL);
if (ret)
return ret;
}
@ -894,8 +894,7 @@ static noinline int replay_one_extent(struct btrfs_trans_handle *trans,
btrfs_update_inode_bytes(BTRFS_I(inode), nbytes, drop_args.bytes_found);
ret = btrfs_update_inode(trans, root, BTRFS_I(inode));
out:
if (inode)
iput(inode);
iput(inode);
return ret;
}
@ -2575,7 +2574,7 @@ static int replay_one_buffer(struct btrfs_root *log, struct extent_buffer *eb,
int i;
int ret;
ret = btrfs_read_buffer(eb, gen, level, NULL);
ret = btrfs_read_extent_buffer(eb, gen, level, NULL);
if (ret)
return ret;
@ -2786,7 +2785,7 @@ static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans,
path->slots[*level]++;
if (wc->free) {
ret = btrfs_read_buffer(next, ptr_gen,
ret = btrfs_read_extent_buffer(next, ptr_gen,
*level - 1, &first_key);
if (ret) {
free_extent_buffer(next);
@ -2815,7 +2814,7 @@ static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans,
free_extent_buffer(next);
continue;
}
ret = btrfs_read_buffer(next, ptr_gen, *level - 1, &first_key);
ret = btrfs_read_extent_buffer(next, ptr_gen, *level - 1, &first_key);
if (ret) {
free_extent_buffer(next);
return ret;

125
fs/btrfs/volumes.c
View File

@ -164,24 +164,12 @@ const struct btrfs_raid_attr btrfs_raid_array[BTRFS_NR_RAID_TYPES] = {
*/
enum btrfs_raid_types __attribute_const__ btrfs_bg_flags_to_raid_index(u64 flags)
{
if (flags & BTRFS_BLOCK_GROUP_RAID10)
return BTRFS_RAID_RAID10;
else if (flags & BTRFS_BLOCK_GROUP_RAID1)
return BTRFS_RAID_RAID1;
else if (flags & BTRFS_BLOCK_GROUP_RAID1C3)
return BTRFS_RAID_RAID1C3;
else if (flags & BTRFS_BLOCK_GROUP_RAID1C4)
return BTRFS_RAID_RAID1C4;
else if (flags & BTRFS_BLOCK_GROUP_DUP)
return BTRFS_RAID_DUP;
else if (flags & BTRFS_BLOCK_GROUP_RAID0)
return BTRFS_RAID_RAID0;
else if (flags & BTRFS_BLOCK_GROUP_RAID5)
return BTRFS_RAID_RAID5;
else if (flags & BTRFS_BLOCK_GROUP_RAID6)
return BTRFS_RAID_RAID6;
const u64 profile = (flags & BTRFS_BLOCK_GROUP_PROFILE_MASK);
return BTRFS_RAID_SINGLE; /* BTRFS_BLOCK_GROUP_SINGLE */
if (!profile)
return BTRFS_RAID_SINGLE;
return BTRFS_BG_FLAG_TO_INDEX(profile);
}
const char *btrfs_bg_type_to_raid_name(u64 flags)
@ -4062,13 +4050,6 @@ static inline int validate_convert_profile(struct btrfs_fs_info *fs_info,
if (!(bargs->flags & BTRFS_BALANCE_ARGS_CONVERT))
return true;
if (fs_info->sectorsize < PAGE_SIZE &&
bargs->target & BTRFS_BLOCK_GROUP_RAID56_MASK) {
btrfs_err(fs_info,
"RAID56 is not yet supported for sectorsize %u with page size %lu",
fs_info->sectorsize, PAGE_SIZE);
return false;
}
/* Profile is valid and does not have bits outside of the allowed set */
if (alloc_profile_is_valid(bargs->target, 1) &&
(bargs->target & ~allowed) == 0)
@ -6312,7 +6293,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
u64 offset;
u64 stripe_offset;
u64 stripe_nr;
u64 stripe_len;
u32 stripe_len;
u64 raid56_full_stripe_start = (u64)-1;
int data_stripes;
@ -6323,19 +6304,13 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
offset = logical - em->start;
/* Len of a stripe in a chunk */
stripe_len = map->stripe_len;
/* Stripe where this block falls in */
stripe_nr = div64_u64(offset, stripe_len);
/* Offset of stripe in the chunk */
stripe_offset = stripe_nr * stripe_len;
if (offset < stripe_offset) {
btrfs_crit(fs_info,
"stripe math has gone wrong, stripe_offset=%llu offset=%llu start=%llu logical=%llu stripe_len=%llu",
stripe_offset, offset, em->start, logical, stripe_len);
return -EINVAL;
}
/*
* Stripe_nr is where this block falls in
* stripe_offset is the offset of this block in its stripe.
*/
stripe_nr = div64_u64_rem(offset, stripe_len, &stripe_offset);
ASSERT(stripe_offset < U32_MAX);
/* stripe_offset is the offset of this block in its stripe */
stripe_offset = offset - stripe_offset;
data_stripes = nr_data_stripes(map);
/* Only stripe based profiles needs to check against stripe length. */
@ -6737,11 +6712,11 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,
bio_op(bio), bio->bi_opf, bio->bi_iter.bi_sector,
(unsigned long)dev->bdev->bd_dev, rcu_str_deref(dev->name),
dev->devid, bio->bi_iter.bi_size);
bio_set_dev(bio, dev->bdev);
btrfs_bio_counter_inc_noblocked(fs_info);
btrfsic_submit_bio(bio);
btrfsic_check_bio(bio);
submit_bio(bio);
}
static void bioc_error(struct btrfs_io_context *bioc, struct bio *bio, u64 logical)
@ -6823,10 +6798,12 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,
continue;
}
if (dev_nr < total_devs - 1)
bio = btrfs_bio_clone(first_bio);
else
if (dev_nr < total_devs - 1) {
bio = btrfs_bio_clone(dev->bdev, first_bio);
} else {
bio = first_bio;
bio_set_dev(bio, dev->bdev);
}
submit_stripe_bio(bioc, bio, bioc->stripes[dev_nr].physical, dev);
}
@ -7359,7 +7336,6 @@ static int read_one_dev(struct extent_buffer *leaf,
int btrfs_read_sys_array(struct btrfs_fs_info *fs_info)
{
struct btrfs_root *root = fs_info->tree_root;
struct btrfs_super_block *super_copy = fs_info->super_copy;
struct extent_buffer *sb;
struct btrfs_disk_key *disk_key;
@ -7375,30 +7351,16 @@ int btrfs_read_sys_array(struct btrfs_fs_info *fs_info)
struct btrfs_key key;
ASSERT(BTRFS_SUPER_INFO_SIZE <= fs_info->nodesize);
/*
* This will create extent buffer of nodesize, superblock size is
* fixed to BTRFS_SUPER_INFO_SIZE. If nodesize > sb size, this will
* overallocate but we can keep it as-is, only the first page is used.
* We allocated a dummy extent, just to use extent buffer accessors.
* There will be unused space after BTRFS_SUPER_INFO_SIZE, but
* that's fine, we will not go beyond system chunk array anyway.
*/
sb = btrfs_find_create_tree_block(fs_info, BTRFS_SUPER_INFO_OFFSET,
root->root_key.objectid, 0);
if (IS_ERR(sb))
return PTR_ERR(sb);
sb = alloc_dummy_extent_buffer(fs_info, BTRFS_SUPER_INFO_OFFSET);
if (!sb)
return -ENOMEM;
set_extent_buffer_uptodate(sb);
/*
* The sb extent buffer is artificial and just used to read the system array.
* set_extent_buffer_uptodate() call does not properly mark all it's
* pages up-to-date when the page is larger: extent does not cover the
* whole page and consequently check_page_uptodate does not find all
* the page's extents up-to-date (the hole beyond sb),
* write_extent_buffer then triggers a WARN_ON.
*
* Regular short extents go through mark_extent_buffer_dirty/writeback cycle,
* but sb spans only this function. Add an explicit SetPageUptodate call
* to silence the warning eg. on PowerPC 64.
*/
if (PAGE_SIZE > BTRFS_SUPER_INFO_SIZE)
SetPageUptodate(sb->pages[0]);
write_extent_buffer(sb, super_copy, 0, BTRFS_SUPER_INFO_SIZE);
array_size = btrfs_super_sys_array_size(super_copy);
@ -7561,6 +7523,7 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)
struct btrfs_key found_key;
int ret;
int slot;
int iter_ret = 0;
u64 total_dev = 0;
u64 last_ra_node = 0;
@ -7604,30 +7567,18 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)
key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
key.offset = 0;
key.type = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto error;
while (1) {
struct extent_buffer *node;
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
struct extent_buffer *node = path->nodes[1];
leaf = path->nodes[0];
slot = path->slots[0];
if (slot >= btrfs_header_nritems(leaf)) {
ret = btrfs_next_leaf(root, path);
if (ret == 0)
continue;
if (ret < 0)
goto error;
break;
}
node = path->nodes[1];
if (node) {
if (last_ra_node != node->start) {
readahead_tree_node_children(node);
last_ra_node = node->start;
}
}
btrfs_item_key_to_cpu(leaf, &found_key, slot);
if (found_key.type == BTRFS_DEV_ITEM_KEY) {
struct btrfs_dev_item *dev_item;
dev_item = btrfs_item_ptr(leaf, slot,
@ -7652,7 +7603,11 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)
if (ret)
goto error;
}
path->slots[0]++;
}
/* Catch error found during iteration */
if (iter_ret < 0) {
ret = iter_ret;
goto error;
}
/*
@ -7660,12 +7615,12 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)
* do another round of validation checks.
*/
if (total_dev != fs_info->fs_devices->total_devices) {
btrfs_err(fs_info,
"super_num_devices %llu mismatch with num_devices %llu found here",
btrfs_warn(fs_info,
"super block num_devices %llu mismatch with DEV_ITEM count %llu, will be repaired on next transaction commit",
btrfs_super_num_devices(fs_info->super_copy),
total_dev);
ret = -EINVAL;
goto error;
fs_info->fs_devices->total_devices = total_dev;
btrfs_set_super_num_devices(fs_info->super_copy, total_dev);
}
if (btrfs_super_total_bytes(fs_info->super_copy) <
fs_info->fs_devices->total_rw_bytes) {
@ -8277,7 +8232,7 @@ bool btrfs_pinned_by_swapfile(struct btrfs_fs_info *fs_info, void *ptr)
static int relocating_repair_kthread(void *data)
{
struct btrfs_block_group *cache = (struct btrfs_block_group *)data;
struct btrfs_block_group *cache = data;
struct btrfs_fs_info *fs_info = cache->fs_info;
u64 target;
int ret = 0;

42
fs/btrfs/volumes.h
View File

@ -17,17 +17,51 @@ extern struct mutex uuid_mutex;
#define BTRFS_STRIPE_LEN SZ_64K
/* Used by sanity check for btrfs_raid_types. */
#define const_ffs(n) (__builtin_ctzll(n) + 1)
/*
* The conversion from BTRFS_BLOCK_GROUP_* bits to btrfs_raid_type requires
* RAID0 always to be the lowest profile bit.
* Although it's part of on-disk format and should never change, do extra
* compile-time sanity checks.
*/
static_assert(const_ffs(BTRFS_BLOCK_GROUP_RAID0) <
const_ffs(BTRFS_BLOCK_GROUP_PROFILE_MASK & ~BTRFS_BLOCK_GROUP_RAID0));
static_assert(const_ilog2(BTRFS_BLOCK_GROUP_RAID0) >
ilog2(BTRFS_BLOCK_GROUP_TYPE_MASK));
/* ilog2() can handle both constants and variables */
#define BTRFS_BG_FLAG_TO_INDEX(profile) \
ilog2((profile) >> (ilog2(BTRFS_BLOCK_GROUP_RAID0) - 1))
enum btrfs_raid_types {
/* SINGLE is the special one as it doesn't have on-disk bit. */
BTRFS_RAID_SINGLE = 0,
BTRFS_RAID_RAID0 = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_RAID0),
BTRFS_RAID_RAID1 = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_RAID1),
BTRFS_RAID_DUP = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_DUP),
BTRFS_RAID_RAID10 = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_RAID10),
BTRFS_RAID_RAID5 = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_RAID5),
BTRFS_RAID_RAID6 = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_RAID6),
BTRFS_RAID_RAID1C3 = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_RAID1C3),
BTRFS_RAID_RAID1C4 = BTRFS_BG_FLAG_TO_INDEX(BTRFS_BLOCK_GROUP_RAID1C4),
BTRFS_NR_RAID_TYPES
};
struct btrfs_io_geometry {
/* remaining bytes before crossing a stripe */
u64 len;
/* offset of logical address in chunk */
u64 offset;
/* length of single IO stripe */
u64 stripe_len;
u32 stripe_len;
/* offset of address in stripe */
u32 stripe_offset;
/* number of stripe where address falls */
u64 stripe_nr;
/* offset of address in stripe */
u64 stripe_offset;
/* offset of raid56 stripe into the chunk */
u64 raid56_stripe_offset;
};
@ -430,7 +464,7 @@ struct map_lookup {
u64 type;
int io_align;
int io_width;
u64 stripe_len;
u32 stripe_len;
int num_stripes;
int sub_stripes;
int verified_stripes; /* For mount time dev extent verification */

40
fs/btrfs/xattr.c
View File

@ -272,10 +272,12 @@ int btrfs_setxattr_trans(struct inode *inode, const char *name,
ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size)
{
struct btrfs_key found_key;
struct btrfs_key key;
struct inode *inode = d_inode(dentry);
struct btrfs_root *root = BTRFS_I(inode)->root;
struct btrfs_path *path;
int iter_ret = 0;
int ret = 0;
size_t total_size = 0, size_left = size;
@ -294,44 +296,23 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size)
path->reada = READA_FORWARD;
/* search for our xattrs */
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto err;
while (1) {
btrfs_for_each_slot(root, &key, &found_key, path, iter_ret) {
struct extent_buffer *leaf;
int slot;
struct btrfs_dir_item *di;
struct btrfs_key found_key;
u32 item_size;
u32 cur;
leaf = path->nodes[0];
slot = path->slots[0];
/* this is where we start walking through the path */
if (slot >= btrfs_header_nritems(leaf)) {
/*
* if we've reached the last slot in this leaf we need
* to go to the next leaf and reset everything
*/
ret = btrfs_next_leaf(root, path);
if (ret < 0)
goto err;
else if (ret > 0)
break;
continue;
}
btrfs_item_key_to_cpu(leaf, &found_key, slot);
/* check to make sure this item is what we want */
if (found_key.objectid != key.objectid)
break;
if (found_key.type > BTRFS_XATTR_ITEM_KEY)
break;
if (found_key.type < BTRFS_XATTR_ITEM_KEY)
goto next_item;
continue;
di = btrfs_item_ptr(leaf, slot, struct btrfs_dir_item);
item_size = btrfs_item_size(leaf, slot);
@ -351,8 +332,8 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size)
goto next;
if (!buffer || (name_len + 1) > size_left) {
ret = -ERANGE;
goto err;
iter_ret = -ERANGE;
break;
}
read_extent_buffer(leaf, buffer, name_ptr, name_len);
@ -364,12 +345,13 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size)
cur += this_len;
di = (struct btrfs_dir_item *)((char *)di + this_len);
}
next_item:
path->slots[0]++;
}
ret = total_size;
err:
if (iter_ret < 0)
ret = iter_ret;
else
ret = total_size;
btrfs_free_path(path);
return ret;

225
fs/btrfs/zoned.c
View File

@ -51,11 +51,13 @@
#define BTRFS_MIN_ACTIVE_ZONES (BTRFS_SUPER_MIRROR_MAX + 5)
/*
* Maximum supported zone size. Currently, SMR disks have a zone size of
* 256MiB, and we are expecting ZNS drives to be in the 1-4GiB range. We do not
* expect the zone size to become larger than 8GiB in the near future.
* Minimum / maximum supported zone size. Currently, SMR disks have a zone
* size of 256MiB, and we are expecting ZNS drives to be in the 1-4GiB range.
* We do not expect the zone size to become larger than 8GiB or smaller than
* 4MiB in the near future.
*/
#define BTRFS_MAX_ZONE_SIZE SZ_8G
#define BTRFS_MIN_ZONE_SIZE SZ_4M
#define SUPER_INFO_SECTORS ((u64)BTRFS_SUPER_INFO_SIZE >> SECTOR_SHIFT)
@ -401,6 +403,13 @@ int btrfs_get_dev_zone_info(struct btrfs_device *device, bool populate_cache)
zone_info->zone_size, BTRFS_MAX_ZONE_SIZE);
ret = -EINVAL;
goto out;
} else if (zone_info->zone_size < BTRFS_MIN_ZONE_SIZE) {
btrfs_err_in_rcu(fs_info,
"zoned: %s: zone size %llu smaller than supported minimum %u",
rcu_str_deref(device->name),
zone_info->zone_size, BTRFS_MIN_ZONE_SIZE);
ret = -EINVAL;
goto out;
}
nr_sectors = bdev_nr_sectors(bdev);
@ -1835,7 +1844,7 @@ bool btrfs_zone_activate(struct btrfs_block_group *block_group)
}
/* No space left */
if (block_group->alloc_offset == block_group->zone_capacity) {
if (btrfs_zoned_bg_is_full(block_group)) {
ret = false;
goto out_unlock;
}
@ -1872,20 +1881,14 @@ bool btrfs_zone_activate(struct btrfs_block_group *block_group)
return ret;
}
int btrfs_zone_finish(struct btrfs_block_group *block_group)
static int do_zone_finish(struct btrfs_block_group *block_group, bool fully_written)
{
struct btrfs_fs_info *fs_info = block_group->fs_info;
struct map_lookup *map;
struct btrfs_device *device;
u64 physical;
bool need_zone_finish;
int ret = 0;
int i;
if (!btrfs_is_zoned(fs_info))
return 0;
map = block_group->physical_map;
spin_lock(&block_group->lock);
if (!block_group->zone_is_active) {
spin_unlock(&block_group->lock);
@ -1895,39 +1898,55 @@ int btrfs_zone_finish(struct btrfs_block_group *block_group)
/* Check if we have unwritten allocated space */
if ((block_group->flags &
(BTRFS_BLOCK_GROUP_METADATA | BTRFS_BLOCK_GROUP_SYSTEM)) &&
block_group->alloc_offset > block_group->meta_write_pointer) {
block_group->start + block_group->alloc_offset > block_group->meta_write_pointer) {
spin_unlock(&block_group->lock);
return -EAGAIN;
}
spin_unlock(&block_group->lock);
ret = btrfs_inc_block_group_ro(block_group, false);
if (ret)
return ret;
/* Ensure all writes in this block group finish */
btrfs_wait_block_group_reservations(block_group);
/* No need to wait for NOCOW writers. Zoned mode does not allow that. */
btrfs_wait_ordered_roots(fs_info, U64_MAX, block_group->start,
block_group->length);
spin_lock(&block_group->lock);
/*
* Bail out if someone already deactivated the block group, or
* allocated space is left in the block group.
* If we are sure that the block group is full (= no more room left for
* new allocation) and the IO for the last usable block is completed, we
* don't need to wait for the other IOs. This holds because we ensure
* the sequential IO submissions using the ZONE_APPEND command for data
* and block_group->meta_write_pointer for metadata.
*/
if (!block_group->zone_is_active) {
if (!fully_written) {
spin_unlock(&block_group->lock);
btrfs_dec_block_group_ro(block_group);
return 0;
ret = btrfs_inc_block_group_ro(block_group, false);
if (ret)
return ret;
/* Ensure all writes in this block group finish */
btrfs_wait_block_group_reservations(block_group);
/* No need to wait for NOCOW writers. Zoned mode does not allow that */
btrfs_wait_ordered_roots(fs_info, U64_MAX, block_group->start,
block_group->length);
spin_lock(&block_group->lock);
/*
* Bail out if someone already deactivated the block group, or
* allocated space is left in the block group.
*/
if (!block_group->zone_is_active) {
spin_unlock(&block_group->lock);
btrfs_dec_block_group_ro(block_group);
return 0;
}
if (block_group->reserved) {
spin_unlock(&block_group->lock);
btrfs_dec_block_group_ro(block_group);
return -EAGAIN;
}
}
if (block_group->reserved) {
spin_unlock(&block_group->lock);
btrfs_dec_block_group_ro(block_group);
return -EAGAIN;
}
/*
* The block group is not fully allocated, so not fully written yet. We
* need to send ZONE_FINISH command to free up an active zone.
*/
need_zone_finish = !btrfs_zoned_bg_is_full(block_group);
block_group->zone_is_active = 0;
block_group->alloc_offset = block_group->zone_capacity;
@ -1936,24 +1955,29 @@ int btrfs_zone_finish(struct btrfs_block_group *block_group)
btrfs_clear_data_reloc_bg(block_group);
spin_unlock(&block_group->lock);
map = block_group->physical_map;
for (i = 0; i < map->num_stripes; i++) {
device = map->stripes[i].dev;
physical = map->stripes[i].physical;
struct btrfs_device *device = map->stripes[i].dev;
const u64 physical = map->stripes[i].physical;
if (device->zone_info->max_active_zones == 0)
continue;
ret = blkdev_zone_mgmt(device->bdev, REQ_OP_ZONE_FINISH,
physical >> SECTOR_SHIFT,
device->zone_info->zone_size >> SECTOR_SHIFT,
GFP_NOFS);
if (need_zone_finish) {
ret = blkdev_zone_mgmt(device->bdev, REQ_OP_ZONE_FINISH,
physical >> SECTOR_SHIFT,
device->zone_info->zone_size >> SECTOR_SHIFT,
GFP_NOFS);
if (ret)
return ret;
if (ret)
return ret;
}
btrfs_dev_clear_active_zone(device, physical);
}
btrfs_dec_block_group_ro(block_group);
if (!fully_written)
btrfs_dec_block_group_ro(block_group);
spin_lock(&fs_info->zone_active_bgs_lock);
ASSERT(!list_empty(&block_group->active_bg_list));
@ -1966,6 +1990,14 @@ int btrfs_zone_finish(struct btrfs_block_group *block_group)
return 0;
}
int btrfs_zone_finish(struct btrfs_block_group *block_group)
{
if (!btrfs_is_zoned(block_group->fs_info))
return 0;
return do_zone_finish(block_group, false);
}
bool btrfs_can_activate_zone(struct btrfs_fs_devices *fs_devices, u64 flags)
{
struct btrfs_fs_info *fs_info = fs_devices->fs_info;
@ -1997,9 +2029,7 @@ bool btrfs_can_activate_zone(struct btrfs_fs_devices *fs_devices, u64 flags)
void btrfs_zone_finish_endio(struct btrfs_fs_info *fs_info, u64 logical, u64 length)
{
struct btrfs_block_group *block_group;
struct map_lookup *map;
struct btrfs_device *device;
u64 physical;
u64 min_alloc_bytes;
if (!btrfs_is_zoned(fs_info))
return;
@ -2007,44 +2037,54 @@ void btrfs_zone_finish_endio(struct btrfs_fs_info *fs_info, u64 logical, u64 len
block_group = btrfs_lookup_block_group(fs_info, logical);
ASSERT(block_group);
if (logical + length < block_group->start + block_group->zone_capacity)
/* No MIXED_BG on zoned btrfs. */
if (block_group->flags & BTRFS_BLOCK_GROUP_DATA)
min_alloc_bytes = fs_info->sectorsize;
else
min_alloc_bytes = fs_info->nodesize;
/* Bail out if we can allocate more data from this block group. */
if (logical + length + min_alloc_bytes <=
block_group->start + block_group->zone_capacity)
goto out;
spin_lock(&block_group->lock);
if (!block_group->zone_is_active) {
spin_unlock(&block_group->lock);
goto out;
}
block_group->zone_is_active = 0;
/* We should have consumed all the free space */
ASSERT(block_group->alloc_offset == block_group->zone_capacity);
ASSERT(block_group->free_space_ctl->free_space == 0);
btrfs_clear_treelog_bg(block_group);
btrfs_clear_data_reloc_bg(block_group);
spin_unlock(&block_group->lock);
map = block_group->physical_map;
device = map->stripes[0].dev;
physical = map->stripes[0].physical;
if (!device->zone_info->max_active_zones)
goto out;
btrfs_dev_clear_active_zone(device, physical);
spin_lock(&fs_info->zone_active_bgs_lock);
ASSERT(!list_empty(&block_group->active_bg_list));
list_del_init(&block_group->active_bg_list);
spin_unlock(&fs_info->zone_active_bgs_lock);
btrfs_put_block_group(block_group);
do_zone_finish(block_group, true);
out:
btrfs_put_block_group(block_group);
}
static void btrfs_zone_finish_endio_workfn(struct work_struct *work)
{
struct btrfs_block_group *bg =
container_of(work, struct btrfs_block_group, zone_finish_work);
wait_on_extent_buffer_writeback(bg->last_eb);
free_extent_buffer(bg->last_eb);
btrfs_zone_finish_endio(bg->fs_info, bg->start, bg->length);
btrfs_put_block_group(bg);
}
void btrfs_schedule_zone_finish_bg(struct btrfs_block_group *bg,
struct extent_buffer *eb)
{
if (!bg->seq_zone || eb->start + eb->len * 2 <= bg->start + bg->zone_capacity)
return;
if (WARN_ON(bg->zone_finish_work.func == btrfs_zone_finish_endio_workfn)) {
btrfs_err(bg->fs_info, "double scheduling of bg %llu zone finishing",
bg->start);
return;
}
/* For the work */
btrfs_get_block_group(bg);
atomic_inc(&eb->refs);
bg->last_eb = eb;
INIT_WORK(&bg->zone_finish_work, btrfs_zone_finish_endio_workfn);
queue_work(system_unbound_wq, &bg->zone_finish_work);
}
void btrfs_clear_data_reloc_bg(struct btrfs_block_group *bg)
{
struct btrfs_fs_info *fs_info = bg->fs_info;
@ -2072,3 +2112,30 @@ void btrfs_free_zone_cache(struct btrfs_fs_info *fs_info)
}
mutex_unlock(&fs_devices->device_list_mutex);
}
bool btrfs_zoned_should_reclaim(struct btrfs_fs_info *fs_info)
{
struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
struct btrfs_device *device;
u64 used = 0;
u64 total = 0;
u64 factor;
ASSERT(btrfs_is_zoned(fs_info));
if (fs_info->bg_reclaim_threshold == 0)
return false;
mutex_lock(&fs_devices->device_list_mutex);
list_for_each_entry(device, &fs_devices->devices, dev_list) {
if (!device->bdev)
continue;
total += device->disk_total_bytes;
used += device->bytes_used;
}
mutex_unlock(&fs_devices->device_list_mutex);
factor = div64_u64(used * 100, total);
return factor >= fs_info->bg_reclaim_threshold;
}

23
fs/btrfs/zoned.h
View File

@ -10,11 +10,7 @@
#include "block-group.h"
#include "btrfs_inode.h"
/*
* Block groups with more than this value (percents) of unusable space will be
* scheduled for background reclaim.
*/
#define BTRFS_DEFAULT_RECLAIM_THRESH 75
#define BTRFS_DEFAULT_RECLAIM_THRESH (75)
struct btrfs_zoned_device_info {
/*
@ -76,8 +72,11 @@ int btrfs_zone_finish(struct btrfs_block_group *block_group);
bool btrfs_can_activate_zone(struct btrfs_fs_devices *fs_devices, u64 flags);
void btrfs_zone_finish_endio(struct btrfs_fs_info *fs_info, u64 logical,
u64 length);
void btrfs_schedule_zone_finish_bg(struct btrfs_block_group *bg,
struct extent_buffer *eb);
void btrfs_clear_data_reloc_bg(struct btrfs_block_group *bg);
void btrfs_free_zone_cache(struct btrfs_fs_info *fs_info);
bool btrfs_zoned_should_reclaim(struct btrfs_fs_info *fs_info);
#else /* CONFIG_BLK_DEV_ZONED */
static inline int btrfs_get_dev_zone(struct btrfs_device *device, u64 pos,
struct blk_zone *zone)
@ -233,9 +232,17 @@ static inline bool btrfs_can_activate_zone(struct btrfs_fs_devices *fs_devices,
static inline void btrfs_zone_finish_endio(struct btrfs_fs_info *fs_info,
u64 logical, u64 length) { }
static inline void btrfs_schedule_zone_finish_bg(struct btrfs_block_group *bg,
struct extent_buffer *eb) { }
static inline void btrfs_clear_data_reloc_bg(struct btrfs_block_group *bg) { }
static inline void btrfs_free_zone_cache(struct btrfs_fs_info *fs_info) { }
static inline bool btrfs_zoned_should_reclaim(struct btrfs_fs_info *fs_info)
{
return false;
}
#endif
static inline bool btrfs_dev_is_sequential(struct btrfs_device *device, u64 pos)
@ -370,4 +377,10 @@ static inline void btrfs_zoned_data_reloc_unlock(struct btrfs_inode *inode)
mutex_unlock(&root->fs_info->zoned_data_reloc_io_lock);
}
static inline bool btrfs_zoned_bg_is_full(const struct btrfs_block_group *bg)
{
ASSERT(btrfs_is_zoned(bg->fs_info));
return (bg->alloc_offset == bg->zone_capacity);
}
#endif

14
fs/btrfs/zstd.c
View File

@ -93,22 +93,26 @@ static inline struct workspace *list_to_workspace(struct list_head *list)
void zstd_free_workspace(struct list_head *ws);
struct list_head *zstd_alloc_workspace(unsigned int level);
/*
* zstd_reclaim_timer_fn - reclaim timer
/**
* Timer callback to free unused workspaces.
*
* @t: timer
*
* This scans the lru_list and attempts to reclaim any workspace that hasn't
* been used for ZSTD_BTRFS_RECLAIM_JIFFIES.
*
* The context is softirq and does not need the _bh locking primitives.
*/
static void zstd_reclaim_timer_fn(struct timer_list *timer)
{
unsigned long reclaim_threshold = jiffies - ZSTD_BTRFS_RECLAIM_JIFFIES;
struct list_head *pos, *next;
spin_lock_bh(&wsm.lock);
spin_lock(&wsm.lock);
if (list_empty(&wsm.lru_list)) {
spin_unlock_bh(&wsm.lock);
spin_unlock(&wsm.lock);
return;
}
@ -137,7 +141,7 @@ static void zstd_reclaim_timer_fn(struct timer_list *timer)
if (!list_empty(&wsm.lru_list))
mod_timer(&wsm.timer, jiffies + ZSTD_BTRFS_RECLAIM_JIFFIES);
spin_unlock_bh(&wsm.lock);
spin_unlock(&wsm.lock);
}
/*

12
fs/cachefiles/Kconfig
View File

@ -26,3 +26,15 @@ config CACHEFILES_ERROR_INJECTION
help
This permits error injection to be enabled in cachefiles whilst a
cache is in service.
config CACHEFILES_ONDEMAND
bool "Support for on-demand read"
depends on CACHEFILES
default n
help
This permits userspace to enable the cachefiles on-demand read mode.
In this mode, when a cache miss occurs, responsibility for fetching
the data lies with the cachefiles backend instead of with the netfs
and is delegated to userspace.
If unsure, say N.

1
fs/cachefiles/Makefile
View File

@ -16,5 +16,6 @@ cachefiles-y := \
xattr.o
cachefiles-$(CONFIG_CACHEFILES_ERROR_INJECTION) += error_inject.o
cachefiles-$(CONFIG_CACHEFILES_ONDEMAND) += ondemand.o
obj-$(CONFIG_CACHEFILES) := cachefiles.o

117
fs/cachefiles/daemon.c
View File

@ -75,6 +75,9 @@ static const struct cachefiles_daemon_cmd cachefiles_daemon_cmds[] = {
{ "inuse", cachefiles_daemon_inuse },
{ "secctx", cachefiles_daemon_secctx },
{ "tag", cachefiles_daemon_tag },
#ifdef CONFIG_CACHEFILES_ONDEMAND
{ "copen", cachefiles_ondemand_copen },
#endif
{ "", NULL }
};
@ -108,6 +111,9 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
INIT_LIST_HEAD(&cache->volumes);
INIT_LIST_HEAD(&cache->object_list);
spin_lock_init(&cache->object_list_lock);
refcount_set(&cache->unbind_pincount, 1);
xa_init_flags(&cache->reqs, XA_FLAGS_ALLOC);
xa_init_flags(&cache->ondemand_ids, XA_FLAGS_ALLOC1);
/* set default caching limits
* - limit at 1% free space and/or free files
@ -126,6 +132,53 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
return 0;
}
static void cachefiles_flush_reqs(struct cachefiles_cache *cache)
{
struct xarray *xa = &cache->reqs;
struct cachefiles_req *req;
unsigned long index;
/*
* Make sure the following two operations won't be reordered.
* 1) set CACHEFILES_DEAD bit
* 2) flush requests in the xarray
* Otherwise the request may be enqueued after xarray has been
* flushed, leaving the orphan request never being completed.
*
* CPU 1 CPU 2
* ===== =====
* flush requests in the xarray
* test CACHEFILES_DEAD bit
* enqueue the request
* set CACHEFILES_DEAD bit
*/
smp_mb();
xa_lock(xa);
xa_for_each(xa, index, req) {
req->error = -EIO;
complete(&req->done);
}
xa_unlock(xa);
xa_destroy(&cache->reqs);
xa_destroy(&cache->ondemand_ids);
}
void cachefiles_put_unbind_pincount(struct cachefiles_cache *cache)
{
if (refcount_dec_and_test(&cache->unbind_pincount)) {
cachefiles_daemon_unbind(cache);
cachefiles_open = 0;
kfree(cache);
}
}
void cachefiles_get_unbind_pincount(struct cachefiles_cache *cache)
{
refcount_inc(&cache->unbind_pincount);
}
/*
* Release a cache.
*/
@ -139,36 +192,27 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)
set_bit(CACHEFILES_DEAD, &cache->flags);
cachefiles_daemon_unbind(cache);
if (cachefiles_in_ondemand_mode(cache))
cachefiles_flush_reqs(cache);
/* clean up the control file interface */
cache->cachefilesd = NULL;
file->private_data = NULL;
cachefiles_open = 0;
kfree(cache);
cachefiles_put_unbind_pincount(cache);
_leave("");
return 0;
}
/*
* Read the cache state.
*/
static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
size_t buflen, loff_t *pos)
static ssize_t cachefiles_do_daemon_read(struct cachefiles_cache *cache,
char __user *_buffer, size_t buflen)
{
struct cachefiles_cache *cache = file->private_data;
unsigned long long b_released;
unsigned f_released;
char buffer[256];
int n;
//_enter(",,%zu,", buflen);
if (!test_bit(CACHEFILES_READY, &cache->flags))
return 0;
/* check how much space the cache has */
cachefiles_has_space(cache, 0, 0, cachefiles_has_space_check);
@ -206,6 +250,25 @@ static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
return n;
}
/*
* Read the cache state.
*/
static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
size_t buflen, loff_t *pos)
{
struct cachefiles_cache *cache = file->private_data;
//_enter(",,%zu,", buflen);
if (!test_bit(CACHEFILES_READY, &cache->flags))
return 0;
if (cachefiles_in_ondemand_mode(cache))
return cachefiles_ondemand_daemon_read(cache, _buffer, buflen);
else
return cachefiles_do_daemon_read(cache, _buffer, buflen);
}
/*
* Take a command from cachefilesd, parse it and act on it.
*/
@ -297,8 +360,13 @@ static __poll_t cachefiles_daemon_poll(struct file *file,
poll_wait(file, &cache->daemon_pollwq, poll);
mask = 0;
if (test_bit(CACHEFILES_STATE_CHANGED, &cache->flags))
mask |= EPOLLIN;
if (cachefiles_in_ondemand_mode(cache)) {
if (!xa_empty(&cache->reqs))
mask |= EPOLLIN;
} else {
if (test_bit(CACHEFILES_STATE_CHANGED, &cache->flags))
mask |= EPOLLIN;
}
if (test_bit(CACHEFILES_CULLING, &cache->flags))
mask |= EPOLLOUT;
@ -687,11 +755,6 @@ static int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args)
cache->brun_percent >= 100)
return -ERANGE;
if (*args) {
pr_err("'bind' command doesn't take an argument\n");
return -EINVAL;
}
if (!cache->rootdirname) {
pr_err("No cache directory specified\n");
return -EINVAL;
@ -703,6 +766,18 @@ static int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args)
return -EBUSY;
}
if (IS_ENABLED(CONFIG_CACHEFILES_ONDEMAND)) {
if (!strcmp(args, "ondemand")) {
set_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags);
} else if (*args) {
pr_err("Invalid argument to the 'bind' command\n");
return -EINVAL;
}
} else if (*args) {
pr_err("'bind' command doesn't take an argument\n");
return -EINVAL;
}
/* Make sure we have copies of the tag string */
if (!cache->tag) {
/*

2
fs/cachefiles/interface.c
View File

@ -362,6 +362,8 @@ static void cachefiles_withdraw_cookie(struct fscache_cookie *cookie)
spin_unlock(&cache->object_list_lock);
}
cachefiles_ondemand_clean_object(object);
if (object->file) {
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_clean_up_object(object, cache);

78
fs/cachefiles/internal.h
View File

@ -15,6 +15,8 @@
#include <linux/fscache-cache.h>
#include <linux/cred.h>
#include <linux/security.h>
#include <linux/xarray.h>
#include <linux/cachefiles.h>
#define CACHEFILES_DIO_BLOCK_SIZE 4096
@ -58,8 +60,13 @@ struct cachefiles_object {
enum cachefiles_content content_info:8; /* Info about content presence */
unsigned long flags;
#define CACHEFILES_OBJECT_USING_TMPFILE 0 /* Have an unlinked tmpfile */
#ifdef CONFIG_CACHEFILES_ONDEMAND
int ondemand_id;
#endif
};
#define CACHEFILES_ONDEMAND_ID_CLOSED -1
/*
* Cache files cache definition
*/
@ -98,11 +105,31 @@ struct cachefiles_cache {
#define CACHEFILES_DEAD 1 /* T if cache dead */
#define CACHEFILES_CULLING 2 /* T if cull engaged */
#define CACHEFILES_STATE_CHANGED 3 /* T if state changed (poll trigger) */
#define CACHEFILES_ONDEMAND_MODE 4 /* T if in on-demand read mode */
char *rootdirname; /* name of cache root directory */
char *secctx; /* LSM security context */
char *tag; /* cache binding tag */
refcount_t unbind_pincount;/* refcount to do daemon unbind */
struct xarray reqs; /* xarray of pending on-demand requests */
struct xarray ondemand_ids; /* xarray for ondemand_id allocation */
u32 ondemand_id_next;
};
static inline bool cachefiles_in_ondemand_mode(struct cachefiles_cache *cache)
{
return IS_ENABLED(CONFIG_CACHEFILES_ONDEMAND) &&
test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags);
}
struct cachefiles_req {
struct cachefiles_object *object;
struct completion done;
int error;
struct cachefiles_msg msg;
};
#define CACHEFILES_REQ_NEW XA_MARK_1
#include <trace/events/cachefiles.h>
static inline
@ -145,6 +172,8 @@ extern int cachefiles_has_space(struct cachefiles_cache *cache,
* daemon.c
*/
extern const struct file_operations cachefiles_daemon_fops;
extern void cachefiles_get_unbind_pincount(struct cachefiles_cache *cache);
extern void cachefiles_put_unbind_pincount(struct cachefiles_cache *cache);
/*
* error_inject.c
@ -201,6 +230,16 @@ extern void cachefiles_put_object(struct cachefiles_object *object,
*/
extern bool cachefiles_begin_operation(struct netfs_cache_resources *cres,
enum fscache_want_state want_state);
extern int __cachefiles_prepare_write(struct cachefiles_object *object,
struct file *file,
loff_t *_start, size_t *_len,
bool no_space_allocated_yet);
extern int __cachefiles_write(struct cachefiles_object *object,
struct file *file,
loff_t start_pos,
struct iov_iter *iter,
netfs_io_terminated_t term_func,
void *term_func_priv);
/*
* key.c
@ -240,6 +279,45 @@ extern struct file *cachefiles_create_tmpfile(struct cachefiles_object *object);
extern bool cachefiles_commit_tmpfile(struct cachefiles_cache *cache,
struct cachefiles_object *object);
/*
* ondemand.c
*/
#ifdef CONFIG_CACHEFILES_ONDEMAND
extern ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
char __user *_buffer, size_t buflen);
extern int cachefiles_ondemand_copen(struct cachefiles_cache *cache,
char *args);
extern int cachefiles_ondemand_init_object(struct cachefiles_object *object);
extern void cachefiles_ondemand_clean_object(struct cachefiles_object *object);
extern int cachefiles_ondemand_read(struct cachefiles_object *object,
loff_t pos, size_t len);
#else
static inline ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
char __user *_buffer, size_t buflen)
{
return -EOPNOTSUPP;
}
static inline int cachefiles_ondemand_init_object(struct cachefiles_object *object)
{
return 0;
}
static inline void cachefiles_ondemand_clean_object(struct cachefiles_object *object)
{
}
static inline int cachefiles_ondemand_read(struct cachefiles_object *object,
loff_t pos, size_t len)
{
return -EOPNOTSUPP;
}
#endif
/*
* security.c
*/

76
fs/cachefiles/io.c
View File

@ -277,36 +277,33 @@ static void cachefiles_write_complete(struct kiocb *iocb, long ret)
/*
* Initiate a write to the cache.
*/
static int cachefiles_write(struct netfs_cache_resources *cres,
loff_t start_pos,
struct iov_iter *iter,
netfs_io_terminated_t term_func,
void *term_func_priv)
int __cachefiles_write(struct cachefiles_object *object,
struct file *file,
loff_t start_pos,
struct iov_iter *iter,
netfs_io_terminated_t term_func,
void *term_func_priv)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct cachefiles_kiocb *ki;
struct inode *inode;
struct file *file;
unsigned int old_nofs;
ssize_t ret = -ENOBUFS;
ssize_t ret;
size_t len = iov_iter_count(iter);
if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
goto presubmission_error;
fscache_count_write();
object = cachefiles_cres_object(cres);
cache = object->volume->cache;
file = cachefiles_cres_file(cres);
_enter("%pD,%li,%llx,%zx/%llx",
file, file_inode(file)->i_ino, start_pos, len,
i_size_read(file_inode(file)));
ret = -ENOMEM;
ki = kzalloc(sizeof(struct cachefiles_kiocb), GFP_KERNEL);
if (!ki)
goto presubmission_error;
if (!ki) {
if (term_func)
term_func(term_func_priv, -ENOMEM, false);
return -ENOMEM;
}
refcount_set(&ki->ki_refcnt, 2);
ki->iocb.ki_filp = file;
@ -314,7 +311,6 @@ static int cachefiles_write(struct netfs_cache_resources *cres,
ki->iocb.ki_flags = IOCB_DIRECT | IOCB_WRITE;
ki->iocb.ki_ioprio = get_current_ioprio();
ki->object = object;
ki->inval_counter = cres->inval_counter;
ki->start = start_pos;
ki->len = len;
ki->term_func = term_func;
@ -369,11 +365,24 @@ static int cachefiles_write(struct netfs_cache_resources *cres,
cachefiles_put_kiocb(ki);
_leave(" = %zd", ret);
return ret;
}
presubmission_error:
if (term_func)
term_func(term_func_priv, ret, false);
return ret;
static int cachefiles_write(struct netfs_cache_resources *cres,
loff_t start_pos,
struct iov_iter *iter,
netfs_io_terminated_t term_func,
void *term_func_priv)
{
if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE)) {
if (term_func)
term_func(term_func_priv, -ENOBUFS, false);
return -ENOBUFS;
}
return __cachefiles_write(cachefiles_cres_object(cres),
cachefiles_cres_file(cres),
start_pos, iter,
term_func, term_func_priv);
}
/*
@ -394,6 +403,7 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *
enum netfs_io_source ret = NETFS_DOWNLOAD_FROM_SERVER;
loff_t off, to;
ino_t ino = file ? file_inode(file)->i_ino : 0;
int rc;
_enter("%zx @%llx/%llx", subreq->len, subreq->start, i_size);
@ -406,7 +416,8 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *
if (test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags)) {
__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
why = cachefiles_trace_read_no_data;
goto out_no_object;
if (!test_bit(NETFS_SREQ_ONDEMAND, &subreq->flags))
goto out_no_object;
}
/* The object and the file may be being created in the background. */
@ -423,7 +434,7 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *
object = cachefiles_cres_object(cres);
cache = object->volume->cache;
cachefiles_begin_secure(cache, &saved_cred);
retry:
off = cachefiles_inject_read_error();
if (off == 0)
off = vfs_llseek(file, subreq->start, SEEK_DATA);
@ -474,6 +485,15 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *
download_and_store:
__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
if (test_bit(NETFS_SREQ_ONDEMAND, &subreq->flags)) {
rc = cachefiles_ondemand_read(object, subreq->start,
subreq->len);
if (!rc) {
__clear_bit(NETFS_SREQ_ONDEMAND, &subreq->flags);
goto retry;
}
ret = NETFS_INVALID_READ;
}
out:
cachefiles_end_secure(cache, saved_cred);
out_no_object:
@ -484,13 +504,12 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *
/*
* Prepare for a write to occur.
*/
static int __cachefiles_prepare_write(struct netfs_cache_resources *cres,
loff_t *_start, size_t *_len, loff_t i_size,
bool no_space_allocated_yet)
int __cachefiles_prepare_write(struct cachefiles_object *object,
struct file *file,
loff_t *_start, size_t *_len,
bool no_space_allocated_yet)
{
struct cachefiles_object *object = cachefiles_cres_object(cres);
struct cachefiles_cache *cache = object->volume->cache;
struct file *file = cachefiles_cres_file(cres);
loff_t start = *_start, pos;
size_t len = *_len, down;
int ret;
@ -577,7 +596,8 @@ static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
}
cachefiles_begin_secure(cache, &saved_cred);
ret = __cachefiles_prepare_write(cres, _start, _len, i_size,
ret = __cachefiles_prepare_write(object, cachefiles_cres_file(cres),
_start, _len,
no_space_allocated_yet);
cachefiles_end_secure(cache, saved_cred);
return ret;

16
fs/cachefiles/namei.c
View File

@ -452,10 +452,9 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
struct dentry *fan = volume->fanout[(u8)object->cookie->key_hash];
struct file *file;
struct path path;
uint64_t ni_size = object->cookie->object_size;
uint64_t ni_size;
long ret;
ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
cachefiles_begin_secure(cache, &saved_cred);
@ -481,6 +480,15 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
goto out_dput;
}
ret = cachefiles_ondemand_init_object(object);
if (ret < 0) {
file = ERR_PTR(ret);
goto out_unuse;
}
ni_size = object->cookie->object_size;
ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
if (ni_size > 0) {
trace_cachefiles_trunc(object, d_backing_inode(path.dentry), 0, ni_size,
cachefiles_trunc_expand_tmpfile);
@ -586,6 +594,10 @@ static bool cachefiles_open_file(struct cachefiles_object *object,
}
_debug("file -> %pd positive", dentry);
ret = cachefiles_ondemand_init_object(object);
if (ret < 0)
goto error_fput;
ret = cachefiles_check_auxdata(object, file);
if (ret < 0)
goto check_failed;

503
fs/cachefiles/ondemand.c Normal file
View File

@ -0,0 +1,503 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <linux/fdtable.h>
#include <linux/anon_inodes.h>
#include <linux/uio.h>
#include "internal.h"
static int cachefiles_ondemand_fd_release(struct inode *inode,
struct file *file)
{
struct cachefiles_object *object = file->private_data;
struct cachefiles_cache *cache = object->volume->cache;
int object_id = object->ondemand_id;
struct cachefiles_req *req;
XA_STATE(xas, &cache->reqs, 0);
xa_lock(&cache->reqs);
object->ondemand_id = CACHEFILES_ONDEMAND_ID_CLOSED;
/*
* Flush all pending READ requests since their completion depends on
* anon_fd.
*/
xas_for_each(&xas, req, ULONG_MAX) {
if (req->msg.opcode == CACHEFILES_OP_READ) {
req->error = -EIO;
complete(&req->done);
xas_store(&xas, NULL);
}
}
xa_unlock(&cache->reqs);
xa_erase(&cache->ondemand_ids, object_id);
trace_cachefiles_ondemand_fd_release(object, object_id);
cachefiles_put_object(object, cachefiles_obj_put_ondemand_fd);
cachefiles_put_unbind_pincount(cache);
return 0;
}
static ssize_t cachefiles_ondemand_fd_write_iter(struct kiocb *kiocb,
struct iov_iter *iter)
{
struct cachefiles_object *object = kiocb->ki_filp->private_data;
struct cachefiles_cache *cache = object->volume->cache;
struct file *file = object->file;
size_t len = iter->count;
loff_t pos = kiocb->ki_pos;
const struct cred *saved_cred;
int ret;
if (!file)
return -ENOBUFS;
cachefiles_begin_secure(cache, &saved_cred);
ret = __cachefiles_prepare_write(object, file, &pos, &len, true);
cachefiles_end_secure(cache, saved_cred);
if (ret < 0)
return ret;
trace_cachefiles_ondemand_fd_write(object, file_inode(file), pos, len);
ret = __cachefiles_write(object, file, pos, iter, NULL, NULL);
if (!ret)
ret = len;
return ret;
}
static loff_t cachefiles_ondemand_fd_llseek(struct file *filp, loff_t pos,
int whence)
{
struct cachefiles_object *object = filp->private_data;
struct file *file = object->file;
if (!file)
return -ENOBUFS;
return vfs_llseek(file, pos, whence);
}
static long cachefiles_ondemand_fd_ioctl(struct file *filp, unsigned int ioctl,
unsigned long arg)
{
struct cachefiles_object *object = filp->private_data;
struct cachefiles_cache *cache = object->volume->cache;
struct cachefiles_req *req;
unsigned long id;
if (ioctl != CACHEFILES_IOC_READ_COMPLETE)
return -EINVAL;
if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
return -EOPNOTSUPP;
id = arg;
req = xa_erase(&cache->reqs, id);
if (!req)
return -EINVAL;
trace_cachefiles_ondemand_cread(object, id);
complete(&req->done);
return 0;
}
static const struct file_operations cachefiles_ondemand_fd_fops = {
.owner = THIS_MODULE,
.release = cachefiles_ondemand_fd_release,
.write_iter = cachefiles_ondemand_fd_write_iter,
.llseek = cachefiles_ondemand_fd_llseek,
.unlocked_ioctl = cachefiles_ondemand_fd_ioctl,
};
/*
* OPEN request Completion (copen)
* - command: "copen <id>,<cache_size>"
* <cache_size> indicates the object size if >=0, error code if negative
*/
int cachefiles_ondemand_copen(struct cachefiles_cache *cache, char *args)
{
struct cachefiles_req *req;
struct fscache_cookie *cookie;
char *pid, *psize;
unsigned long id;
long size;
int ret;
if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
return -EOPNOTSUPP;
if (!*args) {
pr_err("Empty id specified\n");
return -EINVAL;
}
pid = args;
psize = strchr(args, ',');
if (!psize) {
pr_err("Cache size is not specified\n");
return -EINVAL;
}
*psize = 0;
psize++;
ret = kstrtoul(pid, 0, &id);
if (ret)
return ret;
req = xa_erase(&cache->reqs, id);
if (!req)
return -EINVAL;
/* fail OPEN request if copen format is invalid */
ret = kstrtol(psize, 0, &size);
if (ret) {
req->error = ret;
goto out;
}
/* fail OPEN request if daemon reports an error */
if (size < 0) {
if (!IS_ERR_VALUE(size))
size = -EINVAL;
req->error = size;
goto out;
}
cookie = req->object->cookie;
cookie->object_size = size;
if (size)
clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
else
set_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
trace_cachefiles_ondemand_copen(req->object, id, size);
out:
complete(&req->done);
return ret;
}
static int cachefiles_ondemand_get_fd(struct cachefiles_req *req)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
struct cachefiles_open *load;
struct file *file;
u32 object_id;
int ret, fd;
object = cachefiles_grab_object(req->object,
cachefiles_obj_get_ondemand_fd);
cache = object->volume->cache;
ret = xa_alloc_cyclic(&cache->ondemand_ids, &object_id, NULL,
XA_LIMIT(1, INT_MAX),
&cache->ondemand_id_next, GFP_KERNEL);
if (ret < 0)
goto err;
fd = get_unused_fd_flags(O_WRONLY);
if (fd < 0) {
ret = fd;
goto err_free_id;
}
file = anon_inode_getfile("[cachefiles]", &cachefiles_ondemand_fd_fops,
object, O_WRONLY);
if (IS_ERR(file)) {
ret = PTR_ERR(file);
goto err_put_fd;
}
file->f_mode |= FMODE_PWRITE | FMODE_LSEEK;
fd_install(fd, file);
load = (void *)req->msg.data;
load->fd = fd;
req->msg.object_id = object_id;
object->ondemand_id = object_id;
cachefiles_get_unbind_pincount(cache);
trace_cachefiles_ondemand_open(object, &req->msg, load);
return 0;
err_put_fd:
put_unused_fd(fd);
err_free_id:
xa_erase(&cache->ondemand_ids, object_id);
err:
cachefiles_put_object(object, cachefiles_obj_put_ondemand_fd);
return ret;
}
ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
char __user *_buffer, size_t buflen)
{
struct cachefiles_req *req;
struct cachefiles_msg *msg;
unsigned long id = 0;
size_t n;
int ret = 0;
XA_STATE(xas, &cache->reqs, 0);
/*
* Search for a request that has not ever been processed, to prevent
* requests from being processed repeatedly.
*/
xa_lock(&cache->reqs);
req = xas_find_marked(&xas, UINT_MAX, CACHEFILES_REQ_NEW);
if (!req) {
xa_unlock(&cache->reqs);
return 0;
}
msg = &req->msg;
n = msg->len;
if (n > buflen) {
xa_unlock(&cache->reqs);
return -EMSGSIZE;
}
xas_clear_mark(&xas, CACHEFILES_REQ_NEW);
xa_unlock(&cache->reqs);
id = xas.xa_index;
msg->msg_id = id;
if (msg->opcode == CACHEFILES_OP_OPEN) {
ret = cachefiles_ondemand_get_fd(req);
if (ret)
goto error;
}
if (copy_to_user(_buffer, msg, n) != 0) {
ret = -EFAULT;
goto err_put_fd;
}
/* CLOSE request has no reply */
if (msg->opcode == CACHEFILES_OP_CLOSE) {
xa_erase(&cache->reqs, id);
complete(&req->done);
}
return n;
err_put_fd:
if (msg->opcode == CACHEFILES_OP_OPEN)
close_fd(((struct cachefiles_open *)msg->data)->fd);
error:
xa_erase(&cache->reqs, id);
req->error = ret;
complete(&req->done);
return ret;
}
typedef int (*init_req_fn)(struct cachefiles_req *req, void *private);
static int cachefiles_ondemand_send_req(struct cachefiles_object *object,
enum cachefiles_opcode opcode,
size_t data_len,
init_req_fn init_req,
void *private)
{
struct cachefiles_cache *cache = object->volume->cache;
struct cachefiles_req *req;
XA_STATE(xas, &cache->reqs, 0);
int ret;
if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
return 0;
if (test_bit(CACHEFILES_DEAD, &cache->flags))
return -EIO;
req = kzalloc(sizeof(*req) + data_len, GFP_KERNEL);
if (!req)
return -ENOMEM;
req->object = object;
init_completion(&req->done);
req->msg.opcode = opcode;
req->msg.len = sizeof(struct cachefiles_msg) + data_len;
ret = init_req(req, private);
if (ret)
goto out;
do {
/*
* Stop enqueuing the request when daemon is dying. The
* following two operations need to be atomic as a whole.
* 1) check cache state, and
* 2) enqueue request if cache is alive.
* Otherwise the request may be enqueued after xarray has been
* flushed, leaving the orphan request never being completed.
*
* CPU 1 CPU 2
* ===== =====
* test CACHEFILES_DEAD bit
* set CACHEFILES_DEAD bit
* flush requests in the xarray
* enqueue the request
*/
xas_lock(&xas);
if (test_bit(CACHEFILES_DEAD, &cache->flags)) {
xas_unlock(&xas);
ret = -EIO;
goto out;
}
/* coupled with the barrier in cachefiles_flush_reqs() */
smp_mb();
if (opcode != CACHEFILES_OP_OPEN && object->ondemand_id <= 0) {
WARN_ON_ONCE(object->ondemand_id == 0);
xas_unlock(&xas);
ret = -EIO;
goto out;
}
xas.xa_index = 0;
xas_find_marked(&xas, UINT_MAX, XA_FREE_MARK);
if (xas.xa_node == XAS_RESTART)
xas_set_err(&xas, -EBUSY);
xas_store(&xas, req);
xas_clear_mark(&xas, XA_FREE_MARK);
xas_set_mark(&xas, CACHEFILES_REQ_NEW);
xas_unlock(&xas);
} while (xas_nomem(&xas, GFP_KERNEL));
ret = xas_error(&xas);
if (ret)
goto out;
wake_up_all(&cache->daemon_pollwq);
wait_for_completion(&req->done);
ret = req->error;
out:
kfree(req);
return ret;
}
static int cachefiles_ondemand_init_open_req(struct cachefiles_req *req,
void *private)
{
struct cachefiles_object *object = req->object;
struct fscache_cookie *cookie = object->cookie;
struct fscache_volume *volume = object->volume->vcookie;
struct cachefiles_open *load = (void *)req->msg.data;
size_t volume_key_size, cookie_key_size;
void *volume_key, *cookie_key;
/*
* Volume key is a NUL-terminated string. key[0] stores strlen() of the
* string, followed by the content of the string (excluding '\0').
*/
volume_key_size = volume->key[0] + 1;
volume_key = volume->key + 1;
/* Cookie key is binary data, which is netfs specific. */
cookie_key_size = cookie->key_len;
cookie_key = fscache_get_key(cookie);
if (!(object->cookie->advice & FSCACHE_ADV_WANT_CACHE_SIZE)) {
pr_err("WANT_CACHE_SIZE is needed for on-demand mode\n");
return -EINVAL;
}
load->volume_key_size = volume_key_size;
load->cookie_key_size = cookie_key_size;
memcpy(load->data, volume_key, volume_key_size);
memcpy(load->data + volume_key_size, cookie_key, cookie_key_size);
return 0;
}
static int cachefiles_ondemand_init_close_req(struct cachefiles_req *req,
void *private)
{
struct cachefiles_object *object = req->object;
int object_id = object->ondemand_id;
/*
* It's possible that object id is still 0 if the cookie looking up
* phase failed before OPEN request has ever been sent. Also avoid
* sending CLOSE request for CACHEFILES_ONDEMAND_ID_CLOSED, which means
* anon_fd has already been closed.
*/
if (object_id <= 0)
return -ENOENT;
req->msg.object_id = object_id;
trace_cachefiles_ondemand_close(object, &req->msg);
return 0;
}
struct cachefiles_read_ctx {
loff_t off;
size_t len;
};
static int cachefiles_ondemand_init_read_req(struct cachefiles_req *req,
void *private)
{
struct cachefiles_object *object = req->object;
struct cachefiles_read *load = (void *)req->msg.data;
struct cachefiles_read_ctx *read_ctx = private;
int object_id = object->ondemand_id;
/* Stop enqueuing requests when daemon has closed anon_fd. */
if (object_id <= 0) {
WARN_ON_ONCE(object_id == 0);
pr_info_once("READ: anonymous fd closed prematurely.\n");
return -EIO;
}
req->msg.object_id = object_id;
load->off = read_ctx->off;
load->len = read_ctx->len;
trace_cachefiles_ondemand_read(object, &req->msg, load);
return 0;
}
int cachefiles_ondemand_init_object(struct cachefiles_object *object)
{
struct fscache_cookie *cookie = object->cookie;
struct fscache_volume *volume = object->volume->vcookie;
size_t volume_key_size, cookie_key_size, data_len;
/*
* CacheFiles will firstly check the cache file under the root cache
* directory. If the coherency check failed, it will fallback to
* creating a new tmpfile as the cache file. Reuse the previously
* allocated object ID if any.
*/
if (object->ondemand_id > 0)
return 0;
volume_key_size = volume->key[0] + 1;
cookie_key_size = cookie->key_len;
data_len = sizeof(struct cachefiles_open) +
volume_key_size + cookie_key_size;
return cachefiles_ondemand_send_req(object, CACHEFILES_OP_OPEN,
data_len, cachefiles_ondemand_init_open_req, NULL);
}
void cachefiles_ondemand_clean_object(struct cachefiles_object *object)
{
cachefiles_ondemand_send_req(object, CACHEFILES_OP_CLOSE, 0,
cachefiles_ondemand_init_close_req, NULL);
}
int cachefiles_ondemand_read(struct cachefiles_object *object,
loff_t pos, size_t len)
{
struct cachefiles_read_ctx read_ctx = {pos, len};
return cachefiles_ondemand_send_req(object, CACHEFILES_OP_READ,
sizeof(struct cachefiles_read),
cachefiles_ondemand_init_read_req, &read_ctx);
}

10
fs/erofs/Kconfig
View File

@ -98,3 +98,13 @@ config EROFS_FS_ZIP_LZMA
systems will be readable without selecting this option.
If unsure, say N.
config EROFS_FS_ONDEMAND
bool "EROFS fscache-based on-demand read support"
depends on CACHEFILES_ONDEMAND && (EROFS_FS=m && FSCACHE || EROFS_FS=y && FSCACHE=y)
default n
help
This permits EROFS to use fscache-backed data blobs with on-demand
read support.
If unsure, say N.

1
fs/erofs/Makefile
View File

@ -5,3 +5,4 @@ erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o sysfs.o
erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o
erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o zdata.o
erofs-$(CONFIG_EROFS_FS_ZIP_LZMA) += decompressor_lzma.o
erofs-$(CONFIG_EROFS_FS_ONDEMAND) += fscache.o

28
fs/erofs/data.c
View File

@ -6,6 +6,7 @@
*/
#include "internal.h"
#include <linux/prefetch.h>
#include <linux/sched/mm.h>
#include <linux/dax.h>
#include <trace/events/erofs.h>
@ -35,14 +36,20 @@ void *erofs_bread(struct erofs_buf *buf, struct inode *inode,
erofs_off_t offset = blknr_to_addr(blkaddr);
pgoff_t index = offset >> PAGE_SHIFT;
struct page *page = buf->page;
struct folio *folio;
unsigned int nofs_flag;
if (!page || page->index != index) {
erofs_put_metabuf(buf);
page = read_cache_page_gfp(mapping, index,
mapping_gfp_constraint(mapping, ~__GFP_FS));
if (IS_ERR(page))
return page;
nofs_flag = memalloc_nofs_save();
folio = read_cache_folio(mapping, index, NULL, NULL);
memalloc_nofs_restore(nofs_flag);
if (IS_ERR(folio))
return folio;
/* should already be PageUptodate, no need to lock page */
page = folio_file_page(folio, index);
buf->page = page;
}
if (buf->kmap_type == EROFS_NO_KMAP) {
@ -63,6 +70,10 @@ void *erofs_bread(struct erofs_buf *buf, struct inode *inode,
void *erofs_read_metabuf(struct erofs_buf *buf, struct super_block *sb,
erofs_blk_t blkaddr, enum erofs_kmap_type type)
{
if (erofs_is_fscache_mode(sb))
return erofs_bread(buf, EROFS_SB(sb)->s_fscache->inode,
blkaddr, type);
return erofs_bread(buf, sb->s_bdev->bd_inode, blkaddr, type);
}
@ -110,8 +121,8 @@ static int erofs_map_blocks_flatmode(struct inode *inode,
return 0;
}
static int erofs_map_blocks(struct inode *inode,
struct erofs_map_blocks *map, int flags)
int erofs_map_blocks(struct inode *inode,
struct erofs_map_blocks *map, int flags)
{
struct super_block *sb = inode->i_sb;
struct erofs_inode *vi = EROFS_I(inode);
@ -199,6 +210,7 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)
map->m_bdev = sb->s_bdev;
map->m_daxdev = EROFS_SB(sb)->dax_dev;
map->m_dax_part_off = EROFS_SB(sb)->dax_part_off;
map->m_fscache = EROFS_SB(sb)->s_fscache;
if (map->m_deviceid) {
down_read(&devs->rwsem);
@ -210,6 +222,7 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)
map->m_bdev = dif->bdev;
map->m_daxdev = dif->dax_dev;
map->m_dax_part_off = dif->dax_part_off;
map->m_fscache = dif->fscache;
up_read(&devs->rwsem);
} else if (devs->extra_devices) {
down_read(&devs->rwsem);
@ -227,6 +240,7 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)
map->m_bdev = dif->bdev;
map->m_daxdev = dif->dax_dev;
map->m_dax_part_off = dif->dax_part_off;
map->m_fscache = dif->fscache;
break;
}
}
@ -385,7 +399,7 @@ static ssize_t erofs_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
if (!err)
return iomap_dio_rw(iocb, to, &erofs_iomap_ops,
NULL, 0, 0);
NULL, 0, NULL, 0);
if (err < 0)
return err;
}

7
fs/erofs/decompressor.c
View File

@ -46,8 +46,6 @@ int z_erofs_load_lz4_config(struct super_block *sb,
erofs_err(sb, "too large lz4 pclusterblks %u",
sbi->lz4.max_pclusterblks);
return -EINVAL;
} else if (sbi->lz4.max_pclusterblks >= 2) {
erofs_info(sb, "EXPERIMENTAL big pcluster feature in use. Use at your own risk!");
}
} else {
distance = le16_to_cpu(dsb->u1.lz4_max_distance);
@ -322,6 +320,7 @@ static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq,
PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
const unsigned int righthalf = min_t(unsigned int, rq->outputsize,
PAGE_SIZE - rq->pageofs_out);
const unsigned int lefthalf = rq->outputsize - righthalf;
unsigned char *src, *dst;
if (nrpages_out > 2) {
@ -344,10 +343,10 @@ static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq,
if (nrpages_out == 2) {
DBG_BUGON(!rq->out[1]);
if (rq->out[1] == *rq->in) {
memmove(src, src + righthalf, rq->pageofs_out);
memmove(src, src + righthalf, lefthalf);
} else {
dst = kmap_atomic(rq->out[1]);
memcpy(dst, src + righthalf, rq->pageofs_out);
memcpy(dst, src + righthalf, lefthalf);
kunmap_atomic(dst);
}
}

50
fs/erofs/erofs_fs.h
View File

@ -37,12 +37,9 @@
#define EROFS_SB_EXTSLOT_SIZE 16
struct erofs_deviceslot {
union {
u8 uuid[16]; /* used for device manager later */
u8 userdata[64]; /* digest(sha256), etc. */
} u;
__le32 blocks; /* total fs blocks of this device */
__le32 mapped_blkaddr; /* map starting at mapped_blkaddr */
u8 tag[64]; /* digest(sha256), etc. */
__le32 blocks; /* total fs blocks of this device */
__le32 mapped_blkaddr; /* map starting at mapped_blkaddr */
u8 reserved[56];
};
#define EROFS_DEVT_SLOT_SIZE sizeof(struct erofs_deviceslot)
@ -58,8 +55,8 @@ struct erofs_super_block {
__le16 root_nid; /* nid of root directory */
__le64 inos; /* total valid ino # (== f_files - f_favail) */
__le64 build_time; /* inode v1 time derivation */
__le32 build_time_nsec; /* inode v1 time derivation in nano scale */
__le64 build_time; /* compact inode time derivation */
__le32 build_time_nsec; /* compact inode time derivation in ns scale */
__le32 blocks; /* used for statfs */
__le32 meta_blkaddr; /* start block address of metadata area */
__le32 xattr_blkaddr; /* start block address of shared xattr area */
@ -79,15 +76,15 @@ struct erofs_super_block {
/*
* erofs inode datalayout (i_format in on-disk inode):
* 0 - inode plain without inline data A:
* 0 - uncompressed flat inode without tail-packing inline data:
* inode, [xattrs], ... | ... | no-holed data
* 1 - inode VLE compression B (legacy):
* inode, [xattrs], extents ... | ...
* 2 - inode plain with inline data C:
* inode, [xattrs], last_inline_data, ... | ... | no-holed data
* 3 - inode compression D:
* 1 - compressed inode with non-compact indexes:
* inode, [xattrs], [map_header], extents ... | ...
* 2 - uncompressed flat inode with tail-packing inline data:
* inode, [xattrs], tailpacking data, ... | ... | no-holed data
* 3 - compressed inode with compact indexes:
* inode, [xattrs], map_header, extents ... | ...
* 4 - inode chunk-based E:
* 4 - chunk-based inode with (optional) multi-device support:
* inode, [xattrs], chunk indexes ... | ...
* 5~7 - reserved
*/
@ -106,7 +103,7 @@ static inline bool erofs_inode_is_data_compressed(unsigned int datamode)
datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY;
}
/* bit definitions of inode i_advise */
/* bit definitions of inode i_format */
#define EROFS_I_VERSION_BITS 1
#define EROFS_I_DATALAYOUT_BITS 3
@ -140,8 +137,9 @@ struct erofs_inode_compact {
__le32 i_size;
__le32 i_reserved;
union {
/* file total compressed blocks for data mapping 1 */
/* total compressed blocks for compressed inodes */
__le32 compressed_blocks;
/* block address for uncompressed flat inodes */
__le32 raw_blkaddr;
/* for device files, used to indicate old/new device # */
@ -156,9 +154,9 @@ struct erofs_inode_compact {
__le32 i_reserved2;
};
/* 32 bytes on-disk inode */
/* 32-byte on-disk inode */
#define EROFS_INODE_LAYOUT_COMPACT 0
/* 64 bytes on-disk inode */
/* 64-byte on-disk inode */
#define EROFS_INODE_LAYOUT_EXTENDED 1
/* 64-byte complete form of an ondisk inode */
@ -171,8 +169,9 @@ struct erofs_inode_extended {
__le16 i_reserved;
__le64 i_size;
union {
/* file total compressed blocks for data mapping 1 */
/* total compressed blocks for compressed inodes */
__le32 compressed_blocks;
/* block address for uncompressed flat inodes */
__le32 raw_blkaddr;
/* for device files, used to indicate old/new device # */
@ -365,17 +364,16 @@ enum {
struct z_erofs_vle_decompressed_index {
__le16 di_advise;
/* where to decompress in the head cluster */
/* where to decompress in the head lcluster */
__le16 di_clusterofs;
union {
/* for the head cluster */
/* for the HEAD lclusters */
__le32 blkaddr;
/*
* for the rest clusters
* eg. for 4k page-sized cluster, maximum 4K*64k = 256M)
* [0] - pointing to the head cluster
* [1] - pointing to the tail cluster
* for the NONHEAD lclusters
* [0] - distance to its HEAD lcluster
* [1] - distance to the next HEAD lcluster
*/
__le16 delta[2];
} di_u;

521
fs/erofs/fscache.c Normal file
View File

@ -0,0 +1,521 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (C) 2022, Alibaba Cloud
*/
#include <linux/fscache.h>
#include "internal.h"
static struct netfs_io_request *erofs_fscache_alloc_request(struct address_space *mapping,
loff_t start, size_t len)
{
struct netfs_io_request *rreq;
rreq = kzalloc(sizeof(struct netfs_io_request), GFP_KERNEL);
if (!rreq)
return ERR_PTR(-ENOMEM);
rreq->start = start;
rreq->len = len;
rreq->mapping = mapping;
INIT_LIST_HEAD(&rreq->subrequests);
refcount_set(&rreq->ref, 1);
return rreq;
}
static void erofs_fscache_put_request(struct netfs_io_request *rreq)
{
if (!refcount_dec_and_test(&rreq->ref))
return;
if (rreq->cache_resources.ops)
rreq->cache_resources.ops->end_operation(&rreq->cache_resources);
kfree(rreq);
}
static void erofs_fscache_put_subrequest(struct netfs_io_subrequest *subreq)
{
if (!refcount_dec_and_test(&subreq->ref))
return;
erofs_fscache_put_request(subreq->rreq);
kfree(subreq);
}
static void erofs_fscache_clear_subrequests(struct netfs_io_request *rreq)
{
struct netfs_io_subrequest *subreq;
while (!list_empty(&rreq->subrequests)) {
subreq = list_first_entry(&rreq->subrequests,
struct netfs_io_subrequest, rreq_link);
list_del(&subreq->rreq_link);
erofs_fscache_put_subrequest(subreq);
}
}
static void erofs_fscache_rreq_unlock_folios(struct netfs_io_request *rreq)
{
struct netfs_io_subrequest *subreq;
struct folio *folio;
unsigned int iopos = 0;
pgoff_t start_page = rreq->start / PAGE_SIZE;
pgoff_t last_page = ((rreq->start + rreq->len) / PAGE_SIZE) - 1;
bool subreq_failed = false;
XA_STATE(xas, &rreq->mapping->i_pages, start_page);
subreq = list_first_entry(&rreq->subrequests,
struct netfs_io_subrequest, rreq_link);
subreq_failed = (subreq->error < 0);
rcu_read_lock();
xas_for_each(&xas, folio, last_page) {
unsigned int pgpos =
(folio_index(folio) - start_page) * PAGE_SIZE;
unsigned int pgend = pgpos + folio_size(folio);
bool pg_failed = false;
for (;;) {
if (!subreq) {
pg_failed = true;
break;
}
pg_failed |= subreq_failed;
if (pgend < iopos + subreq->len)
break;
iopos += subreq->len;
if (!list_is_last(&subreq->rreq_link,
&rreq->subrequests)) {
subreq = list_next_entry(subreq, rreq_link);
subreq_failed = (subreq->error < 0);
} else {
subreq = NULL;
subreq_failed = false;
}
if (pgend == iopos)
break;
}
if (!pg_failed)
folio_mark_uptodate(folio);
folio_unlock(folio);
}
rcu_read_unlock();
}
static void erofs_fscache_rreq_complete(struct netfs_io_request *rreq)
{
erofs_fscache_rreq_unlock_folios(rreq);
erofs_fscache_clear_subrequests(rreq);
erofs_fscache_put_request(rreq);
}
static void erofc_fscache_subreq_complete(void *priv,
ssize_t transferred_or_error, bool was_async)
{
struct netfs_io_subrequest *subreq = priv;
struct netfs_io_request *rreq = subreq->rreq;
if (IS_ERR_VALUE(transferred_or_error))
subreq->error = transferred_or_error;
if (atomic_dec_and_test(&rreq->nr_outstanding))
erofs_fscache_rreq_complete(rreq);
erofs_fscache_put_subrequest(subreq);
}
/*
* Read data from fscache and fill the read data into page cache described by
* @rreq, which shall be both aligned with PAGE_SIZE. @pstart describes
* the start physical address in the cache file.
*/
static int erofs_fscache_read_folios_async(struct fscache_cookie *cookie,
struct netfs_io_request *rreq, loff_t pstart)
{
enum netfs_io_source source;
struct super_block *sb = rreq->mapping->host->i_sb;
struct netfs_io_subrequest *subreq;
struct netfs_cache_resources *cres = &rreq->cache_resources;
struct iov_iter iter;
loff_t start = rreq->start;
size_t len = rreq->len;
size_t done = 0;
int ret;
atomic_set(&rreq->nr_outstanding, 1);
ret = fscache_begin_read_operation(cres, cookie);
if (ret)
goto out;
while (done < len) {
subreq = kzalloc(sizeof(struct netfs_io_subrequest),
GFP_KERNEL);
if (subreq) {
INIT_LIST_HEAD(&subreq->rreq_link);
refcount_set(&subreq->ref, 2);
subreq->rreq = rreq;
refcount_inc(&rreq->ref);
} else {
ret = -ENOMEM;
goto out;
}
subreq->start = pstart + done;
subreq->len = len - done;
subreq->flags = 1 << NETFS_SREQ_ONDEMAND;
list_add_tail(&subreq->rreq_link, &rreq->subrequests);
source = cres->ops->prepare_read(subreq, LLONG_MAX);
if (WARN_ON(subreq->len == 0))
source = NETFS_INVALID_READ;
if (source != NETFS_READ_FROM_CACHE) {
erofs_err(sb, "failed to fscache prepare_read (source %d)",
source);
ret = -EIO;
subreq->error = ret;
erofs_fscache_put_subrequest(subreq);
goto out;
}
atomic_inc(&rreq->nr_outstanding);
iov_iter_xarray(&iter, READ, &rreq->mapping->i_pages,
start + done, subreq->len);
ret = fscache_read(cres, subreq->start, &iter,
NETFS_READ_HOLE_FAIL,
erofc_fscache_subreq_complete, subreq);
if (ret == -EIOCBQUEUED)
ret = 0;
if (ret) {
erofs_err(sb, "failed to fscache_read (ret %d)", ret);
goto out;
}
done += subreq->len;
}
out:
if (atomic_dec_and_test(&rreq->nr_outstanding))
erofs_fscache_rreq_complete(rreq);
return ret;
}
static int erofs_fscache_meta_readpage(struct file *data, struct page *page)
{
int ret;
struct folio *folio = page_folio(page);
struct super_block *sb = folio_mapping(folio)->host->i_sb;
struct netfs_io_request *rreq;
struct erofs_map_dev mdev = {
.m_deviceid = 0,
.m_pa = folio_pos(folio),
};
ret = erofs_map_dev(sb, &mdev);
if (ret)
goto out;
rreq = erofs_fscache_alloc_request(folio_mapping(folio),
folio_pos(folio), folio_size(folio));
if (IS_ERR(rreq))
goto out;
return erofs_fscache_read_folios_async(mdev.m_fscache->cookie,
rreq, mdev.m_pa);
out:
folio_unlock(folio);
return ret;
}
static int erofs_fscache_readpage_inline(struct folio *folio,
struct erofs_map_blocks *map)
{
struct super_block *sb = folio_mapping(folio)->host->i_sb;
struct erofs_buf buf = __EROFS_BUF_INITIALIZER;
erofs_blk_t blknr;
size_t offset, len;
void *src, *dst;
/* For tail packing layout, the offset may be non-zero. */
offset = erofs_blkoff(map->m_pa);
blknr = erofs_blknr(map->m_pa);
len = map->m_llen;
src = erofs_read_metabuf(&buf, sb, blknr, EROFS_KMAP);
if (IS_ERR(src))
return PTR_ERR(src);
dst = kmap_local_folio(folio, 0);
memcpy(dst, src + offset, len);
memset(dst + len, 0, PAGE_SIZE - len);
kunmap_local(dst);
erofs_put_metabuf(&buf);
return 0;
}
static int erofs_fscache_readpage(struct file *file, struct page *page)
{
struct folio *folio = page_folio(page);
struct inode *inode = folio_mapping(folio)->host;
struct super_block *sb = inode->i_sb;
struct erofs_map_blocks map;
struct erofs_map_dev mdev;
struct netfs_io_request *rreq;
erofs_off_t pos;
loff_t pstart;
int ret;
DBG_BUGON(folio_size(folio) != EROFS_BLKSIZ);
pos = folio_pos(folio);
map.m_la = pos;
ret = erofs_map_blocks(inode, &map, EROFS_GET_BLOCKS_RAW);
if (ret)
goto out_unlock;
if (!(map.m_flags & EROFS_MAP_MAPPED)) {
folio_zero_range(folio, 0, folio_size(folio));
goto out_uptodate;
}
if (map.m_flags & EROFS_MAP_META) {
ret = erofs_fscache_readpage_inline(folio, &map);
goto out_uptodate;
}
mdev = (struct erofs_map_dev) {
.m_deviceid = map.m_deviceid,
.m_pa = map.m_pa,
};
ret = erofs_map_dev(sb, &mdev);
if (ret)
goto out_unlock;
rreq = erofs_fscache_alloc_request(folio_mapping(folio),
folio_pos(folio), folio_size(folio));
if (IS_ERR(rreq))
goto out_unlock;
pstart = mdev.m_pa + (pos - map.m_la);
return erofs_fscache_read_folios_async(mdev.m_fscache->cookie,
rreq, pstart);
out_uptodate:
if (!ret)
folio_mark_uptodate(folio);
out_unlock:
folio_unlock(folio);
return ret;
}
static void erofs_fscache_advance_folios(struct readahead_control *rac,
size_t len, bool unlock)
{
while (len) {
struct folio *folio = readahead_folio(rac);
len -= folio_size(folio);
if (unlock) {
folio_mark_uptodate(folio);
folio_unlock(folio);
}
}
}
static void erofs_fscache_readahead(struct readahead_control *rac)
{
struct inode *inode = rac->mapping->host;
struct super_block *sb = inode->i_sb;
size_t len, count, done = 0;
erofs_off_t pos;
loff_t start, offset;
int ret;
if (!readahead_count(rac))
return;
start = readahead_pos(rac);
len = readahead_length(rac);
do {
struct erofs_map_blocks map;
struct erofs_map_dev mdev;
struct netfs_io_request *rreq;
pos = start + done;
map.m_la = pos;
ret = erofs_map_blocks(inode, &map, EROFS_GET_BLOCKS_RAW);
if (ret)
return;
offset = start + done;
count = min_t(size_t, map.m_llen - (pos - map.m_la),
len - done);
if (!(map.m_flags & EROFS_MAP_MAPPED)) {
struct iov_iter iter;
iov_iter_xarray(&iter, READ, &rac->mapping->i_pages,
offset, count);
iov_iter_zero(count, &iter);
erofs_fscache_advance_folios(rac, count, true);
ret = count;
continue;
}
if (map.m_flags & EROFS_MAP_META) {
struct folio *folio = readahead_folio(rac);
ret = erofs_fscache_readpage_inline(folio, &map);
if (!ret) {
folio_mark_uptodate(folio);
ret = folio_size(folio);
}
folio_unlock(folio);
continue;
}
mdev = (struct erofs_map_dev) {
.m_deviceid = map.m_deviceid,
.m_pa = map.m_pa,
};
ret = erofs_map_dev(sb, &mdev);
if (ret)
return;
rreq = erofs_fscache_alloc_request(rac->mapping, offset, count);
if (IS_ERR(rreq))
return;
/*
* Drop the ref of folios here. Unlock them in
* rreq_unlock_folios() when rreq complete.
*/
erofs_fscache_advance_folios(rac, count, false);
ret = erofs_fscache_read_folios_async(mdev.m_fscache->cookie,
rreq, mdev.m_pa + (pos - map.m_la));
if (!ret)
ret = count;
} while (ret > 0 && ((done += ret) < len));
}
static const struct address_space_operations erofs_fscache_meta_aops = {
.readpage = erofs_fscache_meta_readpage,
};
const struct address_space_operations erofs_fscache_access_aops = {
.readpage = erofs_fscache_readpage,
.readahead = erofs_fscache_readahead,
};
int erofs_fscache_register_cookie(struct super_block *sb,
struct erofs_fscache **fscache,
char *name, bool need_inode)
{
struct fscache_volume *volume = EROFS_SB(sb)->volume;
struct erofs_fscache *ctx;
struct fscache_cookie *cookie;
int ret;
ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
if (!ctx)
return -ENOMEM;
cookie = fscache_acquire_cookie(volume, FSCACHE_ADV_WANT_CACHE_SIZE,
name, strlen(name), NULL, 0, 0);
if (!cookie) {
erofs_err(sb, "failed to get cookie for %s", name);
ret = -EINVAL;
goto err;
}
fscache_use_cookie(cookie, false);
ctx->cookie = cookie;
if (need_inode) {
struct inode *const inode = new_inode(sb);
if (!inode) {
erofs_err(sb, "failed to get anon inode for %s", name);
ret = -ENOMEM;
goto err_cookie;
}
set_nlink(inode, 1);
inode->i_size = OFFSET_MAX;
inode->i_mapping->a_ops = &erofs_fscache_meta_aops;
mapping_set_gfp_mask(inode->i_mapping, GFP_NOFS);
ctx->inode = inode;
}
*fscache = ctx;
return 0;
err_cookie:
fscache_unuse_cookie(ctx->cookie, NULL, NULL);
fscache_relinquish_cookie(ctx->cookie, false);
ctx->cookie = NULL;
err:
kfree(ctx);
return ret;
}
void erofs_fscache_unregister_cookie(struct erofs_fscache **fscache)
{
struct erofs_fscache *ctx = *fscache;
if (!ctx)
return;
fscache_unuse_cookie(ctx->cookie, NULL, NULL);
fscache_relinquish_cookie(ctx->cookie, false);
ctx->cookie = NULL;
iput(ctx->inode);
ctx->inode = NULL;
kfree(ctx);
*fscache = NULL;
}
int erofs_fscache_register_fs(struct super_block *sb)
{
struct erofs_sb_info *sbi = EROFS_SB(sb);
struct fscache_volume *volume;
char *name;
int ret = 0;
name = kasprintf(GFP_KERNEL, "erofs,%s", sbi->opt.fsid);
if (!name)
return -ENOMEM;
volume = fscache_acquire_volume(name, NULL, NULL, 0);
if (IS_ERR_OR_NULL(volume)) {
erofs_err(sb, "failed to register volume for %s", name);
ret = volume ? PTR_ERR(volume) : -EOPNOTSUPP;
volume = NULL;
}
sbi->volume = volume;
kfree(name);
return ret;
}
void erofs_fscache_unregister_fs(struct super_block *sb)
{
struct erofs_sb_info *sbi = EROFS_SB(sb);
fscache_relinquish_volume(sbi->volume, NULL, false);
sbi->volume = NULL;
}

11
fs/erofs/inode.c
View File

@ -8,11 +8,6 @@
#include <trace/events/erofs.h>
/*
* if inode is successfully read, return its inode page (or sometimes
* the inode payload page if it's an extended inode) in order to fill
* inline data if possible.
*/
static void *erofs_read_inode(struct erofs_buf *buf,
struct inode *inode, unsigned int *ofs)
{
@ -297,6 +292,10 @@ static int erofs_fill_inode(struct inode *inode, int isdir)
goto out_unlock;
}
inode->i_mapping->a_ops = &erofs_raw_access_aops;
#ifdef CONFIG_EROFS_FS_ONDEMAND
if (erofs_is_fscache_mode(inode->i_sb))
inode->i_mapping->a_ops = &erofs_fscache_access_aops;
#endif
out_unlock:
erofs_put_metabuf(&buf);
@ -370,7 +369,7 @@ int erofs_getattr(struct user_namespace *mnt_userns, const struct path *path,
stat->attributes_mask |= (STATX_ATTR_COMPRESSED |
STATX_ATTR_IMMUTABLE);
generic_fillattr(&init_user_ns, inode, stat);
generic_fillattr(mnt_userns, inode, stat);
return 0;
}

76
fs/erofs/internal.h
View File

@ -49,6 +49,7 @@ typedef u32 erofs_blk_t;
struct erofs_device_info {
char *path;
struct erofs_fscache *fscache;
struct block_device *bdev;
struct dax_device *dax_dev;
u64 dax_part_off;
@ -74,6 +75,7 @@ struct erofs_mount_opts {
unsigned int max_sync_decompress_pages;
#endif
unsigned int mount_opt;
char *fsid;
};
struct erofs_dev_context {
@ -96,6 +98,11 @@ struct erofs_sb_lz4_info {
u16 max_pclusterblks;
};
struct erofs_fscache {
struct fscache_cookie *cookie;
struct inode *inode;
};
struct erofs_sb_info {
struct erofs_mount_opts opt; /* options */
#ifdef CONFIG_EROFS_FS_ZIP
@ -146,6 +153,10 @@ struct erofs_sb_info {
/* sysfs support */
struct kobject s_kobj; /* /sys/fs/erofs/<devname> */
struct completion s_kobj_unregister;
/* fscache support */
struct fscache_volume *volume;
struct erofs_fscache *s_fscache;
};
#define EROFS_SB(sb) ((struct erofs_sb_info *)(sb)->s_fs_info)
@ -161,6 +172,11 @@ struct erofs_sb_info {
#define set_opt(opt, option) ((opt)->mount_opt |= EROFS_MOUNT_##option)
#define test_opt(opt, option) ((opt)->mount_opt & EROFS_MOUNT_##option)
static inline bool erofs_is_fscache_mode(struct super_block *sb)
{
return IS_ENABLED(CONFIG_EROFS_FS_ONDEMAND) && !sb->s_bdev;
}
enum {
EROFS_ZIP_CACHE_DISABLED,
EROFS_ZIP_CACHE_READAHEAD,
@ -381,31 +397,6 @@ extern const struct super_operations erofs_sops;
extern const struct address_space_operations erofs_raw_access_aops;
extern const struct address_space_operations z_erofs_aops;
/*
* Logical to physical block mapping
*
* Different with other file systems, it is used for 2 access modes:
*
* 1) RAW access mode:
*
* Users pass a valid (m_lblk, m_lofs -- usually 0) pair,
* and get the valid m_pblk, m_pofs and the longest m_len(in bytes).
*
* Note that m_lblk in the RAW access mode refers to the number of
* the compressed ondisk block rather than the uncompressed
* in-memory block for the compressed file.
*
* m_pofs equals to m_lofs except for the inline data page.
*
* 2) Normal access mode:
*
* If the inode is not compressed, it has no difference with
* the RAW access mode. However, if the inode is compressed,
* users should pass a valid (m_lblk, m_lofs) pair, and get
* the needed m_pblk, m_pofs, m_len to get the compressed data
* and the updated m_lblk, m_lofs which indicates the start
* of the corresponding uncompressed data in the file.
*/
enum {
BH_Encoded = BH_PrivateStart,
BH_FullMapped,
@ -467,6 +458,7 @@ static inline int z_erofs_map_blocks_iter(struct inode *inode,
#endif /* !CONFIG_EROFS_FS_ZIP */
struct erofs_map_dev {
struct erofs_fscache *m_fscache;
struct block_device *m_bdev;
struct dax_device *m_daxdev;
u64 m_dax_part_off;
@ -486,6 +478,8 @@ void *erofs_read_metabuf(struct erofs_buf *buf, struct super_block *sb,
int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *dev);
int erofs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
u64 start, u64 len);
int erofs_map_blocks(struct inode *inode,
struct erofs_map_blocks *map, int flags);
/* inode.c */
static inline unsigned long erofs_inode_hash(erofs_nid_t nid)
@ -509,7 +503,7 @@ int erofs_getattr(struct user_namespace *mnt_userns, const struct path *path,
/* namei.c */
extern const struct inode_operations erofs_dir_iops;
int erofs_namei(struct inode *dir, struct qstr *name,
int erofs_namei(struct inode *dir, const struct qstr *name,
erofs_nid_t *nid, unsigned int *d_type);
/* dir.c */
@ -611,6 +605,36 @@ static inline int z_erofs_load_lzma_config(struct super_block *sb,
}
#endif /* !CONFIG_EROFS_FS_ZIP */
/* fscache.c */
#ifdef CONFIG_EROFS_FS_ONDEMAND
int erofs_fscache_register_fs(struct super_block *sb);
void erofs_fscache_unregister_fs(struct super_block *sb);
int erofs_fscache_register_cookie(struct super_block *sb,
struct erofs_fscache **fscache,
char *name, bool need_inode);
void erofs_fscache_unregister_cookie(struct erofs_fscache **fscache);
extern const struct address_space_operations erofs_fscache_access_aops;
#else
static inline int erofs_fscache_register_fs(struct super_block *sb)
{
return 0;
}
static inline void erofs_fscache_unregister_fs(struct super_block *sb) {}
static inline int erofs_fscache_register_cookie(struct super_block *sb,
struct erofs_fscache **fscache,
char *name, bool need_inode)
{
return -EOPNOTSUPP;
}
static inline void erofs_fscache_unregister_cookie(struct erofs_fscache **fscache)
{
}
#endif
#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
#endif /* __EROFS_INTERNAL_H */

5
fs/erofs/namei.c
View File

@ -165,9 +165,8 @@ static void *find_target_block_classic(struct erofs_buf *target,
return candidate;
}
int erofs_namei(struct inode *dir,
struct qstr *name,
erofs_nid_t *nid, unsigned int *d_type)
int erofs_namei(struct inode *dir, const struct qstr *name, erofs_nid_t *nid,
unsigned int *d_type)
{
int ndirents;
struct erofs_buf buf = __EROFS_BUF_INITIALIZER;

219
fs/erofs/super.c
View File

@ -13,6 +13,7 @@
#include <linux/fs_context.h>
#include <linux/fs_parser.h>
#include <linux/dax.h>
#include <linux/exportfs.h>
#include "xattr.h"
#define CREATE_TRACE_POINTS
@ -219,7 +220,52 @@ static int erofs_load_compr_cfgs(struct super_block *sb,
}
#endif
static int erofs_init_devices(struct super_block *sb,
static int erofs_init_device(struct erofs_buf *buf, struct super_block *sb,
struct erofs_device_info *dif, erofs_off_t *pos)
{
struct erofs_sb_info *sbi = EROFS_SB(sb);
struct erofs_deviceslot *dis;
struct block_device *bdev;
void *ptr;
int ret;
ptr = erofs_read_metabuf(buf, sb, erofs_blknr(*pos), EROFS_KMAP);
if (IS_ERR(ptr))
return PTR_ERR(ptr);
dis = ptr + erofs_blkoff(*pos);
if (!dif->path) {
if (!dis->tag[0]) {
erofs_err(sb, "empty device tag @ pos %llu", *pos);
return -EINVAL;
}
dif->path = kmemdup_nul(dis->tag, sizeof(dis->tag), GFP_KERNEL);
if (!dif->path)
return -ENOMEM;
}
if (erofs_is_fscache_mode(sb)) {
ret = erofs_fscache_register_cookie(sb, &dif->fscache,
dif->path, false);
if (ret)
return ret;
} else {
bdev = blkdev_get_by_path(dif->path, FMODE_READ | FMODE_EXCL,
sb->s_type);
if (IS_ERR(bdev))
return PTR_ERR(bdev);
dif->bdev = bdev;
dif->dax_dev = fs_dax_get_by_bdev(bdev, &dif->dax_part_off);
}
dif->blocks = le32_to_cpu(dis->blocks);
dif->mapped_blkaddr = le32_to_cpu(dis->mapped_blkaddr);
sbi->total_blocks += dif->blocks;
*pos += EROFS_DEVT_SLOT_SIZE;
return 0;
}
static int erofs_scan_devices(struct super_block *sb,
struct erofs_super_block *dsb)
{
struct erofs_sb_info *sbi = EROFS_SB(sb);
@ -227,8 +273,6 @@ static int erofs_init_devices(struct super_block *sb,
erofs_off_t pos;
struct erofs_buf buf = __EROFS_BUF_INITIALIZER;
struct erofs_device_info *dif;
struct erofs_deviceslot *dis;
void *ptr;
int id, err = 0;
sbi->total_blocks = sbi->primarydevice_blocks;
@ -237,7 +281,8 @@ static int erofs_init_devices(struct super_block *sb,
else
ondisk_extradevs = le16_to_cpu(dsb->extra_devices);
if (ondisk_extradevs != sbi->devs->extra_devices) {
if (sbi->devs->extra_devices &&
ondisk_extradevs != sbi->devs->extra_devices) {
erofs_err(sb, "extra devices don't match (ondisk %u, given %u)",
ondisk_extradevs, sbi->devs->extra_devices);
return -EINVAL;
@ -248,30 +293,31 @@ static int erofs_init_devices(struct super_block *sb,
sbi->device_id_mask = roundup_pow_of_two(ondisk_extradevs + 1) - 1;
pos = le16_to_cpu(dsb->devt_slotoff) * EROFS_DEVT_SLOT_SIZE;
down_read(&sbi->devs->rwsem);
idr_for_each_entry(&sbi->devs->tree, dif, id) {
struct block_device *bdev;
ptr = erofs_read_metabuf(&buf, sb, erofs_blknr(pos),
EROFS_KMAP);
if (IS_ERR(ptr)) {
err = PTR_ERR(ptr);
break;
if (sbi->devs->extra_devices) {
idr_for_each_entry(&sbi->devs->tree, dif, id) {
err = erofs_init_device(&buf, sb, dif, &pos);
if (err)
break;
}
dis = ptr + erofs_blkoff(pos);
} else {
for (id = 0; id < ondisk_extradevs; id++) {
dif = kzalloc(sizeof(*dif), GFP_KERNEL);
if (!dif) {
err = -ENOMEM;
break;
}
bdev = blkdev_get_by_path(dif->path,
FMODE_READ | FMODE_EXCL,
sb->s_type);
if (IS_ERR(bdev)) {
err = PTR_ERR(bdev);
break;
err = idr_alloc(&sbi->devs->tree, dif, 0, 0, GFP_KERNEL);
if (err < 0) {
kfree(dif);
break;
}
++sbi->devs->extra_devices;
err = erofs_init_device(&buf, sb, dif, &pos);
if (err)
break;
}
dif->bdev = bdev;
dif->dax_dev = fs_dax_get_by_bdev(bdev, &dif->dax_part_off);
dif->blocks = le32_to_cpu(dis->blocks);
dif->mapped_blkaddr = le32_to_cpu(dis->mapped_blkaddr);
sbi->total_blocks += dif->blocks;
pos += EROFS_DEVT_SLOT_SIZE;
}
up_read(&sbi->devs->rwsem);
erofs_put_metabuf(&buf);
@ -358,10 +404,12 @@ static int erofs_read_superblock(struct super_block *sb)
goto out;
/* handle multiple devices */
ret = erofs_init_devices(sb, dsb);
ret = erofs_scan_devices(sb, dsb);
if (erofs_sb_has_ztailpacking(sbi))
erofs_info(sb, "EXPERIMENTAL compressed inline data feature in use. Use at your own risk!");
if (erofs_is_fscache_mode(sb))
erofs_info(sb, "EXPERIMENTAL fscache-based on-demand read feature in use. Use at your own risk!");
out:
erofs_put_metabuf(&buf);
return ret;
@ -390,6 +438,7 @@ enum {
Opt_dax,
Opt_dax_enum,
Opt_device,
Opt_fsid,
Opt_err
};
@ -414,6 +463,7 @@ static const struct fs_parameter_spec erofs_fs_parameters[] = {
fsparam_flag("dax", Opt_dax),
fsparam_enum("dax", Opt_dax_enum, erofs_dax_param_enums),
fsparam_string("device", Opt_device),
fsparam_string("fsid", Opt_fsid),
{}
};
@ -509,6 +559,16 @@ static int erofs_fc_parse_param(struct fs_context *fc,
}
++ctx->devs->extra_devices;
break;
case Opt_fsid:
#ifdef CONFIG_EROFS_FS_ONDEMAND
kfree(ctx->opt.fsid);
ctx->opt.fsid = kstrdup(param->string, GFP_KERNEL);
if (!ctx->opt.fsid)
return -ENOMEM;
#else
errorfc(fc, "fsid option not supported");
#endif
break;
default:
return -ENOPARAM;
}
@ -577,6 +637,44 @@ static int erofs_init_managed_cache(struct super_block *sb)
static int erofs_init_managed_cache(struct super_block *sb) { return 0; }
#endif
static struct inode *erofs_nfs_get_inode(struct super_block *sb,
u64 ino, u32 generation)
{
return erofs_iget(sb, ino, false);
}
static struct dentry *erofs_fh_to_dentry(struct super_block *sb,
struct fid *fid, int fh_len, int fh_type)
{
return generic_fh_to_dentry(sb, fid, fh_len, fh_type,
erofs_nfs_get_inode);
}
static struct dentry *erofs_fh_to_parent(struct super_block *sb,
struct fid *fid, int fh_len, int fh_type)
{
return generic_fh_to_parent(sb, fid, fh_len, fh_type,
erofs_nfs_get_inode);
}
static struct dentry *erofs_get_parent(struct dentry *child)
{
erofs_nid_t nid;
unsigned int d_type;
int err;
err = erofs_namei(d_inode(child), &dotdot_name, &nid, &d_type);
if (err)
return ERR_PTR(err);
return d_obtain_alias(erofs_iget(child->d_sb, nid, d_type == FT_DIR));
}
static const struct export_operations erofs_export_ops = {
.fh_to_dentry = erofs_fh_to_dentry,
.fh_to_parent = erofs_fh_to_parent,
.get_parent = erofs_get_parent,
};
static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct inode *inode;
@ -585,11 +683,9 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)
int err;
sb->s_magic = EROFS_SUPER_MAGIC;
if (!sb_set_blocksize(sb, EROFS_BLKSIZ)) {
erofs_err(sb, "failed to set erofs blksize");
return -EINVAL;
}
sb->s_flags |= SB_RDONLY | SB_NOATIME;
sb->s_maxbytes = MAX_LFS_FILESIZE;
sb->s_op = &erofs_sops;
sbi = kzalloc(sizeof(*sbi), GFP_KERNEL);
if (!sbi)
@ -597,10 +693,36 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)
sb->s_fs_info = sbi;
sbi->opt = ctx->opt;
sbi->dax_dev = fs_dax_get_by_bdev(sb->s_bdev, &sbi->dax_part_off);
ctx->opt.fsid = NULL;
sbi->devs = ctx->devs;
ctx->devs = NULL;
if (erofs_is_fscache_mode(sb)) {
sb->s_blocksize = EROFS_BLKSIZ;
sb->s_blocksize_bits = LOG_BLOCK_SIZE;
err = erofs_fscache_register_fs(sb);
if (err)
return err;
err = erofs_fscache_register_cookie(sb, &sbi->s_fscache,
sbi->opt.fsid, true);
if (err)
return err;
err = super_setup_bdi(sb);
if (err)
return err;
} else {
if (!sb_set_blocksize(sb, EROFS_BLKSIZ)) {
erofs_err(sb, "failed to set erofs blksize");
return -EINVAL;
}
sbi->dax_dev = fs_dax_get_by_bdev(sb->s_bdev,
&sbi->dax_part_off);
}
err = erofs_read_superblock(sb);
if (err)
return err;
@ -613,12 +735,10 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)
clear_opt(&sbi->opt, DAX_ALWAYS);
}
}
sb->s_flags |= SB_RDONLY | SB_NOATIME;
sb->s_maxbytes = MAX_LFS_FILESIZE;
sb->s_time_gran = 1;
sb->s_op = &erofs_sops;
sb->s_time_gran = 1;
sb->s_xattr = erofs_xattr_handlers;
sb->s_export_op = &erofs_export_ops;
if (test_opt(&sbi->opt, POSIX_ACL))
sb->s_flags |= SB_POSIXACL;
@ -661,6 +781,11 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)
static int erofs_fc_get_tree(struct fs_context *fc)
{
struct erofs_fs_context *ctx = fc->fs_private;
if (IS_ENABLED(CONFIG_EROFS_FS_ONDEMAND) && ctx->opt.fsid)
return get_tree_nodev(fc, erofs_fc_fill_super);
return get_tree_bdev(fc, erofs_fc_fill_super);
}
@ -690,6 +815,7 @@ static int erofs_release_device_info(int id, void *ptr, void *data)
fs_put_dax(dif->dax_dev);
if (dif->bdev)
blkdev_put(dif->bdev, FMODE_READ | FMODE_EXCL);
erofs_fscache_unregister_cookie(&dif->fscache);
kfree(dif->path);
kfree(dif);
return 0;
@ -709,6 +835,7 @@ static void erofs_fc_free(struct fs_context *fc)
struct erofs_fs_context *ctx = fc->fs_private;
erofs_free_dev_context(ctx->devs);
kfree(ctx->opt.fsid);
kfree(ctx);
}
@ -749,7 +876,10 @@ static void erofs_kill_sb(struct super_block *sb)
WARN_ON(sb->s_magic != EROFS_SUPER_MAGIC);
kill_block_super(sb);
if (erofs_is_fscache_mode(sb))
generic_shutdown_super(sb);
else
kill_block_super(sb);
sbi = EROFS_SB(sb);
if (!sbi)
@ -757,6 +887,9 @@ static void erofs_kill_sb(struct super_block *sb)
erofs_free_dev_context(sbi->devs);
fs_put_dax(sbi->dax_dev);
erofs_fscache_unregister_cookie(&sbi->s_fscache);
erofs_fscache_unregister_fs(sb);
kfree(sbi->opt.fsid);
kfree(sbi);
sb->s_fs_info = NULL;
}
@ -774,6 +907,7 @@ static void erofs_put_super(struct super_block *sb)
iput(sbi->managed_cache);
sbi->managed_cache = NULL;
#endif
erofs_fscache_unregister_cookie(&sbi->s_fscache);
}
static struct file_system_type erofs_fs_type = {
@ -781,7 +915,7 @@ static struct file_system_type erofs_fs_type = {
.name = "erofs",
.init_fs_context = erofs_init_fs_context,
.kill_sb = erofs_kill_sb,
.fs_flags = FS_REQUIRES_DEV,
.fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP,
};
MODULE_ALIAS_FS("erofs");
@ -857,7 +991,10 @@ static int erofs_statfs(struct dentry *dentry, struct kstatfs *buf)
{
struct super_block *sb = dentry->d_sb;
struct erofs_sb_info *sbi = EROFS_SB(sb);
u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
u64 id = 0;
if (!erofs_is_fscache_mode(sb))
id = huge_encode_dev(sb->s_bdev->bd_dev);
buf->f_type = sb->s_magic;
buf->f_bsize = EROFS_BLKSIZ;
@ -902,6 +1039,10 @@ static int erofs_show_options(struct seq_file *seq, struct dentry *root)
seq_puts(seq, ",dax=always");
if (test_opt(opt, DAX_NEVER))
seq_puts(seq, ",dax=never");
#ifdef CONFIG_EROFS_FS_ONDEMAND
if (opt->fsid)
seq_printf(seq, ",fsid=%s", opt->fsid);
#endif
return 0;
}

4
fs/erofs/sysfs.c
View File

@ -205,8 +205,8 @@ int erofs_register_sysfs(struct super_block *sb)
sbi->s_kobj.kset = &erofs_root;
init_completion(&sbi->s_kobj_unregister);
err = kobject_init_and_add(&sbi->s_kobj, &erofs_sb_ktype, NULL,
"%s", sb->s_id);
err = kobject_init_and_add(&sbi->s_kobj, &erofs_sb_ktype, NULL, "%s",
erofs_is_fscache_mode(sb) ? sbi->opt.fsid : sb->s_id);
if (err)
goto put_sb_kobj;
return 0;

8
fs/exfat/balloc.c
View File

@ -148,7 +148,9 @@ int exfat_set_bitmap(struct inode *inode, unsigned int clu, bool sync)
struct super_block *sb = inode->i_sb;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
WARN_ON(clu < EXFAT_FIRST_CLUSTER);
if (!is_valid_cluster(sbi, clu))
return -EINVAL;
ent_idx = CLUSTER_TO_BITMAP_ENT(clu);
i = BITMAP_OFFSET_SECTOR_INDEX(sb, ent_idx);
b = BITMAP_OFFSET_BIT_IN_SECTOR(sb, ent_idx);
@ -166,7 +168,9 @@ void exfat_clear_bitmap(struct inode *inode, unsigned int clu, bool sync)
struct exfat_sb_info *sbi = EXFAT_SB(sb);
struct exfat_mount_options *opts = &sbi->options;
WARN_ON(clu < EXFAT_FIRST_CLUSTER);
if (!is_valid_cluster(sbi, clu))
return;
ent_idx = CLUSTER_TO_BITMAP_ENT(clu);
i = BITMAP_OFFSET_SECTOR_INDEX(sb, ent_idx);
b = BITMAP_OFFSET_BIT_IN_SECTOR(sb, ent_idx);

7
fs/exfat/exfat_fs.h
View File

@ -203,6 +203,7 @@ struct exfat_mount_options {
/* on error: continue, panic, remount-ro */
enum exfat_error_mode errors;
unsigned utf8:1, /* Use of UTF-8 character set */
sys_tz:1, /* Use local timezone */
discard:1, /* Issue discard requests on deletions */
keep_last_dots:1; /* Keep trailing periods in paths */
int time_offset; /* Offset of timestamps from UTC (in minutes) */
@ -381,6 +382,12 @@ static inline int exfat_sector_to_cluster(struct exfat_sb_info *sbi,
EXFAT_RESERVED_CLUSTERS;
}
static inline bool is_valid_cluster(struct exfat_sb_info *sbi,
unsigned int clus)
{
return clus >= EXFAT_FIRST_CLUSTER && clus < sbi->num_clusters;
}
/* super.c */
int exfat_set_volume_dirty(struct super_block *sb);
int exfat_clear_volume_dirty(struct super_block *sb);

47
fs/exfat/fatent.c
View File

@ -6,6 +6,7 @@
#include <linux/slab.h>
#include <asm/unaligned.h>
#include <linux/buffer_head.h>
#include <linux/blkdev.h>
#include "exfat_raw.h"
#include "exfat_fs.h"
@ -81,12 +82,6 @@ int exfat_ent_set(struct super_block *sb, unsigned int loc,
return 0;
}
static inline bool is_valid_cluster(struct exfat_sb_info *sbi,
unsigned int clus)
{
return clus >= EXFAT_FIRST_CLUSTER && clus < sbi->num_clusters;
}
int exfat_ent_get(struct super_block *sb, unsigned int loc,
unsigned int *content)
{
@ -274,10 +269,9 @@ int exfat_zeroed_cluster(struct inode *dir, unsigned int clu)
{
struct super_block *sb = dir->i_sb;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
struct buffer_head *bhs[MAX_BUF_PER_PAGE];
int nr_bhs = MAX_BUF_PER_PAGE;
struct buffer_head *bh;
sector_t blknr, last_blknr;
int err, i, n;
int i;
blknr = exfat_cluster_to_sector(sbi, clu);
last_blknr = blknr + sbi->sect_per_clus;
@ -291,30 +285,23 @@ int exfat_zeroed_cluster(struct inode *dir, unsigned int clu)
}
/* Zeroing the unused blocks on this cluster */
while (blknr < last_blknr) {
for (n = 0; n < nr_bhs && blknr < last_blknr; n++, blknr++) {
bhs[n] = sb_getblk(sb, blknr);
if (!bhs[n]) {
err = -ENOMEM;
goto release_bhs;
}
memset(bhs[n]->b_data, 0, sb->s_blocksize);
}
for (i = blknr; i < last_blknr; i++) {
bh = sb_getblk(sb, i);
if (!bh)
return -ENOMEM;
err = exfat_update_bhs(bhs, n, IS_DIRSYNC(dir));
if (err)
goto release_bhs;
for (i = 0; i < n; i++)
brelse(bhs[i]);
memset(bh->b_data, 0, sb->s_blocksize);
set_buffer_uptodate(bh);
mark_buffer_dirty(bh);
brelse(bh);
}
return 0;
release_bhs:
exfat_err(sb, "failed zeroed sect %llu\n", (unsigned long long)blknr);
for (i = 0; i < n; i++)
bforget(bhs[i]);
return err;
if (IS_DIRSYNC(dir))
return sync_blockdev_range(sb->s_bdev,
EXFAT_BLK_TO_B(blknr, sb),
EXFAT_BLK_TO_B(last_blknr, sb) - 1);
return 0;
}
int exfat_alloc_cluster(struct inode *inode, unsigned int num_alloc,

10
fs/exfat/misc.c
View File

@ -74,6 +74,13 @@ static void exfat_adjust_tz(struct timespec64 *ts, u8 tz_off)
ts->tv_sec += TIMEZONE_SEC(0x80 - tz_off);
}
static inline int exfat_tz_offset(struct exfat_sb_info *sbi)
{
if (sbi->options.sys_tz)
return -sys_tz.tz_minuteswest;
return sbi->options.time_offset;
}
/* Convert a EXFAT time/date pair to a UNIX date (seconds since 1 1 70). */
void exfat_get_entry_time(struct exfat_sb_info *sbi, struct timespec64 *ts,
u8 tz, __le16 time, __le16 date, u8 time_cs)
@ -96,8 +103,7 @@ void exfat_get_entry_time(struct exfat_sb_info *sbi, struct timespec64 *ts,
/* Adjust timezone to UTC0. */
exfat_adjust_tz(ts, tz & ~EXFAT_TZ_VALID);
else
/* Convert from local time to UTC using time_offset. */
ts->tv_sec -= sbi->options.time_offset * SECS_PER_MIN;
ts->tv_sec -= exfat_tz_offset(sbi) * SECS_PER_MIN;
}
/* Convert linear UNIX date to a EXFAT time/date pair. */

27
fs/exfat/namei.c
View File

@ -1080,6 +1080,7 @@ static int exfat_rename_file(struct inode *inode, struct exfat_chain *p_dir,
exfat_remove_entries(inode, p_dir, oldentry, 0,
num_old_entries);
ei->dir = *p_dir;
ei->entry = newentry;
} else {
if (exfat_get_entry_type(epold) == TYPE_FILE) {
@ -1167,28 +1168,6 @@ static int exfat_move_file(struct inode *inode, struct exfat_chain *p_olddir,
return 0;
}
static void exfat_update_parent_info(struct exfat_inode_info *ei,
struct inode *parent_inode)
{
struct exfat_sb_info *sbi = EXFAT_SB(parent_inode->i_sb);
struct exfat_inode_info *parent_ei = EXFAT_I(parent_inode);
loff_t parent_isize = i_size_read(parent_inode);
/*
* the problem that struct exfat_inode_info caches wrong parent info.
*
* because of flag-mismatch of ei->dir,
* there is abnormal traversing cluster chain.
*/
if (unlikely(parent_ei->flags != ei->dir.flags ||
parent_isize != EXFAT_CLU_TO_B(ei->dir.size, sbi) ||
parent_ei->start_clu != ei->dir.dir)) {
exfat_chain_set(&ei->dir, parent_ei->start_clu,
EXFAT_B_TO_CLU_ROUND_UP(parent_isize, sbi),
parent_ei->flags);
}
}
/* rename or move a old file into a new file */
static int __exfat_rename(struct inode *old_parent_inode,
struct exfat_inode_info *ei, struct inode *new_parent_inode,
@ -1219,8 +1198,6 @@ static int __exfat_rename(struct inode *old_parent_inode,
return -ENOENT;
}
exfat_update_parent_info(ei, old_parent_inode);
exfat_chain_dup(&olddir, &ei->dir);
dentry = ei->entry;
@ -1241,8 +1218,6 @@ static int __exfat_rename(struct inode *old_parent_inode,
goto out;
}
exfat_update_parent_info(new_ei, new_parent_inode);
p_dir = &(new_ei->dir);
new_entry = new_ei->entry;
ep = exfat_get_dentry(sb, p_dir, new_entry, &new_bh);

9
fs/exfat/super.c
View File

@ -170,7 +170,9 @@ static int exfat_show_options(struct seq_file *m, struct dentry *root)
seq_puts(m, ",discard");
if (opts->keep_last_dots)
seq_puts(m, ",keep_last_dots");
if (opts->time_offset)
if (opts->sys_tz)
seq_puts(m, ",sys_tz");
else if (opts->time_offset)
seq_printf(m, ",time_offset=%d", opts->time_offset);
return 0;
}
@ -214,6 +216,7 @@ enum {
Opt_errors,
Opt_discard,
Opt_keep_last_dots,
Opt_sys_tz,
Opt_time_offset,
/* Deprecated options */
@ -241,6 +244,7 @@ static const struct fs_parameter_spec exfat_parameters[] = {
fsparam_enum("errors", Opt_errors, exfat_param_enums),
fsparam_flag("discard", Opt_discard),
fsparam_flag("keep_last_dots", Opt_keep_last_dots),
fsparam_flag("sys_tz", Opt_sys_tz),
fsparam_s32("time_offset", Opt_time_offset),
__fsparam(NULL, "utf8", Opt_utf8, fs_param_deprecated,
NULL),
@ -298,6 +302,9 @@ static int exfat_parse_param(struct fs_context *fc, struct fs_parameter *param)
case Opt_keep_last_dots:
opts->keep_last_dots = 1;
break;
case Opt_sys_tz:
opts->sys_tz = 1;
break;
case Opt_time_offset:
/*
* Make the limit 24 just in case someone invents something

1
fs/ext4/Makefile
View File

@ -17,3 +17,4 @@ ext4-$(CONFIG_EXT4_FS_SECURITY) += xattr_security.o
ext4-inode-test-objs += inode-test.o
obj-$(CONFIG_EXT4_KUNIT_TESTS) += ext4-inode-test.o
ext4-$(CONFIG_FS_VERITY) += verity.o
ext4-$(CONFIG_FS_ENCRYPTION) += crypto.o

246
fs/ext4/crypto.c Normal file
View File

@ -0,0 +1,246 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/quotaops.h>
#include <linux/uuid.h>
#include "ext4.h"
#include "xattr.h"
#include "ext4_jbd2.h"
static void ext4_fname_from_fscrypt_name(struct ext4_filename *dst,
const struct fscrypt_name *src)
{
memset(dst, 0, sizeof(*dst));
dst->usr_fname = src->usr_fname;
dst->disk_name = src->disk_name;
dst->hinfo.hash = src->hash;
dst->hinfo.minor_hash = src->minor_hash;
dst->crypto_buf = src->crypto_buf;
}
int ext4_fname_setup_filename(struct inode *dir, const struct qstr *iname,
int lookup, struct ext4_filename *fname)
{
struct fscrypt_name name;
int err;
err = fscrypt_setup_filename(dir, iname, lookup, &name);
if (err)
return err;
ext4_fname_from_fscrypt_name(fname, &name);
#if IS_ENABLED(CONFIG_UNICODE)
err = ext4_fname_setup_ci_filename(dir, iname, fname);
#endif
return err;
}
int ext4_fname_prepare_lookup(struct inode *dir, struct dentry *dentry,
struct ext4_filename *fname)
{
struct fscrypt_name name;
int err;
err = fscrypt_prepare_lookup(dir, dentry, &name);
if (err)
return err;
ext4_fname_from_fscrypt_name(fname, &name);
#if IS_ENABLED(CONFIG_UNICODE)
err = ext4_fname_setup_ci_filename(dir, &dentry->d_name, fname);
#endif
return err;
}
void ext4_fname_free_filename(struct ext4_filename *fname)
{
struct fscrypt_name name;
name.crypto_buf = fname->crypto_buf;
fscrypt_free_filename(&name);
fname->crypto_buf.name = NULL;
fname->usr_fname = NULL;
fname->disk_name.name = NULL;
#if IS_ENABLED(CONFIG_UNICODE)
kfree(fname->cf_name.name);
fname->cf_name.name = NULL;
#endif
}
static bool uuid_is_zero(__u8 u[16])
{
int i;
for (i = 0; i < 16; i++)
if (u[i])
return false;
return true;
}
int ext4_ioctl_get_encryption_pwsalt(struct file *filp, void __user *arg)
{
struct super_block *sb = file_inode(filp)->i_sb;
struct ext4_sb_info *sbi = EXT4_SB(sb);
int err, err2;
handle_t *handle;
if (!ext4_has_feature_encrypt(sb))
return -EOPNOTSUPP;
if (uuid_is_zero(sbi->s_es->s_encrypt_pw_salt)) {
err = mnt_want_write_file(filp);
if (err)
return err;
handle = ext4_journal_start_sb(sb, EXT4_HT_MISC, 1);
if (IS_ERR(handle)) {
err = PTR_ERR(handle);
goto pwsalt_err_exit;
}
err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh,
EXT4_JTR_NONE);
if (err)
goto pwsalt_err_journal;
lock_buffer(sbi->s_sbh);
generate_random_uuid(sbi->s_es->s_encrypt_pw_salt);
ext4_superblock_csum_set(sb);
unlock_buffer(sbi->s_sbh);
err = ext4_handle_dirty_metadata(handle, NULL, sbi->s_sbh);
pwsalt_err_journal:
err2 = ext4_journal_stop(handle);
if (err2 && !err)
err = err2;
pwsalt_err_exit:
mnt_drop_write_file(filp);
if (err)
return err;
}
if (copy_to_user(arg, sbi->s_es->s_encrypt_pw_salt, 16))
return -EFAULT;
return 0;
}
static int ext4_get_context(struct inode *inode, void *ctx, size_t len)
{
return ext4_xattr_get(inode, EXT4_XATTR_INDEX_ENCRYPTION,
EXT4_XATTR_NAME_ENCRYPTION_CONTEXT, ctx, len);
}
static int ext4_set_context(struct inode *inode, const void *ctx, size_t len,
void *fs_data)
{
handle_t *handle = fs_data;
int res, res2, credits, retries = 0;
/*
* Encrypting the root directory is not allowed because e2fsck expects
* lost+found to exist and be unencrypted, and encrypting the root
* directory would imply encrypting the lost+found directory as well as
* the filename "lost+found" itself.
*/
if (inode->i_ino == EXT4_ROOT_INO)
return -EPERM;
if (WARN_ON_ONCE(IS_DAX(inode) && i_size_read(inode)))
return -EINVAL;
if (ext4_test_inode_flag(inode, EXT4_INODE_DAX))
return -EOPNOTSUPP;
res = ext4_convert_inline_data(inode);
if (res)
return res;
/*
* If a journal handle was specified, then the encryption context is
* being set on a new inode via inheritance and is part of a larger
* transaction to create the inode. Otherwise the encryption context is
* being set on an existing inode in its own transaction. Only in the
* latter case should the "retry on ENOSPC" logic be used.
*/
if (handle) {
res = ext4_xattr_set_handle(handle, inode,
EXT4_XATTR_INDEX_ENCRYPTION,
EXT4_XATTR_NAME_ENCRYPTION_CONTEXT,
ctx, len, 0);
if (!res) {
ext4_set_inode_flag(inode, EXT4_INODE_ENCRYPT);
ext4_clear_inode_state(inode,
EXT4_STATE_MAY_INLINE_DATA);
/*
* Update inode->i_flags - S_ENCRYPTED will be enabled,
* S_DAX may be disabled
*/
ext4_set_inode_flags(inode, false);
}
return res;
}
res = dquot_initialize(inode);
if (res)
return res;
retry:
res = ext4_xattr_set_credits(inode, len, false /* is_create */,
&credits);
if (res)
return res;
handle = ext4_journal_start(inode, EXT4_HT_MISC, credits);
if (IS_ERR(handle))
return PTR_ERR(handle);
res = ext4_xattr_set_handle(handle, inode, EXT4_XATTR_INDEX_ENCRYPTION,
EXT4_XATTR_NAME_ENCRYPTION_CONTEXT,
ctx, len, 0);
if (!res) {
ext4_set_inode_flag(inode, EXT4_INODE_ENCRYPT);
/*
* Update inode->i_flags - S_ENCRYPTED will be enabled,
* S_DAX may be disabled
*/
ext4_set_inode_flags(inode, false);
res = ext4_mark_inode_dirty(handle, inode);
if (res)
EXT4_ERROR_INODE(inode, "Failed to mark inode dirty");
}
res2 = ext4_journal_stop(handle);
if (res == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
goto retry;
if (!res)
res = res2;
return res;
}
static const union fscrypt_policy *ext4_get_dummy_policy(struct super_block *sb)
{
return EXT4_SB(sb)->s_dummy_enc_policy.policy;
}
static bool ext4_has_stable_inodes(struct super_block *sb)
{
return ext4_has_feature_stable_inodes(sb);
}
static void ext4_get_ino_and_lblk_bits(struct super_block *sb,
int *ino_bits_ret, int *lblk_bits_ret)
{
*ino_bits_ret = 8 * sizeof(EXT4_SB(sb)->s_es->s_inodes_count);
*lblk_bits_ret = 8 * sizeof(ext4_lblk_t);
}
const struct fscrypt_operations ext4_cryptops = {
.key_prefix = "ext4:",
.get_context = ext4_get_context,
.set_context = ext4_set_context,
.get_dummy_policy = ext4_get_dummy_policy,
.empty_dir = ext4_empty_dir,
.has_stable_inodes = ext4_has_stable_inodes,
.get_ino_and_lblk_bits = ext4_get_ino_and_lblk_bits,
};

6
fs/ext4/dir.c
View File

@ -412,7 +412,7 @@ struct fname {
};
/*
* This functoin implements a non-recursive way of freeing all of the
* This function implements a non-recursive way of freeing all of the
* nodes in the red-black tree.
*/
static void free_rb_tree_fname(struct rb_root *root)
@ -515,7 +515,7 @@ int ext4_htree_store_dirent(struct file *dir_file, __u32 hash,
/*
* This is a helper function for ext4_dx_readdir. It calls filldir
* for all entres on the fname linked list. (Normally there is only
* for all entries on the fname linked list. (Normally there is only
* one entry on the linked list, unless there are 62 bit hash collisions.)
*/
static int call_filldir(struct file *file, struct dir_context *ctx,
@ -648,7 +648,7 @@ int ext4_check_all_de(struct inode *dir, struct buffer_head *bh, void *buf,
unsigned int offset = 0;
char *top;
de = (struct ext4_dir_entry_2 *)buf;
de = buf;
top = buf + buf_size;
while ((char *) de < top) {
if (ext4_check_dir_entry(dir, NULL, de, bh,

84
fs/ext4/ext4.h
View File

@ -673,6 +673,8 @@ enum {
/* Caller will submit data before dropping transaction handle. This
* allows jbd2 to avoid submitting data before commit. */
#define EXT4_GET_BLOCKS_IO_SUBMIT 0x0400
/* Caller is in the atomic contex, find extent if it has been cached */
#define EXT4_GET_BLOCKS_CACHED_NOWAIT 0x0800
/*
* The bit position of these flags must not overlap with any of the
@ -1440,12 +1442,6 @@ struct ext4_super_block {
#ifdef __KERNEL__
#ifdef CONFIG_FS_ENCRYPTION
#define DUMMY_ENCRYPTION_ENABLED(sbi) ((sbi)->s_dummy_enc_policy.policy != NULL)
#else
#define DUMMY_ENCRYPTION_ENABLED(sbi) (0)
#endif
/* Number of quota types we support */
#define EXT4_MAXQUOTAS 3
@ -2731,74 +2727,20 @@ extern int ext4_fname_setup_ci_filename(struct inode *dir,
struct ext4_filename *fname);
#endif
/* ext4 encryption related stuff goes here crypto.c */
#ifdef CONFIG_FS_ENCRYPTION
static inline void ext4_fname_from_fscrypt_name(struct ext4_filename *dst,
const struct fscrypt_name *src)
{
memset(dst, 0, sizeof(*dst));
extern const struct fscrypt_operations ext4_cryptops;
dst->usr_fname = src->usr_fname;
dst->disk_name = src->disk_name;
dst->hinfo.hash = src->hash;
dst->hinfo.minor_hash = src->minor_hash;
dst->crypto_buf = src->crypto_buf;
}
int ext4_fname_setup_filename(struct inode *dir, const struct qstr *iname,
int lookup, struct ext4_filename *fname);
static inline int ext4_fname_setup_filename(struct inode *dir,
const struct qstr *iname,
int lookup,
struct ext4_filename *fname)
{
struct fscrypt_name name;
int err;
int ext4_fname_prepare_lookup(struct inode *dir, struct dentry *dentry,
struct ext4_filename *fname);
err = fscrypt_setup_filename(dir, iname, lookup, &name);
if (err)
return err;
void ext4_fname_free_filename(struct ext4_filename *fname);
ext4_fname_from_fscrypt_name(fname, &name);
int ext4_ioctl_get_encryption_pwsalt(struct file *filp, void __user *arg);
#if IS_ENABLED(CONFIG_UNICODE)
err = ext4_fname_setup_ci_filename(dir, iname, fname);
#endif
return err;
}
static inline int ext4_fname_prepare_lookup(struct inode *dir,
struct dentry *dentry,
struct ext4_filename *fname)
{
struct fscrypt_name name;
int err;
err = fscrypt_prepare_lookup(dir, dentry, &name);
if (err)
return err;
ext4_fname_from_fscrypt_name(fname, &name);
#if IS_ENABLED(CONFIG_UNICODE)
err = ext4_fname_setup_ci_filename(dir, &dentry->d_name, fname);
#endif
return err;
}
static inline void ext4_fname_free_filename(struct ext4_filename *fname)
{
struct fscrypt_name name;
name.crypto_buf = fname->crypto_buf;
fscrypt_free_filename(&name);
fname->crypto_buf.name = NULL;
fname->usr_fname = NULL;
fname->disk_name.name = NULL;
#if IS_ENABLED(CONFIG_UNICODE)
kfree(fname->cf_name.name);
fname->cf_name.name = NULL;
#endif
}
#else /* !CONFIG_FS_ENCRYPTION */
static inline int ext4_fname_setup_filename(struct inode *dir,
const struct qstr *iname,
@ -2831,6 +2773,12 @@ static inline void ext4_fname_free_filename(struct ext4_filename *fname)
fname->cf_name.name = NULL;
#endif
}
static inline int ext4_ioctl_get_encryption_pwsalt(struct file *filp,
void __user *arg)
{
return -EOPNOTSUPP;
}
#endif /* !CONFIG_FS_ENCRYPTION */
/* dir.c */

20
fs/ext4/extents.c
View File

@ -372,7 +372,7 @@ static int ext4_valid_extent_entries(struct inode *inode,
{
unsigned short entries;
ext4_lblk_t lblock = 0;
ext4_lblk_t prev = 0;
ext4_lblk_t cur = 0;
if (eh->eh_entries == 0)
return 1;
@ -396,11 +396,11 @@ static int ext4_valid_extent_entries(struct inode *inode,
/* Check for overlapping extents */
lblock = le32_to_cpu(ext->ee_block);
if ((lblock <= prev) && prev) {
if (lblock < cur) {
*pblk = ext4_ext_pblock(ext);
return 0;
}
prev = lblock + ext4_ext_get_actual_len(ext) - 1;
cur = lblock + ext4_ext_get_actual_len(ext);
ext++;
entries--;
}
@ -420,13 +420,13 @@ static int ext4_valid_extent_entries(struct inode *inode,
/* Check for overlapping index extents */
lblock = le32_to_cpu(ext_idx->ei_block);
if ((lblock <= prev) && prev) {
if (lblock < cur) {
*pblk = ext4_idx_pblock(ext_idx);
return 0;
}
ext_idx++;
entries--;
prev = lblock;
cur = lblock + 1;
}
}
return 1;
@ -4693,15 +4693,17 @@ long ext4_fallocate(struct file *file, int mode, loff_t offset, loff_t len)
FALLOC_FL_INSERT_RANGE))
return -EOPNOTSUPP;
inode_lock(inode);
ret = ext4_convert_inline_data(inode);
inode_unlock(inode);
if (ret)
goto exit;
if (mode & FALLOC_FL_PUNCH_HOLE) {
ret = ext4_punch_hole(file, offset, len);
goto exit;
}
ret = ext4_convert_inline_data(inode);
if (ret)
goto exit;
if (mode & FALLOC_FL_COLLAPSE_RANGE) {
ret = ext4_collapse_range(file, offset, len);
goto exit;

13
fs/ext4/fast_commit.c
View File

@ -970,7 +970,7 @@ static int ext4_fc_write_inode_data(struct inode *inode, u32 *crc)
/* Submit data for all the fast commit inodes */
static int ext4_fc_submit_inode_data_all(journal_t *journal)
{
struct super_block *sb = (struct super_block *)(journal->j_private);
struct super_block *sb = journal->j_private;
struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_inode_info *ei;
int ret = 0;
@ -1004,7 +1004,7 @@ static int ext4_fc_submit_inode_data_all(journal_t *journal)
/* Wait for completion of data for all the fast commit inodes */
static int ext4_fc_wait_inode_data_all(journal_t *journal)
{
struct super_block *sb = (struct super_block *)(journal->j_private);
struct super_block *sb = journal->j_private;
struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_inode_info *pos, *n;
int ret = 0;
@ -1031,7 +1031,7 @@ static int ext4_fc_commit_dentry_updates(journal_t *journal, u32 *crc)
__acquires(&sbi->s_fc_lock)
__releases(&sbi->s_fc_lock)
{
struct super_block *sb = (struct super_block *)(journal->j_private);
struct super_block *sb = journal->j_private;
struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_fc_dentry_update *fc_dentry, *fc_dentry_n;
struct inode *inode;
@ -1093,7 +1093,7 @@ __releases(&sbi->s_fc_lock)
static int ext4_fc_perform_commit(journal_t *journal)
{
struct super_block *sb = (struct super_block *)(journal->j_private);
struct super_block *sb = journal->j_private;
struct ext4_sb_info *sbi = EXT4_SB(sb);
struct ext4_inode_info *iter;
struct ext4_fc_head head;
@ -1198,7 +1198,7 @@ static void ext4_fc_update_stats(struct super_block *sb, int status,
*/
int ext4_fc_commit(journal_t *journal, tid_t commit_tid)
{
struct super_block *sb = (struct super_block *)(journal->j_private);
struct super_block *sb = journal->j_private;
struct ext4_sb_info *sbi = EXT4_SB(sb);
int nblks = 0, ret, bsize = journal->j_blocksize;
int subtid = atomic_read(&sbi->s_fc_subtid);
@ -1659,8 +1659,7 @@ static int ext4_fc_replay_create(struct super_block *sb, struct ext4_fc_tl *tl,
set_nlink(inode, 1);
ext4_mark_inode_dirty(NULL, inode);
out:
if (inode)
iput(inode);
iput(inode);
return ret;
}

4
fs/ext4/file.c
View File

@ -76,7 +76,7 @@ static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to)
return generic_file_read_iter(iocb, to);
}
ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, 0);
ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, NULL, 0);
inode_unlock_shared(inode);
file_accessed(iocb->ki_filp);
@ -565,7 +565,7 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from)
iomap_ops = &ext4_iomap_overwrite_ops;
ret = iomap_dio_rw(iocb, from, iomap_ops, &ext4_dio_write_ops,
(unaligned_io || extend) ? IOMAP_DIO_FORCE_WAIT : 0,
0);
NULL, 0);
if (ret == -ENOTBLK)
ret = 0;

18
fs/ext4/inline.c
View File

@ -1083,14 +1083,14 @@ static void ext4_update_final_de(void *de_buf, int old_size, int new_size)
void *limit;
int de_len;
de = (struct ext4_dir_entry_2 *)de_buf;
de = de_buf;
if (old_size) {
limit = de_buf + old_size;
do {
prev_de = de;
de_len = ext4_rec_len_from_disk(de->rec_len, old_size);
de_buf += de_len;
de = (struct ext4_dir_entry_2 *)de_buf;
de = de_buf;
} while (de_buf < limit);
prev_de->rec_len = ext4_rec_len_to_disk(de_len + new_size -
@ -1155,7 +1155,7 @@ static int ext4_finish_convert_inline_dir(handle_t *handle,
* First create "." and ".." and then copy the dir information
* back to the block.
*/
de = (struct ext4_dir_entry_2 *)target;
de = target;
de = ext4_init_dot_dotdot(inode, de,
inode->i_sb->s_blocksize, csum_size,
le32_to_cpu(((struct ext4_dir_entry_2 *)buf)->inode), 1);
@ -2005,6 +2005,18 @@ int ext4_convert_inline_data(struct inode *inode)
if (!ext4_has_inline_data(inode)) {
ext4_clear_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA);
return 0;
} else if (!ext4_test_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA)) {
/*
* Inode has inline data but EXT4_STATE_MAY_INLINE_DATA is
* cleared. This means we are in the middle of moving of
* inline data to delay allocated block. Just force writeout
* here to finish conversion.
*/
error = filemap_flush(inode->i_mapping);
if (error)
return error;
if (!ext4_has_inline_data(inode))
return 0;
}
needed_blocks = ext4_writepage_trans_blocks(inode);

37
fs/ext4/inode.c
View File

@ -41,7 +41,6 @@
#include <linux/bitops.h>
#include <linux/iomap.h>
#include <linux/iversion.h>
#include <linux/dax.h>
#include "ext4_jbd2.h"
#include "xattr.h"
@ -199,8 +198,7 @@ void ext4_evict_inode(struct inode *inode)
*/
if (inode->i_ino != EXT4_JOURNAL_INO &&
ext4_should_journal_data(inode) &&
(S_ISLNK(inode->i_mode) || S_ISREG(inode->i_mode)) &&
inode->i_data.nrpages) {
S_ISREG(inode->i_mode) && inode->i_data.nrpages) {
journal_t *journal = EXT4_SB(inode->i_sb)->s_journal;
tid_t commit_tid = EXT4_I(inode)->i_datasync_tid;
@ -545,12 +543,21 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
} else {
BUG();
}
if (flags & EXT4_GET_BLOCKS_CACHED_NOWAIT)
return retval;
#ifdef ES_AGGRESSIVE_TEST
ext4_map_blocks_es_recheck(handle, inode, map,
&orig_map, flags);
#endif
goto found;
}
/*
* In the query cache no-wait mode, nothing we can do more if we
* cannot find extent in the cache.
*/
if (flags & EXT4_GET_BLOCKS_CACHED_NOWAIT)
return 0;
/*
* Try to see if we can get the block without requesting a new
@ -837,10 +844,12 @@ struct buffer_head *ext4_getblk(handle_t *handle, struct inode *inode,
struct ext4_map_blocks map;
struct buffer_head *bh;
int create = map_flags & EXT4_GET_BLOCKS_CREATE;
bool nowait = map_flags & EXT4_GET_BLOCKS_CACHED_NOWAIT;
int err;
ASSERT((EXT4_SB(inode->i_sb)->s_mount_state & EXT4_FC_REPLAY)
|| handle != NULL || create == 0);
ASSERT(create == 0 || !nowait);
map.m_lblk = block;
map.m_len = 1;
@ -851,6 +860,9 @@ struct buffer_head *ext4_getblk(handle_t *handle, struct inode *inode,
if (err < 0)
return ERR_PTR(err);
if (nowait)
return sb_find_get_block(inode->i_sb, map.m_pblk);
bh = sb_getblk(inode->i_sb, map.m_pblk);
if (unlikely(!bh))
return ERR_PTR(-ENOMEM);
@ -2944,8 +2956,7 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping,
index = pos >> PAGE_SHIFT;
if (ext4_nonda_switch(inode->i_sb) || S_ISLNK(inode->i_mode) ||
ext4_verity_in_progress(inode)) {
if (ext4_nonda_switch(inode->i_sb) || ext4_verity_in_progress(inode)) {
*fsdata = (void *)FALL_BACK_TO_NONDELALLOC;
return ext4_write_begin(file, mapping, pos,
len, flags, pagep, fsdata);
@ -3967,15 +3978,6 @@ int ext4_punch_hole(struct file *file, loff_t offset, loff_t length)
trace_ext4_punch_hole(inode, offset, length, 0);
ext4_clear_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA);
if (ext4_has_inline_data(inode)) {
filemap_invalidate_lock(mapping);
ret = ext4_convert_inline_data(inode);
filemap_invalidate_unlock(mapping);
if (ret)
return ret;
}
/*
* Write out all dirty pages to avoid race conditions
* Then release them.
@ -4991,7 +4993,6 @@ struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
}
if (IS_ENCRYPTED(inode)) {
inode->i_op = &ext4_encrypted_symlink_inode_operations;
ext4_set_aops(inode);
} else if (ext4_inode_is_fast_symlink(inode)) {
inode->i_link = (char *)ei->i_data;
inode->i_op = &ext4_fast_symlink_inode_operations;
@ -4999,9 +5000,7 @@ struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
sizeof(ei->i_data) - 1);
} else {
inode->i_op = &ext4_symlink_inode_operations;
ext4_set_aops(inode);
}
inode_nohighmem(inode);
} else if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode) ||
S_ISFIFO(inode->i_mode) || S_ISSOCK(inode->i_mode)) {
inode->i_op = &ext4_special_inode_operations;
@ -5398,6 +5397,7 @@ int ext4_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,
if (attr->ia_valid & ATTR_SIZE) {
handle_t *handle;
loff_t oldsize = inode->i_size;
loff_t old_disksize;
int shrink = (attr->ia_size < inode->i_size);
if (!(ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))) {
@ -5469,6 +5469,7 @@ int ext4_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,
inode->i_sb->s_blocksize_bits);
down_write(&EXT4_I(inode)->i_data_sem);
old_disksize = EXT4_I(inode)->i_disksize;
EXT4_I(inode)->i_disksize = attr->ia_size;
rc = ext4_mark_inode_dirty(handle, inode);
if (!error)
@ -5480,6 +5481,8 @@ int ext4_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,
*/
if (!error)
i_size_write(inode, attr->ia_size);
else
EXT4_I(inode)->i_disksize = old_disksize;
up_write(&EXT4_I(inode)->i_data_sem);
ext4_journal_stop(handle);
if (error)

59
fs/ext4/ioctl.c
View File

@ -16,7 +16,6 @@
#include <linux/file.h>
#include <linux/quotaops.h>
#include <linux/random.h>
#include <linux/uuid.h>
#include <linux/uaccess.h>
#include <linux/delay.h>
#include <linux/iversion.h>
@ -504,18 +503,6 @@ static long swap_inode_boot_loader(struct super_block *sb,
return err;
}
#ifdef CONFIG_FS_ENCRYPTION
static int uuid_is_zero(__u8 u[16])
{
int i;
for (i = 0; i < 16; i++)
if (u[i])
return 0;
return 1;
}
#endif
/*
* If immutable is set and we are not clearing it, we're not allowed to change
* anything else in the inode. Don't error out if we're only trying to set
@ -1428,51 +1415,9 @@ static long __ext4_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
return -EOPNOTSUPP;
return fscrypt_ioctl_set_policy(filp, (const void __user *)arg);
case FS_IOC_GET_ENCRYPTION_PWSALT: {
#ifdef CONFIG_FS_ENCRYPTION
int err, err2;
struct ext4_sb_info *sbi = EXT4_SB(sb);
handle_t *handle;
case FS_IOC_GET_ENCRYPTION_PWSALT:
return ext4_ioctl_get_encryption_pwsalt(filp, (void __user *)arg);
if (!ext4_has_feature_encrypt(sb))
return -EOPNOTSUPP;
if (uuid_is_zero(sbi->s_es->s_encrypt_pw_salt)) {
err = mnt_want_write_file(filp);
if (err)
return err;
handle = ext4_journal_start_sb(sb, EXT4_HT_MISC, 1);
if (IS_ERR(handle)) {
err = PTR_ERR(handle);
goto pwsalt_err_exit;
}
err = ext4_journal_get_write_access(handle, sb,
sbi->s_sbh,
EXT4_JTR_NONE);
if (err)
goto pwsalt_err_journal;
lock_buffer(sbi->s_sbh);
generate_random_uuid(sbi->s_es->s_encrypt_pw_salt);
ext4_superblock_csum_set(sb);
unlock_buffer(sbi->s_sbh);
err = ext4_handle_dirty_metadata(handle, NULL,
sbi->s_sbh);
pwsalt_err_journal:
err2 = ext4_journal_stop(handle);
if (err2 && !err)
err = err2;
pwsalt_err_exit:
mnt_drop_write_file(filp);
if (err)
return err;
}
if (copy_to_user((void __user *) arg,
sbi->s_es->s_encrypt_pw_salt, 16))
return -EFAULT;
return 0;
#else
return -EOPNOTSUPP;
#endif
}
case FS_IOC_GET_ENCRYPTION_POLICY:
if (!ext4_has_feature_encrypt(sb))
return -EOPNOTSUPP;

25
fs/ext4/mballoc.c
View File

@ -695,13 +695,10 @@ static int __mb_check_buddy(struct ext4_buddy *e4b, char *file,
for (i = 0; i < max; i++) {
if (mb_test_bit(i, buddy)) {
/* only single bit in buddy2 may be 1 */
/* only single bit in buddy2 may be 0 */
if (!mb_test_bit(i << 1, buddy2)) {
MB_CHECK_ASSERT(
mb_test_bit((i<<1)+1, buddy2));
} else if (!mb_test_bit((i << 1) + 1, buddy2)) {
MB_CHECK_ASSERT(
mb_test_bit(i << 1, buddy2));
}
continue;
}
@ -2919,7 +2916,7 @@ const struct seq_operations ext4_mb_seq_groups_ops = {
int ext4_seq_mb_stats_show(struct seq_file *seq, void *offset)
{
struct super_block *sb = (struct super_block *)seq->private;
struct super_block *sb = seq->private;
struct ext4_sb_info *sbi = EXT4_SB(sb);
seq_puts(seq, "mballoc:\n");
@ -6398,6 +6395,7 @@ __releases(ext4_group_lock_ptr(sb, e4b->bd_group))
* @start: first group block to examine
* @max: last group block to examine
* @minblocks: minimum extent block count
* @set_trimmed: set the trimmed flag if at least one block is trimmed
*
* ext4_trim_all_free walks through group's block bitmap searching for free
* extents. When the free extent is found, mark it as used in group buddy
@ -6407,7 +6405,7 @@ __releases(ext4_group_lock_ptr(sb, e4b->bd_group))
static ext4_grpblk_t
ext4_trim_all_free(struct super_block *sb, ext4_group_t group,
ext4_grpblk_t start, ext4_grpblk_t max,
ext4_grpblk_t minblocks)
ext4_grpblk_t minblocks, bool set_trimmed)
{
struct ext4_buddy e4b;
int ret;
@ -6426,7 +6424,7 @@ ext4_trim_all_free(struct super_block *sb, ext4_group_t group,
if (!EXT4_MB_GRP_WAS_TRIMMED(e4b.bd_info) ||
minblocks < EXT4_SB(sb)->s_last_trim_minblks) {
ret = ext4_try_to_trim_range(sb, &e4b, start, max, minblocks);
if (ret >= 0)
if (ret >= 0 && set_trimmed)
EXT4_MB_GRP_SET_TRIMMED(e4b.bd_info);
} else {
ret = 0;
@ -6463,6 +6461,7 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range)
ext4_fsblk_t first_data_blk =
le32_to_cpu(EXT4_SB(sb)->s_es->s_first_data_block);
ext4_fsblk_t max_blks = ext4_blocks_count(EXT4_SB(sb)->s_es);
bool whole_group, eof = false;
int ret = 0;
start = range->start >> sb->s_blocksize_bits;
@ -6481,8 +6480,10 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range)
if (minlen > EXT4_CLUSTERS_PER_GROUP(sb))
goto out;
}
if (end >= max_blks)
if (end >= max_blks - 1) {
end = max_blks - 1;
eof = true;
}
if (end <= first_data_blk)
goto out;
if (start < first_data_blk)
@ -6496,6 +6497,7 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range)
/* end now represents the last cluster to discard in this group */
end = EXT4_CLUSTERS_PER_GROUP(sb) - 1;
whole_group = true;
for (group = first_group; group <= last_group; group++) {
grp = ext4_get_group_info(sb, group);
@ -6512,12 +6514,13 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range)
* change it for the last group, note that last_cluster is
* already computed earlier by ext4_get_group_no_and_offset()
*/
if (group == last_group)
if (group == last_group) {
end = last_cluster;
whole_group = eof ? true : end == EXT4_CLUSTERS_PER_GROUP(sb) - 1;
}
if (grp->bb_free >= minlen) {
cnt = ext4_trim_all_free(sb, group, first_cluster,
end, minlen);
end, minlen, whole_group);
if (cnt < 0) {
ret = cnt;
break;

2
fs/ext4/mmp.c
View File

@ -127,7 +127,7 @@ void __dump_mmp_msg(struct super_block *sb, struct mmp_struct *mmp,
*/
static int kmmpd(void *data)
{
struct super_block *sb = (struct super_block *) data;
struct super_block *sb = data;
struct ext4_super_block *es = EXT4_SB(sb)->s_es;
struct buffer_head *bh = EXT4_SB(sb)->s_mmp_bh;
struct mmp_struct *mmp;

214
fs/ext4/namei.c
View File

@ -277,9 +277,9 @@ static struct dx_frame *dx_probe(struct ext4_filename *fname,
struct dx_hash_info *hinfo,
struct dx_frame *frame);
static void dx_release(struct dx_frame *frames);
static int dx_make_map(struct inode *dir, struct ext4_dir_entry_2 *de,
unsigned blocksize, struct dx_hash_info *hinfo,
struct dx_map_entry map[]);
static int dx_make_map(struct inode *dir, struct buffer_head *bh,
struct dx_hash_info *hinfo,
struct dx_map_entry *map_tail);
static void dx_sort_map(struct dx_map_entry *map, unsigned count);
static struct ext4_dir_entry_2 *dx_move_dirents(struct inode *dir, char *from,
char *to, struct dx_map_entry *offsets,
@ -777,12 +777,14 @@ static struct dx_frame *
dx_probe(struct ext4_filename *fname, struct inode *dir,
struct dx_hash_info *hinfo, struct dx_frame *frame_in)
{
unsigned count, indirect;
unsigned count, indirect, level, i;
struct dx_entry *at, *entries, *p, *q, *m;
struct dx_root *root;
struct dx_frame *frame = frame_in;
struct dx_frame *ret_err = ERR_PTR(ERR_BAD_DX_DIR);
u32 hash;
ext4_lblk_t block;
ext4_lblk_t blocks[EXT4_HTREE_LEVEL];
memset(frame_in, 0, EXT4_HTREE_LEVEL * sizeof(frame_in[0]));
frame->bh = ext4_read_dirblock(dir, 0, INDEX);
@ -854,6 +856,8 @@ dx_probe(struct ext4_filename *fname, struct inode *dir,
}
dxtrace(printk("Look up %x", hash));
level = 0;
blocks[0] = 0;
while (1) {
count = dx_get_count(entries);
if (!count || count > dx_get_limit(entries)) {
@ -882,15 +886,27 @@ dx_probe(struct ext4_filename *fname, struct inode *dir,
dx_get_block(at)));
frame->entries = entries;
frame->at = at;
if (!indirect--)
block = dx_get_block(at);
for (i = 0; i <= level; i++) {
if (blocks[i] == block) {
ext4_warning_inode(dir,
"dx entry: tree cycle block %u points back to block %u",
blocks[level], block);
goto fail;
}
}
if (++level > indirect)
return frame;
blocks[level] = block;
frame++;
frame->bh = ext4_read_dirblock(dir, dx_get_block(at), INDEX);
frame->bh = ext4_read_dirblock(dir, block, INDEX);
if (IS_ERR(frame->bh)) {
ret_err = (struct dx_frame *) frame->bh;
frame->bh = NULL;
goto fail;
}
entries = ((struct dx_node *) frame->bh->b_data)->entries;
if (dx_get_limit(entries) != dx_node_limit(dir)) {
@ -1249,15 +1265,23 @@ static inline int search_dirblock(struct buffer_head *bh,
* Create map of hash values, offsets, and sizes, stored at end of block.
* Returns number of entries mapped.
*/
static int dx_make_map(struct inode *dir, struct ext4_dir_entry_2 *de,
unsigned blocksize, struct dx_hash_info *hinfo,
static int dx_make_map(struct inode *dir, struct buffer_head *bh,
struct dx_hash_info *hinfo,
struct dx_map_entry *map_tail)
{
int count = 0;
char *base = (char *) de;
struct ext4_dir_entry_2 *de = (struct ext4_dir_entry_2 *)bh->b_data;
unsigned int buflen = bh->b_size;
char *base = bh->b_data;
struct dx_hash_info h = *hinfo;
while ((char *) de < base + blocksize) {
if (ext4_has_metadata_csum(dir->i_sb))
buflen -= sizeof(struct ext4_dir_entry_tail);
while ((char *) de < base + buflen) {
if (ext4_check_dir_entry(dir, NULL, de, bh, base, buflen,
((char *)de) - base))
return -EFSCORRUPTED;
if (de->name_len && de->inode) {
if (ext4_hash_in_dirent(dir))
h.hash = EXT4_DIRENT_HASH(de);
@ -1270,8 +1294,7 @@ static int dx_make_map(struct inode *dir, struct ext4_dir_entry_2 *de,
count++;
cond_resched();
}
/* XXX: do we need to check rec_len == 0 case? -Chris */
de = ext4_next_entry(de, blocksize);
de = ext4_next_entry(de, dir->i_sb->s_blocksize);
}
return count;
}
@ -1943,8 +1966,11 @@ static struct ext4_dir_entry_2 *do_split(handle_t *handle, struct inode *dir,
/* create map in the end of data2 block */
map = (struct dx_map_entry *) (data2 + blocksize);
count = dx_make_map(dir, (struct ext4_dir_entry_2 *) data1,
blocksize, hinfo, map);
count = dx_make_map(dir, *bh, hinfo, map);
if (count < 0) {
err = count;
goto journal_error;
}
map -= count;
dx_sort_map(map, count);
/* Ensure that neither split block is over half full */
@ -2031,7 +2057,7 @@ int ext4_find_dest_de(struct inode *dir, struct inode *inode,
unsigned int offset = 0;
char *top;
de = (struct ext4_dir_entry_2 *)buf;
de = buf;
top = buf + buf_size - reclen;
while ((char *) de <= top) {
if (ext4_check_dir_entry(dir, NULL, de, bh,
@ -2587,7 +2613,7 @@ int ext4_generic_delete_entry(struct inode *dir,
i = 0;
pde = NULL;
de = (struct ext4_dir_entry_2 *)entry_buf;
de = entry_buf;
while (i < buf_size - csum_size) {
if (ext4_check_dir_entry(dir, NULL, de, bh,
entry_buf, buf_size, i))
@ -3249,6 +3275,32 @@ static int ext4_unlink(struct inode *dir, struct dentry *dentry)
return retval;
}
static int ext4_init_symlink_block(handle_t *handle, struct inode *inode,
struct fscrypt_str *disk_link)
{
struct buffer_head *bh;
char *kaddr;
int err = 0;
bh = ext4_bread(handle, inode, 0, EXT4_GET_BLOCKS_CREATE);
if (IS_ERR(bh))
return PTR_ERR(bh);
BUFFER_TRACE(bh, "get_write_access");
err = ext4_journal_get_write_access(handle, inode->i_sb, bh, EXT4_JTR_NONE);
if (err)
goto out;
kaddr = (char *)bh->b_data;
memcpy(kaddr, disk_link->name, disk_link->len);
inode->i_size = disk_link->len - 1;
EXT4_I(inode)->i_disksize = inode->i_size;
err = ext4_handle_dirty_metadata(handle, inode, bh);
out:
brelse(bh);
return err;
}
static int ext4_symlink(struct user_namespace *mnt_userns, struct inode *dir,
struct dentry *dentry, const char *symname)
{
@ -3257,6 +3309,7 @@ static int ext4_symlink(struct user_namespace *mnt_userns, struct inode *dir,
int err, len = strlen(symname);
int credits;
struct fscrypt_str disk_link;
int retries = 0;
if (unlikely(ext4_forced_shutdown(EXT4_SB(dir->i_sb))))
return -EIO;
@ -3270,26 +3323,15 @@ static int ext4_symlink(struct user_namespace *mnt_userns, struct inode *dir,
if (err)
return err;
if ((disk_link.len > EXT4_N_BLOCKS * 4)) {
/*
* For non-fast symlinks, we just allocate inode and put it on
* orphan list in the first transaction => we need bitmap,
* group descriptor, sb, inode block, quota blocks, and
* possibly selinux xattr blocks.
*/
credits = 4 + EXT4_MAXQUOTAS_INIT_BLOCKS(dir->i_sb) +
EXT4_XATTR_TRANS_BLOCKS;
} else {
/*
* Fast symlink. We have to add entry to directory
* (EXT4_DATA_TRANS_BLOCKS + EXT4_INDEX_EXTRA_TRANS_BLOCKS),
* allocate new inode (bitmap, group descriptor, inode block,
* quota blocks, sb is already counted in previous macros).
*/
credits = EXT4_DATA_TRANS_BLOCKS(dir->i_sb) +
EXT4_INDEX_EXTRA_TRANS_BLOCKS + 3;
}
/*
* EXT4_INDEX_EXTRA_TRANS_BLOCKS for addition of entry into the
* directory. +3 for inode, inode bitmap, group descriptor allocation.
* EXT4_DATA_TRANS_BLOCKS for the data block allocation and
* modification.
*/
credits = EXT4_DATA_TRANS_BLOCKS(dir->i_sb) +
EXT4_INDEX_EXTRA_TRANS_BLOCKS + 3;
retry:
inode = ext4_new_inode_start_handle(mnt_userns, dir, S_IFLNK|S_IRWXUGO,
&dentry->d_name, 0, NULL,
EXT4_HT_DIR, credits);
@ -3297,7 +3339,8 @@ static int ext4_symlink(struct user_namespace *mnt_userns, struct inode *dir,
if (IS_ERR(inode)) {
if (handle)
ext4_journal_stop(handle);
return PTR_ERR(inode);
err = PTR_ERR(inode);
goto out_retry;
}
if (IS_ENCRYPTED(inode)) {
@ -3305,75 +3348,44 @@ static int ext4_symlink(struct user_namespace *mnt_userns, struct inode *dir,
if (err)
goto err_drop_inode;
inode->i_op = &ext4_encrypted_symlink_inode_operations;
} else {
if ((disk_link.len > EXT4_N_BLOCKS * 4)) {
inode->i_op = &ext4_symlink_inode_operations;
} else {
inode->i_op = &ext4_fast_symlink_inode_operations;
inode->i_link = (char *)&EXT4_I(inode)->i_data;
}
}
if ((disk_link.len > EXT4_N_BLOCKS * 4)) {
if (!IS_ENCRYPTED(inode))
inode->i_op = &ext4_symlink_inode_operations;
inode_nohighmem(inode);
ext4_set_aops(inode);
/*
* We cannot call page_symlink() with transaction started
* because it calls into ext4_write_begin() which can wait
* for transaction commit if we are running out of space
* and thus we deadlock. So we have to stop transaction now
* and restart it when symlink contents is written.
*
* To keep fs consistent in case of crash, we have to put inode
* to orphan list in the mean time.
*/
drop_nlink(inode);
err = ext4_orphan_add(handle, inode);
if (handle)
ext4_journal_stop(handle);
handle = NULL;
if (err)
goto err_drop_inode;
err = __page_symlink(inode, disk_link.name, disk_link.len, 1);
if (err)
goto err_drop_inode;
/*
* Now inode is being linked into dir (EXT4_DATA_TRANS_BLOCKS
* + EXT4_INDEX_EXTRA_TRANS_BLOCKS), inode is also modified
*/
handle = ext4_journal_start(dir, EXT4_HT_DIR,
EXT4_DATA_TRANS_BLOCKS(dir->i_sb) +
EXT4_INDEX_EXTRA_TRANS_BLOCKS + 1);
if (IS_ERR(handle)) {
err = PTR_ERR(handle);
handle = NULL;
goto err_drop_inode;
}
set_nlink(inode, 1);
err = ext4_orphan_del(handle, inode);
/* alloc symlink block and fill it */
err = ext4_init_symlink_block(handle, inode, &disk_link);
if (err)
goto err_drop_inode;
} else {
/* clear the extent format for fast symlink */
ext4_clear_inode_flag(inode, EXT4_INODE_EXTENTS);
if (!IS_ENCRYPTED(inode)) {
inode->i_op = &ext4_fast_symlink_inode_operations;
inode->i_link = (char *)&EXT4_I(inode)->i_data;
}
memcpy((char *)&EXT4_I(inode)->i_data, disk_link.name,
disk_link.len);
inode->i_size = disk_link.len - 1;
EXT4_I(inode)->i_disksize = inode->i_size;
}
EXT4_I(inode)->i_disksize = inode->i_size;
err = ext4_add_nondir(handle, dentry, &inode);
if (handle)
ext4_journal_stop(handle);
if (inode)
iput(inode);
goto out_free_encrypted_link;
iput(inode);
goto out_retry;
err_drop_inode:
clear_nlink(inode);
ext4_orphan_add(handle, inode);
unlock_new_inode(inode);
if (handle)
ext4_journal_stop(handle);
clear_nlink(inode);
unlock_new_inode(inode);
iput(inode);
out_free_encrypted_link:
out_retry:
if (err == -ENOSPC && ext4_should_retry_alloc(dir->i_sb, &retries))
goto retry;
if (disk_link.name != (unsigned char *)symname)
kfree(disk_link.name);
return err;
@ -3455,6 +3467,9 @@ static struct buffer_head *ext4_get_first_dir_block(handle_t *handle,
struct buffer_head *bh;
if (!ext4_has_inline_data(inode)) {
struct ext4_dir_entry_2 *de;
unsigned int offset;
/* The first directory block must not be a hole, so
* treat it as DIRENT_HTREE
*/
@ -3463,9 +3478,30 @@ static struct buffer_head *ext4_get_first_dir_block(handle_t *handle,
*retval = PTR_ERR(bh);
return NULL;
}
*parent_de = ext4_next_entry(
(struct ext4_dir_entry_2 *)bh->b_data,
inode->i_sb->s_blocksize);
de = (struct ext4_dir_entry_2 *) bh->b_data;
if (ext4_check_dir_entry(inode, NULL, de, bh, bh->b_data,
bh->b_size, 0) ||
le32_to_cpu(de->inode) != inode->i_ino ||
strcmp(".", de->name)) {
EXT4_ERROR_INODE(inode, "directory missing '.'");
brelse(bh);
*retval = -EFSCORRUPTED;
return NULL;
}
offset = ext4_rec_len_from_disk(de->rec_len,
inode->i_sb->s_blocksize);
de = ext4_next_entry(de, inode->i_sb->s_blocksize);
if (ext4_check_dir_entry(inode, NULL, de, bh, bh->b_data,
bh->b_size, offset) ||
le32_to_cpu(de->inode) == 0 || strcmp("..", de->name)) {
EXT4_ERROR_INODE(inode, "directory missing '..'");
brelse(bh);
*retval = -EFSCORRUPTED;
return NULL;
}
*parent_de = de;
return bh;
}

217
fs/ext4/super.c
View File

@ -1211,6 +1211,9 @@ static void ext4_put_super(struct super_block *sb)
*/
ext4_unregister_sysfs(sb);
if (___ratelimit(&ext4_mount_msg_ratelimit, "EXT4-fs unmount"))
ext4_msg(sb, KERN_INFO, "unmounting filesystem.");
ext4_unregister_li_request(sb);
ext4_quota_off_umount(sb);
@ -1397,7 +1400,7 @@ static void ext4_destroy_inode(struct inode *inode)
static void init_once(void *foo)
{
struct ext4_inode_info *ei = (struct ext4_inode_info *) foo;
struct ext4_inode_info *ei = foo;
INIT_LIST_HEAD(&ei->i_orphan);
init_rwsem(&ei->xattr_sem);
@ -1492,128 +1495,6 @@ static int ext4_nfs_commit_metadata(struct inode *inode)
return ext4_write_inode(inode, &wbc);
}
#ifdef CONFIG_FS_ENCRYPTION
static int ext4_get_context(struct inode *inode, void *ctx, size_t len)
{
return ext4_xattr_get(inode, EXT4_XATTR_INDEX_ENCRYPTION,
EXT4_XATTR_NAME_ENCRYPTION_CONTEXT, ctx, len);
}
static int ext4_set_context(struct inode *inode, const void *ctx, size_t len,
void *fs_data)
{
handle_t *handle = fs_data;
int res, res2, credits, retries = 0;
/*
* Encrypting the root directory is not allowed because e2fsck expects
* lost+found to exist and be unencrypted, and encrypting the root
* directory would imply encrypting the lost+found directory as well as
* the filename "lost+found" itself.
*/
if (inode->i_ino == EXT4_ROOT_INO)
return -EPERM;
if (WARN_ON_ONCE(IS_DAX(inode) && i_size_read(inode)))
return -EINVAL;
if (ext4_test_inode_flag(inode, EXT4_INODE_DAX))
return -EOPNOTSUPP;
res = ext4_convert_inline_data(inode);
if (res)
return res;
/*
* If a journal handle was specified, then the encryption context is
* being set on a new inode via inheritance and is part of a larger
* transaction to create the inode. Otherwise the encryption context is
* being set on an existing inode in its own transaction. Only in the
* latter case should the "retry on ENOSPC" logic be used.
*/
if (handle) {
res = ext4_xattr_set_handle(handle, inode,
EXT4_XATTR_INDEX_ENCRYPTION,
EXT4_XATTR_NAME_ENCRYPTION_CONTEXT,
ctx, len, 0);
if (!res) {
ext4_set_inode_flag(inode, EXT4_INODE_ENCRYPT);
ext4_clear_inode_state(inode,
EXT4_STATE_MAY_INLINE_DATA);
/*
* Update inode->i_flags - S_ENCRYPTED will be enabled,
* S_DAX may be disabled
*/
ext4_set_inode_flags(inode, false);
}
return res;
}
res = dquot_initialize(inode);
if (res)
return res;
retry:
res = ext4_xattr_set_credits(inode, len, false /* is_create */,
&credits);
if (res)
return res;
handle = ext4_journal_start(inode, EXT4_HT_MISC, credits);
if (IS_ERR(handle))
return PTR_ERR(handle);
res = ext4_xattr_set_handle(handle, inode, EXT4_XATTR_INDEX_ENCRYPTION,
EXT4_XATTR_NAME_ENCRYPTION_CONTEXT,
ctx, len, 0);
if (!res) {
ext4_set_inode_flag(inode, EXT4_INODE_ENCRYPT);
/*
* Update inode->i_flags - S_ENCRYPTED will be enabled,
* S_DAX may be disabled
*/
ext4_set_inode_flags(inode, false);
res = ext4_mark_inode_dirty(handle, inode);
if (res)
EXT4_ERROR_INODE(inode, "Failed to mark inode dirty");
}
res2 = ext4_journal_stop(handle);
if (res == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
goto retry;
if (!res)
res = res2;
return res;
}
static const union fscrypt_policy *ext4_get_dummy_policy(struct super_block *sb)
{
return EXT4_SB(sb)->s_dummy_enc_policy.policy;
}
static bool ext4_has_stable_inodes(struct super_block *sb)
{
return ext4_has_feature_stable_inodes(sb);
}
static void ext4_get_ino_and_lblk_bits(struct super_block *sb,
int *ino_bits_ret, int *lblk_bits_ret)
{
*ino_bits_ret = 8 * sizeof(EXT4_SB(sb)->s_es->s_inodes_count);
*lblk_bits_ret = 8 * sizeof(ext4_lblk_t);
}
static const struct fscrypt_operations ext4_cryptops = {
.key_prefix = "ext4:",
.get_context = ext4_get_context,
.set_context = ext4_set_context,
.get_dummy_policy = ext4_get_dummy_policy,
.empty_dir = ext4_empty_dir,
.has_stable_inodes = ext4_has_stable_inodes,
.get_ino_and_lblk_bits = ext4_get_ino_and_lblk_bits,
};
#endif
#ifdef CONFIG_QUOTA
static const char * const quotatypes[] = INITQFNAMES;
#define QTYPE2NAME(t) (quotatypes[t])
@ -1867,7 +1748,6 @@ static const struct fs_parameter_spec ext4_param_specs[] = {
};
#define DEFAULT_JOURNAL_IOPRIO (IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, 3))
#define DEFAULT_MB_OPTIMIZE_SCAN (-1)
static const char deprecated_msg[] =
"Mount option \"%s\" will be removed by %s\n"
@ -1913,6 +1793,7 @@ static const struct mount_opts {
MOPT_EXT4_ONLY | MOPT_CLEAR},
{Opt_warn_on_error, EXT4_MOUNT_WARN_ON_ERROR, MOPT_SET},
{Opt_nowarn_on_error, EXT4_MOUNT_WARN_ON_ERROR, MOPT_CLEAR},
{Opt_commit, 0, MOPT_NO_EXT2},
{Opt_nojournal_checksum, EXT4_MOUNT_JOURNAL_CHECKSUM,
MOPT_EXT4_ONLY | MOPT_CLEAR},
{Opt_journal_checksum, EXT4_MOUNT_JOURNAL_CHECKSUM,
@ -2427,11 +2308,12 @@ static int ext4_parse_param(struct fs_context *fc, struct fs_parameter *param)
ctx->spec |= EXT4_SPEC_DUMMY_ENCRYPTION;
ctx->test_dummy_enc_arg = kmemdup_nul(param->string, param->size,
GFP_KERNEL);
return 0;
#else
ext4_msg(NULL, KERN_WARNING,
"Test dummy encryption mount option ignored");
"test_dummy_encryption option not supported");
return -EINVAL;
#endif
return 0;
case Opt_dax:
case Opt_dax_type:
#ifdef CONFIG_FS_DAX
@ -2625,8 +2507,10 @@ static int parse_apply_sb_mount_options(struct super_block *sb,
ret = ext4_apply_options(fc, sb);
out_free:
kfree(s_ctx);
kfree(fc);
if (fc) {
ext4_fc_free(fc);
kfree(fc);
}
kfree(s_mount_opts);
return ret;
}
@ -2786,12 +2670,44 @@ static int ext4_check_quota_consistency(struct fs_context *fc,
#endif
}
static int ext4_check_test_dummy_encryption(const struct fs_context *fc,
struct super_block *sb)
{
#ifdef CONFIG_FS_ENCRYPTION
const struct ext4_fs_context *ctx = fc->fs_private;
const struct ext4_sb_info *sbi = EXT4_SB(sb);
if (!(ctx->spec & EXT4_SPEC_DUMMY_ENCRYPTION))
return 0;
if (!ext4_has_feature_encrypt(sb)) {
ext4_msg(NULL, KERN_WARNING,
"test_dummy_encryption requires encrypt feature");
return -EINVAL;
}
/*
* This mount option is just for testing, and it's not worthwhile to
* implement the extra complexity (e.g. RCU protection) that would be
* needed to allow it to be set or changed during remount. We do allow
* it to be specified during remount, but only if there is no change.
*/
if (fc->purpose == FS_CONTEXT_FOR_RECONFIGURE &&
!sbi->s_dummy_enc_policy.policy) {
ext4_msg(NULL, KERN_WARNING,
"Can't set test_dummy_encryption on remount");
return -EINVAL;
}
#endif /* CONFIG_FS_ENCRYPTION */
return 0;
}
static int ext4_check_opt_consistency(struct fs_context *fc,
struct super_block *sb)
{
struct ext4_fs_context *ctx = fc->fs_private;
struct ext4_sb_info *sbi = fc->s_fs_info;
int is_remount = fc->purpose == FS_CONTEXT_FOR_RECONFIGURE;
int err;
if ((ctx->opt_flags & MOPT_NO_EXT2) && IS_EXT2_SB(sb)) {
ext4_msg(NULL, KERN_ERR,
@ -2821,20 +2737,9 @@ static int ext4_check_opt_consistency(struct fs_context *fc,
"for blocksize < PAGE_SIZE");
}
#ifdef CONFIG_FS_ENCRYPTION
/*
* This mount option is just for testing, and it's not worthwhile to
* implement the extra complexity (e.g. RCU protection) that would be
* needed to allow it to be set or changed during remount. We do allow
* it to be specified during remount, but only if there is no change.
*/
if ((ctx->spec & EXT4_SPEC_DUMMY_ENCRYPTION) &&
is_remount && !sbi->s_dummy_enc_policy.policy) {
ext4_msg(NULL, KERN_WARNING,
"Can't set test_dummy_encryption on remount");
return -1;
}
#endif
err = ext4_check_test_dummy_encryption(fc, sb);
if (err)
return err;
if ((ctx->spec & EXT4_SPEC_DATAJ) && is_remount) {
if (!sbi->s_journal) {
@ -3837,7 +3742,7 @@ static struct task_struct *ext4_lazyinit_task;
*/
static int ext4_lazyinit_thread(void *arg)
{
struct ext4_lazy_init *eli = (struct ext4_lazy_init *)arg;
struct ext4_lazy_init *eli = arg;
struct list_head *pos, *n;
struct ext4_li_request *elr;
unsigned long next_wakeup, cur;
@ -4409,7 +4314,8 @@ static int __ext4_fill_super(struct fs_context *fc, struct super_block *sb)
int silent = fc->sb_flags & SB_SILENT;
/* Set defaults for the variables that will be set during parsing */
ctx->journal_ioprio = DEFAULT_JOURNAL_IOPRIO;
if (!(ctx->spec & EXT4_SPEC_JOURNAL_IOPRIO))
ctx->journal_ioprio = DEFAULT_JOURNAL_IOPRIO;
sbi->s_inode_readahead_blks = EXT4_DEF_INODE_READAHEAD_BLKS;
sbi->s_sectors_written_start =
@ -4886,7 +4792,7 @@ static int __ext4_fill_super(struct fs_context *fc, struct super_block *sb)
sbi->s_inodes_per_block;
sbi->s_desc_per_block = blocksize / EXT4_DESC_SIZE(sb);
sbi->s_sbh = bh;
sbi->s_mount_state = le16_to_cpu(es->s_state);
sbi->s_mount_state = le16_to_cpu(es->s_state) & ~EXT4_FC_REPLAY;
sbi->s_addr_per_block_bits = ilog2(EXT4_ADDR_PER_BLOCK(sb));
sbi->s_desc_per_block_bits = ilog2(EXT4_DESC_PER_BLOCK(sb));
@ -5279,12 +5185,6 @@ static int __ext4_fill_super(struct fs_context *fc, struct super_block *sb)
goto failed_mount_wq;
}
if (DUMMY_ENCRYPTION_ENABLED(sbi) && !sb_rdonly(sb) &&
!ext4_has_feature_encrypt(sb)) {
ext4_set_feature_encrypt(sb);
ext4_commit_super(sb);
}
/*
* Get the # of file system overhead blocks from the
* superblock if present.
@ -6272,7 +6172,6 @@ static int __ext4_remount(struct fs_context *fc, struct super_block *sb)
char *to_free[EXT4_MAXQUOTAS];
#endif
ctx->journal_ioprio = DEFAULT_JOURNAL_IOPRIO;
/* Store the original options */
old_sb_flags = sb->s_flags;
@ -6298,9 +6197,14 @@ static int __ext4_remount(struct fs_context *fc, struct super_block *sb)
} else
old_opts.s_qf_names[i] = NULL;
#endif
if (sbi->s_journal && sbi->s_journal->j_task->io_context)
ctx->journal_ioprio =
sbi->s_journal->j_task->io_context->ioprio;
if (!(ctx->spec & EXT4_SPEC_JOURNAL_IOPRIO)) {
if (sbi->s_journal && sbi->s_journal->j_task->io_context)
ctx->journal_ioprio =
sbi->s_journal->j_task->io_context->ioprio;
else
ctx->journal_ioprio = DEFAULT_JOURNAL_IOPRIO;
}
ext4_apply_options(fc, sb);
@ -6441,7 +6345,8 @@ static int __ext4_remount(struct fs_context *fc, struct super_block *sb)
if (err)
goto restore_opts;
}
sbi->s_mount_state = le16_to_cpu(es->s_state);
sbi->s_mount_state = (le16_to_cpu(es->s_state) &
~EXT4_FC_REPLAY);
err = ext4_setup_super(sb, es, 0);
if (err)

51
fs/ext4/symlink.c
View File

@ -27,7 +27,7 @@ static const char *ext4_encrypted_get_link(struct dentry *dentry,
struct inode *inode,
struct delayed_call *done)
{
struct page *cpage = NULL;
struct buffer_head *bh = NULL;
const void *caddr;
unsigned int max_size;
const char *paddr;
@ -39,16 +39,19 @@ static const char *ext4_encrypted_get_link(struct dentry *dentry,
caddr = EXT4_I(inode)->i_data;
max_size = sizeof(EXT4_I(inode)->i_data);
} else {
cpage = read_mapping_page(inode->i_mapping, 0, NULL);
if (IS_ERR(cpage))
return ERR_CAST(cpage);
caddr = page_address(cpage);
bh = ext4_bread(NULL, inode, 0, 0);
if (IS_ERR(bh))
return ERR_CAST(bh);
if (!bh) {
EXT4_ERROR_INODE(inode, "bad symlink.");
return ERR_PTR(-EFSCORRUPTED);
}
caddr = bh->b_data;
max_size = inode->i_sb->s_blocksize;
}
paddr = fscrypt_get_symlink(inode, caddr, max_size, done);
if (cpage)
put_page(cpage);
brelse(bh);
return paddr;
}
@ -62,6 +65,38 @@ static int ext4_encrypted_symlink_getattr(struct user_namespace *mnt_userns,
return fscrypt_symlink_getattr(path, stat);
}
static void ext4_free_link(void *bh)
{
brelse(bh);
}
static const char *ext4_get_link(struct dentry *dentry, struct inode *inode,
struct delayed_call *callback)
{
struct buffer_head *bh;
if (!dentry) {
bh = ext4_getblk(NULL, inode, 0, EXT4_GET_BLOCKS_CACHED_NOWAIT);
if (IS_ERR(bh))
return ERR_CAST(bh);
if (!bh || !ext4_buffer_uptodate(bh))
return ERR_PTR(-ECHILD);
} else {
bh = ext4_bread(NULL, inode, 0, 0);
if (IS_ERR(bh))
return ERR_CAST(bh);
if (!bh) {
EXT4_ERROR_INODE(inode, "bad symlink.");
return ERR_PTR(-EFSCORRUPTED);
}
}
set_delayed_call(callback, ext4_free_link, bh);
nd_terminate_link(bh->b_data, inode->i_size,
inode->i_sb->s_blocksize - 1);
return bh->b_data;
}
const struct inode_operations ext4_encrypted_symlink_inode_operations = {
.get_link = ext4_encrypted_get_link,
.setattr = ext4_setattr,
@ -70,7 +105,7 @@ const struct inode_operations ext4_encrypted_symlink_inode_operations = {
};
const struct inode_operations ext4_symlink_inode_operations = {
.get_link = page_get_link,
.get_link = ext4_get_link,
.setattr = ext4_setattr,
.getattr = ext4_getattr,
.listxattr = ext4_listxattr,

4
fs/f2fs/file.c
View File

@ -4308,7 +4308,7 @@ static ssize_t f2fs_dio_read_iter(struct kiocb *iocb, struct iov_iter *to)
*/
inc_page_count(sbi, F2FS_DIO_READ);
dio = __iomap_dio_rw(iocb, to, &f2fs_iomap_ops,
&f2fs_iomap_dio_read_ops, 0, 0);
&f2fs_iomap_dio_read_ops, 0, NULL, 0);
if (IS_ERR_OR_NULL(dio)) {
ret = PTR_ERR_OR_ZERO(dio);
if (ret != -EIOCBQUEUED)
@ -4526,7 +4526,7 @@ static ssize_t f2fs_dio_write_iter(struct kiocb *iocb, struct iov_iter *from,
if (pos + count > inode->i_size)
dio_flags |= IOMAP_DIO_FORCE_WAIT;
dio = __iomap_dio_rw(iocb, from, &f2fs_iomap_ops,
&f2fs_iomap_dio_write_ops, dio_flags, 0);
&f2fs_iomap_dio_write_ops, dio_flags, NULL, 0);
if (IS_ERR_OR_NULL(dio)) {
ret = PTR_ERR_OR_ZERO(dio);
if (ret == -ENOTBLK)

8
fs/gfs2/file.c
View File

@ -835,11 +835,12 @@ static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to,
pagefault_disable();
to->nofault = true;
ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL,
IOMAP_DIO_PARTIAL, read);
IOMAP_DIO_PARTIAL, NULL, read);
to->nofault = false;
pagefault_enable();
if (ret <= 0 && ret != -EFAULT)
goto out_unlock;
/* No increment (+=) because iomap_dio_rw returns a cumulative value. */
if (ret > 0)
read = ret;
@ -854,6 +855,7 @@ static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to,
gfs2_glock_dq(gh);
out_uninit:
gfs2_holder_uninit(gh);
/* User space doesn't expect partial success. */
if (ret < 0)
return ret;
return read;
@ -898,7 +900,7 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from,
from->nofault = true;
ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL,
IOMAP_DIO_PARTIAL, written);
IOMAP_DIO_PARTIAL, NULL, written);
from->nofault = false;
if (ret <= 0) {
if (ret == -ENOTBLK)
@ -906,6 +908,7 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from,
if (ret != -EFAULT)
goto out_unlock;
}
/* No increment (+=) because iomap_dio_rw returns a cumulative value. */
if (ret > 0)
written = ret;
@ -920,6 +923,7 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from,
gfs2_glock_dq(gh);
out_uninit:
gfs2_holder_uninit(gh);
/* User space doesn't expect partial success. */
if (ret < 0)
return ret;
return written;

Some files were not shown because too many files have changed in this diff Show More