mirror of
https://github.com/torvalds/linux.git
synced 2026-11-07 06:17:02 +01:00
- dm
- fix a bug in dm_accept_partial_bio that broke dm-writecache
in persistent-memory mode
- use assign_bit()
- dm-vdo
- fix misbehavior on invalid metadata
- dm-crypt
- use memzero_explicit
- reject malformed keys
- allow sector size up to page size
- dm-delay
- advertise flush support if the user wants to delay flushes
- dm-writecache
- reduce per-block metadata overhead by 8 bytes
- report I/O error when flush failed
- fix a quirk in statistics
- fix a crash if we attempt to use a non-dax device
- fix uncommitted_blocks underflow
- fix REQ_FUA processing
- dm-verity
- add the option DM_VERITY_VERIFY_ROOTHASH_SIG_FORCE
- clear LSM integrity state on suspend
- dm-thin
- remove unnecessary locking in pool_map
- dm-mpath
- small refactoring of the historical service time path selector
- dm-snapshot
- fix crash if dm_io_client_create fails
- make the chunk list being sorted by the chunk number
- dm-bufio
- fix non-working memory limit
- fix a race leading to an unsigned integer underflow
- clean-up cache limit reading
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRnH8MwLyZDhyYfesYTAyx9YGnhbQUCatdtwhQcbXBhdG9ja2FA
cmVkaGF0LmNvbQAKCRATAyx9YGnhbR3yAP0VGMxhAZZt4AYgEukfh5v1ivCNX10l
ldE3aRh74G7OYAEApC9G2sm5EtQsvQ+SZvnsCCKgvl5l+J7ObprpLO8adg4=
=XKui
-----END PGP SIGNATURE-----
Merge tag 'for-7.4/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm
Pull device mapper updates from Mikulas Patocka:
"dm:
- fix a bug in dm_accept_partial_bio that broke dm-writecache
in persistent-memory mode
- use assign_bit()
dm-vdo:
- fix misbehavior on invalid metadata
dm-crypt:
- use memzero_explicit
- reject malformed keys
- allow sector size up to page size
dm-delay:
- advertise flush support if the user wants to delay flushes
dm-writecache:
- reduce per-block metadata overhead by 8 bytes
- report I/O error when flush failed
- fix a quirk in statistics
- fix a crash if we attempt to use a non-dax device
- fix uncommitted_blocks underflow
- fix REQ_FUA processing
dm-verity:
- add the option DM_VERITY_VERIFY_ROOTHASH_SIG_FORCE
- clear LSM integrity state on suspend
dm-thin:
- remove unnecessary locking in pool_map
dm-mpath:
- small refactoring of the historical service time path selector
dm-snapshot:
- fix crash if dm_io_client_create fails
- make the chunk list being sorted by the chunk number
dm-bufio:
- fix non-working memory limit
- fix a race leading to an unsigned integer underflow
- clean-up cache limit reading"
* tag 'for-7.4/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (29 commits)
dm-verity: clear LSM integrity state on suspend
dm-bufio: clean-up cache size reading
dm-bufio: fix a race leading to an unsigned integer underflow
dm-bufio: fix non-working memory limit
dm-snapshot: make the chunk list sorted by the chunk number
dm-snapshot: fix crash if dm_io_client_create fails
dm-crypt: allow encryption sector size up to PAGE_SIZE
dm-mpath: check values in hst_status while holding spinlock
dm-mpath: standardize repeat_count handling in the HST path selector
dm-mpath: streamline setup work in the HST path selector
dm-writecache: fix REQ_FUA processing
dm-writecache: fix uncommitted_blocks underflow
dm-writecache: fix a crash if we attempt to use non-dax device
dm-writecache: fix a quirk in statistics
dm-writecache: report I/O error if flush failed
dm: fix a bug in dm_accept_partial_bio
dm thin: Remove unnecessary locking from pool_map()
dm-verity: add DM_VERITY_VERIFY_ROOTHASH_SIG_FORCE
dm: use assign_bit() where applicable
dm-writecache: Reduce metadata memory use by 8 bytes per cache block
...
This commit is contained in:
Linus Torvalds
commit
696e75b3a6
|
|
@ -153,9 +153,27 @@ integrity_key_size:<bytes>
|
|||
|
||||
sector_size:<bytes>
|
||||
Use <bytes> as the encryption unit instead of 512 bytes sectors.
|
||||
This option can be in range 512 - 4096 bytes and must be power of two.
|
||||
This option can be in range 512 - PAGE_SIZE bytes, further limited by
|
||||
the block layer's maximum block size, and must be power of two.
|
||||
Virtual device will announce this size as a minimal IO and logical sector.
|
||||
|
||||
An encryption unit larger than 4096 bytes can only be used on a system
|
||||
whose PAGE_SIZE is at least that large, so such a mapping is not
|
||||
portable across architectures and is unsuitable for portable on-disk
|
||||
formats such as LUKS.
|
||||
|
||||
An encryption sector larger than the unit the underlying device writes
|
||||
atomically can be torn by a power failure, leaving part of the sector
|
||||
written and part not. A device that advertises no atomic write unit
|
||||
gives no such guarantee beyond a single logical block, so this is
|
||||
already possible at 4096 bytes; a larger sector widens the window.
|
||||
With XTS and ECB the torn sector decrypts to a mixture of old and new
|
||||
data, as a torn write does on an unencrypted device. With chaining
|
||||
modes the block at the tear also decrypts to garbage, with AEAD the
|
||||
whole sector fails authentication, and with the wide-block diffusers
|
||||
the whole sector decrypts to garbage. Use a large sector only where
|
||||
losing a sector to a power failure is acceptable.
|
||||
|
||||
iv_large_sectors
|
||||
IV generators will use sector number counted in <sector_size> units
|
||||
instead of default 512 bytes sectors.
|
||||
|
|
|
|||
|
|
@ -163,6 +163,11 @@ root_hash_sig_key_desc <key_description>
|
|||
also gain new certificates at run time if they are signed by a certificate
|
||||
already in the secondary trusted keyring.
|
||||
|
||||
Whether a signature is required for every dm-verity device is controlled by
|
||||
the dm_verity.require_signatures parameter which defaults to off. Setting
|
||||
DM_VERITY_VERIFY_ROOTHASH_SIG_FORCE makes it default to on in which case it
|
||||
can no longer be turned off.
|
||||
|
||||
try_verify_in_tasklet
|
||||
If verity hashes are in cache and the IO size does not exceed the limit,
|
||||
verify data blocks in bottom half instead of workqueue. This option can
|
||||
|
|
|
|||
|
|
@ -610,6 +610,20 @@ config DM_VERITY_VERIFY_ROOTHASH_SIG_PLATFORM_KEYRING
|
|||
|
||||
If unsure, say N.
|
||||
|
||||
config DM_VERITY_VERIFY_ROOTHASH_SIG_FORCE
|
||||
bool "Require dm-verity root hash signature verification"
|
||||
depends on DM_VERITY_VERIFY_ROOTHASH_SIG
|
||||
help
|
||||
Reject dm-verity devices that are created without a valid root hash
|
||||
signature. Without this, whether a signature is required is decided
|
||||
at boot time by the dm_verity.require_signatures parameter which
|
||||
defaults to off.
|
||||
|
||||
Enabling this makes that parameter default to on and it can then no
|
||||
longer be turned off.
|
||||
|
||||
If unsure, say N.
|
||||
|
||||
config DM_VERITY_FEC
|
||||
bool "Verity forward error correction support"
|
||||
depends on DM_VERITY
|
||||
|
|
|
|||
|
|
@ -1044,12 +1044,6 @@ static unsigned long dm_bufio_default_cache_size;
|
|||
*/
|
||||
static unsigned long dm_bufio_cache_size;
|
||||
|
||||
/*
|
||||
* A copy of dm_bufio_cache_size because dm_bufio_cache_size can change
|
||||
* at any time. If it disagrees, the user has changed cache size.
|
||||
*/
|
||||
static unsigned long dm_bufio_cache_size_latch;
|
||||
|
||||
static DEFINE_SPINLOCK(global_spinlock);
|
||||
|
||||
static unsigned int dm_bufio_max_age; /* No longer does anything */
|
||||
|
|
@ -1076,7 +1070,7 @@ static int dm_bufio_client_count;
|
|||
static LIST_HEAD(dm_bufio_all_clients);
|
||||
|
||||
/*
|
||||
* This mutex protects dm_bufio_cache_size_latch and dm_bufio_client_count
|
||||
* This mutex protects dm_bufio_client_count
|
||||
*/
|
||||
static DEFINE_MUTEX(dm_bufio_clients_lock);
|
||||
|
||||
|
|
@ -1093,6 +1087,17 @@ static void buffer_record_stack(struct dm_buffer *b)
|
|||
|
||||
/*----------------------------------------------------------------*/
|
||||
|
||||
static unsigned long dm_budio_get_cache_size(void)
|
||||
{
|
||||
unsigned long s = READ_ONCE(dm_bufio_cache_size);
|
||||
if (unlikely(!s)) {
|
||||
unsigned long d = dm_bufio_default_cache_size;
|
||||
(void)cmpxchg(&dm_bufio_cache_size, 0, d);
|
||||
s = d;
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
static void adjust_total_allocated(struct dm_buffer *b, bool unlink)
|
||||
{
|
||||
unsigned char data_mode;
|
||||
|
|
@ -1120,35 +1125,13 @@ static void adjust_total_allocated(struct dm_buffer *b, bool unlink)
|
|||
dm_bufio_peak_allocated = dm_bufio_current_allocated;
|
||||
|
||||
if (!unlink) {
|
||||
if (dm_bufio_current_allocated > dm_bufio_cache_size)
|
||||
if (dm_bufio_current_allocated > dm_budio_get_cache_size())
|
||||
queue_work(dm_bufio_wq, &dm_bufio_replacement_work);
|
||||
}
|
||||
|
||||
spin_unlock(&global_spinlock);
|
||||
}
|
||||
|
||||
/*
|
||||
* Change the number of clients and recalculate per-client limit.
|
||||
*/
|
||||
static void __cache_size_refresh(void)
|
||||
{
|
||||
if (WARN_ON(!mutex_is_locked(&dm_bufio_clients_lock)))
|
||||
return;
|
||||
if (WARN_ON(dm_bufio_client_count < 0))
|
||||
return;
|
||||
|
||||
dm_bufio_cache_size_latch = READ_ONCE(dm_bufio_cache_size);
|
||||
|
||||
/*
|
||||
* Use default if set to 0 and report the actual cache size used.
|
||||
*/
|
||||
if (!dm_bufio_cache_size_latch) {
|
||||
(void)cmpxchg(&dm_bufio_cache_size, 0,
|
||||
dm_bufio_default_cache_size);
|
||||
dm_bufio_cache_size_latch = dm_bufio_default_cache_size;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* Allocating buffer data.
|
||||
*
|
||||
|
|
@ -1618,7 +1601,7 @@ static struct dm_buffer *__alloc_buffer_wait_no_callback(struct dm_bufio_client
|
|||
* be allocated.
|
||||
*/
|
||||
while (1) {
|
||||
if (dm_bufio_cache_size_latch != 1) {
|
||||
if (dm_budio_get_cache_size() != 1) {
|
||||
b = alloc_buffer(c, GFP_NOWAIT | __GFP_NORETRY | __GFP_NOMEMALLOC);
|
||||
if (b)
|
||||
return b;
|
||||
|
|
@ -1627,7 +1610,7 @@ static struct dm_buffer *__alloc_buffer_wait_no_callback(struct dm_bufio_client
|
|||
if (nf == NF_PREFETCH)
|
||||
return NULL;
|
||||
|
||||
if (dm_bufio_cache_size_latch != 1 && !tried_noio_alloc) {
|
||||
if (dm_budio_get_cache_size() != 1 && !tried_noio_alloc) {
|
||||
dm_bufio_unlock(c);
|
||||
b = alloc_buffer(c, GFP_NOIO | __GFP_NORETRY | __GFP_NOMEMALLOC | __GFP_NOWARN);
|
||||
dm_bufio_lock(c);
|
||||
|
|
@ -2426,8 +2409,10 @@ static void __scan(struct dm_bufio_client *c)
|
|||
|
||||
for (l = 0; l < LIST_SIZE; l++) {
|
||||
while (true) {
|
||||
if (count - freed <= retain_target)
|
||||
if (count - freed <= retain_target) {
|
||||
atomic_long_set(&c->need_shrink, 0);
|
||||
break;
|
||||
}
|
||||
if (!atomic_long_read(&c->need_shrink))
|
||||
break;
|
||||
|
||||
|
|
@ -2545,6 +2530,13 @@ struct dm_bufio_client *dm_bufio_client_create(struct block_device *bdev, unsign
|
|||
init_waitqueue_head(&c->free_buffer_wait);
|
||||
c->async_write_error = 0;
|
||||
|
||||
/*
|
||||
* A client with no buffers yet is not holding the oldest buffer in
|
||||
* the system; leaving this zeroed would claim exactly that and keep
|
||||
* the client pinned at the front of dm_bufio_all_clients.
|
||||
*/
|
||||
c->oldest_buffer = jiffies;
|
||||
|
||||
c->dm_io = dm_io_client_create();
|
||||
if (IS_ERR(c->dm_io)) {
|
||||
r = PTR_ERR(c->dm_io);
|
||||
|
|
@ -2607,7 +2599,6 @@ struct dm_bufio_client *dm_bufio_client_create(struct block_device *bdev, unsign
|
|||
mutex_lock(&dm_bufio_clients_lock);
|
||||
dm_bufio_client_count++;
|
||||
list_add(&c->client_list, &dm_bufio_all_clients);
|
||||
__cache_size_refresh();
|
||||
mutex_unlock(&dm_bufio_clients_lock);
|
||||
|
||||
return c;
|
||||
|
|
@ -2649,7 +2640,6 @@ void dm_bufio_client_destroy(struct dm_bufio_client *c)
|
|||
|
||||
list_del(&c->client_list);
|
||||
dm_bufio_client_count--;
|
||||
__cache_size_refresh();
|
||||
|
||||
mutex_unlock(&dm_bufio_clients_lock);
|
||||
|
||||
|
|
@ -2779,9 +2769,23 @@ static unsigned long __evict_a_few(unsigned long nr_buffers)
|
|||
|
||||
dm_bufio_unlock(c);
|
||||
|
||||
if (count)
|
||||
if (count) {
|
||||
c->oldest_buffer = oldest_buffer;
|
||||
__insert_client(c);
|
||||
__insert_client(c);
|
||||
} else {
|
||||
/*
|
||||
* We evicted nothing, so 'oldest_buffer' tells us nothing new
|
||||
* and the client keeps its old, possibly very stale, value.
|
||||
* Requeue it at the back rather than by that value, otherwise
|
||||
* __insert_client() would put it straight back at the front
|
||||
* and every subsequent call would pop this same client again,
|
||||
* never reaching any of the others.
|
||||
*
|
||||
* This leaves the list slightly out of order, which is fine -
|
||||
* it is only ever an approximation.
|
||||
*/
|
||||
list_add_tail(&c->client_list, &dm_bufio_all_clients);
|
||||
}
|
||||
|
||||
return count;
|
||||
}
|
||||
|
|
@ -2804,13 +2808,25 @@ static void check_watermarks(void)
|
|||
|
||||
static void evict_old(void)
|
||||
{
|
||||
unsigned long threshold = dm_bufio_cache_size -
|
||||
dm_bufio_cache_size / DM_BUFIO_LOW_WATERMARK_RATIO;
|
||||
unsigned long cs = dm_budio_get_cache_size();
|
||||
unsigned long threshold = cs - cs / DM_BUFIO_LOW_WATERMARK_RATIO;
|
||||
int unproductive = 0;
|
||||
|
||||
mutex_lock(&dm_bufio_clients_lock);
|
||||
while (dm_bufio_current_allocated > threshold) {
|
||||
if (!__evict_a_few(64))
|
||||
break;
|
||||
if (__evict_a_few(64)) {
|
||||
unproductive = 0;
|
||||
} else {
|
||||
/*
|
||||
* This client had nothing to give. Try the others
|
||||
* before giving up: one idle client, or one whose
|
||||
* buffers are all held, must not stop us evicting
|
||||
* from everybody else. Stop once a whole sweep of
|
||||
* the client list has freed nothing.
|
||||
*/
|
||||
if (++unproductive >= dm_bufio_client_count)
|
||||
break;
|
||||
}
|
||||
cond_resched();
|
||||
}
|
||||
mutex_unlock(&dm_bufio_clients_lock);
|
||||
|
|
@ -2854,10 +2870,8 @@ static int __init dm_bufio_init(void)
|
|||
#endif
|
||||
|
||||
dm_bufio_default_cache_size = mem;
|
||||
|
||||
mutex_lock(&dm_bufio_clients_lock);
|
||||
__cache_size_refresh();
|
||||
mutex_unlock(&dm_bufio_clients_lock);
|
||||
if (!dm_bufio_cache_size)
|
||||
dm_bufio_cache_size = mem;
|
||||
|
||||
dm_bufio_wq = alloc_workqueue("dm_bufio_cache",
|
||||
WQ_MEM_RECLAIM | WQ_PERCPU, 0);
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ enum superblock_flag_bits {
|
|||
enum mapping_bits {
|
||||
/*
|
||||
* A valid mapping. Because we're using an array we clear this
|
||||
* flag for an non existant mapping.
|
||||
* flag for an non existent mapping.
|
||||
*/
|
||||
M_VALID = 1,
|
||||
|
||||
|
|
|
|||
|
|
@ -300,9 +300,6 @@ static void __prepare_superblock(struct dm_clone_metadata *cmd,
|
|||
{
|
||||
sb->flags = cpu_to_le32(0UL);
|
||||
|
||||
/* FIXME: UUID is currently unused */
|
||||
memset(sb->uuid, 0, sizeof(sb->uuid));
|
||||
|
||||
sb->magic = cpu_to_le64(SUPERBLOCK_MAGIC);
|
||||
sb->version = cpu_to_le32(DM_CLONE_MAX_METADATA_VERSION);
|
||||
|
||||
|
|
|
|||
|
|
@ -182,7 +182,7 @@ struct crypt_config {
|
|||
} iv_gen_private;
|
||||
u64 iv_offset;
|
||||
unsigned int iv_size;
|
||||
unsigned short sector_size;
|
||||
unsigned int sector_size;
|
||||
unsigned char sector_shift;
|
||||
|
||||
union {
|
||||
|
|
@ -241,6 +241,24 @@ struct crypt_config {
|
|||
#define MAX_TAG_SIZE 480
|
||||
#define POOL_ENTRY_SIZE 512
|
||||
|
||||
/*
|
||||
* Largest encryption sector size that can be requested with the
|
||||
* "sector_size:<bytes>" option.
|
||||
*
|
||||
* A sector is handed to the crypto API as a single scatterlist entry, so it
|
||||
* has to be covered by one bio_vec. bio_iter_iovec() never returns more than
|
||||
* PAGE_SIZE bytes, and crypt_alloc_buffer() may fall back to order-0 pages
|
||||
* for the write bounce buffer, so PAGE_SIZE is the ceiling.
|
||||
*
|
||||
* crypt_io_hints() announces the sector size as the logical block size, which
|
||||
* the block layer caps at BLK_MAX_BLOCK_SIZE. That cap is never below
|
||||
* PAGE_SIZE in any configuration today, so it does not lower the limit; take
|
||||
* the minimum anyway so that this target cannot announce a block size
|
||||
* blk_validate_limits() would reject.
|
||||
*/
|
||||
#define DM_CRYPT_MAX_SECTOR_SIZE MIN_T(unsigned int, PAGE_SIZE, \
|
||||
BLK_MAX_BLOCK_SIZE)
|
||||
|
||||
static DEFINE_SPINLOCK(dm_crypt_clients_lock);
|
||||
static unsigned int dm_crypt_clients_n;
|
||||
static volatile unsigned long dm_crypt_pages_per_client;
|
||||
|
|
@ -513,7 +531,7 @@ static void crypt_iv_lmk_wipe(struct crypt_config *cc)
|
|||
struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
|
||||
|
||||
if (lmk->seed)
|
||||
memset(lmk->seed, 0, LMK_SEED_SIZE);
|
||||
memzero_explicit(lmk->seed, LMK_SEED_SIZE);
|
||||
}
|
||||
|
||||
static void crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
|
||||
|
|
@ -630,8 +648,8 @@ static void crypt_iv_tcw_wipe(struct crypt_config *cc)
|
|||
{
|
||||
struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
|
||||
|
||||
memset(tcw->iv_seed, 0, cc->iv_size);
|
||||
memset(tcw->whitening, 0, TCW_WHITENING_SIZE);
|
||||
memzero_explicit(tcw->iv_seed, cc->iv_size);
|
||||
memzero_explicit(tcw->whitening, TCW_WHITENING_SIZE);
|
||||
}
|
||||
|
||||
static void crypt_iv_tcw_whitening(struct crypt_config *cc,
|
||||
|
|
@ -2365,7 +2383,7 @@ static unsigned int crypt_authenckey_size(struct crypt_config *cc)
|
|||
/*
|
||||
* If AEAD is composed like authenc(hmac(sha256),xts(aes)),
|
||||
* the key must be for some reason in special format.
|
||||
* This funcion converts cc->key to this special format.
|
||||
* This function converts cc->key to this special format.
|
||||
*/
|
||||
static void crypt_copy_authenckey(char *p, const void *key,
|
||||
unsigned int enckeylen, unsigned int authkeylen)
|
||||
|
|
@ -2595,7 +2613,7 @@ static int get_key_size(char **key_string)
|
|||
static int crypt_set_key(struct crypt_config *cc, char *key)
|
||||
{
|
||||
int r = -EINVAL;
|
||||
int key_string_len = strlen(key);
|
||||
size_t key_string_len = strlen(key);
|
||||
|
||||
/* Hyphen (which gives a key_size of zero) means there is no key. */
|
||||
if (!cc->key_size && strcmp(key, "-"))
|
||||
|
|
@ -2614,6 +2632,9 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
|
|||
kfree_sensitive(cc->key_string);
|
||||
cc->key_string = NULL;
|
||||
|
||||
if (cc->key_size && key_string_len != cc->key_size * 2)
|
||||
goto out;
|
||||
|
||||
/* Decode key from its hex representation. */
|
||||
if (cc->key_size && hex2bin(cc->key, key, cc->key_size) < 0)
|
||||
goto out;
|
||||
|
|
@ -2624,7 +2645,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
|
|||
|
||||
out:
|
||||
/* Hex key string not needed after here, so wipe it. */
|
||||
memset(key, '0', key_string_len);
|
||||
memzero_explicit(key, key_string_len);
|
||||
|
||||
return r;
|
||||
}
|
||||
|
|
@ -2643,7 +2664,7 @@ static int crypt_wipe_key(struct crypt_config *cc)
|
|||
kfree_sensitive(cc->key_string);
|
||||
cc->key_string = NULL;
|
||||
r = crypt_setkey(cc);
|
||||
memset(&cc->key, 0, cc->key_size * sizeof(u8));
|
||||
memzero_explicit(cc->key, cc->key_size);
|
||||
|
||||
return r;
|
||||
}
|
||||
|
|
@ -3069,7 +3090,7 @@ static int crypt_ctr_cipher(struct dm_target *ti, char *cipher_in, char *key)
|
|||
|
||||
/* wipe the kernel key payload copy */
|
||||
if (cc->key_string)
|
||||
memset(cc->key, 0, cc->key_size * sizeof(u8));
|
||||
memzero_explicit(cc->key, cc->key_size);
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
|
@ -3139,9 +3160,9 @@ static int crypt_ctr_optional(struct dm_target *ti, unsigned int argc, char **ar
|
|||
}
|
||||
cc->key_mac_size = val;
|
||||
set_bit(CRYPT_KEY_MAC_SIZE_SET, &cc->cipher_flags);
|
||||
} else if (sscanf(opt_string, "sector_size:%hu%c", &cc->sector_size, &dummy) == 1) {
|
||||
} else if (sscanf(opt_string, "sector_size:%u%c", &cc->sector_size, &dummy) == 1) {
|
||||
if (cc->sector_size < (1 << SECTOR_SHIFT) ||
|
||||
cc->sector_size > 4096 ||
|
||||
cc->sector_size > DM_CRYPT_MAX_SECTOR_SIZE ||
|
||||
(cc->sector_size & (cc->sector_size - 1))) {
|
||||
ti->error = "Invalid feature value for sector_size";
|
||||
return -EINVAL;
|
||||
|
|
@ -3561,7 +3582,7 @@ static void crypt_status(struct dm_target *ti, status_type_t type,
|
|||
if (cc->used_tag_size)
|
||||
DMEMIT(" integrity:%u:%s", cc->used_tag_size, cc->cipher_auth);
|
||||
if (cc->sector_size != (1 << SECTOR_SHIFT))
|
||||
DMEMIT(" sector_size:%d", cc->sector_size);
|
||||
DMEMIT(" sector_size:%u", cc->sector_size);
|
||||
if (test_bit(CRYPT_IV_LARGE_SECTORS, &cc->cipher_flags))
|
||||
DMEMIT(" iv_large_sectors");
|
||||
if (test_bit(CRYPT_KEY_MAC_SIZE_SET, &cc->cipher_flags))
|
||||
|
|
@ -3587,7 +3608,7 @@ static void crypt_status(struct dm_target *ti, status_type_t type,
|
|||
DMEMIT(",integrity_tag_size=%u,cipher_auth=%s",
|
||||
cc->used_tag_size, cc->cipher_auth);
|
||||
if (cc->sector_size != (1 << SECTOR_SHIFT))
|
||||
DMEMIT(",sector_size=%d", cc->sector_size);
|
||||
DMEMIT(",sector_size=%u", cc->sector_size);
|
||||
if (cc->cipher_string)
|
||||
DMEMIT(",cipher_string=%s", cc->cipher_string);
|
||||
|
||||
|
|
@ -3648,7 +3669,7 @@ static int crypt_message(struct dm_target *ti, unsigned int argc, char **argv,
|
|||
/* The key size may not be changed. */
|
||||
key_size = get_key_size(&argv[2]);
|
||||
if (key_size < 0 || cc->key_size != key_size) {
|
||||
memset(argv[2], '0', strlen(argv[2]));
|
||||
memzero_explicit(argv[2], strlen(argv[2]));
|
||||
return -EINVAL;
|
||||
}
|
||||
|
||||
|
|
@ -3659,7 +3680,7 @@ static int crypt_message(struct dm_target *ti, unsigned int argc, char **argv,
|
|||
ret = cc->iv_gen_ops->init(cc);
|
||||
/* wipe the kernel key payload copy */
|
||||
if (cc->key_string)
|
||||
memset(cc->key, 0, cc->key_size * sizeof(u8));
|
||||
memzero_explicit(cc->key, cc->key_size);
|
||||
return ret;
|
||||
}
|
||||
if (argc == 2 && !strcasecmp(argv[1], "wipe"))
|
||||
|
|
@ -3705,7 +3726,7 @@ static void crypt_io_hints(struct dm_target *ti, struct queue_limits *limits)
|
|||
|
||||
static struct target_type crypt_target = {
|
||||
.name = "crypt",
|
||||
.version = {1, 29, 0},
|
||||
.version = {1, 30, 0},
|
||||
.module = THIS_MODULE,
|
||||
.ctr = crypt_ctr,
|
||||
.dtr = crypt_dtr,
|
||||
|
|
|
|||
|
|
@ -301,6 +301,7 @@ static int delay_ctr(struct dm_target *ti, unsigned int argc, char **argv)
|
|||
}
|
||||
|
||||
ti->num_flush_bios = 1;
|
||||
ti->flush_supported = dc->flush.delay != 0;
|
||||
ti->num_discard_bios = 1;
|
||||
ti->accounts_remapped_io = true;
|
||||
ti->per_io_data_size = sizeof(struct dm_delay_info);
|
||||
|
|
@ -451,7 +452,7 @@ static int delay_iterate_devices(struct dm_target *ti,
|
|||
|
||||
static struct target_type delay_target = {
|
||||
.name = "delay",
|
||||
.version = {1, 5, 0},
|
||||
.version = {1, 6, 0},
|
||||
.features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_ZONED_HM,
|
||||
.module = THIS_MODULE,
|
||||
.ctr = delay_ctr,
|
||||
|
|
|
|||
|
|
@ -135,7 +135,7 @@ struct dm_dev *dm_snap_origin(struct dm_snapshot *snap);
|
|||
struct dm_dev *dm_snap_cow(struct dm_snapshot *snap);
|
||||
|
||||
/*
|
||||
* Funtions to manipulate consecutive chunks
|
||||
* Functions to manipulate consecutive chunks
|
||||
*/
|
||||
#define DM_CHUNK_CONSECUTIVE_BITS 8
|
||||
#define DM_CHUNK_NUMBER_BITS 56
|
||||
|
|
|
|||
|
|
@ -1604,10 +1604,7 @@ static void pg_init_done(void *data, int errors)
|
|||
goto out;
|
||||
|
||||
if (test_bit(MPATHF_PG_INIT_REQUIRED, &m->flags)) {
|
||||
if (delay_retry)
|
||||
set_bit(MPATHF_PG_INIT_DELAY_RETRY, &m->flags);
|
||||
else
|
||||
clear_bit(MPATHF_PG_INIT_DELAY_RETRY, &m->flags);
|
||||
assign_bit(MPATHF_PG_INIT_DELAY_RETRY, &m->flags, delay_retry);
|
||||
|
||||
if (__pg_init_all_paths(m))
|
||||
goto out;
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ void backing_dev_stop(struct dm_pcache *pcache)
|
|||
struct pcache_backing_dev *backing_dev = &pcache->backing_dev;
|
||||
|
||||
/*
|
||||
* There should not be any new request comming, just wait
|
||||
* There should not be any new request coming, just wait
|
||||
* inflight requests done.
|
||||
*/
|
||||
wait_event(backing_dev->inflight_wq,
|
||||
|
|
|
|||
|
|
@ -31,11 +31,12 @@
|
|||
|
||||
#define DM_MSG_PREFIX "multipath historical-service-time"
|
||||
#define HST_MIN_IO 1
|
||||
#define HST_VERSION "0.1.1"
|
||||
#define HST_VERSION "0.2.0"
|
||||
|
||||
#define HST_FIXED_SHIFT 10 /* 10 bits of decimal precision */
|
||||
#define HST_FIXED_MAX (ULLONG_MAX >> HST_FIXED_SHIFT)
|
||||
#define HST_FIXED_1 (1 << HST_FIXED_SHIFT)
|
||||
#define HST_FIXED_POINT_5 (1 << (HST_FIXED_SHIFT - 1))
|
||||
#define HST_FIXED_95 972
|
||||
#define HST_MAX_INFLIGHT HST_FIXED_1
|
||||
#define HST_BUCKET_SHIFT 24 /* Buckets are ~ 16ms */
|
||||
|
|
@ -66,46 +67,6 @@ struct path_info {
|
|||
u64 outstanding;
|
||||
};
|
||||
|
||||
/**
|
||||
* fixed_power - compute: x^n, in O(log n) time
|
||||
*
|
||||
* @x: base of the power
|
||||
* @frac_bits: fractional bits of @x
|
||||
* @n: power to raise @x to.
|
||||
*
|
||||
* By exploiting the relation between the definition of the natural power
|
||||
* function: x^n := x*x*...*x (x multiplied by itself for n times), and
|
||||
* the binary encoding of numbers used by computers: n := \Sum n_i * 2^i,
|
||||
* (where: n_i \elem {0, 1}, the binary vector representing n),
|
||||
* we find: x^n := x^(\Sum n_i * 2^i) := \Prod x^(n_i * 2^i), which is
|
||||
* of course trivially computable in O(log_2 n), the length of our binary
|
||||
* vector.
|
||||
*
|
||||
* (see: kernel/sched/loadavg.c)
|
||||
*/
|
||||
static u64 fixed_power(u64 x, unsigned int frac_bits, unsigned int n)
|
||||
{
|
||||
unsigned long result = 1UL << frac_bits;
|
||||
|
||||
if (n) {
|
||||
for (;;) {
|
||||
if (n & 1) {
|
||||
result *= x;
|
||||
result += 1UL << (frac_bits - 1);
|
||||
result >>= frac_bits;
|
||||
}
|
||||
n >>= 1;
|
||||
if (!n)
|
||||
break;
|
||||
x *= x;
|
||||
x += 1UL << (frac_bits - 1);
|
||||
x >>= frac_bits;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/*
|
||||
* Calculate the next value of an exponential moving average
|
||||
* a_1 = a_0 * e + a * (1 - e)
|
||||
|
|
@ -123,7 +84,7 @@ static u64 fixed_ema(u64 last, u64 next, u64 weight)
|
|||
{
|
||||
last *= weight;
|
||||
last += next * (HST_FIXED_1 - weight);
|
||||
last += 1ULL << (HST_FIXED_SHIFT - 1);
|
||||
last += HST_FIXED_POINT_5;
|
||||
return last >> HST_FIXED_SHIFT;
|
||||
}
|
||||
|
||||
|
|
@ -167,8 +128,10 @@ static void hst_set_weights(struct path_selector *ps, unsigned int base)
|
|||
if (base >= HST_FIXED_1)
|
||||
return;
|
||||
|
||||
for (i = 0; i < HST_WEIGHT_COUNT - 1; i++)
|
||||
s->weights[i] = fixed_power(base, HST_FIXED_SHIFT, i + 1);
|
||||
s->weights[0] = base;
|
||||
for (i = 1; i < HST_WEIGHT_COUNT - 1; i++)
|
||||
s->weights[i] = (base * s->weights[i - 1] +
|
||||
HST_FIXED_POINT_5) >> HST_FIXED_SHIFT;
|
||||
s->weights[HST_WEIGHT_COUNT - 1] = 0;
|
||||
}
|
||||
|
||||
|
|
@ -239,6 +202,7 @@ static int hst_status(struct path_selector *ps, struct dm_path *path,
|
|||
{
|
||||
unsigned int sz = 0;
|
||||
struct path_info *pi;
|
||||
unsigned long flags;
|
||||
|
||||
if (!path) {
|
||||
struct selector *s = ps->context;
|
||||
|
|
@ -249,11 +213,13 @@ static int hst_status(struct path_selector *ps, struct dm_path *path,
|
|||
|
||||
switch (type) {
|
||||
case STATUSTYPE_INFO:
|
||||
spin_lock_irqsave(&pi->lock, flags);
|
||||
DMEMIT("%llu %llu %llu ", pi->historical_service_time,
|
||||
pi->outstanding, pi->stale_after);
|
||||
spin_unlock_irqrestore(&pi->lock, flags);
|
||||
break;
|
||||
case STATUSTYPE_TABLE:
|
||||
DMEMIT("0 ");
|
||||
DMEMIT("%u ", pi->repeat_count);
|
||||
break;
|
||||
case STATUSTYPE_IMA:
|
||||
*result = '\0';
|
||||
|
|
@ -288,6 +254,11 @@ static int hst_add_path(struct path_selector *ps, struct dm_path *path,
|
|||
return -EINVAL;
|
||||
}
|
||||
|
||||
if (repeat_count > 1) {
|
||||
DMWARN_LIMIT("repeat_count > 1 is deprecated, using 1 instead");
|
||||
repeat_count = 1;
|
||||
}
|
||||
|
||||
/* allocate the path */
|
||||
pi = kmalloc_obj(*pi);
|
||||
if (!pi) {
|
||||
|
|
|
|||
|
|
@ -1784,7 +1784,7 @@ static int rs_check_takeover(struct raid_set *rs)
|
|||
|
||||
/* raid10* -> raid0 */
|
||||
if (mddev->new_level == 0) {
|
||||
/* Can takeover raid10_near with raid disks divisable by data copies! */
|
||||
/* Can takeover raid10_near with raid disks divisible by data copies! */
|
||||
if (near_copies > 1 &&
|
||||
!(mddev->raid_disks % near_copies)) {
|
||||
mddev->raid_disks /= near_copies;
|
||||
|
|
|
|||
|
|
@ -1393,7 +1393,7 @@ static void mirror_resume(struct dm_target *ti)
|
|||
* we have encountered.
|
||||
* A => Alive - No failures
|
||||
* D => Dead - A write failure occurred leaving mirror out-of-sync
|
||||
* S => Sync - A sychronization failure occurred, mirror out-of-sync
|
||||
* S => Sync - A synchronization failure occurred, mirror out-of-sync
|
||||
* R => Read - A read failure occurred, mirror data unaffected
|
||||
*
|
||||
* Returns: <char>
|
||||
|
|
|
|||
|
|
@ -325,8 +325,11 @@ static int read_header(struct pstore *ps, int *new_snapshot)
|
|||
}
|
||||
|
||||
ps->io_client = dm_io_client_create();
|
||||
if (IS_ERR(ps->io_client))
|
||||
return PTR_ERR(ps->io_client);
|
||||
if (IS_ERR(ps->io_client)) {
|
||||
r = PTR_ERR(ps->io_client);
|
||||
ps->io_client = NULL;
|
||||
return r;
|
||||
}
|
||||
|
||||
r = alloc_area(ps);
|
||||
if (r)
|
||||
|
|
|
|||
|
|
@ -766,7 +766,7 @@ static void dm_insert_exception(struct dm_exception_table *eh,
|
|||
struct dm_exception *new_e)
|
||||
{
|
||||
struct hlist_head *l;
|
||||
struct dm_exception *e = NULL;
|
||||
struct dm_exception *e = NULL, *prev = NULL;
|
||||
|
||||
l = &eh->table[exception_hash(eh, new_e->old_chunk)].head;
|
||||
|
||||
|
|
@ -798,21 +798,23 @@ static void dm_insert_exception(struct dm_exception_table *eh,
|
|||
|
||||
if (new_e->old_chunk < e->old_chunk)
|
||||
break;
|
||||
|
||||
prev = e;
|
||||
}
|
||||
|
||||
out:
|
||||
if (!e) {
|
||||
if (e && new_e->old_chunk < e->old_chunk) {
|
||||
/* Add before an existing exception */
|
||||
hlist_add_before(&new_e->hash_list, &e->hash_list);
|
||||
} else if (prev) {
|
||||
/* Add to l's tail: prev is the last exception in this slot */
|
||||
hlist_add_behind(&new_e->hash_list, &prev->hash_list);
|
||||
} else {
|
||||
/*
|
||||
* Either the table doesn't support consecutive chunks or slot
|
||||
* l is empty.
|
||||
*/
|
||||
hlist_add_head(&new_e->hash_list, l);
|
||||
} else if (new_e->old_chunk < e->old_chunk) {
|
||||
/* Add before an existing exception */
|
||||
hlist_add_before(&new_e->hash_list, &e->hash_list);
|
||||
} else {
|
||||
/* Add to l's tail: e is the last exception in this slot */
|
||||
hlist_add_behind(&new_e->hash_list, &e->hash_list);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -3438,14 +3438,11 @@ static int pool_ctr(struct dm_target *ti, unsigned int argc, char **argv)
|
|||
static int pool_map(struct dm_target *ti, struct bio *bio)
|
||||
{
|
||||
struct pool_c *pt = ti->private;
|
||||
struct pool *pool = pt->pool;
|
||||
|
||||
/*
|
||||
* As this is a singleton target, ti->begin is always zero.
|
||||
*/
|
||||
spin_lock_irq(&pool->lock);
|
||||
bio_set_dev(bio, pt->data_dev->bdev);
|
||||
spin_unlock_irq(&pool->lock);
|
||||
|
||||
return DM_MAPIO_REMAPPED;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ static void cleanup_unstripe(struct unstripe_c *uc, struct dm_target *ti)
|
|||
}
|
||||
|
||||
/*
|
||||
* Contruct an unstriped mapping.
|
||||
* Construct an unstriped mapping.
|
||||
* <number of stripes> <chunk size> <stripe #> <dev_path> <offset>
|
||||
*/
|
||||
static int unstripe_ctr(struct dm_target *ti, unsigned int argc, char **argv)
|
||||
|
|
|
|||
|
|
@ -240,7 +240,7 @@ static int __must_check allocate_cache_components(struct vdo_page_cache *cache)
|
|||
* assert_on_cache_thread() - Assert that a function has been called on the VDO page cache's
|
||||
* thread.
|
||||
* @cache: The page cache.
|
||||
* @function_name: The funtion name to report if the assertion fails.
|
||||
* @function_name: The function name to report if the assertion fails.
|
||||
*/
|
||||
static inline void assert_on_cache_thread(struct vdo_page_cache *cache,
|
||||
const char *function_name)
|
||||
|
|
|
|||
|
|
@ -443,7 +443,7 @@ static int __must_check parse_slab_size(const char *slab_str, block_count_t *sla
|
|||
|
||||
result = kstrtoull(slab_str, 10, &value);
|
||||
if (result) {
|
||||
vdo_log_error("optional parameter error: invalid slab size, must be a postive integer");
|
||||
vdo_log_error("optional parameter error: invalid slab size, must be a positive integer");
|
||||
return -EINVAL;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -284,10 +284,17 @@ int uds_search_chapter_index_page(struct delta_index_page *index_page,
|
|||
if (result != UDS_SUCCESS)
|
||||
return result;
|
||||
|
||||
if (was_entry_found(&entry, address))
|
||||
*record_page_ptr = uds_get_delta_entry_value(&entry);
|
||||
else
|
||||
if (!was_entry_found(&entry, address)) {
|
||||
*record_page_ptr = NO_CHAPTER_INDEX_ENTRY;
|
||||
return UDS_SUCCESS;
|
||||
}
|
||||
|
||||
*record_page_ptr = uds_get_delta_entry_value(&entry);
|
||||
result = VDO_ASSERT(*record_page_ptr < geometry->record_pages_per_chapter,
|
||||
"0 <= %d < %u", *record_page_ptr,
|
||||
geometry->record_pages_per_chapter);
|
||||
if (result != VDO_SUCCESS)
|
||||
*record_page_ptr = NO_CHAPTER_INDEX_ENTRY;
|
||||
|
||||
return UDS_SUCCESS;
|
||||
return result;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -800,7 +800,7 @@ static void compute_new_list_offsets(struct delta_zone *delta_zone, u32 growing_
|
|||
(delta_zone->size * BITS_PER_BYTE - delta_lists[tail_guard_index].size);
|
||||
}
|
||||
|
||||
static void rebalance_lists(struct delta_zone *delta_zone)
|
||||
static int rebalance_lists(struct delta_zone *delta_zone)
|
||||
{
|
||||
struct delta_list *delta_lists;
|
||||
u32 i;
|
||||
|
|
@ -811,9 +811,17 @@ static void rebalance_lists(struct delta_zone *delta_zone)
|
|||
for (i = 0; i <= delta_zone->list_count + 1; i++)
|
||||
used_space += get_delta_list_byte_size(&delta_lists[i]);
|
||||
|
||||
if (delta_zone->size < used_space) {
|
||||
return vdo_log_warning_strerror(UDS_CORRUPT_DATA,
|
||||
"delta zone lists overflow zone size %zu",
|
||||
delta_zone->size);
|
||||
}
|
||||
|
||||
compute_new_list_offsets(delta_zone, 0, 0, used_space);
|
||||
for (i = 1; i <= delta_zone->list_count + 1; i++)
|
||||
delta_lists[i].start = delta_zone->new_offsets[i];
|
||||
|
||||
return UDS_SUCCESS;
|
||||
}
|
||||
|
||||
/* Start restoring a delta index from multiple input streams. */
|
||||
|
|
@ -941,8 +949,11 @@ int uds_start_restoring_delta_index(struct delta_index *delta_index,
|
|||
}
|
||||
|
||||
/* Prepare each zone to start receiving the delta list data. */
|
||||
for (z = 0; z < delta_index->zone_count; z++)
|
||||
rebalance_lists(&delta_index->delta_zones[z]);
|
||||
for (z = 0; z < delta_index->zone_count; z++) {
|
||||
result = rebalance_lists(&delta_index->delta_zones[z]);
|
||||
if (result != UDS_SUCCESS)
|
||||
return result;
|
||||
}
|
||||
|
||||
return UDS_SUCCESS;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -891,12 +891,6 @@ int uds_search_cached_record_page(struct volume *volume, struct uds_request *req
|
|||
if (record_page_number == NO_CHAPTER_INDEX_ENTRY)
|
||||
return UDS_SUCCESS;
|
||||
|
||||
result = VDO_ASSERT(record_page_number < geometry->record_pages_per_chapter,
|
||||
"0 <= %d < %u", record_page_number,
|
||||
geometry->record_pages_per_chapter);
|
||||
if (result != VDO_SUCCESS)
|
||||
return result;
|
||||
|
||||
page_number = geometry->index_pages_per_chapter + record_page_number;
|
||||
|
||||
physical_page = map_to_physical_page(&volume->geometry, chapter, page_number);
|
||||
|
|
|
|||
|
|
@ -825,13 +825,6 @@ static int verity_map(struct dm_target *ti, struct bio *bio)
|
|||
return DM_MAPIO_SUBMITTED;
|
||||
}
|
||||
|
||||
static void verity_postsuspend(struct dm_target *ti)
|
||||
{
|
||||
struct dm_verity *v = ti->private;
|
||||
flush_workqueue(v->verify_wq);
|
||||
dm_bufio_client_reset(v->bufio);
|
||||
}
|
||||
|
||||
/*
|
||||
* Status: V (valid) or C (corruption found)
|
||||
*/
|
||||
|
|
@ -1733,6 +1726,11 @@ static int verity_security_set_signature(struct block_device *bdev,
|
|||
v->sig_size);
|
||||
}
|
||||
|
||||
static void verity_security_clear_signature(struct block_device *bdev)
|
||||
{
|
||||
security_bdev_setintegrity(bdev, LSM_INT_DMVERITY_SIG_VALID, NULL, 0);
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
static inline int verity_security_set_signature(struct block_device *bdev,
|
||||
|
|
@ -1741,6 +1739,10 @@ static inline int verity_security_set_signature(struct block_device *bdev,
|
|||
return 0;
|
||||
}
|
||||
|
||||
static inline void verity_security_clear_signature(struct block_device *bdev)
|
||||
{
|
||||
}
|
||||
|
||||
#endif /* CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG */
|
||||
|
||||
/*
|
||||
|
|
@ -1779,8 +1781,31 @@ static int verity_preresume(struct dm_target *ti)
|
|||
return r;
|
||||
}
|
||||
|
||||
static void verity_security_clear_integrity(struct dm_target *ti)
|
||||
{
|
||||
struct block_device *bdev = dm_disk(dm_table_get_md(ti->table))->part0;
|
||||
|
||||
security_bdev_setintegrity(bdev, LSM_INT_DMVERITY_ROOTHASH, NULL, 0);
|
||||
verity_security_clear_signature(bdev);
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
static inline void verity_security_clear_integrity(struct dm_target *ti)
|
||||
{
|
||||
}
|
||||
|
||||
#endif /* CONFIG_SECURITY */
|
||||
|
||||
static void verity_postsuspend(struct dm_target *ti)
|
||||
{
|
||||
struct dm_verity *v = ti->private;
|
||||
|
||||
flush_workqueue(v->verify_wq);
|
||||
dm_bufio_client_reset(v->bufio);
|
||||
verity_security_clear_integrity(ti);
|
||||
}
|
||||
|
||||
static struct target_type verity_target = {
|
||||
.name = "verity",
|
||||
/* Note: the LSMs depend on the singleton and immutable features */
|
||||
|
|
|
|||
|
|
@ -21,8 +21,8 @@ static bool dm_verity_keyring_unsealed __ro_after_init;
|
|||
module_param_named(keyring_unsealed, dm_verity_keyring_unsealed, bool, 0444);
|
||||
MODULE_PARM_DESC(keyring_unsealed, "Leave the dm-verity keyring unsealed");
|
||||
|
||||
static bool require_signatures;
|
||||
module_param(require_signatures, bool, 0444);
|
||||
static bool require_signatures = IS_ENABLED(CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG_FORCE);
|
||||
module_param(require_signatures, bool_enable_only, 0444);
|
||||
MODULE_PARM_DESC(require_signatures,
|
||||
"Verify the roothash of dm-verity hash tree");
|
||||
|
||||
|
|
|
|||
|
|
@ -81,15 +81,9 @@ struct wc_memory_superblock {
|
|||
struct wc_entry {
|
||||
struct rb_node rb_node;
|
||||
struct list_head lru;
|
||||
u32 age; // jiffies
|
||||
unsigned short wc_list_contiguous;
|
||||
#if BITS_PER_LONG == 64
|
||||
bool write_in_progress : 1;
|
||||
unsigned long index : 47;
|
||||
#else
|
||||
bool write_in_progress;
|
||||
unsigned long index;
|
||||
#endif
|
||||
unsigned long age;
|
||||
#ifdef DM_WRITECACHE_HANDLE_HARDWARE_ERRORS
|
||||
uint64_t original_sector;
|
||||
uint64_t seq_count;
|
||||
|
|
@ -194,6 +188,7 @@ struct dm_writecache {
|
|||
struct task_struct *endio_thread;
|
||||
|
||||
struct task_struct *flush_thread;
|
||||
spinlock_t flush_list_lock;
|
||||
struct bio_list flush_list;
|
||||
|
||||
struct dm_kcopyd_client *dm_kcopyd;
|
||||
|
|
@ -389,20 +384,28 @@ static struct wc_memory_superblock *sb(struct dm_writecache *wc)
|
|||
return wc->memory_map;
|
||||
}
|
||||
|
||||
static inline unsigned long wc_entry_index(const struct wc_entry *base, const struct wc_entry *entry)
|
||||
{
|
||||
return entry - base;
|
||||
}
|
||||
|
||||
static struct wc_memory_entry *memory_entry(struct dm_writecache *wc, struct wc_entry *e)
|
||||
{
|
||||
return &sb(wc)->entries[e->index];
|
||||
const unsigned long index = wc_entry_index(wc->entries, e);
|
||||
return &sb(wc)->entries[index];
|
||||
}
|
||||
|
||||
static void *memory_data(struct dm_writecache *wc, struct wc_entry *e)
|
||||
{
|
||||
return (char *)wc->block_start + (e->index << wc->block_size_bits);
|
||||
const unsigned long index = wc_entry_index(wc->entries, e);
|
||||
return (char *)wc->block_start + (index << wc->block_size_bits);
|
||||
}
|
||||
|
||||
static sector_t cache_sector(struct dm_writecache *wc, struct wc_entry *e)
|
||||
{
|
||||
const unsigned long index = wc_entry_index(wc->entries, e);
|
||||
return wc->start_sector + wc->metadata_sectors +
|
||||
((sector_t)e->index << (wc->block_size_bits - SECTOR_SHIFT));
|
||||
((sector_t)index << (wc->block_size_bits - SECTOR_SHIFT));
|
||||
}
|
||||
|
||||
static uint64_t read_original_sector(struct dm_writecache *wc, struct wc_entry *e)
|
||||
|
|
@ -668,7 +671,7 @@ static void writecache_insert_entry(struct dm_writecache *wc, struct wc_entry *i
|
|||
rb_link_node(&ins->rb_node, parent, node);
|
||||
rb_insert_color(&ins->rb_node, &wc->tree);
|
||||
list_add(&ins->lru, &wc->lru);
|
||||
ins->age = jiffies;
|
||||
ins->age = (u32)jiffies;
|
||||
}
|
||||
|
||||
static void writecache_unlink(struct dm_writecache *wc, struct wc_entry *e)
|
||||
|
|
@ -899,8 +902,6 @@ static void writecache_discard(struct dm_writecache *wc, sector_t start, sector_
|
|||
}
|
||||
discarded_something = true;
|
||||
}
|
||||
if (!writecache_entry_is_committed(wc, e))
|
||||
wc->uncommitted_blocks--;
|
||||
writecache_free_entry(wc, e);
|
||||
}
|
||||
|
||||
|
|
@ -969,7 +970,6 @@ static int writecache_alloc_entries(struct dm_writecache *wc)
|
|||
for (b = 0; b < wc->n_blocks; b++) {
|
||||
struct wc_entry *e = &wc->entries[b];
|
||||
|
||||
e->index = b;
|
||||
e->write_in_progress = false;
|
||||
cond_resched();
|
||||
}
|
||||
|
|
@ -1290,9 +1290,11 @@ static int writecache_flush_thread(void *data)
|
|||
struct bio *bio;
|
||||
|
||||
wc_lock(wc);
|
||||
spin_lock_irq(&wc->flush_list_lock);
|
||||
bio = bio_list_pop(&wc->flush_list);
|
||||
if (!bio) {
|
||||
set_current_state(TASK_INTERRUPTIBLE);
|
||||
spin_unlock_irq(&wc->flush_list_lock);
|
||||
wc_unlock(wc);
|
||||
|
||||
if (unlikely(kthread_should_stop())) {
|
||||
|
|
@ -1303,6 +1305,7 @@ static int writecache_flush_thread(void *data)
|
|||
schedule();
|
||||
continue;
|
||||
}
|
||||
spin_unlock_irq(&wc->flush_list_lock);
|
||||
|
||||
if (bio_op(bio) == REQ_OP_DISCARD) {
|
||||
writecache_discard(wc, bio->bi_iter.bi_sector,
|
||||
|
|
@ -1324,9 +1327,12 @@ static int writecache_flush_thread(void *data)
|
|||
|
||||
static void writecache_offload_bio(struct dm_writecache *wc, struct bio *bio)
|
||||
{
|
||||
unsigned long flags;
|
||||
spin_lock_irqsave(&wc->flush_list_lock, flags);
|
||||
if (bio_list_empty(&wc->flush_list))
|
||||
wake_up_process(wc->flush_thread);
|
||||
bio_list_add(&wc->flush_list, bio);
|
||||
spin_unlock_irqrestore(&wc->flush_list_lock, flags);
|
||||
}
|
||||
|
||||
enum wc_map_op {
|
||||
|
|
@ -1424,7 +1430,8 @@ static void writecache_bio_copy_ssd(struct dm_writecache *wc, struct bio *bio,
|
|||
dm_accept_partial_bio(bio, bio_size >> SECTOR_SHIFT);
|
||||
|
||||
wc->stats.writes += bio->bi_iter.bi_size >> wc->block_size_bits;
|
||||
wc->stats.writes_allocate += (bio->bi_iter.bi_size - wc->block_size) >> wc->block_size_bits;
|
||||
if (!search_used)
|
||||
wc->stats.writes_allocate += (bio->bi_iter.bi_size - wc->block_size) >> wc->block_size_bits;
|
||||
|
||||
if (unlikely(wc->uncommitted_blocks >= wc->autocommit_blocks)) {
|
||||
wc->uncommitted_blocks = 0;
|
||||
|
|
@ -1437,6 +1444,8 @@ static void writecache_bio_copy_ssd(struct dm_writecache *wc, struct bio *bio,
|
|||
static enum wc_map_op writecache_map_write(struct dm_writecache *wc, struct bio *bio)
|
||||
{
|
||||
struct wc_entry *e;
|
||||
enum wc_map_op ret = WC_MAP_SUBMIT;
|
||||
bool need_flush = false;
|
||||
|
||||
do {
|
||||
bool found_entry = false;
|
||||
|
|
@ -1473,7 +1482,8 @@ static enum wc_map_op writecache_map_write(struct dm_writecache *wc, struct bio
|
|||
writecache_map_remap_origin(wc, bio, e);
|
||||
wc->stats.writes_around += bio->bi_iter.bi_size >> wc->block_size_bits;
|
||||
wc->stats.writes += bio->bi_iter.bi_size >> wc->block_size_bits;
|
||||
return WC_MAP_REMAP_ORIGIN;
|
||||
ret = WC_MAP_REMAP_ORIGIN;
|
||||
goto flush_pmem_and_ret;
|
||||
}
|
||||
wc->stats.writes_blocked_on_freelist++;
|
||||
writecache_wait_on_freelist(wc);
|
||||
|
|
@ -1487,18 +1497,25 @@ static enum wc_map_op writecache_map_write(struct dm_writecache *wc, struct bio
|
|||
if (WC_MODE_PMEM(wc)) {
|
||||
bio_copy_block(wc, bio, memory_data(wc, e));
|
||||
wc->stats.writes++;
|
||||
need_flush = true;
|
||||
} else {
|
||||
writecache_bio_copy_ssd(wc, bio, e, search_used);
|
||||
return WC_MAP_REMAP;
|
||||
}
|
||||
} while (bio->bi_iter.bi_size);
|
||||
|
||||
if (unlikely(bio->bi_opf & REQ_FUA || wc->uncommitted_blocks >= wc->autocommit_blocks))
|
||||
writecache_flush(wc);
|
||||
else
|
||||
writecache_schedule_autocommit(wc);
|
||||
flush_pmem_and_ret:
|
||||
if (need_flush) {
|
||||
if (unlikely(bio->bi_opf & REQ_FUA || wc->uncommitted_blocks >= wc->autocommit_blocks)) {
|
||||
writecache_flush(wc);
|
||||
if (writecache_has_error(wc))
|
||||
ret = WC_MAP_ERROR;
|
||||
} else {
|
||||
writecache_schedule_autocommit(wc);
|
||||
}
|
||||
}
|
||||
|
||||
return WC_MAP_SUBMIT;
|
||||
return ret;
|
||||
}
|
||||
|
||||
static enum wc_map_op writecache_map_flush(struct dm_writecache *wc, struct bio *bio)
|
||||
|
|
@ -1588,7 +1605,10 @@ static int writecache_map(struct dm_target *ti, struct bio *bio)
|
|||
|
||||
case WC_MAP_REMAP:
|
||||
/* make sure that writecache_end_io decrements bio_in_progress: */
|
||||
bio->bi_private = (void *)1;
|
||||
if (unlikely(bio->bi_opf & REQ_FUA) && bio_op(bio) == REQ_OP_WRITE)
|
||||
bio->bi_private = (void *)3;
|
||||
else
|
||||
bio->bi_private = (void *)1;
|
||||
atomic_inc(&wc->bio_in_progress[bio_data_dir(bio)]);
|
||||
wc_unlock(wc);
|
||||
return DM_MAPIO_REMAPPED;
|
||||
|
|
@ -1618,12 +1638,19 @@ static int writecache_end_io(struct dm_target *ti, struct bio *bio, blk_status_t
|
|||
{
|
||||
struct dm_writecache *wc = ti->private;
|
||||
|
||||
if (bio->bi_private == (void *)1) {
|
||||
if (bio->bi_private == (void *)1 || bio->bi_private == (void *)3) {
|
||||
int dir = bio_data_dir(bio);
|
||||
bool fua = bio->bi_private == (void *)3;
|
||||
|
||||
if (atomic_dec_and_test(&wc->bio_in_progress[dir]))
|
||||
if (unlikely(waitqueue_active(&wc->bio_in_progress_wait[dir])))
|
||||
wake_up(&wc->bio_in_progress_wait[dir]);
|
||||
|
||||
if (fua && likely(*status == BLK_STS_OK)) {
|
||||
bio->bi_private = NULL;
|
||||
writecache_offload_bio(wc, bio);
|
||||
return DM_ENDIO_INCOMPLETE;
|
||||
}
|
||||
} else if (bio->bi_private == (void *)2) {
|
||||
dm_iot_io_end(&wc->iot, 1);
|
||||
}
|
||||
|
|
@ -1986,7 +2013,7 @@ static void writecache_writeback(struct work_struct *work)
|
|||
while (!list_empty(&wc->lru) &&
|
||||
(wc->writeback_all ||
|
||||
wc->freelist_size + wc->writeback_size <= wc->freelist_low_watermark ||
|
||||
(jiffies - container_of(wc->lru.prev, struct wc_entry, lru)->age >=
|
||||
((u32)(jiffies - container_of(wc->lru.prev, struct wc_entry, lru)->age) >=
|
||||
wc->max_age - wc->max_age / MAX_AGE_DIV))) {
|
||||
|
||||
n_walked++;
|
||||
|
|
@ -2107,7 +2134,6 @@ static int calculate_memory_size(uint64_t device_size, unsigned int block_size,
|
|||
size_t *n_blocks_p, size_t *n_metadata_blocks_p)
|
||||
{
|
||||
uint64_t n_blocks, offset;
|
||||
struct wc_entry e;
|
||||
|
||||
n_blocks = device_size;
|
||||
do_div(n_blocks, block_size + sizeof(struct wc_memory_entry));
|
||||
|
|
@ -2126,11 +2152,6 @@ static int calculate_memory_size(uint64_t device_size, unsigned int block_size,
|
|||
n_blocks--;
|
||||
}
|
||||
|
||||
/* check if the bit field overflows */
|
||||
e.index = n_blocks;
|
||||
if (e.index != n_blocks)
|
||||
return -EFBIG;
|
||||
|
||||
if (n_blocks_p)
|
||||
*n_blocks_p = n_blocks;
|
||||
if (n_metadata_blocks_p)
|
||||
|
|
@ -2444,13 +2465,15 @@ static int writecache_ctr(struct dm_target *ti, unsigned int argc, char **argv)
|
|||
wc->autocommit_time_set = true;
|
||||
} else if (!strcasecmp(string, "max_age") && opt_params >= 1) {
|
||||
unsigned int max_age_msecs;
|
||||
unsigned long max_age_jiffies;
|
||||
|
||||
string = dm_shift_arg(&as), opt_params--;
|
||||
if (sscanf(string, "%u%c", &max_age_msecs, &dummy) != 1)
|
||||
goto invalid_optional;
|
||||
if (max_age_msecs > 86400000)
|
||||
max_age_jiffies = msecs_to_jiffies(max_age_msecs);
|
||||
if (max_age_jiffies >= min(MAX_JIFFY_OFFSET, (7 << 28))) // 7/8ths of 1 << 31
|
||||
goto invalid_optional;
|
||||
wc->max_age = msecs_to_jiffies(max_age_msecs);
|
||||
wc->max_age = max_age_jiffies;
|
||||
wc->max_age_set = true;
|
||||
wc->max_age_value = max_age_msecs;
|
||||
} else if (!strcasecmp(string, "cleaner")) {
|
||||
|
|
@ -2498,6 +2521,11 @@ static int writecache_ctr(struct dm_target *ti, unsigned int argc, char **argv)
|
|||
}
|
||||
|
||||
if (WC_MODE_PMEM(wc)) {
|
||||
if (!wc->ssd_dev->dax_dev) {
|
||||
r = -EOPNOTSUPP;
|
||||
ti->error = "Cache device is not a DAX device";
|
||||
goto bad;
|
||||
}
|
||||
if (!dax_synchronous(wc->ssd_dev->dax_dev)) {
|
||||
r = -EOPNOTSUPP;
|
||||
ti->error = "Asynchronous persistent memory not supported as pmem cache";
|
||||
|
|
@ -2515,6 +2543,7 @@ static int writecache_ctr(struct dm_target *ti, unsigned int argc, char **argv)
|
|||
|
||||
wc->memory_map_size -= (uint64_t)wc->start_sector << SECTOR_SHIFT;
|
||||
|
||||
spin_lock_init(&wc->flush_list_lock);
|
||||
bio_list_init(&wc->flush_list);
|
||||
wc->flush_thread = kthread_run(writecache_flush_thread, wc, "dm_writecache_flush");
|
||||
if (IS_ERR(wc->flush_thread)) {
|
||||
|
|
|
|||
|
|
@ -447,10 +447,8 @@ void dm_finalize_zone_settings(struct dm_table *t, struct queue_limits *lim)
|
|||
struct mapped_device *md = t->md;
|
||||
|
||||
if (lim->features & BLK_FEAT_ZONED) {
|
||||
if (dm_table_supports_zone_append(t))
|
||||
clear_bit(DMF_EMULATE_ZONE_APPEND, &md->flags);
|
||||
else
|
||||
set_bit(DMF_EMULATE_ZONE_APPEND, &md->flags);
|
||||
assign_bit(DMF_EMULATE_ZONE_APPEND, &md->flags,
|
||||
!dm_table_supports_zone_append(t));
|
||||
} else {
|
||||
clear_bit(DMF_EMULATE_ZONE_APPEND, &md->flags);
|
||||
md->disk->nr_zones = 0;
|
||||
|
|
|
|||
|
|
@ -1350,7 +1350,7 @@ void dm_accept_partial_bio(struct bio *bio, unsigned int n_sectors)
|
|||
* for accounting but it is being reduced so update accordingly.
|
||||
*/
|
||||
dm_io_set_flag(io, DM_IO_WAS_SPLIT);
|
||||
io->sectors = n_sectors;
|
||||
io->sectors = *tio->len_ptr;
|
||||
io->sector_offset = bio_sectors(io->orig_bio);
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(dm_accept_partial_bio);
|
||||
|
|
|
|||
|
|
@ -59,10 +59,7 @@ static int pack_bits(uint32_t index, void *value, void *context)
|
|||
if (r)
|
||||
return r;
|
||||
|
||||
if (bv)
|
||||
set_bit(bit, (unsigned long *) &word);
|
||||
else
|
||||
clear_bit(bit, (unsigned long *) &word);
|
||||
assign_bit(bit, (unsigned long *)&word, bv);
|
||||
}
|
||||
|
||||
*((__le64 *) value) = cpu_to_le64(word);
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@
|
|||
* least MIN_ENTRIES + 1. We do this in the following ways:
|
||||
*
|
||||
* [A] No siblings => this can only happen if the node is the root, in which
|
||||
* case we copy the childs contents over the root.
|
||||
* case we copy the children contents over the root.
|
||||
*
|
||||
* [B] No left sibling
|
||||
* ==> rebalance(node, right sibling)
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user