kpriv/linux-hardened
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
linux-hardened/drivers/md
History
Jens Axboe c4cf5261f8 bio: skip atomic inc/dec of ->bi_remaining for non-chains 
Struct bio has an atomic ref count for chained bio's, and we use this
to know when to end IO on the bio. However, most bio's are not chained,
so we don't need to always introduce this atomic operation as part of
ending IO.

Add a helper to elevate the bi_remaining count, and flag the bio as
now actually needing the decrement at end_io time. Rename the field
to __bi_remaining to catch any current users of this doing the
incrementing manually.

For high IOPS workloads, this reduces the overhead of bio_endio()
substantially.

Tested-by: Robert Elliott <elliott@hp.com>
Acked-by: Kent Overstreet <kent.overstreet@gmail.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jens Axboe <axboe@fb.com>
2015-05-05 13:32:47 -06:00
..
bcache md/bcache: use generic io stats accounting functions to simplify io stat accounting 2014-11-24 08:05:12 -07:00
persistent-data - Significant dm-crypt CPU scalability performance improvements thanks 2015-02-21 13:28:45 -08:00
bitmap.c md-cluster: re-add capabilities 2015-04-22 07:59:39 +10:00
bitmap.h md-cluster: re-add capabilities 2015-04-22 07:59:39 +10:00
dm-bio-prison.c dm bio prison: introduce support for locking ranges of blocks 2014-11-10 15:25:30 -05:00
dm-bio-prison.h dm bio prison: introduce support for locking ranges of blocks 2014-11-10 15:25:30 -05:00
dm-bio-record.h dm: Refactor for new bio cloning/splitting 2013-11-23 22:33:55 -08:00
dm-bufio.c dm bufio: fix time comparison to use time_after_eq() 2015-02-09 13:06:48 -05:00
dm-bufio.h dm snapshot: use dm-bufio prefetch 2014-01-14 23:23:03 -05:00
dm-builtin.c dm sysfs: fix a module unload race 2014-01-14 23:23:04 -05:00
dm-cache-block-types.h dm cache: revert "remove remainder of distinct discard block size" 2014-11-10 15:25:30 -05:00
dm-cache-metadata.c dm cache: fix missing ERR_PTR returns and handling 2015-01-28 09:59:20 -05:00
dm-cache-metadata.h dm cache: revert "remove remainder of distinct discard block size" 2014-11-10 15:25:30 -05:00
dm-cache-policy-cleaner.c
dm-cache-policy-internal.h
dm-cache-policy-mq.c dm cache policy mq: try not to writeback data that changed in the last second 2015-03-31 12:03:48 -04:00
dm-cache-policy.c dm cache: add policy name to status output 2014-01-16 13:44:11 -05:00
dm-cache-policy.h dm cache: add policy name to status output 2014-01-16 13:44:11 -05:00
dm-cache-target.c bio: skip atomic inc/dec of ->bi_remaining for non-chains 2015-05-05 13:32:47 -06:00
dm-crypt.c dm crypt: fix missing error code return from crypt_ctr error path 2015-04-16 22:00:50 -04:00
dm-delay.c dm delay: use msecs_to_jiffies for time conversion 2015-04-15 12:10:21 -04:00
dm-era-target.c dm era: check for a non-NULL metadata object before closing it 2014-06-03 13:44:08 -04:00
dm-exception-store.c
dm-exception-store.h
dm-flakey.c
dm-io.c dm io: deal with wandering queue limits when handling REQ_DISCARD and REQ_WRITE_SAME 2015-02-27 14:53:32 -05:00
dm-ioctl.c dm: only initialize the request_queue once 2015-04-30 10:25:21 -04:00
dm-kcopyd.c
dm-linear.c
dm-log-userspace-base.c dm log userspace base: fix compile warning 2015-04-15 12:10:20 -04:00
dm-log-userspace-transfer.c dm log userspace transfer: match wait_for_completion_timeout return type 2015-04-15 12:10:20 -04:00
dm-log-userspace-transfer.h
dm-log-writes.c dm: add log writes target 2015-04-15 12:10:24 -04:00
dm-log.c
dm-mpath.c dm: optimize dm_mq_queue_rq to _not_ use kthread if using pure blk-mq 2015-04-15 12:10:17 -04:00
dm-mpath.h
dm-path-selector.c
dm-path-selector.h
dm-queue-length.c
dm-raid.c - Most significant change this cycle is request-based DM now supports 2015-02-12 16:36:31 -08:00
dm-raid1.c bio: skip atomic inc/dec of ->bi_remaining for non-chains 2015-05-05 13:32:47 -06:00
dm-region-hash.c
dm-round-robin.c
dm-service-time.c
dm-snap-persistent.c dm snapshot: remove unnecessary NULL checks before vfree() calls 2015-02-09 13:06:49 -05:00
dm-snap-transient.c
dm-snap.c bio: skip atomic inc/dec of ->bi_remaining for non-chains 2015-05-05 13:32:47 -06:00
dm-stats.c - Significant DM thin-provisioning performance improvements to meet 2014-12-08 21:10:03 -08:00
dm-stats.h
dm-stripe.c dm stripe: fix potential for leak in stripe_ctr error path 2014-10-10 22:05:18 -04:00
dm-switch.c dm switch: efficiently support repetitive patterns 2014-08-01 12:30:37 -04:00
dm-sysfs.c dm: add 'use_blk_mq' module param and expose in per-device ro sysfs attr 2015-04-15 12:10:17 -04:00
dm-table.c dm table: use bool function return values of true/false not 1/0 2015-04-15 12:10:23 -04:00
dm-target.c dm: allocate requests in target when stacking on blk-mq devices 2015-02-09 13:06:47 -05:00
dm-thin-metadata.c dm thin metadata: remove unused dm_pool_get_data_block_size() 2015-02-09 13:06:49 -05:00
dm-thin-metadata.h dm thin metadata: remove unused dm_pool_get_data_block_size() 2015-02-09 13:06:49 -05:00
dm-thin.c bio: skip atomic inc/dec of ->bi_remaining for non-chains 2015-05-05 13:32:47 -06:00
dm-uevent.c
dm-uevent.h
dm-verity.c dm verity: add error handling modes for corrupted blocks 2015-04-15 12:10:22 -04:00
dm-zero.c dm crypt, dm zero: update author name following legal name change 2014-07-10 16:44:14 -04:00
dm.c dm: fix free_rq_clone() NULL pointer when requeueing unmapped request 2015-04-30 10:25:21 -04:00
dm.h dm: add 'use_blk_mq' module param and expose in per-device ro sysfs attr 2015-04-15 12:10:17 -04:00
faulty.c md: rename ->stop to ->free 2015-02-04 08:35:52 +11:00
Kconfig md updates for 4.1 2015-04-24 09:28:01 -07:00
linear.c md: rename ->stop to ->free 2015-02-04 08:35:52 +11:00
linear.h
Makefile md updates for 4.1 2015-04-24 09:28:01 -07:00
md-cluster.c md-cluster: re-add capabilities 2015-04-22 07:59:39 +10:00
md-cluster.h md-cluster: re-add capabilities 2015-04-22 07:59:39 +10:00
md.c md: allow resync to go faster when there is competing IO. 2015-04-22 08:00:40 +10:00
md.h md: remove 'go_faster' option from ->sync_request() 2015-04-22 08:00:40 +10:00
multipath.c md: rename ->stop to ->free 2015-02-04 08:35:52 +11:00
multipath.h
raid0.c md raid0: access mddev->queue (request queue member) conditionally because it is not set when accessed from dm-raid 2015-04-22 08:00:41 +10:00
raid0.h
raid1.c md: remove 'go_faster' option from ->sync_request() 2015-04-22 08:00:40 +10:00
raid1.h md: make ->congested robust against personality changes. 2015-02-04 08:35:52 +11:00
raid5.c md/raid5: don't do chunk aligned read on degraded array. 2015-04-22 08:00:43 +10:00
raid5.h md/raid5: allow the stripe_cache to grow and shrink. 2015-04-22 08:00:43 +10:00
raid10.c md: remove 'go_faster' option from ->sync_request() 2015-04-22 08:00:40 +10:00
raid10.h md: make ->congested robust against personality changes. 2015-02-04 08:35:52 +11:00