2005-04-17 00:20:36 +02:00
|
|
|
/*
|
2005-11-02 04:58:39 +01:00
|
|
|
* Copyright (c) 2000,2002,2005 Silicon Graphics, Inc.
|
|
|
|
|
* All Rights Reserved.
|
2005-04-17 00:20:36 +02:00
|
|
|
*
|
2005-11-02 04:58:39 +01:00
|
|
|
* This program is free software; you can redistribute it and/or
|
|
|
|
|
* modify it under the terms of the GNU General Public License as
|
2005-04-17 00:20:36 +02:00
|
|
|
* published by the Free Software Foundation.
|
|
|
|
|
*
|
2005-11-02 04:58:39 +01:00
|
|
|
* This program is distributed in the hope that it would be useful,
|
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
* GNU General Public License for more details.
|
2005-04-17 00:20:36 +02:00
|
|
|
*
|
2005-11-02 04:58:39 +01:00
|
|
|
* You should have received a copy of the GNU General Public License
|
|
|
|
|
* along with this program; if not, write the Free Software Foundation,
|
|
|
|
|
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
2005-04-17 00:20:36 +02:00
|
|
|
*/
|
|
|
|
|
#ifndef __XFS_TRANS_PRIV_H__
|
|
|
|
|
#define __XFS_TRANS_PRIV_H__
|
|
|
|
|
|
|
|
|
|
struct xfs_log_item;
|
|
|
|
|
struct xfs_log_item_desc;
|
|
|
|
|
struct xfs_mount;
|
|
|
|
|
struct xfs_trans;
|
2010-12-20 02:02:19 +01:00
|
|
|
struct xfs_ail;
|
|
|
|
|
struct xfs_log_vec;
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2013-08-12 12:49:31 +02:00
|
|
|
|
|
|
|
|
void xfs_trans_init(struct xfs_mount *);
|
2010-06-23 10:11:15 +02:00
|
|
|
void xfs_trans_add_item(struct xfs_trans *, struct xfs_log_item *);
|
|
|
|
|
void xfs_trans_del_item(struct xfs_log_item *);
|
2010-08-24 03:42:52 +02:00
|
|
|
void xfs_trans_free_items(struct xfs_trans *tp, xfs_lsn_t commit_lsn,
|
2015-06-04 05:47:43 +02:00
|
|
|
bool abort);
|
xfs: Introduce delayed logging core code
The delayed logging code only changes in-memory structures and as
such can be enabled and disabled with a mount option. Add the mount
option and emit a warning that this is an experimental feature that
should not be used in production yet.
We also need infrastructure to track committed items that have not
yet been written to the log. This is what the Committed Item List
(CIL) is for.
The log item also needs to be extended to track the current log
vector, the associated memory buffer and it's location in the Commit
Item List. Extend the log item and log vector structures to enable
this tracking.
To maintain the current log format for transactions with delayed
logging, we need to introduce a checkpoint transaction and a context
for tracking each checkpoint from initiation to transaction
completion. This includes adding a log ticket for tracking space
log required/used by the context checkpoint.
To track all the changes we need an io vector array per log item,
rather than a single array for the entire transaction. Using the new
log vector structure for this requires two passes - the first to
allocate the log vector structures and chain them together, and the
second to fill them out. This log vector chain can then be passed
to the CIL for formatting, pinning and insertion into the CIL.
Formatting of the log vector chain is relatively simple - it's just
a loop over the iovecs on each log vector, but it is made slightly
more complex because we re-write the iovec after the copy to point
back at the memory buffer we just copied into.
This code also needs to pin log items. If the log item is not
already tracked in this checkpoint context, then it needs to be
pinned. Otherwise it is already pinned and we don't need to pin it
again.
The only other complexity is calculating the amount of new log space
the formatting has consumed. This needs to be accounted to the
transaction in progress, and the accounting is made more complex
becase we need also to steal space from it for log metadata in the
checkpoint transaction. Calculate all this at insert time and update
all the tickets, counters, etc correctly.
Once we've formatted all the log items in the transaction, attach
the busy extents to the checkpoint context so the busy extents live
until checkpoint completion and can be processed at that point in
time. Transactions can then be freed at this point in time.
Now we need to issue checkpoints - we are tracking the amount of log space
used by the items in the CIL, so we can trigger background checkpoints when the
space usage gets to a certain threshold. Otherwise, checkpoints need ot be
triggered when a log synchronisation point is reached - a log force event.
Because the log write code already handles chained log vectors, writing the
transaction is trivial, too. Construct a transaction header, add it
to the head of the chain and write it into the log, then issue a
commit record write. Then we can release the checkpoint log ticket
and attach the context to the log buffer so it can be called during
Io completion to complete the checkpoint.
We also need to allow for synchronising multiple in-flight
checkpoints. This is needed for two things - the first is to ensure
that checkpoint commit records appear in the log in the correct
sequence order (so they are replayed in the correct order). The
second is so that xfs_log_force_lsn() operates correctly and only
flushes and/or waits for the specific sequence it was provided with.
To do this we need a wait variable and a list tracking the
checkpoint commits in progress. We can walk this list and wait for
the checkpoints to change state or complete easily, an this provides
the necessary synchronisation for correct operation in both cases.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Alex Elder <aelder@sgi.com>
2010-05-21 06:37:18 +02:00
|
|
|
void xfs_trans_unreserve_and_mod_sb(struct xfs_trans *tp);
|
2005-04-17 00:20:36 +02:00
|
|
|
|
2010-12-20 02:02:19 +01:00
|
|
|
void xfs_trans_committed_bulk(struct xfs_ail *ailp, struct xfs_log_vec *lv,
|
|
|
|
|
xfs_lsn_t commit_lsn, int aborted);
|
2005-04-17 00:20:36 +02:00
|
|
|
/*
|
2008-10-30 07:38:39 +01:00
|
|
|
* AIL traversal cursor.
|
|
|
|
|
*
|
|
|
|
|
* Rather than using a generation number for detecting changes in the ail, use
|
|
|
|
|
* a cursor that is protected by the ail lock. The aild cursor exists in the
|
|
|
|
|
* struct xfs_ail, but other traversals can declare it on the stack and link it
|
|
|
|
|
* to the ail list.
|
|
|
|
|
*
|
|
|
|
|
* When an object is deleted from or moved int the AIL, the cursor list is
|
|
|
|
|
* searched to see if the object is a designated cursor item. If it is, it is
|
|
|
|
|
* deleted from the cursor so that the next time the cursor is used traversal
|
|
|
|
|
* will return to the start.
|
|
|
|
|
*
|
|
|
|
|
* This means a traversal colliding with a removal will cause a restart of the
|
|
|
|
|
* list scan, rather than any insertion or deletion anywhere in the list. The
|
|
|
|
|
* low bit of the item pointer is set if the cursor has been invalidated so
|
|
|
|
|
* that we can tell the difference between invalidation and reaching the end
|
|
|
|
|
* of the list to trigger traversal restarts.
|
2005-04-17 00:20:36 +02:00
|
|
|
*/
|
2008-10-30 07:38:39 +01:00
|
|
|
struct xfs_ail_cursor {
|
2011-07-18 05:40:18 +02:00
|
|
|
struct list_head list;
|
2008-10-30 07:38:39 +01:00
|
|
|
struct xfs_log_item *item;
|
|
|
|
|
};
|
2005-04-17 00:20:36 +02:00
|
|
|
|
[XFS] Move AIL pushing into it's own thread
When many hundreds to thousands of threads all try to do simultaneous
transactions and the log is in a tail-pushing situation (i.e. full), we
can get multiple threads walking the AIL list and contending on the AIL
lock.
The AIL push is, in effect, a simple I/O dispatch algorithm complicated by
the ordering constraints placed on it by the transaction subsystem. It
really does not need multiple threads to push on it - even when only a
single CPU is pushing the AIL, it can push the I/O out far faster that
pretty much any disk subsystem can handle.
So, to avoid contention problems stemming from multiple list walkers, move
the list walk off into another thread and simply provide a "target" to
push to. When a thread requires a push, it sets the target and wakes the
push thread, then goes to sleep waiting for the required amount of space
to become available in the log.
This mechanism should also be a lot fairer under heavy load as the waiters
will queue in arrival order, rather than queuing in "who completed a push
first" order.
Also, by moving the pushing to a separate thread we can do more
effectively overload detection and prevention as we can keep context from
loop iteration to loop iteration. That is, we can push only part of the
list each loop and not have to loop back to the start of the list every
time we run. This should also help by reducing the number of items we try
to lock and/or push items that we cannot move.
Note that this patch is not intended to solve the inefficiencies in the
AIL structure and the associated issues with extremely large list
contents. That needs to be addresses separately; parallel access would
cause problems to any new structure as well, so I'm only aiming to isolate
the structure from unbounded parallelism here.
SGI-PV: 972759
SGI-Modid: xfs-linux-melb:xfs-kern:30371a
Signed-off-by: David Chinner <dgc@sgi.com>
Signed-off-by: Lachlan McIlroy <lachlan@sgi.com>
2008-02-05 02:13:32 +01:00
|
|
|
/*
|
2008-10-30 07:38:39 +01:00
|
|
|
* Private AIL structures.
|
|
|
|
|
*
|
|
|
|
|
* Eventually we need to drive the locking in here as well.
|
[XFS] Move AIL pushing into it's own thread
When many hundreds to thousands of threads all try to do simultaneous
transactions and the log is in a tail-pushing situation (i.e. full), we
can get multiple threads walking the AIL list and contending on the AIL
lock.
The AIL push is, in effect, a simple I/O dispatch algorithm complicated by
the ordering constraints placed on it by the transaction subsystem. It
really does not need multiple threads to push on it - even when only a
single CPU is pushing the AIL, it can push the I/O out far faster that
pretty much any disk subsystem can handle.
So, to avoid contention problems stemming from multiple list walkers, move
the list walk off into another thread and simply provide a "target" to
push to. When a thread requires a push, it sets the target and wakes the
push thread, then goes to sleep waiting for the required amount of space
to become available in the log.
This mechanism should also be a lot fairer under heavy load as the waiters
will queue in arrival order, rather than queuing in "who completed a push
first" order.
Also, by moving the pushing to a separate thread we can do more
effectively overload detection and prevention as we can keep context from
loop iteration to loop iteration. That is, we can push only part of the
list each loop and not have to loop back to the start of the list every
time we run. This should also help by reducing the number of items we try
to lock and/or push items that we cannot move.
Note that this patch is not intended to solve the inefficiencies in the
AIL structure and the associated issues with extremely large list
contents. That needs to be addresses separately; parallel access would
cause problems to any new structure as well, so I'm only aiming to isolate
the structure from unbounded parallelism here.
SGI-PV: 972759
SGI-Modid: xfs-linux-melb:xfs-kern:30371a
Signed-off-by: David Chinner <dgc@sgi.com>
Signed-off-by: Lachlan McIlroy <lachlan@sgi.com>
2008-02-05 02:13:32 +01:00
|
|
|
*/
|
2008-10-30 07:38:26 +01:00
|
|
|
struct xfs_ail {
|
|
|
|
|
struct xfs_mount *xa_mount;
|
2011-10-11 17:14:10 +02:00
|
|
|
struct task_struct *xa_task;
|
2008-10-30 07:38:26 +01:00
|
|
|
struct list_head xa_ail;
|
|
|
|
|
xfs_lsn_t xa_target;
|
2012-06-28 12:52:56 +02:00
|
|
|
xfs_lsn_t xa_target_prev;
|
2011-07-18 05:40:18 +02:00
|
|
|
struct list_head xa_cursors;
|
2008-10-30 07:39:23 +01:00
|
|
|
spinlock_t xa_lock;
|
2011-04-08 04:45:07 +02:00
|
|
|
xfs_lsn_t xa_last_pushed_lsn;
|
2011-09-30 06:45:03 +02:00
|
|
|
int xa_log_flush;
|
xfs: on-stack delayed write buffer lists
Queue delwri buffers on a local on-stack list instead of a per-buftarg one,
and write back the buffers per-process instead of by waking up xfsbufd.
This is now easily doable given that we have very few places left that write
delwri buffers:
- log recovery:
Only done at mount time, and already forcing out the buffers
synchronously using xfs_flush_buftarg
- quotacheck:
Same story.
- dquot reclaim:
Writes out dirty dquots on the LRU under memory pressure. We might
want to look into doing more of this via xfsaild, but it's already
more optimal than the synchronous inode reclaim that writes each
buffer synchronously.
- xfsaild:
This is the main beneficiary of the change. By keeping a local list
of buffers to write we reduce latency of writing out buffers, and
more importably we can remove all the delwri list promotions which
were hitting the buffer cache hard under sustained metadata loads.
The implementation is very straight forward - xfs_buf_delwri_queue now gets
a new list_head pointer that it adds the delwri buffers to, and all callers
need to eventually submit the list using xfs_buf_delwi_submit or
xfs_buf_delwi_submit_nowait. Buffers that already are on a delwri list are
skipped in xfs_buf_delwri_queue, assuming they already are on another delwri
list. The biggest change to pass down the buffer list was done to the AIL
pushing. Now that we operate on buffers the trylock, push and pushbuf log
item methods are merged into a single push routine, which tries to lock the
item, and if possible add the buffer that needs writeback to the buffer list.
This leads to much simpler code than the previous split but requires the
individual IOP_PUSH instances to unlock and reacquire the AIL around calls
to blocking routines.
Given that xfsailds now also handle writing out buffers, the conditions for
log forcing and the sleep times needed some small changes. The most
important one is that we consider an AIL busy as long we still have buffers
to push, and the other one is that we do increment the pushed LSN for
buffers that are under flushing at this moment, but still count them towards
the stuck items for restart purposes. Without this we could hammer on stuck
items without ever forcing the log and not make progress under heavy random
delete workloads on fast flash storage devices.
[ Dave Chinner:
- rebase on previous patches.
- improved comments for XBF_DELWRI_Q handling
- fix XBF_ASYNC handling in queue submission (test 106 failure)
- rename delwri submit function buffer list parameters for clarity
- xfs_efd_item_push() should return XFS_ITEM_PINNED ]
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Mark Tinguely <tinguely@sgi.com>
Signed-off-by: Ben Myers <bpm@sgi.com>
2012-04-23 07:58:39 +02:00
|
|
|
struct list_head xa_buf_list;
|
2012-04-23 07:58:34 +02:00
|
|
|
wait_queue_head_t xa_empty;
|
2008-10-30 07:38:26 +01:00
|
|
|
};
|
|
|
|
|
|
2008-10-30 07:38:39 +01:00
|
|
|
/*
|
|
|
|
|
* From xfs_trans_ail.c
|
|
|
|
|
*/
|
2010-12-20 02:34:26 +01:00
|
|
|
void xfs_trans_ail_update_bulk(struct xfs_ail *ailp,
|
2011-07-18 05:40:16 +02:00
|
|
|
struct xfs_ail_cursor *cur,
|
2010-12-20 02:34:26 +01:00
|
|
|
struct xfs_log_item **log_items, int nr_items,
|
|
|
|
|
xfs_lsn_t lsn) __releases(ailp->xa_lock);
|
2013-08-15 07:08:35 +02:00
|
|
|
/*
|
|
|
|
|
* Return a pointer to the first item in the AIL. If the AIL is empty, then
|
|
|
|
|
* return NULL.
|
|
|
|
|
*/
|
|
|
|
|
static inline struct xfs_log_item *
|
|
|
|
|
xfs_ail_min(
|
|
|
|
|
struct xfs_ail *ailp)
|
|
|
|
|
{
|
|
|
|
|
return list_first_entry_or_null(&ailp->xa_ail, struct xfs_log_item,
|
|
|
|
|
li_ail);
|
|
|
|
|
}
|
|
|
|
|
|
2010-12-20 02:34:26 +01:00
|
|
|
static inline void
|
|
|
|
|
xfs_trans_ail_update(
|
|
|
|
|
struct xfs_ail *ailp,
|
|
|
|
|
struct xfs_log_item *lip,
|
|
|
|
|
xfs_lsn_t lsn) __releases(ailp->xa_lock)
|
|
|
|
|
{
|
2011-07-18 05:40:16 +02:00
|
|
|
xfs_trans_ail_update_bulk(ailp, NULL, &lip, 1, lsn);
|
2010-12-20 02:34:26 +01:00
|
|
|
}
|
|
|
|
|
|
2010-12-20 02:36:15 +01:00
|
|
|
void xfs_trans_ail_delete_bulk(struct xfs_ail *ailp,
|
2012-04-23 07:58:41 +02:00
|
|
|
struct xfs_log_item **log_items, int nr_items,
|
|
|
|
|
int shutdown_type)
|
2010-12-20 02:36:15 +01:00
|
|
|
__releases(ailp->xa_lock);
|
|
|
|
|
static inline void
|
|
|
|
|
xfs_trans_ail_delete(
|
|
|
|
|
struct xfs_ail *ailp,
|
2012-04-23 07:58:41 +02:00
|
|
|
xfs_log_item_t *lip,
|
|
|
|
|
int shutdown_type) __releases(ailp->xa_lock)
|
2010-12-20 02:36:15 +01:00
|
|
|
{
|
2012-04-23 07:58:41 +02:00
|
|
|
xfs_trans_ail_delete_bulk(ailp, &lip, 1, shutdown_type);
|
2010-12-20 02:36:15 +01:00
|
|
|
}
|
|
|
|
|
|
2015-08-19 02:01:08 +02:00
|
|
|
static inline void
|
|
|
|
|
xfs_trans_ail_remove(
|
|
|
|
|
struct xfs_log_item *lip,
|
|
|
|
|
int shutdown_type)
|
|
|
|
|
{
|
|
|
|
|
struct xfs_ail *ailp = lip->li_ailp;
|
|
|
|
|
|
|
|
|
|
spin_lock(&ailp->xa_lock);
|
|
|
|
|
/* xfs_trans_ail_delete() drops the AIL lock */
|
|
|
|
|
if (lip->li_flags & XFS_LI_IN_AIL)
|
|
|
|
|
xfs_trans_ail_delete(ailp, lip, shutdown_type);
|
|
|
|
|
else
|
|
|
|
|
spin_unlock(&ailp->xa_lock);
|
|
|
|
|
}
|
|
|
|
|
|
2011-04-08 04:45:07 +02:00
|
|
|
void xfs_ail_push(struct xfs_ail *, xfs_lsn_t);
|
|
|
|
|
void xfs_ail_push_all(struct xfs_ail *);
|
2012-04-23 07:58:34 +02:00
|
|
|
void xfs_ail_push_all_sync(struct xfs_ail *);
|
2012-04-23 07:58:33 +02:00
|
|
|
struct xfs_log_item *xfs_ail_min(struct xfs_ail *ailp);
|
2011-04-08 04:45:07 +02:00
|
|
|
xfs_lsn_t xfs_ail_min_lsn(struct xfs_ail *ailp);
|
|
|
|
|
|
2011-07-18 05:40:16 +02:00
|
|
|
struct xfs_log_item * xfs_trans_ail_cursor_first(struct xfs_ail *ailp,
|
2008-10-30 07:39:00 +01:00
|
|
|
struct xfs_ail_cursor *cur,
|
|
|
|
|
xfs_lsn_t lsn);
|
2011-07-18 05:40:16 +02:00
|
|
|
struct xfs_log_item * xfs_trans_ail_cursor_last(struct xfs_ail *ailp,
|
|
|
|
|
struct xfs_ail_cursor *cur,
|
|
|
|
|
xfs_lsn_t lsn);
|
|
|
|
|
struct xfs_log_item * xfs_trans_ail_cursor_next(struct xfs_ail *ailp,
|
2008-10-30 07:38:39 +01:00
|
|
|
struct xfs_ail_cursor *cur);
|
2014-04-14 11:06:05 +02:00
|
|
|
void xfs_trans_ail_cursor_done(struct xfs_ail_cursor *cur);
|
2008-10-30 07:38:39 +01:00
|
|
|
|
2008-10-30 07:39:12 +01:00
|
|
|
#if BITS_PER_LONG != 64
|
|
|
|
|
static inline void
|
|
|
|
|
xfs_trans_ail_copy_lsn(
|
|
|
|
|
struct xfs_ail *ailp,
|
|
|
|
|
xfs_lsn_t *dst,
|
|
|
|
|
xfs_lsn_t *src)
|
|
|
|
|
{
|
|
|
|
|
ASSERT(sizeof(xfs_lsn_t) == 8); /* don't lock if it shrinks */
|
2008-10-30 07:39:23 +01:00
|
|
|
spin_lock(&ailp->xa_lock);
|
2008-10-30 07:39:12 +01:00
|
|
|
*dst = *src;
|
2008-10-30 07:39:23 +01:00
|
|
|
spin_unlock(&ailp->xa_lock);
|
2008-10-30 07:39:12 +01:00
|
|
|
}
|
|
|
|
|
#else
|
|
|
|
|
static inline void
|
|
|
|
|
xfs_trans_ail_copy_lsn(
|
|
|
|
|
struct xfs_ail *ailp,
|
|
|
|
|
xfs_lsn_t *dst,
|
|
|
|
|
xfs_lsn_t *src)
|
|
|
|
|
{
|
|
|
|
|
ASSERT(sizeof(xfs_lsn_t) == 8);
|
|
|
|
|
*dst = *src;
|
|
|
|
|
}
|
|
|
|
|
#endif
|
2005-04-17 00:20:36 +02:00
|
|
|
#endif /* __XFS_TRANS_PRIV_H__ */
|
