Commit graph

44 commits

Author SHA1 Message Date
Daniel Drake
65e5a0e18e jffs2: Dynamically choose inocache hash size
When JFFS2 is used for large volumes, the mount times are quite long.
Increasing the hash size provides a significant speed boost on the OLPC
XO-1 laptop.

Add logic that dynamically selects a hash size based on the size of
the medium. A 64mb medium will result in a hash size of 128, and a 512mb
medium will result in a hash size of 1024.

Signed-off-by: Daniel Drake <dsd@laptop.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-10-25 00:57:19 +01:00
Tejun Heo
5a0e3ad6af include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-30 22:02:32 +09:00
David Woodhouse
15953580e7 [JFFS2] Improve getdents vs. f_pos handling on NOR flash.
Commit a491486a20 started obliterating
dirents directly on the medium, when jffs2_can_mark_obsolete(). Removing
them immediately from the f->dents list, however, screws up handling of
f_pos within a directory -- because the offset is equivalent to the
number of entries through the list we are, and the existence of
deletion dirents served to provide 'placeholders' for unlinked
entries. Now, 'rm -r' doesn't even manage to unlink everything in the
directory.

Revert to keeping 'deletion' dirents in the list, at least in memory
even though we no longer write anything to the medium.

Spotted, debugged and mostly fixed by Joakim Tjernlund

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2007-11-01 16:25:56 -04:00
David Woodhouse
61c4b23770 [JFFS2] Handle inodes with only a single metadata node with non-zero isize
This should never happen unless there's corruption on the medium and the
actual data nodes go missing. But the failure mode (an oops when we assume
the fragtree isn't empty and go looking for its last node) isn't useful.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2007-04-25 17:04:23 +01:00
David Woodhouse
c00c310eac [JFFS2] Tidy up licensing/copyright boilerplate.
In particular, remove the bit in the LICENCE file about contacting
Red Hat for alternative arrangements. Their errant IS department broke
that arrangement a long time ago -- the policy of collecting copyright
assignments from contributors came to an end when the plug was pulled on
the servers hosting the project, without notice or reason.

We do still dual-license it for use with eCos, with the GPL+exception
licence approved by the FSF as being GPL-compatible. It's just that nobody
has the right to license it differently.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2007-04-25 14:16:47 +01:00
David Woodhouse
df8e96f391 [JFFS2] Improve read_inode memory usage, v2.
We originally used to read every node and allocate a jffs2_tmp_dnode_info
structure for each, before processing them in (reverse) version order
and discarding the ones which are obsoleted by later nodes.

With huge logfiles, this behaviour caused memory problems. For example, a
file involved in OLPC trac #1292 has 1822391 nodes, and would cause the XO
machine to run out of memory during the first stage of read_inode().

Instead of just inserting nodes into a tree in version order as we find
them, we now put them into a tree in order of their offset within the
file, which allows us to immediately discard nodes which are completely
obsoleted.

We don't use a full tree with 'fragments' pointing to the real data
structure, as we do in the normal fragtree. We sort only on the start
address, and add an 'overlapped' flag to the tmp_dnode_info to indicate
that the node in question is (partially) overlapped by another.

When the scan is complete, we start at the end of the file, adding each
node to a real fragtree as before. Where the node is non-overlapped, we
just add it (it doesn't matter that it's not the latest version; there is
no overlap). When the node at the end of the tree _is_ overlapped, we sort
it and all its overlapping nodes into version order and then add them to
the fragtree in that order.

This 'early discard' reduces the peak allocation of tmp_dnode_info
structures from 1.8M to a mere 62872 (3.5%) in the degenerate case
referenced above.

This version of the patch also correctly rememembers the highest node
version# seen for an inode when it's scanned.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2007-04-25 03:23:42 +01:00
Adrian Bunk
90a18fab4a make fs/jffs2/nodelist.c:jffs2_obsolete_node_frag() static
This patch makes the needlessly global jffs2_obsolete_node_frag()
static.

Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-07-07 00:02:10 +01:00
KaiGai Kohei
355ed4e141 [JFFS2][XATTR] Fix memory leak with jffs2_xattr_ref
If xattr_ref is associated with an orphan inode_cache
on filesystem mounting, those xattr_refs are not
released even if this inode_cache is released.

This patch enables to call jffs2_xattr_delete_inode()
for such a irregular inode_cachde too.

Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-06-27 16:18:30 +01:00
David Woodhouse
2ebf09c249 [JFFS2] Fix oops when marking space dirty in scan, but no previous node exists.
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-28 22:13:25 +01:00
David Woodhouse
9bfeb691e7 [JFFS2] Switch to using an array of jffs2_raw_node_refs instead of a list.
This allows us to drop another pointer from the struct jffs2_raw_node_ref,
shrinking it to 8 bytes on 32-bit machines (if the TEST_TOTLEN) paranoia
check is turned off, which will be committed soon).

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-26 21:19:05 +01:00
David Woodhouse
99988f7bbd [JFFS2] Introduce ref_next() macro for finding next physical node
Another part of the preparation for switching to an array...

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-24 09:04:17 +01:00
David Woodhouse
2f785402f3 [JFFS2] Reduce visibility of raw_node_ref to upper layers of JFFS2 code.
As the first step towards eliminating the ref->next_phys member and saving
memory by using an _array_ of struct jffs2_raw_node_ref per eraseblock,
stop the write functions from allocating their own refs; have them just
_reserve_ the appropriate number instead. Then jffs2_link_node_ref() can
just fill them in.

Use a linked list of pre-allocated refs in the superblock, for now. Once
we switch to an array, it'll just be a case of extending that array.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-24 02:04:45 +01:00
David Woodhouse
fcb7578719 [JFFS2] Extend jffs2_link_node_ref() to link into per-inode list too.
Let's avoid the potential for forgetting to set ref->next_in_ino, by doing
it within jffs2_link_node_ref() instead.

This highlights the ugliness of what we're currently doing with
xattr_datum and xattr_ref structures -- we should find a nicer way of
dealing with that.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-22 15:23:10 +01:00
David Woodhouse
3b79673cfa [JFFS2] Fix accounting error in jffs2_link_node_ref()
When filing REF_OBSOLETE nodes, we'd add their size to the global
'dirty_size' count, but then to the eraseblock's 'used_size' count.
That's not clever.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-22 12:15:47 +01:00
David Woodhouse
ca89a517fa [JFFS2] Finally eliminate __totlen field from struct jffs2_raw_node_ref
Well, almost. We'll actually keep a 'TEST_TOTLEN' macro set for now, and keep
doing some paranoia checks to make sure it's all working correctly. But if
TEST_TOTLEN is unset, the size of struct jffs2_raw_node_ref drops from 16
bytes to 12 on 32-bit machines. That's a saving of about half a megabyte of
memory on the OLPC prototype board, with 125K or so nodes in its 512MiB of
flash.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-21 13:29:11 +01:00
David Woodhouse
68270995f2 [JFFS2] Introduce jffs2_scan_dirty_space() function.
To eliminate the __totlen field from struct jffs2_raw_node_ref, we need
to allocate nodes for dirty space instead of just tweaking the accounting
data. Introduce jffs2_scan_dirty_space() in preparation for that.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-21 03:46:05 +01:00
David Woodhouse
f1f9671bd8 [JFFS2] Introduce jffs2_link_node_ref() function to reduce code duplication
The same sequence of code was repeated in many places, to add a new
struct jffs2_raw_node_ref to an eraseblock and adjust the space accounting
accordingly. Move it out-of-line.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-20 19:45:26 +01:00
KaiGai Kohei
20a92fc74c Merge git://git.infradead.org/mtd-2.6 2006-05-19 00:43:53 +09:00
Andrew Morton
184f565210 [JFFS2] Fix printk format in some error messages.
fs/jffs2/nodelist.c: In function `check_node_data':
fs/jffs2/nodelist.c:441: warning: unsigned int format, different type arg (arg 4)
fs/jffs2/nodelist.c:464: warning: int format, different type arg (arg 5)

Modified from Andrew's original fix because while his terminal may indeed
only have eighty columns, mine only has _TWENTYFOUR_ lines. So the
cosmetic fluff is perfectly OK out past column 80 where it was -- the
casual reader doesn't _care_ about anything more than the fact that it
goes 'if (foo) JFFS2_WARNING...', and there's no point wasting a whole
line to display the tail end of the printk which nobody actually cares
about.

Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-15 13:45:58 +01:00
KaiGai Kohei
aa98d7cf59 [JFFS2][XATTR] XATTR support on JFFS2 (version. 5)
This attached patches provide xattr support including POSIX-ACL and
SELinux support on JFFS2 (version.5).

There are some significant differences from previous version posted
at last December.
The biggest change is addition of EBS(Erase Block Summary) support.
Currently, both kernel and usermode utility (sumtool) can recognize
xattr nodes which have JFFS2_NODETYPE_XATTR/_XREF nodetype.

In addition, some bugs are fixed.
- A potential race condition was fixed.
- Unexpected fail when updating a xattr by same name/value pair was fixed.
- A bug when removing xattr name/value pair was fixed.

The fundamental structures (such as using two new nodetypes and exclusion
mechanism by rwsem) are unchanged. But most of implementation were reviewed
and updated if necessary.
Espacially, we had to change several internal implementations related to
load_xattr_datum() to avoid a potential race condition.

[1/2] xattr_on_jffs2.kernel.version-5.patch
[2/2] xattr_on_jffs2.utils.version-5.patch

Signed-off-by: KaiGai Kohei <kaigai@ak.jp.nec.com>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2006-05-13 15:09:47 +09:00
Atsushi Nemoto
0ef675d491 [PATCH] mtd: 64 bit fixes
Fix some bugs in mtd/jffs2 on 64bit platform.

The MEMGETBADBLOCK/MEMSETBADBLOCK ioctl are not listed in compat_ioctl.h.

And some variables in jffs2 are declared as uint32_t but used to hold
size_t values.

Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-03-09 19:47:37 -08:00
Arjan van de Ven
858119e159 [PATCH] Unlinline a bunch of other functions
Remove the "inline" keyword from a bunch of big functions in the kernel with
the goal of shrinking it by 30kb to 40kb

Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Acked-by: Jeff Garzik <jgarzik@pobox.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-01-14 18:27:06 -08:00
Thomas Gleixner
182ec4eee3 [JFFS2] Clean up trailing white spaces
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-07 14:18:56 +01:00
Artem B. Bityutskiy
733802d974 [JFFS2] Debug code simplification, update TODO
Simplify the debugging code further.
Update the TODO list

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 22:20:33 +01:00
Artem B. Bityutskiy
f0507530cb [JFFS2] Solve BUG caused by frag->node representing a hole in fragtree
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 20:32:36 +01:00
Artem B. Bityutskiy
280562b210 [JFFS2] Calculate CRC check starting point correctly
When data starts from the beginning of NAND page, 'len' must be zero, not
c->wbuf_page.

Thanks to Zoltan Sogor for reporting this problem.

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 20:29:56 +01:00
Artem B. Bityutskiy
8d5df40954 [JFFS2] More message formatting cleanups
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 20:27:14 +01:00
Artem B. Bityutskiy
45ca1b509e [JFFS2] Debug code clean up - step 7
Remove more noisy debugs. Add current->pid to debug messages.
Remove bogus includes.

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 19:14:35 +01:00
Artem B. Bityutskiy
3c09133739 [JFFS2] Correct buggy length checks
The previous changes introduced wrong length calculations.

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 18:35:36 +01:00
Artem B. Bityutskiy
392435081e [JFFS2] Debug code clean up - step 6
Remove extra noisy debugs

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 18:33:09 +01:00
Artem B. Bityutskiy
1e0da3cb6c [JFFS2] Build fragtree in reverse order
Instead of building fragtree starting from node with the smallest version
number, start from the highest. This helps to avoid reading and checking
obsolete nodes.

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 18:22:17 +01:00
Artem B. Bityutskiy
1e900979a7 [JFFS2] Move another fragtree-related function to nodelist.c
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 18:11:59 +01:00
Artem B. Bityutskiy
e0d601373b [JFFS2] Debug code clean up - step 5
Replace the D1(printk()) style debugging with the new debug macros

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 18:01:24 +01:00
Artem B. Bityutskiy
f97117d153 [JFFS2] Move scattered function into related files
Move functions to read inodes into readinode.c
Move functions to handle fragtree and dentry lists into nodelist.[ch]

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 17:50:45 +01:00
Artem B. Bityutskiy
e0c8e42f8f [JFFS2] Debug code clean up - step 3
Various simplifiactions. printk format corrections.
Convert more code to use the new debug functions.

Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 17:06:49 +01:00
Artem B. Bityutskiy
dae6227f71 [JFFS2] Split a large routine on several smaller.
Signed-off-by: Artem B. Bityutskiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-11-06 15:40:55 +01:00
Artem B. Bityuckiy
e4fef66189 [JFFS2] Rename function and update comments
We recently changed the method of collecting and sorting of
tmp_dnode objects to use a temporary RB-tree instead of a
temporary list. Rename function and update comments.

Signed-off-by: Artem B. Bityuckiy <dedekind@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-07-12 23:58:26 +02:00
Artem B. Bityuckiy
6430a8def1 [JFFS2] Simplify the tree insert code.
It isn't _normal_ that we allow key collision in rbtrees, 
but it does not matter as long as the two nodes with the same
version are together.

Signed-off-by: Artem B. Bityuckiy <dedekind@infradead.org>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-07-06 18:30:00 +02:00
David Woodhouse
9dee7503ce [JFFS2] Optimise jffs2_add_tn_to_list
Use an rbtree instead of a simple linked list. We were wasting 
an amazing amount of time in jffs2_add_tn_to_list(). 
Thanks to Artem Bityuckiy and Jarkko Jlavinen  for noticing.

Signed-off-by: David Woodhouse <dwmw2@infradead.org> 
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-07-06 14:07:54 +02:00
Thomas Gleixner
7d27c8143c [JFFS2] Whitespace cleanup. Fix missing debug message
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-05-23 13:49:40 +02:00
David Woodhouse
7d200960d4 [JFFS2] Fix inode allocation race
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-05-23 13:17:49 +02:00
David Woodhouse
67e345d17f [JFFS2] Prevent ino cache removal for inodes in use
Don't remove inocache for inodes which are in read_inode() or
clear_inode() until they're done.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-05-23 12:46:14 +02:00
Todd Poynor
8fabed4a0f [JFFS2] Avoid warning for empty filesystems
Avoid "Eep. No valid nodes for ino #1" message for just-created filesystem.

Signed-off-by: Todd Poynor <tpoynor@mvista.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2005-05-23 11:30:31 +02:00
Linus Torvalds
1da177e4c3 Linux-2.6.12-rc2
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!
2005-04-16 15:20:36 -07:00