David Oberhollenzer | 25 Jul 17:02 2016

mtd-utils: Swtich to autotools based build system

This patch series is based on Richards RFC to replace the hand
written Makefiles with an autoconf/automake based build system
to gain all the advantages of having an _actual_ build system
that is high level, widely used, well known and mature/stable.

Switching to the GNU buildsystem should make it a lot easier for
people to [cross] compile the mtd-utils and integrating them into
existing, automated [cross] build toolchains, as the autotools are
very widely used/known, effectively representing a pseudo-standard
that people are familiar with and that a lot of tools have been
adapted to.

The GNU build system also had a few more decades time and a _lot_
more testing to mature in regards to cross compilation setups and
toolchains than the previous mtd-utils build system that would have
to be further supported and maintained in its entirety.

The proposed autotools based build system should also be easier to
maintain than a hurdle of makefiles. The automake files are more
high level and only contain a list of input files for each program.
Generating the makefiles with precise dependencies and build steps
is done by the build system.

Furthermore, files like the test scripts can fairly easily be
processed with autoconf at compile time to (for instance) replace
file locations with the actual path where a program is configured
to be installed.

Linux MTD discussion mailing list
(Continue reading)

Iosif Harutyunov | 21 Jul 04:07 2016

[PATCH] mtd: ubi: Fix race condition between ubi device creation and udev

While implementing udev rules for UBI device I ran into the problem when udev would
ignore UBI rules I created to process attachment of volume. Interestingly, when I trigger
udev ubi subsystem "change" event manually after the attachment, rules would work just fine.

I traced problem down to UBI sysfs processing, which turned out to be racing condition.
See patch below to address the problem.


Signed-off-by: Iosif Harutyunov <iharutyunov <at> sonicwall.com>
Once ubi is attached, make sure ubi_devices[] is initialized early before being
used in the dev_attribute_show().

This is to prevent race condition between udev ubi rules processing and ubi device
creation, which manifests itself ignoring udev ATTR rules.

 drivers/mtd/ubi/build.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index ef36182..e6117d7 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
 <at>  <at>  -986,6 +986,8  <at>  <at>  int ubi_attach_mtd_dev(struct mtd_info *mtd, int ubi_num,
 		goto out_free;
(Continue reading)

Daniel Golle | 19 Jul 00:26 2016

[PATCH for-4.8] ubifs: silence early error messages if MS_SILENT is set

Probe-mounting a volume too small for UBIFS results in kernel log
polution which might irritate users.
Address this by silencing errors which may happen during boot if the
rootfs is e.g. squashfs (and thus rather small) stored on a UBI volume.
This commit should be seen in context with
commit 65fb47721925628bf85ec08c4302ac5e7224a66a
commit 90bea5a3f0bf680b87b90516f3c231997f4b8f3b

Signed-off-by: Daniel Golle <daniel <at> makrotopia.org>
 fs/ubifs/super.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/fs/ubifs/super.c b/fs/ubifs/super.c
index 736dd58..4ec0510 100644
--- a/fs/ubifs/super.c
+++ b/fs/ubifs/super.c
 <at>  <at>  -520,19 +520,19  <at>  <at>  static int init_constants_early(struct ubifs_info *c)
 	c->max_write_shift = fls(c->max_write_size) - 1;

 	if (c->leb_size < UBIFS_MIN_LEB_SZ) {
-		ubifs_err(c, "too small LEBs (%d bytes), min. is %d bytes",
-			  c->leb_size, UBIFS_MIN_LEB_SZ);
+		ubifs_errc(c, "too small LEBs (%d bytes), min. is %d bytes",
+			   c->leb_size, UBIFS_MIN_LEB_SZ);
 		return -EINVAL;

 	if (c->leb_cnt < UBIFS_MIN_LEB_CNT) {
(Continue reading)

kbuild test robot | 18 Jul 21:43 2016

[mtd-next:master 30/33] drivers/mtd/spi-nor/cadence-quadspi.c:529:4: error: implicit declaration of function 'readsl'

tree:   git://git.infradead.org/linux-mtd-next.git master
head:   f78921b9020c510ed222a6c2402e2aa126432415
commit: 140623410536905fa6ab737b625decfde6c64a72 [30/33] mtd: spi-nor: Add driver for Cadence Quad
SPI Flash Controller
config: x86_64-allmodconfig (attached as .config)
compiler: gcc-6 (Debian 6.1.1-1) 6.1.1 20160430
        git checkout 140623410536905fa6ab737b625decfde6c64a72
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

   drivers/mtd/spi-nor/cadence-quadspi.c: In function 'cqspi_indirect_read_execute':
>> drivers/mtd/spi-nor/cadence-quadspi.c:529:4: error: implicit declaration of function 'readsl' [-Werror=implicit-function-declaration]
       readsl(ahb_base, rxbuf, DIV_ROUND_UP(bytes_to_read, 4));
   drivers/mtd/spi-nor/cadence-quadspi.c: In function 'cqspi_indirect_write_execute':
>> drivers/mtd/spi-nor/cadence-quadspi.c:613:3: error: implicit declaration of function 'writesl' [-Werror=implicit-function-declaration]
      writesl(cqspi->ahb_base, txbuf, DIV_ROUND_UP(write_bytes, 4));
   cc1: some warnings being treated as errors

vim +/readsl +529 drivers/mtd/spi-nor/cadence-quadspi.c

   523			}
   525			while (bytes_to_read != 0) {
   526				bytes_to_read *= cqspi->fifo_width;
   527				bytes_to_read = bytes_to_read > remaining ?
(Continue reading)

Rafał Miłecki | 18 Jul 11:18 2016

[PATCH mtd-utils] nanddump: check write function result for errors

Errors may happen, it's e.g. easy on embedded devices to run out of space
when dumping big partitions. This patch adds a helper function for
writing. It deals with partial writes and just returns 0 on success or
error number.

The old code didn't check for errors at all which could result in
incomplete dumps without exiting with an error.

Signed-off-by: Rafał Miłecki <zajec5 <at> gmail.com>
 nand-utils/nanddump.c | 50 ++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 44 insertions(+), 6 deletions(-)

diff --git a/nand-utils/nanddump.c b/nand-utils/nanddump.c
index 4ee7ed4..0bbc97f 100644
--- a/nand-utils/nanddump.c
+++ b/nand-utils/nanddump.c
 <at>  <at>  -293,6 +293,25  <at>  <at>  nil:
 	linebuf[lx++] = '\0';

+ * ofd_write - writes whole buffer to the file associated with a descriptor
+ *
+ * On failure an error (negative number) is returned. Otherwise 0 is returned.
+ */
+static int ofd_write(int ofd, const unsigned char *buf, size_t nbyte)
+	ssize_t bytes;
(Continue reading)

Hector Palacios | 18 Jul 10:39 2016

[PATCH] mtd: nand: fix bug writing 1 byte less than page size

nand_do_write_ops() determines if it is writing a partial page with the
	part_pagewr = (column || writelen < (mtd->writesize - 1))

When 'writelen' is exactly 1 byte less than the NAND page size the formula
equates to zero, so the code doesn't process it as a partial write,
although it should.
As a consequence the function remains in the while(1) loop with 'writelen'
becoming 0xffffffff and iterating endlessly.

The bug may not be easy to reproduce in Linux since user space tools
usually force the padding or round-up the write size to a page-size
This was discovered in U-Boot where the issue can be reproduced by
writing any size that is 1 byte less than a page-size multiple.
For example, on a NAND with 2K page (0x800):
	=> nand erase.part <partition>
	=> nand write $loadaddr <partition> 7ff

Signed-off-by: Hector Palacios <hector.palacios <at> digi.com>
 drivers/mtd/nand/nand_base.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index 0b0dc29d2af7..77533f7f2429 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
 <at>  <at>  -2610,7 +2610,7  <at>  <at>  static int nand_do_write_ops(struct mtd_info *mtd, loff_t to,
 		int cached = writelen > bytes && page != blockmask;
(Continue reading)

Thomas Petazzoni | 16 Jul 11:48 2016

[PATCH mtd-utils] integck: only use execinfo.h when INTEGCK_DEBUG is enabled

Guard the usage of execinfo.h by INTEGCK_DEBUG so that by defaut,
integck builds properly on systems without <execinfo.h> (uClibc and
musl based systems). As stated in the code, the backtrace()
functionality of <execinfo.h> will anyway only work properly when
INTEGCK_DEBUG is defined (it makes all functions non-static, which is
needed for backtrace to provide some useful information).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni <at> free-electrons.com>
 tests/fs-tests/integrity/integck.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tests/fs-tests/integrity/integck.c b/tests/fs-tests/integrity/integck.c
index 8badd1f..6ef817e 100644
--- a/tests/fs-tests/integrity/integck.c
+++ b/tests/fs-tests/integrity/integck.c
 <at>  <at>  -31,7 +31,9  <at>  <at> 
 #include <getopt.h>
 #include <assert.h>
 #include <mntent.h>
 #include <execinfo.h>
 #include <bits/stdio_lim.h>
 #include <sys/mman.h>
 #include <sys/vfs.h>
 <at>  <at>  -248,14 +250,18  <at>  <at>  static char *random_name_buf;
 static void check_failed(const char *cond, const char *func, const char *file,
 			 int line)
(Continue reading)

Richard Weinberger | 15 Jul 14:57 2016

[RFC][PATCH] ubi: Implement a read cache

Implement a simple read cache such that small adjacent reads
with in the same NAND page can be cached.
When a read smaller than a NAND page is requested, UBI reads the full
page and caches it.
NAND core has already such a cache but it will go away soon.

To be able to benchmark better, there is also a debugfs file which reports
cache stats, misses, hits and no_query.
Misses and hits should be clear, no_query is incremented when the cache
cannot be used, i.e. for reads larger than a page size.

This patch is just a PoC.

Signed-off-by: Richard Weinberger <richard <at> nod.at>

This is an initial draft for a read cache implemented in UBI.

The cache sits in ubi_eba_read_leb() and is used when the read length is
smaller than a NAND page and does not cross a page boundary, this can be
improved later.
Currently the cache consists of 61 cache lines where the PEB acts
as selector.
Maybe less cache lines would also do it, this needs to be benchmarked
in more detail.

I did already some benchmarks.
The system I've used has a 4GiB MLC NAND with 16KiB page size.
Userspace is a Debian 8 with systemd.
(Continue reading)

Sgt Debra Moore | 13 Jul 19:15 2016


Mutual benefit for the both of us from Camp Stanley, get back to me for more info.

Linux MTD discussion mailing list

Richard Weinberger | 13 Jul 14:29 2016

UBIFS orphans and ro-mounts


I wonder why UBIFS processes orphan inodes also when mounting read-only.
Depending on the workload before a power cut this can take a few seconds
and since we mount read-only the updated TNC will not written back to flash,
so upon next mount the time is again wasted.

This hurts boot performance a lot on systems where u-boot loads the kernel
from UBIFS.
u-boot always mounts UBIFS read-only.
So even when Linux mounts UBIFS rw and a powercut happens orphan processing
will happen twice.

I'd suggest calling ubifs_mount_orphans() only when we mount rw or remount to
What do you think?


sigma star gmbh - Eduard-Bodem-Gasse 6 - 6020 Innsbruck - Austria
ATU66964118 - FN 374287y

Linux MTD discussion mailing list

Richard Weinberger | 13 Jul 14:30 2016

Cached NAND reads and UBIFS


As discussed on IRC, Boris and I figured that on our target UBIFS is sometimes
very slow.
i.e. deleting a 1GiB file right after a reboot takes more than 30 seconds.

When deleting a file with a cold TNC UBIFS has to lookup a lot of znodes
on the flash.
For every single znode lookup UBIFS requests a few bytes from the flash.
This is slow.

After some investigation we found out that the NAND read cache is disabled
when the NAND driver supports reading subpages.
So we removed the NAND_SUBPAGE_READ flag from the driver and suddenly
lookups were fast. Really fast. Deleting a 1GiB took less than 5 seconds.
Since on our MLC NAND a page is 16KiB many znodes can be read very fast
directly out of the NAND read cache.
The read cache helps here a lot because in the regular case UBIFS' index
nodes are linearly stored in a LEB.

The TNC seems to assume that it can do a lot of short reads since the NAND
read cache will help.
But as soon subpage reads are possible this assumption is no longer true.

Now we're not sure what do do, should we implement bulk reading in the TNC
code or improve NAND read caching?


(Continue reading)