In int-ll64.h, we always use the following typedefs:
typedef unsigned int u32;
typedef unsigned long uintptr_t;
typedef unsigned long long u64;
This does not need to match to the compiler's <inttypes.h>.
Do not include it.
The use of PRI* makes the code super-ugly. You can simply use
"l" for printing uintptr_t, "ll" for u64, and no modifier for u32.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.
In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.
This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.
Signed-off-by: Tom Rini <trini@konsulko.com>
Some fixes when reading EXT files and directory entries were identified
after using e2fuzz to corrupt an EXT3 filesystem:
- Stop reading directory entries if the offset becomes badly aligned.
- Avoid overwriting memory by clamping the length used to zero the buffer
in ext4fs_read_file. Also sanity check blocksize.
Signed-off-by: Ian Ray <ian.ray@ge.com>
Signed-off-by: Martyn Welch <martyn.welch@collabora.co.uk>
Reviewed-by: Stefano Babic <sbabic@denx.de>
The current code doesn't compute the group descriptor checksum correctly
for the filesystems that e2fsprogs 1.43.4 creates (they have
'Group descriptor size: 64' as reported by tune2fs). Extend the checksum
calculation to be done as ext4_group_desc_csum() does in Linux.
This fixes these errors in dmesg from running fs-test.sh and makes it
succeed again:
[1671902.620699] EXT4-fs (loop1): ext4_check_descriptors: Checksum for group 0 failed (35782!=10965)
[1671902.620706] EXT4-fs (loop1): group descriptors corrupted!
Signed-off-by: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
genext2fs creates revision level 0 filesystems, which are not readable
by u-boot due to the initialized group descriptor size field.
f798b1dda1
Reported-by: Kever Yang <kever.yang@rock-chips.com>
Reported-by: FrostyBytes@protonmail.com
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Tested-by: Kever Yang <kever.yang@rock-chips.com>
A sparse file may have regions not mapped by any extents, at the start
or at the end of the file, or anywhere between, thus not finding a
matching extent region is never an error.
Found by python filesystem tests.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Instead of creating a journal entry for each directory block, even
if the block is unmodified, only log the modified block.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
The direntlen checks were quite bogus, i.e. the loop termination used
"len + offset == blocksize" (exact match only), and checked for a
direntlen less than 0. The latter can never happen as the len is
unsigned, this has been reported by Coverity, CID 153384.
Use the same code as in search_dir for directory traversal. This code
has the correct checks for direntlen >= sizeof(struct dirent), and
offset < blocksize.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reported-by: Coverity (CID: 153383, 153384)
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
Use the same variable names as in search_dir, to make purpose of variables
more obvious.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Tom Rini <trini@konsulko.com>
Enable mounting of ext4 fs with 64bit feature, as it is supported now.
These had been disabled in 6f94ab6656.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
The descriptor size is variable, thus array indices are not generically
applicable. The larger group descriptors also contain e.g. high parts
of block numbers, which have to be read and written.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
The correct descriptor size must be used when calculating offsets, and
also to read the correct amount of data.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
The helper functions encapsulate access of the block group descriptors,
independent of group descriptor size. The helpers also deal with the
endianess of the fields, and with split fields like free_blocks/
free_blocks_high.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
If EXT4_FEATURE_INCOMPAT_64BIT is set, the descriptor can be read from
the superblocks, otherwise it defaults to 32.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
read_allocated block may return block number 0, which is just an indicator
a chunk of the file is not backed by a block, i.e. it is sparse.
During file deletions, just continue with the next logical block, for other
operations treat blocknumber <= 0 as an error.
For writes, blocknumber 0 should never happen, as U-Boot always allocates
blocks for the whole file. Reading already handles this correctly, i.e. the
read buffer is 0-fillled.
Not treating block 0 as sparse block leads to FS corruption, e.g.
./sandbox/u-boot -c 'host bind 0 ./sandbox/test/fs/3GB.ext4.img ;
ext4write host 0 0 /2.5GB.file 1 '
The 2.5GB.file from the fs test is actually a sparse file.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
If the blocksize is 1024, count is initialized with 1. Incrementing count
by 8 will never match (count == fs->blksz * 8), and ptr may be
incremented beyond the buffer end if the bitmap is filled. Add the
startblock offset after the loop.
Remove the second loop, as only the first iteration will be done.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
The last free block of a block group may be in its middle. After it has
been allocated, the next block group should be scanned from its beginning.
The following command triggers the bad behaviour (on a blocksize 1024 fs):
./sandbox/u-boot -c 'i=0; host bind 0 ./disk.raw ;
while test $i -lt 260 ; do echo $i; setexpr i $i + 1;
ext4write host 0:2 0 /X${i} 0x1450; done ;
ext4write host 0:2 0 /X240 0x2000 ; '
When 'X240' is extended from 5200 byte to 8192 byte, the new blocks should
start from the first free block (8811), but it uses the blocks 8098-8103
and 16296-16297 -- 8103 + 1 + 8192 = 16296. This can be shown with
debugfs, commands 'ffb' and 'stat X240'.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
zero_buffer is never written, thus clearing it is pointless.
journal_buffer is completely initialized by ext4fs_devread (or in case
of failure, not used).
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
e2fsck warns about "Group descriptor 0 marked uninitialized without
feature set."
The bg_itable_unused field is only defined if FEATURE_RO_COMPAT_GDT_CSUM
is set, and should be set (kept) zero otherwise.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
Scanning only the direct blocks of the directory file may falsely report
an existing file as nonexisting, and worse can also lead to creation
of a duplicate entry on file creation.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
Previously, only the last directory block was scanned for available space.
Instead, scan all blocks back to front, and if no sufficient space is
found, eventually append a new block.
Blocks are only appended if the directory does not use extents or the new
block would require insertion of indirect blocks, as the old code does.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
The following command crashes u-boot:
./sandbox/u-boot -c 'i=0; host bind 0 ./sandbox/test/fs/3GB.ext4.img ;
while test $i -lt 200 ; do echo $i; setexpr i $i + 1;
ext4write host 0 0 /foobar${i} 0; done'
Previously, the code updated the direct_block even for extents, and
fortunately crashed before pushing garbage to the disk.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
In case the dir entry creation failed, ext4fs_write would later overwrite
a random inode, as inodeno was never initialized.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
The following command triggers a segfault in search_dir:
./sandbox/u-boot -c 'host bind 0 ./sandbox/test/fs/3GB.ext4.img ;
ext4write host 0 0 /./foo 0x10'
The following command triggers a segfault in check_filename:
./sandbox/u-boot -c 'host bind 0 ./sandbox/test/fs/3GB.ext4.img ;
ext4write host 0 0 /. 0x10'
"." is the first entry in the directory, thus previous_dir is NULL. The
whole previous_dir block in search_dir seems to be a bad copy from
check_filename(...). As the changed data is not written to disk, the
statement is mostly harmless, save the possible NULL-ptr reference.
Typically a file is unlinked by extending the direntlen of the previous
entry. If the entry is the first entry in the directory block, it is
invalidated by setting inode=0.
The inode==0 case is hard to trigger without crafted filesystems. It only
hits if the first entry in a directory block is deleted and later a lookup
for the entry (by name) is done.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
All fields were accessed directly instead of using the proper byte swap
functions. Thus, ext4 write support was only usable on little-endian
architectures. Fix this.
Signed-off-by: Michael Walle <michael@walle.cc>
With e2fsprogs after 1.43 the 64bit and metadata_csum features are
enabled by default. The metadata_csum feature changes how
ext4_group_desc->bg_checksum is calculated, which would break write
support. The 64bit feature however introduces changes such that it
cannot be read by implementations that do not support it. Since we do
not support this, we must not mount it.
Cc: Stephen Warren <swarren@nvidia.com>
Cc: Simon Glass <sjg@chromium.org>
Cc: Lukasz Majewski <l.majewski@samsung.com>
Cc: Stefan Roese <sr@denx.de>
Reported-by: Andrew Bradford <andrew.bradford@kodakalaris.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
The function ext4fs_read_symlink was unable to handle a symlink
which had target name of exactly 60 characters.
Signed-off-by: Ronald Zachariah <rozachar@cisco.com>
Signed-off-by: Stefan Roese <sr@denx.de>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Cc: Tom Rini <trini@konsulko.com>
To ease conversion to driver model, add helper functions which deal with
calling each block device method. With driver model we can reimplement these
functions with the same arguments.
Use inline functions to avoid increasing code size on some boards.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
As noted by Coverity, when we have an error in
alloc_triple_indirect_block we will leak ti_pbuff_start_addr as it's not
being freed. Further inspection here shows that we could also leak
ti_cbuff_start_addr in one corner case so free that as well.
Reported-by: Coverity (CID 131205, 131206)
Signed-off-by: Tom Rini <trini@konsulko.com>
This will allow the implementation to make use of data in the block_dev
structure beyond the base device number. This will be useful so that eMMC
block devices can encompass the HW partition ID rather than treating this
out-of-band. Equally, the existence of the priv field is crying out for
this patch to exist.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
If the ext3 journal gets out of sync with what is written on disk, for
example because of an unexpected power cut, ext4fs_read_file can
return an all-zero directory entry. In that case, ext4fs_iterate_dir
would infinite loop.
This patch detects when a directory entry's direntlen member is 0 and
returns a failure status, which breaks out of the infinite loop. As a
result, U-Boot will not find files that may subsequently be recovered
when the journal is replayed.
This is better behaviour than hanging in an infinite loop, but as a
further improvement maybe U-Boot could interpret the ext3 journal and
actually find the unsynced entries.
Signed-off-by: Thomas Fitzsimmons <fitzsim@cisco.com>
Reviewed-by: Stefan Roese <sr@denx.de>
Since last API changes for files >2GB, the read of symlink is broken as
ext4fs_read_file now returns 0 instead of the length of the actual read.
Signed-off-by: Gary Bisson <gary.bisson@boundarydevices.com>
root_first_block_buffer should be free()d in all cases, not just when an
error occurs. Fix the success exit path of the function to do this.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Tested-by: Lukasz Majewski <l.majewski@samsung.com>
parse_path() malloc()s the entries in the array it's passed. Those
allocations must be free()d by the caller, ext4fs_get_parent_inode_num().
Add code to do this.
For this to work, all the array entries must be dynamically allocated,
rather than a mix of dynamic and static allocations. Fix parse_path() not
to over-write arr[0] with a pointer to statically allocated data.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Tested-by: Lukasz Majewski <l.majewski@samsung.com>
Now that we have a new header file for cache-aligned allocation, we should
move the stack-based allocation macro there also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Change the internal EXT4 functions to use loff_t for offsets.
Signed-off-by: Suriyan Ramasami <suriyan.r@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
[trini: Update common/spl/spl_ext.c]
Signed-off-by: Tom Rini <trini@ti.com>
On 64-bit platforms (like sandbox) 64-bit integers may be 'long' rather
than 'long long'. Use the inttypes header to avoid compiler warnings.
Signed-off-by: Simon Glass <sjg@chromium.org>
ext4fs_allocate_blocks() always allocates at least one block for a file.
If the file size is zero, this causes total_remaining_blocks to
underflow, which then causes an apparent hang while 2^32 blocks are
allocated.
To solve this, check that total_remaining_blocks is non-zero as part of
the loop condition (i.e. before each loop) rather than at the end of
the loop.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
This bug shows up when file stored on the ext4 file system is updated.
The ext4fs_delete_file() is responsible for deleting file's (e.g. uImage)
data.
However some global data (especially ext4fs_indir2_block), which is used
during file deletion are left unchanged.
The ext4fs_indir2_block pointer stores reference to old ext4 double
indirect allocated blocks. When it is unchanged, after file deletion,
ext4fs_write_file() uses the same pointer (since it is already initialized
- i.e. not NULL) to return number of blocks to write. This trunks larger
file when previous one was smaller.
Lets consider following scenario:
1. Flash target with ext4 formatted boot.img (which has uImage [*] on itself)
2. Developer wants to upload their custom uImage [**]
- When new uImage [**] is smaller than the [*] - everything works
correctly - we are able to store the whole smaller file with corrupted
ext4fs_indir2_block pointer
- When new uImage [**] is larger than the [*] - theCRC is corrupted,
since truncation on data stored at eMMC was done.
3. When uImage CRC error appears, then reboot and LTHOR/DFU reflashing causes
proper setting of ext4fs_indir2_block() and after that uImage[**]
is successfully stored (correct uImage [*] metadata is stored at an
eMMC on the first flashing).
Due to above the bug was very difficult to reproduce.
This patch sets default values for all ext4fs_indir* pointers/variables.
Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Code responsible for handling situation when ext4 has block size of 1024B
can be ordered to take less space.
This patch does that for ext4 common and write files.
Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
This reverts commit fc0fc50f38.
The author has asked on the mailing list that we revert this for now as
it breaks write support.
Reported-by: Łukasz Majewski <l.majewski@samsung.com>
Signed-off-by: Tom Rini <trini@ti.com>
In an ext4 filesystem, the inode corresponding to a file has a 60-byte
area which contains an extent header structure and up to 4 extent
structures (5 x 12 bytes).
For files that need more than 4 extents to be represented (either files
larger than 4 x 128MB = 512MB or smaller files but very fragmented),
ext4 creates extent index structures. Each extent index points to a 4KB
physical block where one extent header and additional 340 extents could
be stored.
The current u-boot ext4 code is very inefficient when it tries to load a
file which has extent indexes. For each logical file block the code will
read over and over again the same blocks of 4096 bytes from the disk.
Since the extent tree in a file is always the same, we can cache the
extent structures in memory before actually starting to read the file.
This patch creates a simple linked list of structures holding information
about all the extents used to represent a file. The list is sorted by
the logical block number (ee_block) so that we can easily find the
proper extent information for any file block.
Without this patch, a 69MB file which had just one extent index pointing
to a block with another 6 extents was read in approximately 3 minutes.
With this patch applied the same file can be read in almost 20 seconds.
Signed-off-by: Ionut Nicu <ioan.nicu.ext@nsn.com>
For files where we actually have extent indexes following
an extent header (ext_block->eh_depth != 0), the do/while
loop from ext4fs_get_extent_block() does not select the
proper extent index structure.
For example, if we have:
ext_block->eh_depth = 1
ext_block->eh_entries = 1
fileblock = 0
index[0].ei_block = 0
the do/while loop will exit with i set to 0 and the
ext4fs_get_extent_block() function will return 0, even if
there was a valid extent index structure following the
header.
Signed-off-by: Ionut Nicu <ioan.nicu.ext@nsn.com>
Signed-off-by: Mathias Rulf <mathias.rulf@nsn.com>
Using fs->blksz in ext4fs_get_extent_block() is not
correct since fs->blksz is not initialized on the
read path. Use EXT2_BLOCK_SIZE() instead which will
produce the desired output.
Signed-off-by: Ionut Nicu <ioan.nicu.ext@nsn.com>
Signed-off-by: Mathias Rulf <mathias.rulf@nsn.com>
Curently, we are using 32 bit multiplication to calculate the offset,
so the result will always be 32 bit.
This can silently cause file system corruption when performing a write
operation on partition larger than 4 GiB.
This patch address the issue by simply promoting the terms to 64 bit,
and let compilers decide how to do the multiplication efficiently.
Signed-off-by: Ma Haijun <mahaijuns@gmail.com>
Masahiro Yamada
Tom Rini
Marek Behún
Ian Ray
Tuomas Tynkkynen
Stefan Brüns
Michael Walle
Ronald Zachariah
Simon Glass
Stephen Warren
Thomas Fitzsimmons
Gary Bisson
Suriyan Ramasami
Łukasz Majewski
Tom Rini
Ionut Nicu
Ma Haijun
Wolfgang Denk