The keyword here is 'twice' to explain the trick.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Add SPDX License Identifier to all .gitignore files.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Even though INITRAMFS_SOURCE kconfig option isn't set in most of
defconfigs it is used (set) extensively by various build systems.
Commit f26661e1 ("initramfs: make initramfs compression choice
non-optional") has changed default compression mode. Previously we
compress initramfs using available compression algorithm. Now
we don't use any compression at all by default.
It significantly increases the image size in case of build system
chooses embedded initramfs. Initially I faced with this issue while
using buildroot.

As of today it's not possible to set preferred compression mode
in target defconfig as this option depends on INITRAMFS_SOURCE
being set. Modification of all build systems either doesn't look
like good option.

Let's instead rewrite initramfs compression mode choices list
the way that "INITRAMFS_COMPRESSION_NONE" will be the last option
in the list. In that case it will be chosen only if all other
options (which implements any compression) are not available.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Since commit ddd09bcc ("initramfs: make compression options not
depend on INITRAMFS_SOURCE"), Kconfig asks the compression mode for
the built-in initramfs regardless of INITRAMFS_SOURCE.

It is technically simpler, but pointless from a UI perspective,
Linus says [1].

When INITRAMFS_SOURCE is empty, usr/Makefile creates a tiny default
cpio, which is so small that nobody cares about the compression.

This commit hides the Kconfig choice in that case. The default cpio
is embedded without compression, which was the original behavior.

[1]: https://lkml.org/lkml/2020/2/1/160



Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

In old days, the "host-progs" syntax was used for specifying host
programs. It was renamed to the current "hostprogs-y" in 2004.

It is typically useful in scripts/Makefile because it allows Kbuild to
selectively compile host programs based on the kernel configuration.

This commit renames like follows:

  always       ->  always-y
  hostprogs-y  ->  hostprogs

So, scripts/Makefile will look like this:

  always-$(CONFIG_BUILD_BIN2C) += ...
  always-$(CONFIG_KALLSYMS)    += ...
      ...
  hostprogs := $(always-y) $(always-m)

I think this makes more sense because a host program is always a host
program, irrespective of the kernel configuration. We want to specify
which ones to compile by CONFIG options, so always-y will be handier.

The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
compatibility for a while.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

This script sets the -e option, so it exits on any error, in which
case it exits without cleaning up the intermediate cpio_list.

Make sure to delete it on exit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Currently, usr/gen_initramfs.sh takes care of all the use-cases:

 [1] generates a cpio file unless CONFIG_INITRAMFS_SOURCE points to
     a single cpio archive

 [2] If CONFIG_INITRAMFS_SOURCE is the path to a cpio archive,
     use it as-is.

 [3] Compress the cpio file according to CONFIG_INITRAMFS_COMPRESSION_*
     unless it is passed a compressed archive.

To simplify the script, move [2] and [3] to usr/Makefile.

If CONFIG_INITRAMFS_SOURCE is the path to a cpio archive, there is
no need to run this shell script.

For the cpio archive compression, you can re-use the rules from
scripts/Makefile.lib .

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Currently, this script outputs a cpio file when -o option is
given, but otherwise a text file in the format recognized by
gen_init_cpio.

This behavior is unclear. Make it always output a cpio file.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

When CONFIG_INITRAMFS_SOURCE is empty, the Makefile passes the -d
option to gen_initramfs.sh to create the default initramfs, which
contains /dev, /dev/console, and /root.

This commit simplifies the default behavior; remove the -d option,
and add the default cpio list.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Currently, this script is run twice, for the dependency list, and then
for the cpio archive.

The first one is re-run every time although its build log is suppressed
so nobody notices it.

Make it work more efficiently by generating the cpio and the dependency
list at the same time.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Specify the dependency directly in the Makefile.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Greg Thelen <gthelen@google.com>

Currently, the choice of the initramfs compression mode is too complex
because users are allowed to not specify the compression mode at all.

I think it makes more sense to require users to choose the compression
mode explicitly, and delete the fallback defaults of INITRAMFS_COMPRESSION.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Even if INITRAMFS_SOURCE is empty, usr/gen_initramfs.sh generates a
tiny default initramfs, which is embedded in vmlinux.

So, defining INITRAMFS_COMPRESSION* options should be valid irrespective
of INITRAMFS_SOURCE.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

init/Kconfig includes usr/Kconfig inside the "if BLK_DEV_INITRD" ...
"endif" block:

    if BLK_DEV_INITRD

    source "usr/Kconfig"

    endif

Hence, all the defines in usr/Kconfig depend on BLK_DEV_INITRD.

Remove the redundant "depends on BLK_DEV_INITRD".

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Greg Thelen <gthelen@google.com>

The comments in usr/Makefile wrongly refer to the script name (twice).

Line 37:
    # The dependency list is generated by gen_initramfs.sh -l

Line 54:
    # 4) Arguments to gen_initramfs.sh changes

There does not exist such a script.

I was going to fix the comments, but after some consideration, I thought
"gen_initramfs.sh" would be more suitable than "gen_initramfs_list.sh"
because it generates an initramfs image in the common usage.

The script generates a list that can be fed to gen_init_cpio only when
it is directly run without -o or -l option.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

There is no tool named "gen_initramfs". The correct name is
"gen_init_cpio".

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Greg Thelen <gthelen@google.com>

This is assigned, but not referenced.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Greg Thelen <gthelen@google.com>

'klibcdirs' was added by commit d39a206b ("kbuild: rebuild initramfs
if content of initramfs changes"). If this is just a matter of forcing
execution of the recipe line, we can replace it with FORCE.

The following code is currently useless:

   $(deps_initramfs): klibcdirs

The original intent could be a hook for the klibc integration into the
kernel tree, but klibc is a separate project, which can be built
independently. Clean it up.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Greg Thelen <gthelen@google.com>

Since commit d2a99dbd ("kbuild: update compile-test header list for
v5.5-rc1"), this does not depend on any CONFIG option.

no-header-test is clearer.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Adding an include guard to every header file is good practice in case
it is included multiple times.

Exported headers are compile-tested for the comprehensive sanity checks.
Let's include the same header twice. If an include guard is missing,
the header will fail to build due to redefinition of something.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>

Prior to commit 858805b3 ("kbuild: add $(BASH) to run scripts with
bash-extension"), this shell script was almost always run by bash since
bash is usually installed on the system by default.

Now, this script is run by sh, which might be a symlink to dash. On such
distributions, the following code emits an error:

  local dev=`LC_ALL=C ls -l "${location}"`

You can reproduce the build error, for example by setting
CONFIG_INITRAMFS_SOURCE="/dev".

    GEN     usr/initramfs_data.cpio.gz
  ./usr/gen_initramfs_list.sh: 131: local: 1: bad variable name
  make[1]: *** [usr/Makefile:61: usr/initramfs_data.cpio.gz] Error 2

This is because `LC_ALL=C ls -l "${location}"` contains spaces.
Surrounding it with double-quotes fixes the error.

Fixes: 858805b3 ("kbuild: add $(BASH) to run scripts with bash-extension")
Reported-by: Jory A. Pratt <anarchy@gentoo.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Since commit fcbb8461 ("kbuild: remove header compile test"),
'make clean' with O= option in the pristine source tree emits
'No such file or directory' warning.

$ git clean -d -f -x
$ make O=foo clean
make[1]: Entering directory '/home/masahiro/linux/foo'
find: ‘usr/include’: No such file or directory
make[1]: Leaving directory '/home/masahiro/linux/foo'

Fixes: fcbb8461 ("kbuild: remove header compile test")
Reported-by: kbuild test robot <lkp@intel.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Userspace cannot compile <asm/sembuf.h> due to some missing type
definitions.  For example, building it for x86 fails as follows:

    CC      usr/include/asm/sembuf.h.s
  In file included from <command-line>:32:0:
  usr/include/asm/sembuf.h:17:20: error: field `sem_perm' has incomplete type
    struct ipc64_perm sem_perm; /* permissions .. see ipc.h */
                      ^~~~~~~~
  usr/include/asm/sembuf.h:24:2: error: unknown type name `__kernel_time_t'
    __kernel_time_t sem_otime; /* last semop time */
    ^~~~~~~~~~~~~~~
  usr/include/asm/sembuf.h:25:2: error: unknown type name `__kernel_ulong_t'
    __kernel_ulong_t __unused1;
    ^~~~~~~~~~~~~~~~
  usr/include/asm/sembuf.h:26:2: error: unknown type name `__kernel_time_t'
    __kernel_time_t sem_ctime; /* last change time */
    ^~~~~~~~~~~~~~~
  usr/include/asm/sembuf.h:27:2: error: unknown type name `__kernel_ulong_t'
    __kernel_ulong_t __unused2;
    ^~~~~~~~~~~~~~~~
  usr/include/asm/sembuf.h:29:2: error: unknown type name `__kernel_ulong_t'
    __kernel_ulong_t sem_nsems; /* no. of semaphores in array */
    ^~~~~~~~~~~~~~~~
  usr/include/asm/sembuf.h:30:2: error: unknown type name `__kernel_ulong_t'
    __kernel_ulong_t __unused3;
    ^~~~~~~~~~~~~~~~
  usr/include/asm/sembuf.h:31:2: error: unknown type name `__kernel_ulong_t'
    __kernel_ulong_t __unused4;
    ^~~~~~~~~~~~~~~~

It is just a matter of missing include directive.

Include <asm/ipcbuf.h> to make it self-contained, and add it to
the compile-test coverage.

Link: http://lkml.kernel.org/r/20191030063855.9989-3-yamada.masahiro@socionext.com


Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Userspace cannot compile <asm/msgbuf.h> due to some missing type
definitions.  For example, building it for x86 fails as follows:

    CC      usr/include/asm/msgbuf.h.s
  In file included from usr/include/asm/msgbuf.h:6:0,
                   from <command-line>:32:
  usr/include/asm-generic/msgbuf.h:25:20: error: field `msg_perm' has incomplete type
    struct ipc64_perm msg_perm;
                      ^~~~~~~~
  usr/include/asm-generic/msgbuf.h:27:2: error: unknown type name `__kernel_time_t'
    __kernel_time_t msg_stime; /* last msgsnd time */
    ^~~~~~~~~~~~~~~
  usr/include/asm-generic/msgbuf.h:28:2: error: unknown type name `__kernel_time_t'
    __kernel_time_t msg_rtime; /* last msgrcv time */
    ^~~~~~~~~~~~~~~
  usr/include/asm-generic/msgbuf.h:29:2: error: unknown type name `__kernel_time_t'
    __kernel_time_t msg_ctime; /* last change time */
    ^~~~~~~~~~~~~~~
  usr/include/asm-generic/msgbuf.h:41:2: error: unknown type name `__kernel_pid_t'
    __kernel_pid_t msg_lspid; /* pid of last msgsnd */
    ^~~~~~~~~~~~~~
  usr/include/asm-generic/msgbuf.h:42:2: error: unknown type name `__kernel_pid_t'
    __kernel_pid_t msg_lrpid; /* last receive pid */
    ^~~~~~~~~~~~~~

It is just a matter of missing include directive.

Include <asm/ipcbuf.h> to make it self-contained, and add it to
the compile-test coverage.

Link: http://lkml.kernel.org/r/20191030063855.9989-2-yamada.masahiro@socionext.com


Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Userspace cannot compile <asm/ipcbuf.h> due to some missing type
definitions.  For example, building it for x86 fails as follows:

    CC      usr/include/asm/ipcbuf.h.s
  In file included from usr/include/asm/ipcbuf.h:1:0,
                   from <command-line>:32:
  usr/include/asm-generic/ipcbuf.h:21:2: error: unknown type name `__kernel_key_t'
    __kernel_key_t  key;
    ^~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:22:2: error: unknown type name `__kernel_uid32_t'
    __kernel_uid32_t uid;
    ^~~~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:23:2: error: unknown type name `__kernel_gid32_t'
    __kernel_gid32_t gid;
    ^~~~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:24:2: error: unknown type name `__kernel_uid32_t'
    __kernel_uid32_t cuid;
    ^~~~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:25:2: error: unknown type name `__kernel_gid32_t'
    __kernel_gid32_t cgid;
    ^~~~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:26:2: error: unknown type name `__kernel_mode_t'
    __kernel_mode_t  mode;
    ^~~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:28:35: error: `__kernel_mode_t' undeclared here (not in a function)
    unsigned char  __pad1[4 - sizeof(__kernel_mode_t)];
                                     ^~~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:31:2: error: unknown type name `__kernel_ulong_t'
    __kernel_ulong_t __unused1;
    ^~~~~~~~~~~~~~~~
  usr/include/asm-generic/ipcbuf.h:32:2: error: unknown type name `__kernel_ulong_t'
    __kernel_ulong_t __unused2;
    ^~~~~~~~~~~~~~~~

It is just a matter of missing include directive.

Include <linux/posix_types.h> to make it self-contained, and add it to
the compile-test coverage.

Link: http://lkml.kernel.org/r/20191030063855.9989-1-yamada.masahiro@socionext.com


Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Userspace cannot compile <linux/scc.h>

    CC      usr/include/linux/scc.h.s
  In file included from <command-line>:32:0:
  usr/include/linux/scc.h:20:20: error: `SIOCDEVPRIVATE' undeclared here (not in a function)
    SIOCSCCRESERVED = SIOCDEVPRIVATE,
                      ^~~~~~~~~~~~~~

Include <linux/sockios.h> to make it self-contained, and add it to the
compile-test coverage.

Link: http://lkml.kernel.org/r/20191108055809.26969-1-yamada.masahiro@socionext.com


Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Currently, some sanity checks for uapi headers are done by
scripts/headers_check.pl, which is wired up to the 'headers_check'
target in the top Makefile.

It is true compiling headers has better test coverage, but there
are still several headers excluded from the compile test. I like
to keep headers_check.pl for a while, but we can delete a lot of
code by moving the build rule to usr/include/Makefile.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

There are both positive and negative options about this feature.
At first, I thought it was a good idea, but actually Linus stated a
negative opinion (https://lkml.org/lkml/2019/9/29/227

). I admit it
is ugly and annoying.

The baseline I'd like to keep is the compile-test of uapi headers.
(Otherwise, kernel developers have no way to ensure the correctness
of the exported headers.)

I will maintain a small build rule in usr/include/Makefile.
Remove the other header test functionality.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Since commit 707816c8 ("netfilter: remove deprecation warnings from
uapi headers."), you can compile linux/netfilter_ipv4/ipt_LOG.h and
linux/netfilter_ipv6/ip6t_LOG.h without warnings.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit 6dc280eb ("coda: remove uapi/linux/coda_psdev.h") removed
a header in question. Some more build errors were fixed. Add more
headers into the test coverage.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

When CONFIG_UAPI_HEADER_TEST=y, exported headers are compile-tested to
make sure they can be included from user-space.

Currently, linux/netfilter_bridge/ebtables.h is excluded from the test
coverage. To make it join the compile-test, we need to fix the build
errors attached below.

For a case like this, we decided to use __u{8,16,32,64} variable types
in this discussion:

  https://lkml.org/lkml/2019/6/5/18



Build log:

  CC      usr/include/linux/netfilter_bridge/ebtables.h.s
In file included from <command-line>:32:0:
./usr/include/linux/netfilter_bridge/ebtables.h:126:4: error: unknown type name ‘uint8_t’
    uint8_t revision;
    ^~~~~~~
./usr/include/linux/netfilter_bridge/ebtables.h:139:4: error: unknown type name ‘uint8_t’
    uint8_t revision;
    ^~~~~~~
./usr/include/linux/netfilter_bridge/ebtables.h:152:4: error: unknown type name ‘uint8_t’
    uint8_t revision;
    ^~~~~~~

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>

Since 9e3596b0 ("kbuild: initramfs cleanup, set target from Kconfig")
"make clean" leaves behind compressed initramfs images.  Example:

  $ make defconfig
  $ sed -i 's|CONFIG_INITRAMFS_SOURCE=""|CONFIG_INITRAMFS_SOURCE="/tmp/ir.cpio"|' .config
  $ make olddefconfig
  $ make -s
  $ make -s clean
  $ git clean -ndxf | grep initramfs
  Would remove usr/initramfs_data.cpio.gz

clean rules do not have CONFIG_* context so they do not know which
compression format was used.  Thus they don't know which files to delete.

Tell clean to delete all possible compression formats.

Once patched usr/initramfs_data.cpio.gz and friends are deleted by
"make clean".

Link: http://lkml.kernel.org/r/20190722063251.55541-1-gthelen@google.com


Fixes: 9e3596b0 ("kbuild: initramfs cleanup, set target from Kconfig")
Signed-off-by: Greg Thelen <gthelen@google.com>
Cc: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

The only the difference between clean-files and clean-dirs is the -r
option passed to the 'rm' command.

You can always pass -r, and then remove the clean-dirs syntax.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Masahiro Yamada changed the zcrypt.h header file to use __u{16,32,64}
instead of uint{16,32,64}_t with ("s390: use __u{16,32,64} instead of
uint{16,32,64}_t in uapi header").

This makes all s390 header files pass - remove zcrypt.h from the blacklist.

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>

 - Some headers graduated from the blacklist

 - hyperv_timer.h joined the header-test when CONFIG_X86=y

 - nf_tables*.h joined the header-test when CONFIG_NF_TABLES is
   enabled.

 - The entry for nf_tables_offload.h was added to fix build error for
   the combination of CONFIG_NF_TABLES=n and CONFIG_KERNEL_HEADER_TEST=y.

 - The entry for iomap.h was added because this header is supposed to
   be included only when CONFIG_BLOCK=y

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Those documents describe a kAPI. So, add to the driver-api
book.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

The two files there describes a Kernel API feature, used to
support early userspace stuff. Prepare for moving them to
the kernel API book by converting to ReST format.

The conversion itself was quite trivial: just add/mark a few
titles as such, add a literal block markup, add a table markup
and a few blank lines, in order to make Sphinx to properly parse it.

At its new index.rst, let's add a :orphan: while this is not linked to
the main index.rst file, in order to avoid build warnings.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

Multiple people have suggested compile-testing UAPI headers to ensure
they can be really included from user-space. "make headers_check" is
obviously not enough to catch bugs, and we often leak unresolved
references to user-space.

Use the new header-test-y syntax to implement it. Please note exported
headers are compile-tested with a completely different set of compiler
flags. The header search path is set to $(objtree)/usr/include since
exported headers should not include unexported ones.

We use -std=gnu89 for the kernel space since the kernel code highly
depends on GNU extensions. On the other hand, UAPI headers should be
written in more standardized C, so they are compiled with -std=c90.
This will emit errors if C++ style comments, the keyword 'inline', etc.
are used. Please use C style comments (/* ... */), '__inline__', etc.
in UAPI headers.

There is additional compiler requirement to enable this test because
many of UAPI headers include <stdlib.h>, <sys/ioctl.h>, <sys/time.h>,
etc. directly or indirectly. You cannot use kernel.org pre-built
toolchains [1] since they lack <stdlib.h>.

I reused CONFIG_CC_CAN_LINK to check the system header availability.
The intention is slightly different, but a compiler that can link
userspace programs provide system headers.

For now, a lot of headers need to be excluded because they cannot
be compiled standalone, but this is a good start point.

[1] https://mirrors.edge.kernel.org/pub/tools/crosstool/index.html



Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>

Signed-off-by: Eugen Cazacu <green182day@hotmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

scripts/gen_initramfs_list.sh is only invoked from usr/Makefile.
Move it so that all tools to create initramfs are self-contained
in the usr/ directory.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>