1. 31 Mar, 2018 3 commits
    • Andrey Ignatov's avatar
      selftests/bpf: Selftest for sys_connect hooks · 622adafb
      Andrey Ignatov authored
      Add selftest for BPF_CGROUP_INET4_CONNECT and BPF_CGROUP_INET6_CONNECT
      attach types.
      
      Try to connect(2) to specified IP:port and test that:
      * remote IP:port pair is overridden;
      * local end of connection is bound to specified IP.
      
      All combinations of IPv4/IPv6 and TCP/UDP are tested.
      
      Example:
        # tcpdump -pn -i lo -w connect.pcap 2>/dev/null &
        [1] 478
        # strace -qqf -e connect -o connect.trace ./test_sock_addr.sh
        Wait for testing IPv4/IPv6 to become available ... OK
        Load bind4 with invalid type (can pollute stderr) ... REJECTED
        Load bind4 with valid type ... OK
        Attach bind4 with invalid type ... REJECTED
        Attach bind4 with valid type ... OK
        Load connect4 with invalid type (can pollute stderr) libbpf: load bpf \
          program failed: Permission denied
        libbpf: -- BEGIN DUMP LOG ---
        libbpf:
        0: (b7) r2 = 23569
        1: (63) *(u32 *)(r1 +24) = r2
        2: (b7) r2 = 16777343
        3: (63) *(u32 *)(r1 +4) = r2
        invalid bpf_context access off=4 size=4
        [ 1518.404609] random: crng init done
      
        libbpf: -- END LOG --
        libbpf: failed to load program 'cgroup/connect4'
        libbpf: failed to load object './connect4_prog.o'
        ... REJECTED
        Load connect4 with valid type ... OK
        Attach connect4 with invalid type ... REJECTED
        Attach connect4 with valid type ... OK
        Test case #1 (IPv4/TCP):
                Requested: bind(192.168.1.254, 4040) ..
                   Actual: bind(127.0.0.1, 4444)
                Requested: connect(192.168.1.254, 4040) from (*, *) ..
                   Actual: connect(127.0.0.1, 4444) from (127.0.0.4, 56068)
        Test case #2 (IPv4/UDP):
                Requested: bind(192.168.1.254, 4040) ..
                   Actual: bind(127.0.0.1, 4444)
                Requested: connect(192.168.1.254, 4040) from (*, *) ..
                   Actual: connect(127.0.0.1, 4444) from (127.0.0.4, 56447)
        Load bind6 with invalid type (can pollute stderr) ... REJECTED
        Load bind6 with valid type ... OK
        Attach bind6 with invalid type ... REJECTED
        Attach bind6 with valid type ... OK
        Load connect6 with invalid type (can pollute stderr) libbpf: load bpf \
          program failed: Permission denied
        libbpf: -- BEGIN DUMP LOG ---
        libbpf:
        0: (b7) r6 = 0
        1: (63) *(u32 *)(r1 +12) = r6
        invalid bpf_context access off=12 size=4
      
        libbpf: -- END LOG --
        libbpf: failed to load program 'cgroup/connect6'
        libbpf: failed to load object './connect6_prog.o'
        ... REJECTED
        Load connect6 with valid type ... OK
        Attach connect6 with invalid type ... REJECTED
        Attach connect6 with valid type ... OK
        Test case #3 (IPv6/TCP):
                Requested: bind(face:b00c:1234:5678::abcd, 6060) ..
                   Actual: bind(::1, 6666)
                Requested: connect(face:b00c:1234:5678::abcd, 6060) from (*, *)
                   Actual: connect(::1, 6666) from (::6, 37458)
        Test case #4 (IPv6/UDP):
                Requested: bind(face:b00c:1234:5678::abcd, 6060) ..
                   Actual: bind(::1, 6666)
                Requested: connect(face:b00c:1234:5678::abcd, 6060) from (*, *)
                   Actual: connect(::1, 6666) from (::6, 39315)
        ### SUCCESS
        # egrep 'connect\(.*AF_INET' connect.trace | \
        > egrep -vw 'htons\(1025\)' | fold -b -s -w 72
        502   connect(7, {sa_family=AF_INET, sin_port=htons(4040),
        sin_addr=inet_addr("192.168.1.254")}, 128) = 0
        502   connect(8, {sa_family=AF_INET, sin_port=htons(4040),
        sin_addr=inet_addr("192.168.1.254")}, 128) = 0
        502   connect(9, {sa_family=AF_INET6, sin6_port=htons(6060),
        inet_pton(AF_INET6, "face:b00c:1234:5678::abcd", &sin6_addr),
        sin6_flowinfo=0, sin6_scope_id=0}, 128) = 0
        502   connect(10, {sa_family=AF_INET6, sin6_port=htons(6060),
        inet_pton(AF_INET6, "face:b00c:1234:5678::abcd", &sin6_addr),
        sin6_flowinfo=0, sin6_scope_id=0}, 128) = 0
        # fg
        tcpdump -pn -i lo -w connect.pcap 2> /dev/null
        # tcpdump -r connect.pcap -n tcp | cut -c 1-72
        reading from file connect.pcap, link-type EN10MB (Ethernet)
        17:57:40.383533 IP 127.0.0.4.56068 > 127.0.0.1.4444: Flags [S], seq 1333
        17:57:40.383566 IP 127.0.0.1.4444 > 127.0.0.4.56068: Flags [S.], seq 112
        17:57:40.383589 IP 127.0.0.4.56068 > 127.0.0.1.4444: Flags [.], ack 1, w
        17:57:40.384578 IP 127.0.0.1.4444 > 127.0.0.4.56068: Flags [R.], seq 1,
        17:57:40.403327 IP6 ::6.37458 > ::1.6666: Flags [S], seq 406513443, win
        17:57:40.403357 IP6 ::1.6666 > ::6.37458: Flags [S.], seq 2448389240, ac
        17:57:40.403376 IP6 ::6.37458 > ::1.6666: Flags [.], ack 1, win 342, opt
        17:57:40.404263 IP6 ::1.6666 > ::6.37458: Flags [R.], seq 1, ack 1, win
      Signed-off-by: default avatarAndrey Ignatov <rdna@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      622adafb
    • Andrey Ignatov's avatar
      selftests/bpf: Selftest for sys_bind hooks · e50b0a6f
      Andrey Ignatov authored
      Add selftest to work with bpf_sock_addr context from
      `BPF_PROG_TYPE_CGROUP_SOCK_ADDR` programs.
      
      Try to bind(2) on IP:port and apply:
      * loads to make sure context can be read correctly, including narrow
        loads (byte, half) for IP and full-size loads (word) for all fields;
      * stores to those fields allowed by verifier.
      
      All combination from IPv4/IPv6 and TCP/UDP are tested.
      
      Both scenarios are tested:
      * valid programs can be loaded and attached;
      * invalid programs can be neither loaded nor attached.
      
      Test passes when expected data can be read from context in the
      BPF-program, and after the call to bind(2) socket is bound to IP:port
      pair that was written by BPF-program to the context.
      
      Example:
        # ./test_sock_addr
        Attached bind4 program.
        Test case #1 (IPv4/TCP):
                Requested: bind(192.168.1.254, 4040) ..
                   Actual: bind(127.0.0.1, 4444)
        Test case #2 (IPv4/UDP):
                Requested: bind(192.168.1.254, 4040) ..
                   Actual: bind(127.0.0.1, 4444)
        Attached bind6 program.
        Test case #3 (IPv6/TCP):
                Requested: bind(face:b00c:1234:5678::abcd, 6060) ..
                   Actual: bind(::1, 6666)
        Test case #4 (IPv6/UDP):
                Requested: bind(face:b00c:1234:5678::abcd, 6060) ..
                   Actual: bind(::1, 6666)
        ### SUCCESS
      Signed-off-by: default avatarAndrey Ignatov <rdna@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      e50b0a6f
    • Andrey Ignatov's avatar
      libbpf: Support expected_attach_type at prog load · d7be143b
      Andrey Ignatov authored
      Support setting `expected_attach_type` at prog load time in both
      `bpf/bpf.h` and `bpf/libbpf.h`.
      
      Since both headers already have API to load programs, new functions are
      added not to break backward compatibility for existing ones:
      * `bpf_load_program_xattr()` is added to `bpf/bpf.h`;
      * `bpf_prog_load_xattr()` is added to `bpf/libbpf.h`.
      
      Both new functions accept structures, `struct bpf_load_program_attr` and
      `struct bpf_prog_load_attr` correspondingly, where new fields can be
      added in the future w/o changing the API.
      
      Standard `_xattr` suffix is used to name the new API functions.
      
      Since `bpf_load_program_name()` is not used as heavily as
      `bpf_load_program()`, it was removed in favor of more generic
      `bpf_load_program_xattr()`.
      Signed-off-by: default avatarAndrey Ignatov <rdna@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      d7be143b
  2. 28 Mar, 2018 1 commit
  3. 19 Mar, 2018 2 commits
  4. 20 Feb, 2018 1 commit
  5. 16 Feb, 2018 3 commits
  6. 08 Feb, 2018 3 commits
    • Jesper Dangaard Brouer's avatar
      tools/libbpf: handle issues with bpf ELF objects containing .eh_frames · e3d91b0c
      Jesper Dangaard Brouer authored
      V3: More generic skipping of relo-section (suggested by Daniel)
      
      If clang >= 4.0.1 is missing the option '-target bpf', it will cause
      llc/llvm to create two ELF sections for "Exception Frames", with
      section names '.eh_frame' and '.rel.eh_frame'.
      
      The BPF ELF loader library libbpf fails when loading files with these
      sections.  The other in-kernel BPF ELF loader in samples/bpf/bpf_load.c,
      handle this gracefully. And iproute2 loader also seems to work with these
      "eh" sections.
      
      The issue in libbpf is caused by bpf_object__elf_collect() skipping
      some sections, and later when performing relocation it will be
      pointing to a skipped section, as these sections cannot be found by
      bpf_object__find_prog_by_idx() in bpf_object__collect_reloc().
      
      This is a general issue that also occurs for other sections, like
      debug sections which are also skipped and can have relo section.
      
      As suggested by Daniel.  To avoid keeping state about all skipped
      sections, instead perform a direct qlookup in the ELF object.  Lookup
      the section that the relo-section points to and check if it contains
      executable machine instructions (denoted by the sh_flags
      SHF_EXECINSTR).  Use this check to also skip irrelevant relo-sections.
      
      Note, for samples/bpf/ the '-target bpf' parameter to clang cannot be used
      due to incompatibility with asm embedded headers, that some of the samples
      include. This is explained in more details by Yonghong Song in bpf_devel_QA.
      Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      e3d91b0c
    • Jesper Dangaard Brouer's avatar
      tools/libbpf: improve the pr_debug statements to contain section numbers · 077c066a
      Jesper Dangaard Brouer authored
      While debugging a bpf ELF loading issue, I needed to correlate the
      ELF section number with the failed relocation section reference.
      Thus, add section numbers/index to the pr_debug.
      
      In debug mode, also print section that were skipped.  This helped
      me identify that a section (.eh_frame) was skipped, and this was
      the reason the relocation section (.rel.eh_frame) could not find
      that section number.
      
      The section numbers corresponds to the readelf tools Section Headers [Nr].
      Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      077c066a
    • Quentin Monnet's avatar
      libbpf: complete list of strings for guessing program type · 0badd331
      Quentin Monnet authored
      It seems that the type guessing feature for libbpf, based on the name of
      the ELF section the program is located in, was inspired from
      samples/bpf/prog_load.c, which was not used by any sample for loading
      programs of certain types such as TC actions and classifiers, or
      LWT-related types. As a consequence, libbpf is not able to guess the
      type of such programs and to load them automatically if type is not
      provided to the `bpf_load_prog()` function.
      
      Add ELF section names associated to those eBPF program types so that
      they can be loaded with e.g. bpftool as well.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      0badd331
  7. 07 Feb, 2018 2 commits
    • Clement Courbet's avatar
      lib: optimize cpumask_next_and() · 0ade34c3
      Clement Courbet authored
      We've measured that we spend ~0.6% of sys cpu time in cpumask_next_and().
      It's essentially a joined iteration in search for a non-zero bit, which is
      currently implemented as a lookup join (find a nonzero bit on the lhs,
      lookup the rhs to see if it's set there).
      
      Implement a direct join (find a nonzero bit on the incrementally built
      join).  Also add generic bitmap benchmarks in the new `test_find_bit`
      module for new function (see `find_next_and_bit` in [2] and [3] below).
      
      For cpumask_next_and, direct benchmarking shows that it's 1.17x to 14x
      faster with a geometric mean of 2.1 on 32 CPUs [1].  No impact on memory
      usage.  Note that on Arm, the new pure-C implementation still outperforms
      the old one that uses a mix of C and asm (`find_next_bit`) [3].
      
      [1] Approximate benchmark code:
      
      ```
        unsigned long src1p[nr_cpumask_longs] = {pattern1};
        unsigned long src2p[nr_cpumask_longs] = {pattern2};
        for (/*a bunch of repetitions*/) {
          for (int n = -1; n <= nr_cpu_ids; ++n) {
            asm volatile("" : "+rm"(src1p)); // prevent any optimization
            asm volatile("" : "+rm"(src2p));
            unsigned long result = cpumask_next_and(n, src1p, src2p);
            asm volatile("" : "+rm"(result));
          }
        }
      ```
      
      Results:
      pattern1    pattern2     time_before/time_after
      0x0000ffff  0x0000ffff   1.65
      0x0000ffff  0x00005555   2.24
      0x0000ffff  0x00001111   2.94
      0x0000ffff  0x00000000   14.0
      0x00005555  0x0000ffff   1.67
      0x00005555  0x00005555   1.71
      0x00005555  0x00001111   1.90
      0x00005555  0x00000000   6.58
      0x00001111  0x0000ffff   1.46
      0x00001111  0x00005555   1.49
      0x00001111  0x00001111   1.45
      0x00001111  0x00000000   3.10
      0x00000000  0x0000ffff   1.18
      0x00000000  0x00005555   1.18
      0x00000000  0x00001111   1.17
      0x00000000  0x00000000   1.25
      -----------------------------
                     geo.mean  2.06
      
      [2] test_find_next_bit, X86 (skylake)
      
       [ 3913.477422] Start testing find_bit() with random-filled bitmap
       [ 3913.477847] find_next_bit: 160868 cycles, 16484 iterations
       [ 3913.477933] find_next_zero_bit: 169542 cycles, 16285 iterations
       [ 3913.478036] find_last_bit: 201638 cycles, 16483 iterations
       [ 3913.480214] find_first_bit: 4353244 cycles, 16484 iterations
       [ 3913.480216] Start testing find_next_and_bit() with random-filled
       bitmap
       [ 3913.481074] find_next_and_bit: 89604 cycles, 8216 iterations
       [ 3913.481075] Start testing find_bit() with sparse bitmap
       [ 3913.481078] find_next_bit: 2536 cycles, 66 iterations
       [ 3913.481252] find_next_zero_bit: 344404 cycles, 32703 iterations
       [ 3913.481255] find_last_bit: 2006 cycles, 66 iterations
       [ 3913.481265] find_first_bit: 17488 cycles, 66 iterations
       [ 3913.481266] Start testing find_next_and_bit() with sparse bitmap
       [ 3913.481272] find_next_and_bit: 764 cycles, 1 iterations
      
      [3] test_find_next_bit, arm (v7 odroid XU3).
      
      [  267.206928] Start testing find_bit() with random-filled bitmap
      [  267.214752] find_next_bit: 4474 cycles, 16419 iterations
      [  267.221850] find_next_zero_bit: 5976 cycles, 16350 iterations
      [  267.229294] find_last_bit: 4209 cycles, 16419 iterations
      [  267.279131] find_first_bit: 1032991 cycles, 16420 iterations
      [  267.286265] Start testing find_next_and_bit() with random-filled
      bitmap
      [  267.302386] find_next_and_bit: 2290 cycles, 8140 iterations
      [  267.309422] Start testing find_bit() with sparse bitmap
      [  267.316054] find_next_bit: 191 cycles, 66 iterations
      [  267.322726] find_next_zero_bit: 8758 cycles, 32703 iterations
      [  267.329803] find_last_bit: 84 cycles, 66 iterations
      [  267.336169] find_first_bit: 4118 cycles, 66 iterations
      [  267.342627] Start testing find_next_and_bit() with sparse bitmap
      [  267.356919] find_next_and_bit: 91 cycles, 1 iterations
      
      [courbet@google.com: v6]
        Link: http://lkml.kernel.org/r/20171129095715.23430-1-courbet@google.com
      [geert@linux-m68k.org: m68k/bitops: always include <asm-generic/bitops/find.h>]
        Link: http://lkml.kernel.org/r/1512556816-28627-1-git-send-email-geert@linux-m68k.org
      Link: http://lkml.kernel.org/r/20171128131334.23491-1-courbet@google.comSigned-off-by: default avatarClement Courbet <courbet@google.com>
      Signed-off-by: default avatarGeert Uytterhoeven <geert@linux-m68k.org>
      Cc: Yury Norov <ynorov@caviumnetworks.com>
      Cc: Geert Uytterhoeven <geert@linux-m68k.org>
      Cc: Alexey Dobriyan <adobriyan@gmail.com>
      Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      0ade34c3
    • Sergey Senozhatsky's avatar
      tools/lib/subcmd/pager.c: do not alias select() params · ad343a98
      Sergey Senozhatsky authored
      Use a separate fd set for select()-s exception fds param to fix the
      following gcc warning:
      
        pager.c:36:12: error: passing argument 2 to restrict-qualified parameter aliases with argument 4 [-Werror=restrict]
          select(1, &in, NULL, &in, NULL);
                    ^~~        ~~~
      
      Link: http://lkml.kernel.org/r/20180101105626.7168-1-sergey.senozhatsky@gmail.comSigned-off-by: default avatarSergey Senozhatsky <sergey.senozhatsky@gmail.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      ad343a98
  8. 03 Feb, 2018 4 commits
  9. 17 Jan, 2018 14 commits
  10. 20 Dec, 2017 1 commit
  11. 17 Dec, 2017 2 commits
  12. 14 Dec, 2017 2 commits
    • Roman Gushchin's avatar
      libbpf: prefer global symbols as bpf program name source · fe4d44b2
      Roman Gushchin authored
      Libbpf picks the name of the first symbol in the corresponding
      elf section to use as a program name. But without taking symbol's
      scope into account it may end's up with some local label
      as a program name. E.g.:
      
      $ bpftool prog
      1: type 15  name LBB0_10    tag 0390a5136ba23f5c
      	loaded_at Dec 07/17:22  uid 0
      	xlated 456B  not jited  memlock 4096B
      
      Fix this by preferring global symbols as program name.
      
      For instance:
      $ bpftool prog
      1: type 15  name bpf_prog1  tag 0390a5136ba23f5c
      	loaded_at Dec 07/17:26  uid 0
      	xlated 456B  not jited  memlock 4096B
      Signed-off-by: default avatarRoman Gushchin <guro@fb.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: Jakub Kicinski <jakub.kicinski@netronome.com>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: Quentin Monnet <quentin.monnet@netronome.com>
      Cc: David Ahern <dsahern@gmail.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fe4d44b2
    • Roman Gushchin's avatar
      libbpf: add ability to guess program type based on section name · 583c9009
      Roman Gushchin authored
      The bpf_prog_load() function will guess program type if it's not
      specified explicitly. This functionality will be used to implement
      loading of different programs without asking a user to specify
      the program type. In first order it will be used by bpftool.
      Signed-off-by: default avatarRoman Gushchin <guro@fb.com>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: Jakub Kicinski <jakub.kicinski@netronome.com>
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: Quentin Monnet <quentin.monnet@netronome.com>
      Cc: David Ahern <dsahern@gmail.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      583c9009
  13. 18 Nov, 2017 1 commit
  14. 02 Nov, 2017 1 commit
    • Greg Kroah-Hartman's avatar
      License cleanup: add SPDX GPL-2.0 license identifier to files with no license · b2441318
      Greg Kroah-Hartman authored
      Many source files in the tree are missing licensing information, which
      makes it harder for compliance tools to determine the correct license.
      
      By default all files without license information are under the default
      license of the kernel, which is GPL version 2.
      
      Update the files which contain no license information with the 'GPL-2.0'
      SPDX license identifier.  The SPDX identifier is a legally binding
      shorthand, which can be used instead of the full boiler plate text.
      
      This patch is based on work done by Thomas Gleixner and Kate Stewart and
      Philippe Ombredanne.
      
      How this work was done:
      
      Patches were generated and checked against linux-4.14-rc6 for a subset of
      the use cases:
       - file had no licensing information it it.
       - file was a */uapi/* one with no licensing information in it,
       - file was a */uapi/* one with existing licensing information,
      
      Further patches will be generated in subsequent months to fix up cases
      where non-standard license headers were used, and references to license
      had to be inferred by heuristics based on keywords.
      
      The analysis to determine which SPDX License Identifier to be applied to
      a file was done in a spreadsheet of side by side results from of the
      output of two independent scanners (ScanCode & Windriver) producing SPDX
      tag:value files created by Philippe Ombredanne.  Philippe prepared the
      base worksheet, and did an initial spot review of a few 1000 files.
      
      The 4.13 kernel was the starting point of the analysis with 60,537 files
      assessed.  Kate Stewart did a file by file comparison of the scanner
      results in the spreadsheet to determine which SPDX license identifier(s)
      to be applied to the file. She confirmed any determination that was not
      immediately clear with lawyers working with the Linux Foundation.
      
      Criteria used to select files for SPDX license identifier tagging was:
       - Files considered eligible had to be source code files.
       - Make and config files were included as candidates if they contained >5
         lines of source
       - File already had some variant of a license header in it (even if <5
         lines).
      
      All documentation files were explicitly excluded.
      
      The following heuristics were used to determine which SPDX license
      identifiers to apply.
      
       - when both scanners couldn't find any license traces, file was
         considered to have no license information in it, and the top level
         COPYING file license applied.
      
         For non */uapi/* files that summary was:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|-------
         GPL-2.0                                              11139
      
         and resulted in the first patch in this series.
      
         If that file was a */uapi/* path one, it was "GPL-2.0 WITH
         Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|-------
         GPL-2.0 WITH Linux-syscall-note                        930
      
         and resulted in the second patch in this series.
      
       - if a file had some form of licensing information in it, and was one
         of the */uapi/* ones, it was denoted with the Linux-syscall-note if
         any GPL family license was found in the file or had no licensing in
         it (per prior point).  Results summary:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|------
         GPL-2.0 WITH Linux-syscall-note                       270
         GPL-2.0+ WITH Linux-syscall-note                      169
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
         LGPL-2.1+ WITH Linux-syscall-note                      15
         GPL-1.0+ WITH Linux-syscall-note                       14
         ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
         LGPL-2.0+ WITH Linux-syscall-note                       4
         LGPL-2.1 WITH Linux-syscall-note                        3
         ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
         ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1
      
         and that resulted in the third patch in this series.
      
       - when the two scanners agreed on the detected license(s), that became
         the concluded license(s).
      
       - when there was disagreement between the two scanners (one detected a
         license but the other didn't, or they both detected different
         licenses) a manual inspection of the file occurred.
      
       - In most cases a manual inspection of the information in the file
         resulted in a clear resolution of the license that should apply (and
         which scanner probably needed to revisit its heuristics).
      
       - When it was not immediately clear, the license identifier was
         confirmed with lawyers working with the Linux Foundation.
      
       - If there was any question as to the appropriate license identifier,
         the file was flagged for further research and to be revisited later
         in time.
      
      In total, over 70 hours of logged manual review was done on the
      spreadsheet to determine the SPDX license identifiers to apply to the
      source files by Kate, Philippe, Thomas and, in some cases, confirmation
      by lawyers working with the Linux Foundation.
      
      Kate also obtained a third independent scan of the 4.13 code base from
      FOSSology, and compared selected files where the other two scanners
      disagreed against that SPDX file, to see if there was new insights.  The
      Windriver scanner is based on an older version of FOSSology in part, so
      they are related.
      
      Thomas did random spot checks in about 500 files from the spreadsheets
      for the uapi headers and agreed with SPDX license identifier in the
      files he inspected. For the non-uapi files Thomas did random spot checks
      in about 15000 files.
      
      In initial set of patches against 4.14-rc6, 3 files were found to have
      copy/paste license identifier errors, and have been fixed to reflect the
      correct identifier.
      
      Additionally Philippe spent 10 hours this week doing a detailed manual
      inspection and review of the 12,461 patched files from the initial patch
      version early this week with:
       - a full scancode scan run, collecting the matched texts, detected
         license ids and scores
       - reviewing anything where there was a license detected (about 500+
         files) to ensure that the applied SPDX license was correct
       - reviewing anything where there was no detection but the patch license
         was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
         SPDX license was correct
      
      This produced a worksheet with 20 files needing minor correction.  This
      worksheet was then exported into 3 different .csv files for the
      different types of files to be modified.
      
      These .csv files were then reviewed by Greg.  Thomas wrote a script to
      parse the csv files and add the proper SPDX tag to the file, in the
      format that the file expected.  This script was further refined by Greg
      based on the output to detect more types of files automatically and to
      distinguish between header and source .c files (which need different
      comment types.)  Finally Greg ran the script using the .csv files to
      generate the patches.
      Reviewed-by: default avatarKate Stewart <kstewart@linuxfoundation.org>
      Reviewed-by: default avatarPhilippe Ombredanne <pombredanne@nexb.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      b2441318