1. 30 Aug, 2019 4 commits
    • Quentin Monnet's avatar
      tools: bpf: account for generated feature/ and libbpf/ directories · fbdb620b
      Quentin Monnet authored
      When building "tools/bpf" from the top of the Linux repository, the
      build system passes a value for the $(OUTPUT) Makefile variable to
      tools/bpf/Makefile and tools/bpf/bpftool/Makefile, which results in
      generating "libbpf/" (for bpftool) and "feature/" (bpf and bpftool)
      directories inside the tree.
      This commit adds such directories to the relevant .gitignore files, and
      edits the Makefiles to ensure they are removed on "make clean". The use
      of "rm" is also made consistent throughout those Makefiles (relies on
      the $(RM) variable, use "--" to prevent interpreting
      $(OUTPUT)/$(DESTDIR) as options.
      - New patch.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    • Quentin Monnet's avatar
      tools: bpftool: improve and check builds for different make invocations · 45c5589d
      Quentin Monnet authored
      There are a number of alternative "make" invocations that can be used to
      compile bpftool. The following invocations are expected to work:
        - through the kbuild system, from the top of the repository
          (make tools/bpf)
        - by telling make to change to the bpftool directory
          (make -C tools/bpf/bpftool)
        - by building the BPF tools from tools/
          (cd tools && make bpf)
        - by running make from bpftool directory
          (cd tools/bpf/bpftool && make)
      Additionally, setting the O or OUTPUT variables should tell the build
      system to use a custom output path, for each of these alternatives.
      The following patch fixes the following invocations:
        $ make tools/bpf
        $ make tools/bpf O=<dir>
        $ make -C tools/bpf/bpftool OUTPUT=<dir>
        $ make -C tools/bpf/bpftool O=<dir>
        $ cd tools/ && make bpf O=<dir>
        $ cd tools/bpf/bpftool && make OUTPUT=<dir>
        $ cd tools/bpf/bpftool && make O=<dir>
      After this commit, the build still fails for two variants when passing
      the OUTPUT variable:
        $ make tools/bpf OUTPUT=<dir>
        $ cd tools/ && make bpf OUTPUT=<dir>
      In order to remember and check what make invocations are supposed to
      work, and to document the ones which do not, a new script is added to
      the BPF selftests. Note that some invocations require the kernel to be
      configured, so the script skips them if no .config file is found.
      - In make_and_clean(), set $ERROR to 1 when "make" returns non-zero,
        even if the binary was produced.
      - Run "make clean" from the correct directory (bpf/ instead of bpftool/,
        when relevant).
      Reported-by: default avatarLorenz Bauer <lmb@cloudflare.com>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    • Quentin Monnet's avatar
      tools: bpftool: ignore make built-in rules for getting kernel version · e0a43aa3
      Quentin Monnet authored
      Bpftool calls the toplevel Makefile to get the kernel version for the
      sources it is built from. But when the utility is built from the top of
      the kernel repository, it may dump the following error message for
      certain architectures (including x86):
          $ make tools/bpf
          make[3]: *** [checkbin] Error 1
      This does not prevent bpftool compilation, but may feel disconcerting.
      The "checkbin" arch-dependent target is not supposed to be called for
      target "kernelversion", which is a simple "echo" of the version number.
      It turns out this is caused by the make invocation in tools/bpf/bpftool,
      which attempts to find implicit rules to apply. Extract from debug
          Reading makefiles...
          Reading makefile 'Makefile'...
          Reading makefile 'scripts/Kbuild.include' (search path) (no ~ expansion)...
          Reading makefile 'scripts/subarch.include' (search path) (no ~ expansion)...
          Reading makefile 'arch/x86/Makefile' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.kcov' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.gcc-plugins' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.kasan' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.extrawarn' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.ubsan' (search path) (no ~ expansion)...
          Updating makefiles....
           Considering target file 'scripts/Makefile.ubsan'.
            Looking for an implicit rule for 'scripts/Makefile.ubsan'.
            Trying pattern rule with stem 'Makefile.ubsan'.
            Trying pattern rule with stem 'Makefile.ubsan'.
            Trying implicit prerequisite 'scripts/Makefile.ubsan.o'.
            Looking for a rule with intermediate file 'scripts/Makefile.ubsan.o'.
             Avoiding implicit rule recursion.
             Trying pattern rule with stem 'Makefile.ubsan'.
             Trying rule prerequisite 'prepare'.
             Trying rule prerequisite 'FORCE'.
            Found an implicit rule for 'scripts/Makefile.ubsan'.
              Considering target file 'prepare'.
               File 'prepare' does not exist.
                Considering target file 'prepare0'.
                 File 'prepare0' does not exist.
                  Considering target file 'archprepare'.
                   File 'archprepare' does not exist.
                    Considering target file 'archheaders'.
                     File 'archheaders' does not exist.
                     Finished prerequisites of target file 'archheaders'.
                    Must remake target 'archheaders'.
          Putting child 0x55976f4f6980 (archheaders) PID 31743 on the chain.
      To avoid that, pass the -r and -R flags to eliminate the use of make
      built-in rules (and while at it, built-in variables) when running
      command "make kernelversion" from bpftool's Makefile.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    • Yauheni Kaliuta's avatar
      bpf: s390: add JIT support for multi-function programs · 1c8f9b91
      Yauheni Kaliuta authored
      This adds support for bpf-to-bpf function calls in the s390 JIT
      compiler. The JIT compiler converts the bpf call instructions to
      native branch instructions. After a round of the usual passes, the
      start addresses of the JITed images for the callee functions are
      known. Finally, to fixup the branch target addresses, we need to
      perform an extra pass.
      Because of the address range in which JITed images are allocated on
      s390, the offsets of the start addresses of these images from
      __bpf_call_base are as large as 64 bits. So, for a function call,
      the imm field of the instruction cannot be used to determine the
      callee's address. Use bpf_jit_get_func_addr() helper instead.
      The patch borrows a lot from:
      commit 8c11ea5c ("bpf, arm64: fix getting subprog addr from aux
      for calls")
      commit e2c95a61 ("bpf, ppc64: generalize fetching subprog into
      commit 8484ce83 ("bpf: powerpc64: add JIT support for
      multi-function programs")
      (including the commit message).
      test_verifier (5.3-rc6 with CONFIG_BPF_JIT_ALWAYS_ON=y):
      without patch:
      Summary: 1501 PASSED, 0 SKIPPED, 47 FAILED
      with patch:
      Summary: 1540 PASSED, 0 SKIPPED, 8 FAILED
      Signed-off-by: default avatarYauheni Kaliuta <yauheni.kaliuta@redhat.com>
      Acked-by: default avatarIlya Leoshkevich <iii@linux.ibm.com>
      Tested-by: default avatarIlya Leoshkevich <iii@linux.ibm.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
  2. 27 Aug, 2019 11 commits
  3. 21 Aug, 2019 10 commits
  4. 20 Aug, 2019 10 commits
  5. 17 Aug, 2019 5 commits
    • Daniel Borkmann's avatar
      Merge branch 'bpf-af-xdp-xskmap-improvements' · 1f726723
      Daniel Borkmann authored
      Björn Töpel says:
      This series (v5 and counting) add two improvements for the XSKMAP,
      used by AF_XDP sockets.
      1. Automatic cleanup when an AF_XDP socket goes out of scope/is
         released. Instead of require that the user manually clears the
         "released" state socket from the map, this is done
         automatically. Each socket tracks which maps it resides in, and
         remove itself from those maps at relase. A notable implementation
         change, is that the sockets references the map, instead of the map
         referencing the sockets. Which implies that when the XSKMAP is
         freed, it is by definition cleared of sockets.
      2. The XSKMAP did not honor the BPF_EXIST/BPF_NOEXIST flag on insert,
         which this patch addresses.
      v1->v2: Fixed deadlock and broken cleanup. (Daniel)
      v2->v3: Rebased onto bpf-next
      v3->v4: {READ, WRITE}_ONCE consistency. (Daniel)
              Socket release/map update race. (Daniel)
      v4->v5: Avoid use-after-free on XSKMAP self-assignment [1]. (Daniel)
              Removed redundant assignment in xsk_map_update_elem().
              Variable name consistency; Use map_entry everywhere.
      [1] https://lore.kernel.org/bpf/20190802081154.30962-1-bjorn.topel@gmail.com/T/#mc68439e97bc07fa301dad9fc4850ed5aa392f385
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    • Björn Töpel's avatar
      xsk: support BPF_EXIST and BPF_NOEXIST flags in XSKMAP · 36cc3435
      Björn Töpel authored
      The XSKMAP did not honor the BPF_EXIST/BPF_NOEXIST flags when updating
      an entry. This patch addresses that.
      Signed-off-by: default avatarBjörn Töpel <bjorn.topel@intel.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    • Björn Töpel's avatar
      xsk: remove AF_XDP socket from map when the socket is released · 0402acd6
      Björn Töpel authored
      When an AF_XDP socket is released/closed the XSKMAP still holds a
      reference to the socket in a "released" state. The socket will still
      use the netdev queue resource, and block newly created sockets from
      attaching to that queue, but no user application can access the
      fill/complete/rx/tx queues. This results in that all applications need
      to explicitly clear the map entry from the old "zombie state"
      socket. This should be done automatically.
      In this patch, the sockets tracks, and have a reference to, which maps
      it resides in. When the socket is released, it will remove itself from
      all maps.
      Suggested-by: default avatarBruce Richardson <bruce.richardson@intel.com>
      Signed-off-by: default avatarBjörn Töpel <bjorn.topel@intel.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    • Daniel Borkmann's avatar
      Merge branch 'bpf-sk-storage-clone' · 8e46c353
      Daniel Borkmann authored
      Stanislav Fomichev says:
      Currently there is no way to propagate sk storage from the listener
      socket to a newly accepted one. Consider the following use case:
              fd = socket();
              setsockopt(fd, SOL_IP, IP_TOS,...);
              /* ^^^ setsockopt BPF program triggers here and saves something
               * into sk storage of the listener.
              listen(fd, ...);
              while (client = accept(fd)) {
                      /* At this point all association between listener
                       * socket and newly accepted one is gone. New
                       * socket will not have any sk storage attached.
      Let's add new BPF_F_CLONE flag that can be specified when creating
      a socket storage map. This new flag indicates that map contents
      should be cloned when the socket is cloned.
      * drop 'goto err' in bpf_sk_storage_clone (Yonghong Song)
      * add comment about race with bpf_sk_storage_map_free to the
        bpf_sk_storage_clone side as well (Daniel Borkmann)
      * make sure BPF_F_NO_PREALLOC is always present when creating
        a map (Martin KaFai Lau)
      * don't call bpf_sk_storage_free explicitly, rely on
        sk_free_unlock_clone to do the cleanup (Martin KaFai Lau)
      * remove spinlocks around selem_link_map/sk (Martin KaFai Lau)
      * BPF_F_CLONE on a map, not selem (Martin KaFai Lau)
      * hold a map while cloning (Martin KaFai Lau)
      * use BTF maps in selftests (Yonghong Song)
      * do proper cleanup selftests; don't call close(-1) (Yonghong Song)
      * export bpf_map_inc_not_zero
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    • Stanislav Fomichev's avatar
      selftests/bpf: add sockopt clone/inheritance test · c3bbf176
      Stanislav Fomichev authored
      Add a test that calls setsockopt on the listener socket which triggers
      BPF program. This BPF program writes to the sk storage and sets
      clone flag. Make sure that sk storage is cloned for a newly
      accepted connection.
      We have two cloned maps in the tests to make sure we hit both cases
      in bpf_sk_storage_clone: first element (sk_storage_alloc) and
      non-first element(s) (selem_link_map).
      Cc: Martin KaFai Lau <kafai@fb.com>
      Cc: Yonghong Song <yhs@fb.com>
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarStanislav Fomichev <sdf@google.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>