1. 30 Sep, 2019 1 commit
  2. 30 Aug, 2019 4 commits
    • Quentin Monnet's avatar
      tools: bpftool: do not link twice against libbpf.a in Makefile · 5b84ad2e
      Quentin Monnet authored
      In bpftool's Makefile, $(LIBS) includes $(LIBBPF), therefore the library
      is used twice in the linking command. No need to have $(LIBBPF) (from
      $^) on that command, let's do with "$(OBJS) $(LIBS)" (but move $(LIBBPF)
      _before_ the -l flags in $(LIBS)).
      Signed-off-by: default avatarIlya Leoshkevich <iii@linux.ibm.com>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      5b84ad2e
    • Quentin Monnet's avatar
      tools: bpf: account for generated feature/ and libbpf/ directories · fbdb620b
      Quentin Monnet authored
      When building "tools/bpf" from the top of the Linux repository, the
      build system passes a value for the $(OUTPUT) Makefile variable to
      tools/bpf/Makefile and tools/bpf/bpftool/Makefile, which results in
      generating "libbpf/" (for bpftool) and "feature/" (bpf and bpftool)
      directories inside the tree.
      
      This commit adds such directories to the relevant .gitignore files, and
      edits the Makefiles to ensure they are removed on "make clean". The use
      of "rm" is also made consistent throughout those Makefiles (relies on
      the $(RM) variable, use "--" to prevent interpreting
      $(OUTPUT)/$(DESTDIR) as options.
      
      v2:
      - New patch.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      fbdb620b
    • Quentin Monnet's avatar
      tools: bpftool: improve and check builds for different make invocations · 45c5589d
      Quentin Monnet authored
      There are a number of alternative "make" invocations that can be used to
      compile bpftool. The following invocations are expected to work:
      
        - through the kbuild system, from the top of the repository
          (make tools/bpf)
        - by telling make to change to the bpftool directory
          (make -C tools/bpf/bpftool)
        - by building the BPF tools from tools/
          (cd tools && make bpf)
        - by running make from bpftool directory
          (cd tools/bpf/bpftool && make)
      
      Additionally, setting the O or OUTPUT variables should tell the build
      system to use a custom output path, for each of these alternatives.
      
      The following patch fixes the following invocations:
      
        $ make tools/bpf
        $ make tools/bpf O=<dir>
        $ make -C tools/bpf/bpftool OUTPUT=<dir>
        $ make -C tools/bpf/bpftool O=<dir>
        $ cd tools/ && make bpf O=<dir>
        $ cd tools/bpf/bpftool && make OUTPUT=<dir>
        $ cd tools/bpf/bpftool && make O=<dir>
      
      After this commit, the build still fails for two variants when passing
      the OUTPUT variable:
      
        $ make tools/bpf OUTPUT=<dir>
        $ cd tools/ && make bpf OUTPUT=<dir>
      
      In order to remember and check what make invocations are supposed to
      work, and to document the ones which do not, a new script is added to
      the BPF selftests. Note that some invocations require the kernel to be
      configured, so the script skips them if no .config file is found.
      
      v2:
      - In make_and_clean(), set $ERROR to 1 when "make" returns non-zero,
        even if the binary was produced.
      - Run "make clean" from the correct directory (bpf/ instead of bpftool/,
        when relevant).
      Reported-by: default avatarLorenz Bauer <lmb@cloudflare.com>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      45c5589d
    • Quentin Monnet's avatar
      tools: bpftool: ignore make built-in rules for getting kernel version · e0a43aa3
      Quentin Monnet authored
      Bpftool calls the toplevel Makefile to get the kernel version for the
      sources it is built from. But when the utility is built from the top of
      the kernel repository, it may dump the following error message for
      certain architectures (including x86):
      
          $ make tools/bpf
          [...]
          make[3]: *** [checkbin] Error 1
          [...]
      
      This does not prevent bpftool compilation, but may feel disconcerting.
      The "checkbin" arch-dependent target is not supposed to be called for
      target "kernelversion", which is a simple "echo" of the version number.
      
      It turns out this is caused by the make invocation in tools/bpf/bpftool,
      which attempts to find implicit rules to apply. Extract from debug
      output:
      
          Reading makefiles...
          Reading makefile 'Makefile'...
          Reading makefile 'scripts/Kbuild.include' (search path) (no ~ expansion)...
          Reading makefile 'scripts/subarch.include' (search path) (no ~ expansion)...
          Reading makefile 'arch/x86/Makefile' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.kcov' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.gcc-plugins' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.kasan' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.extrawarn' (search path) (no ~ expansion)...
          Reading makefile 'scripts/Makefile.ubsan' (search path) (no ~ expansion)...
          Updating makefiles....
           Considering target file 'scripts/Makefile.ubsan'.
            Looking for an implicit rule for 'scripts/Makefile.ubsan'.
            Trying pattern rule with stem 'Makefile.ubsan'.
          [...]
            Trying pattern rule with stem 'Makefile.ubsan'.
            Trying implicit prerequisite 'scripts/Makefile.ubsan.o'.
            Looking for a rule with intermediate file 'scripts/Makefile.ubsan.o'.
             Avoiding implicit rule recursion.
             Trying pattern rule with stem 'Makefile.ubsan'.
             Trying rule prerequisite 'prepare'.
             Trying rule prerequisite 'FORCE'.
            Found an implicit rule for 'scripts/Makefile.ubsan'.
              Considering target file 'prepare'.
               File 'prepare' does not exist.
                Considering target file 'prepare0'.
                 File 'prepare0' does not exist.
                  Considering target file 'archprepare'.
                   File 'archprepare' does not exist.
                    Considering target file 'archheaders'.
                     File 'archheaders' does not exist.
                     Finished prerequisites of target file 'archheaders'.
                    Must remake target 'archheaders'.
          Putting child 0x55976f4f6980 (archheaders) PID 31743 on the chain.
      
      To avoid that, pass the -r and -R flags to eliminate the use of make
      built-in rules (and while at it, built-in variables) when running
      command "make kernelversion" from bpftool's Makefile.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      e0a43aa3
  3. 21 Aug, 2019 2 commits
  4. 20 Aug, 2019 1 commit
    • Quentin Monnet's avatar
      tools: bpftool: implement "bpftool btf show|list" · 4d374ba0
      Quentin Monnet authored
      Add a "btf list" (alias: "btf show") subcommand to bpftool in order to
      dump all BTF objects loaded on a system.
      
      When running the command, hash tables are built in bpftool to retrieve
      all the associations between BTF objects and BPF maps and programs. This
      allows for printing all such associations when listing the BTF objects.
      
      The command is added at the top of the subcommands for "bpftool btf", so
      that typing only "bpftool btf" also comes down to listing the programs.
      We could not have this with the previous command ("dump"), which
      required a BTF object id, so it should not break any previous behaviour.
      This also makes the "btf" command behaviour consistent with "prog" or
      "map".
      
      Bash completion is updated to use "bpftool btf" instead of "bpftool
      prog" to list the BTF ids, as it looks more consistent.
      
      Example output (plain):
      
          # bpftool btf show
          9: size 2989B  prog_ids 21  map_ids 15
          17: size 2847B  prog_ids 36  map_ids 30,29,28
          26: size 2847B
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      4d374ba0
  5. 16 Aug, 2019 11 commits
  6. 14 Aug, 2019 1 commit
  7. 13 Aug, 2019 1 commit
  8. 12 Aug, 2019 1 commit
  9. 09 Aug, 2019 2 commits
  10. 31 Jul, 2019 1 commit
    • Jakub Kicinski's avatar
      tools: bpftool: add support for reporting the effective cgroup progs · a98bf573
      Jakub Kicinski authored
      Takshak said in the original submission:
      
      With different bpf attach_flags available to attach bpf programs specially
      with BPF_F_ALLOW_OVERRIDE and BPF_F_ALLOW_MULTI, the list of effective
      bpf-programs available to any sub-cgroups really needs to be available for
      easy debugging.
      
      Using BPF_F_QUERY_EFFECTIVE flag, one can get the list of not only attached
      bpf-programs to a cgroup but also the inherited ones from parent cgroup.
      
      So a new option is introduced to use BPF_F_QUERY_EFFECTIVE query flag here
      to list all the effective bpf-programs available for execution at a specified
      cgroup.
      
      Reused modified test program test_cgroup_attach from tools/testing/selftests/bpf:
        # ./test_cgroup_attach
      
      With old bpftool:
      
       # bpftool cgroup show /sys/fs/cgroup/cgroup-test-work-dir/cg1/
        ID       AttachType      AttachFlags     Name
        271      egress          multi           pkt_cntr_1
        272      egress          multi           pkt_cntr_2
      
      Attached new program pkt_cntr_4 in cg2 gives following:
      
       # bpftool cgroup show /sys/fs/cgroup/cgroup-test-work-dir/cg1/cg2
        ID       AttachType      AttachFlags     Name
        273      egress          override        pkt_cntr_4
      
      And with new "effective" option it shows all effective programs for cg2:
      
       # bpftool cgroup show /sys/fs/cgroup/cgroup-test-work-dir/cg1/cg2 effective
        ID       AttachType      AttachFlags     Name
        273      egress          override        pkt_cntr_4
        271      egress          override        pkt_cntr_1
        272      egress          override        pkt_cntr_2
      
      Compared to original submission use a local flag instead of global
      option.
      
      We need to clear query_flags on every command, in case batch mode
      wants to use varying settings.
      
      v2: (Takshak)
       - forbid duplicated flags;
       - fix cgroup path freeing.
      Signed-off-by: default avatarTakshak Chahande <ctakshak@fb.com>
      Signed-off-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Reviewed-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarTakshak Chahande <ctakshak@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      a98bf573
  11. 29 Jul, 2019 1 commit
  12. 12 Jul, 2019 1 commit
  13. 08 Jul, 2019 2 commits
  14. 05 Jul, 2019 2 commits
    • Jiri Olsa's avatar
      tools: bpftool: Fix json dump crash on powerpc · aa52bcbe
      Jiri Olsa authored
      Michael reported crash with by bpf program in json mode on powerpc:
      
        # bpftool prog -p dump jited id 14
        [{
              "name": "0xd00000000a9aa760",
              "insns": [{
                      "pc": "0x0",
                      "operation": "nop",
                      "operands": [null
                      ]
                  },{
                      "pc": "0x4",
                      "operation": "nop",
                      "operands": [null
                      ]
                  },{
                      "pc": "0x8",
                      "operation": "mflr",
        Segmentation fault (core dumped)
      
      The code is assuming char pointers in format, which is not always
      true at least for powerpc. Fixing this by dumping the whole string
      into buffer based on its format.
      
      Please note that libopcodes code does not check return values from
      fprintf callback, but as per Jakub suggestion returning -1 on allocation
      failure so we do the best effort to propagate the error.
      
      Fixes: 107f0412 ("tools: bpftool: add JSON output for `bpftool prog dump jited *` command")
      Reported-by: default avatarMichael Petlan <mpetlan@redhat.com>
      Signed-off-by: default avatarJiri Olsa <jolsa@kernel.org>
      Reviewed-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      aa52bcbe
    • Quentin Monnet's avatar
      tools: bpftool: add "prog run" subcommand to test-run programs · ba95c745
      Quentin Monnet authored
      Add a new "bpftool prog run" subcommand to run a loaded program on input
      data (and possibly with input context) passed by the user.
      
      Print output data (and output context if relevant) into a file or into
      the console. Print return value and duration for the test run into the
      console.
      
      A "repeat" argument can be passed to run the program several times in a
      row.
      
      The command does not perform any kind of verification based on program
      type (Is this program type allowed to use an input context?) or on data
      consistency (Can I work with empty input data?), this is left to the
      kernel.
      
      Example invocation:
      
          # perl -e 'print "\x0" x 14' | ./bpftool prog run \
                  pinned /sys/fs/bpf/sample_ret0 \
                  data_in - data_out - repeat 5
          0000000 0000 0000 0000 0000 0000 0000 0000      | ........ ......
          Return value: 0, duration (average): 260ns
      
      When one of data_in or ctx_in is "-", bpftool reads from standard input,
      in binary format. Other formats (JSON, hexdump) might be supported (via
      an optional command line keyword like "data_fmt_in") in the future if
      relevant, but this would require doing more parsing in bpftool.
      
      v2:
      - Fix argument names for function check_single_stdin(). (Yonghong)
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      ba95c745
  15. 27 Jun, 2019 1 commit
  16. 26 Jun, 2019 1 commit
  17. 19 Jun, 2019 1 commit
  18. 11 Jun, 2019 1 commit
  19. 06 Jun, 2019 1 commit
  20. 05 Jun, 2019 1 commit
    • Krzesimir Nowak's avatar
      tools: bpftool: Fix JSON output when lookup fails · 1884c066
      Krzesimir Nowak authored
      In commit 9a5ab8bf ("tools: bpftool: turn err() and info() macros
      into functions") one case of error reporting was special cased, so it
      could report a lookup error for a specific key when dumping the map
      element. What the code forgot to do is to wrap the key and value keys
      into a JSON object, so an example output of pretty JSON dump of a
      sockhash map (which does not support looking up its values) is:
      
      [
          "key": ["0x0a","0x41","0x00","0x02","0x1f","0x78","0x00","0x00"
          ],
          "value": {
              "error": "Operation not supported"
          },
          "key": ["0x0a","0x41","0x00","0x02","0x1f","0x78","0x00","0x01"
          ],
          "value": {
              "error": "Operation not supported"
          }
      ]
      
      Note the key-value pairs inside the toplevel array. They should be
      wrapped inside a JSON object, otherwise it is an invalid JSON. This
      commit fixes this, so the output now is:
      
      [{
              "key": ["0x0a","0x41","0x00","0x02","0x1f","0x78","0x00","0x00"
              ],
              "value": {
                  "error": "Operation not supported"
              }
          },{
              "key": ["0x0a","0x41","0x00","0x02","0x1f","0x78","0x00","0x01"
              ],
              "value": {
                  "error": "Operation not supported"
              }
          }
      ]
      
      Fixes: 9a5ab8bf ("tools: bpftool: turn err() and info() macros into functions")
      Cc: Quentin Monnet <quentin.monnet@netronome.com>
      Signed-off-by: default avatarKrzesimir Nowak <krzesimir@kinvolk.io>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      1884c066
  21. 28 May, 2019 3 commits
    • Quentin Monnet's avatar
      tools: bpftool: make -d option print debug output from verifier · 55d77807
      Quentin Monnet authored
      The "-d" option is used to require all logs available for bpftool. So
      far it meant telling libbpf to print even debug-level information. But
      there is another source of info that can be made more verbose: when we
      attemt to load programs with bpftool, we can pass a log_level parameter
      to the verifier in order to control the amount of information that is
      printed to the console.
      
      Reuse the "-d" option to print all information the verifier can tell. At
      this time, this means logs related to BPF_LOG_LEVEL1, BPF_LOG_LEVEL2 and
      BPF_LOG_STATS. As mentioned in the discussion on the first version of
      this set, these macros are internal to the kernel
      (include/linux/bpf_verifier.h) and are not meant to be part of the
      stable user API, therefore we simply use the related constants to print
      whatever we can at this time, without trying to tell users what is
      log_level1 or what is statistics.
      
      Verifier logs are only used when loading programs for now (In the
      future: for loading BTF objects with bpftool? Although libbpf does not
      currently offer to print verifier info at debug level if no error
      occurred when loading BTF objects), so bpftool.rst and bpftool-prog.rst
      are the only man pages to get the update.
      
      v3:
      - Add details on log level and BTF loading at the end of commit log.
      
      v2:
      - Remove the possibility to select the log levels to use (v1 offered a
        combination of "log_level1", "log_level2" and "stats").
      - The macros from kernel header bpf_verifier.h are not used (and
        therefore not moved to UAPI header).
      - In v1 this was a distinct option, but is now merged in the only "-d"
        switch to activate libbpf and verifier debug-level logs all at the
        same time.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      55d77807
    • Quentin Monnet's avatar
      tools: bpftool: add -d option to get debug output from libbpf · 775bc8ad
      Quentin Monnet authored
      libbpf has three levels of priority for output messages: warn, info,
      debug. By default, debug output is not printed to the console.
      
      Add a new "--debug" (short name: "-d") option to bpftool to print libbpf
      logs for all three levels.
      
      Internally, we simply use the function provided by libbpf to replace the
      default printing function by one that prints logs regardless of their
      level.
      
      v2:
      - Remove the possibility to select the log-levels to use (v1 offered a
        combination of "warn", "info" and "debug").
      - Rename option and offer a short name: -d|--debug.
      - Add option description to all bpftool manual pages (instead of
        bpftool-prog.rst only), as all commands use libbpf.
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      775bc8ad
    • Chang-Hsien Tsai's avatar
      bpf: style fix in while(!feof()) loop · 92bd6820
      Chang-Hsien Tsai authored
      Use fgets() as the while loop condition.
      Signed-off-by: default avatarChang-Hsien Tsai <luke.tw@gmail.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      92bd6820