bpftool no longer compiles on i686

[michel-slm]

When building retsnoop 0.10, which bundles an very recent copy of bpftool (commit 64402b8 from July 16), bpftool fails to compile on i686, while it previously succeeded

make[1]: Leaving directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/bpftool/src'
Makefile.feature:34: Probing: feature-clang-bpf-co-re
Makefile.feature:35: printf '%s\n' 'struct s { int i; } __attribute__((preserve_access_index)); struct s foo = {};' | clang -g -target bpf -S -o - -x c -  | grep -q BTF_KIND_VAR && (echo 1 && >&2 echo result: 1) || (echo 0 && >&2 echo result: 0)
result: 1
Makefile.feature:54: Probing: feature-libbfd
Makefile.feature:55: printf '%b\n' '#include <bfd.h>\n' 'int main(void) {' '	bfd_demangle(0, 0, 0);' '	return 0;' '}' | clang -O2 -flto=thin -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS --config=/usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection  -DSKEL_NAME=retsnoop_bpf -DSKEL_HEADER=retsnoop.skel.h -DSKEL_EXTRA_HEADER=retsnoop.h -Wno-sign-compare -Wall -Werror -x c - -lbfd -ldl -o /dev/null >/dev/null && (echo 1 && >&2 echo result: 1) || (echo 0 && >&2 echo result: 0)
<stdin>:1:10: fatal error: 'bfd.h' file not found
    1 | #include <bfd.h>
      |          ^~~~~~~
1 error generated.

[michel-slm]

https://koji.fedoraproject.org/koji/taskinfo?taskID=122896647

build.log

[michel-slm]

The bizarre thing is - for both x86_64 and i686, the feature detection correctly says libbfd is not found (since I did not add binutils-devel as a build requirement), but on i686 it then tries using it

make[1]: Entering directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/bpftool/src'
...                        libbfd: [ OFF ]
...               clang-bpf-co-re: [ on  ]
...                          llvm: [ OFF ]
...                        libcap: [ OFF ]

[qmonnet]

I think there's a misunderstanding. Bpftool compiles fine in these build logs, there's no error when make leaves bpftool's directory at line 573. The error that you quoted above is expected: it's a failure from the libbfd probe, which is printed because you build in verbose mode, and that just means that libbfd won't be supported indeed. The fact that the features are displayed before the actual probes could be a consequence of the parallel build messing up with the order of the output?

As far as I can tell, your build error is elsewhere in the logs:

make: Entering directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/src'
clang -O2 -flto=thin -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS --config=/usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection  -DSKEL_NAME=retsnoop_bpf -DSKEL_HEADER=retsnoop.skel.h -DSKEL_EXTRA_HEADER=retsnoop.h -I.output -I.output/include -c fnargs.c -o .output/fnargs.o
make: Leaving directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/src'
In file included from fnargs.c:17:
.output/kmem_reader.skel.h:266:17: error: static assertion failed due to requirement 'sizeof (s->bss->value) == 8': unexpected size of 'value'
  266 |         _Static_assert(sizeof(s->bss->value) == 8, "unexpected size of 'value'");
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~
.output/kmem_reader.skel.h:266:39: note: expression evaluates to '4 == 8'
  266 |         _Static_assert(sizeof(s->bss->value) == 8, "unexpected size of 'value'");
      |                        ~~~~~~~~~~~~~~~~~~~~~~^~~~
.output/kmem_reader.skel.h:267:17: error: static assertion failed due to requirement 'sizeof (s->bss->addr) == 8': unexpected size of 'addr'
  267 |         _Static_assert(sizeof(s->bss->addr) == 8, "unexpected size of 'addr'");
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
.output/kmem_reader.skel.h:267:38: note: expression evaluates to '4 == 8'
  267 |         _Static_assert(sizeof(s->bss->addr) == 8, "unexpected size of 'addr'");
      |                        ~~~~~~~~~~~~~~~~~~~~~^~~~
2 errors generated.
make: *** [Makefile:132: .output/fnargs.o] Error 1
make: *** Waiting for unfinished jobs....
make: Entering directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/src'
clang -O2 -flto=thin -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS --config=/usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection  -DSKEL_NAME=retsnoop_bpf -DSKEL_HEADER=retsnoop.skel.h -DSKEL_EXTRA_HEADER=retsnoop.h -I.output -I.output/include -c retsnoop.c -o .output/retsnoop.o
make: Leaving directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/src'
In file included from retsnoop.c:19:
.output/retsnoop.skel.h:6784:17: error: static assertion failed due to requirement 'sizeof (s->bss->stats) == 16': unexpected size of 'stats'
 6784 |         _Static_assert(sizeof(s->bss->stats) == 16, "unexpected size of 'stats'");
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~
.output/retsnoop.skel.h:6784:39: note: expression evaluates to '8 == 16'
 6784 |         _Static_assert(sizeof(s->bss->stats) == 16, "unexpected size of 'stats'");
      |                        ~~~~~~~~~~~~~~~~~~~~~~^~~~~
1 error generated.
make: *** [Makefile:132: .output/retsnoop.o] Error 1
make: Entering directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/src'
clang -O2 -flto=thin -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS --config=/usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection  -DSKEL_NAME=retsnoop_bpf -DSKEL_HEADER=retsnoop.skel.h -DSKEL_EXTRA_HEADER=retsnoop.h -I.output -I.output/include -c mass_attacher.c -o .output/mass_attacher.o
make: Leaving directory '/builddir/build/BUILD/retsnoop-0.10-build/retsnoop-0.10/src'
In file included from mass_attacher.c:39:
.output/retsnoop.skel.h:6784:17: error: static assertion failed due to requirement 'sizeof (s->bss->stats) == 16': unexpected size of 'stats'
 6784 |         _Static_assert(sizeof(s->bss->stats) == 16, "unexpected size of 'stats'");
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~
.output/retsnoop.skel.h:6784:39: note: expression evaluates to '8 == 16'
 6784 |         _Static_assert(sizeof(s->bss->stats) == 16, "unexpected size of 'stats'");
      |                        ~~~~~~~~~~~~~~~~~~~~~~^~~~~
1 error generated.
make: *** [Makefile:132: .output/mass_attacher.o] Error 1

This is in retsnoop, in the BPF skeletons. This skeleton and in particular the _Static_assert() is generated from bpftool, but the related code is apparently from 2022, so no new change to expect in there - or how old was the previous bpftool version you were using? Seems to be e17d6cf from 2023, looking at the changes in retsnoop (and retsnoop added bpftool as a submodule after torvalds/linux@08d4dba was here anyway).

So I'd look at changes in retsnoop, maybe. I'm not sure, but anakryiko/retsnoop@6dd197b and anakryiko/retsnoop@98e61e6 are interesting ones - I wouldn't be surprised if they mess up with the -m32 -march=i686 you use when compiling with the BPF skeleton.