Tensorflow does not build with Abseil 20240116.2 headers and later [383173946]

Assigned

Bug

Status Update

No update yet.

Description

aj...@google.com

created issue #1

Dec 9, 2024 10:24PM

When upgrading abseil to 20240116 for sys-devel/autofdo (see parent bug), we can't build tensorflow because it has incorrect includes for abseil.

tensorflow-2.17.0-r155:   /usr/bin/x86_64-cros-linux-gnu-clang -MD -MF bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/lite/kernels/_objs/combined_all_kernel_tests_lib/random_ops_test.d '-frandom-seed=bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/lite/kernels/_objs/combined_all_kernel_tests_lib/random_ops_test.o' -DTFLITE_KERNEL_USE_XNNPACK '-DXNNPACK_DELEGATE_ENABLE_QS8=1' '-DXNNPACK_DELEGATE_ENABLE_QU8=1' '-DEIGEN_MAX_ALIGN_BYTES=64' -DEIGEN_ALLOW_UNALIGNED_SCALARS '-DEIGEN_USE_AVX512_GEMM_KERNELS=0' -DFC_4BIT_SSE -DPTHREADPOOL_NO_DEPRECATED_API '-DEIGEN_NEON_GEBP_NR=4' '-DXNN_LOG_LEVEL=0' '-DXNN_ENABLE_GEMM_M_SPECIALIZATION=1' '-DXNN_ENABLE_JIT=0' '-DXNN_ENABLE_ASSEMBLY=1' '-DXNN_ENABLE_AVX512AMX=1' '-DXNN_ENABLE_AVX512FP16=1' '-DXNN_ENABLE_AVXVNNI=1' '-DXNN_ENABLE_DWCONV_MULTIPASS=1' '-DXNN_ENABLE_ARM_FP16_SCALAR=0' '-DXNN_ENABLE_ARM_FP16_VECTOR=0' '-DXNN_ENABLE_ARM_BF16=0' '-DXNN_ENABLE_ARM_DOTPROD=0' '-DXNN_ENABLE_ARM_I8MM=0' '-DXNN_ENABLE_RISCV_FP16_VECTOR=0' '-DXNN_ENABLE_AVX512VNNIGFNI=1' '-DXNN_ENABLE_SPARSE=1' '-DXNN_ENABLE_MEMOPT=1' -DTFLITE_BUILD_WITH_XNNPACK_DELEGATE '-DEIGEN_ALTIVEC_USE_CUSTOM_PACK=0' -DBENCHMARK_STATIC_DEFINE '-DBAZEL_CURRENT_REPOSITORY="org_tensorflow"' -iquote external/org_tensorflow -iquote bazel-out/k8-opt/bin/external/org_tensorflow -iquote external/flatbuffers -iquote bazel-out/k8-opt/bin/external/flatbuffers -iquote external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/ruy -iquote bazel-out/k8-opt/bin/external/ruy -iquote external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/FP16 -iquote bazel-out/k8-opt/bin/external/FP16 -iquote external/arm_neon_2_x86_sse -iquote bazel-out/k8-opt/bin/external/arm_neon_2_x86_sse -iquote external/gemmlowp -iquote bazel-out/k8-opt/bin/external/gemmlowp -iquote external/cpuinfo -iquote bazel-out/k8-opt/bin/external/cpuinfo -iquote external/pthreadpool -iquote bazel-out/k8-opt/bin/external/pthreadpool -iquote external/FXdiv -iquote bazel-out/k8-opt/bin/external/FXdiv -iquote external/XNNPACK -iquote bazel-out/k8-opt/bin/external/XNNPACK -iquote external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/local_tsl -iquote bazel-out/k8-opt/bin/external/local_tsl -iquote external/local_xla -iquote bazel-out/k8-opt/bin/external/local_xla -iquote external/com_google_googletest -iquote bazel-out/k8-opt/bin/external/com_google_googletest -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/ml_dtypes -iquote bazel-out/k8-opt/bin/external/ml_dtypes -iquote external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/com_google_benchmark -iquote bazel-out/k8-opt/bin/external/com_google_benchmark -Ibazel-out/k8-opt/bin/external/flatbuffers/_virtual_includes/runtime_cc -Ibazel-out/k8-opt/bin/external/flatbuffers/_virtual_includes/flatbuffers -Ibazel-out/k8-opt/bin/external/flatbuffers/src/_virtual_includes/flatbuffers -Ibazel-out/k8-opt/bin/external/flatbuffers/src/_virtual_includes/code_generators -Ibazel-out/k8-opt/bin/external/flatbuffers/src/_virtual_includes/generate_fbs -Ibazel-out/k8-opt/bin/external/FP16/_virtual_includes/FP16 -Ibazel-out/k8-opt/bin/external/cpuinfo/_virtual_includes/cpuinfo -Ibazel-out/k8-opt/bin/external/pthreadpool/_virtual_includes/pthreadpool -Ibazel-out/k8-opt/bin/external/FXdiv/_virtual_includes/FXdiv -Ibazel-out/k8-opt/bin/external/ml_dtypes/_virtual_includes/float8 -Ibazel-out/k8-opt/bin/external/ml_dtypes/_virtual_includes/intn -Ibazel-out/k8-opt/bin/external/com_google_benchmark/_virtual_includes/benchmark -isystem external/org_tensorflow/tensorflow/lite/schema -isystem bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/lite/schema -isystem external/org_tensorflow/tensorflow/compiler/mlir/lite/schema -isystem bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/compiler/mlir/lite/schema -isystem external/org_tensorflow/tensorflow/lite/acceleration/configuration -isystem bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/lite/acceleration/configuration -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/eigen_archive/mkl_include -isystem bazel-out/k8-opt/bin/external/eigen_archive/mkl_include -isystem external/FP16/include -isystem bazel-out/k8-opt/bin/external/FP16/include -isystem external/cpuinfo/include -isystem bazel-out/k8-opt/bin/external/cpuinfo/include -isystem external/cpuinfo/src -isystem bazel-out/k8-opt/bin/external/cpuinfo/src -isystem external/pthreadpool/include -isystem bazel-out/k8-opt/bin/external/pthreadpool/include -isystem external/FXdiv/include -isystem bazel-out/k8-opt/bin/external/FXdiv/include -isystem external/org_tensorflow/tensorflow/lite/delegates/xnnpack -isystem bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/lite/delegates/xnnpack -isystem external/XNNPACK/include -isystem bazel-out/k8-opt/bin/external/XNNPACK/include -isystem external/XNNPACK/src -isystem bazel-out/k8-opt/bin/external/XNNPACK/src -isystem external/com_google_googletest/googlemock -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googlemock -isystem external/com_google_googletest/googlemock/include -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googlemock/include -isystem external/com_google_googletest/googletest -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googletest -isystem external/com_google_googletest/googletest/include -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googletest/include -isystem external/ml_dtypes -isystem bazel-out/k8-opt/bin/external/ml_dtypes -isystem external/ml_dtypes/ml_dtypes -isystem bazel-out/k8-opt/bin/external/ml_dtypes/ml_dtypes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -no-canonical-prefixes -g0 -O2 -ffunction-sections -fdata-sections -DEGL_NO_X11 -DFORCE_ENABLE_GPU_DELEGATE -Werror '-Wno-error=unused-variable' '-Wno-error=unused-but-set-variable' '-Wno-error=unused-function' -fno-sanitize-blacklist '-std=gnu++20' -Os -pipe '-march=goldmont' -mno-sha -g -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -ffunction-sections -fdata-sections '-D_FILE_OFFSET_BITS=64' -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Wno-everything '--sysroot=/build/hatch' -c external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc -o bazel-out/k8-opt/bin/external/org_tensorflow/tensorflow/lite/kernels/_objs/combined_all_kernel_tests_lib/random_ops_test.o)
tensorflow-2.17.0-r155: # Configuration: f153655d0020dd6423f618b51d7fd6875d227c101144d23826f0bef7f148b77f
tensorflow-2.17.0-r155: # Execution platform: @local_execution_config_platform//:platform
tensorflow-2.17.0-r155: external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc:171:32: error: no member named 'StrCat' in namespace 'absl'
tensorflow-2.17.0-r155:   171 |       std::string name = absl::StrCat(
tensorflow-2.17.0-r155:       |                          ~~~~~~^
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:469:11: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
tensorflow-2.17.0-r155:   469 |           __VA_ARGS__,                                                       \
tensorflow-2.17.0-r155:       |           ^~~~~~~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:458:47: note: expanded from macro 'GTEST_GET_SECOND_'
tensorflow-2.17.0-r155:   458 | #define GTEST_GET_SECOND_(first, second, ...) second
tensorflow-2.17.0-r155:       |                                               ^~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:456:28: note: expanded from macro 'GTEST_EXPAND_'
tensorflow-2.17.0-r155:   456 | #define GTEST_EXPAND_(arg) arg
tensorflow-2.17.0-r155:       |                            ^~~
tensorflow-2.17.0-r155: external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc:171:32: error: no member named 'StrCat' in namespace 'absl'
tensorflow-2.17.0-r155:   171 |       std::string name = absl::StrCat(
tensorflow-2.17.0-r155:       |                          ~~~~~~^
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:472:32: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
tensorflow-2.17.0-r155:   472 |       auto t = std::make_tuple(__VA_ARGS__);                                 \
tensorflow-2.17.0-r155:       |                                ^~~~~~~~~~~
tensorflow-2.17.0-r155: external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc:171:32: error: no member named 'StrCat' in namespace 'absl'
tensorflow-2.17.0-r155:   171 |       std::string name = absl::StrCat(
tensorflow-2.17.0-r155:       |                          ~~~~~~^
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:477:9: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
tensorflow-2.17.0-r155:   477 |         __VA_ARGS__,                                                         \
tensorflow-2.17.0-r155:       |         ^~~~~~~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:458:47: note: expanded from macro 'GTEST_GET_SECOND_'
tensorflow-2.17.0-r155:   458 | #define GTEST_GET_SECOND_(first, second, ...) second
tensorflow-2.17.0-r155:       |                                               ^~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:456:28: note: expanded from macro 'GTEST_EXPAND_'
tensorflow-2.17.0-r155:   456 | #define GTEST_EXPAND_(arg) arg
tensorflow-2.17.0-r155:       |                            ^~~
tensorflow-2.17.0-r155: external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc:318:32: error: no member named 'StrCat' in namespace 'absl'
tensorflow-2.17.0-r155:   318 |       std::string name = absl::StrCat(
tensorflow-2.17.0-r155:       |                          ~~~~~~^
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:469:11: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
tensorflow-2.17.0-r155:   469 |           __VA_ARGS__,                                                       \
tensorflow-2.17.0-r155:       |           ^~~~~~~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:458:47: note: expanded from macro 'GTEST_GET_SECOND_'
tensorflow-2.17.0-r155:   458 | #define GTEST_GET_SECOND_(first, second, ...) second
tensorflow-2.17.0-r155:       |                                               ^~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:456:28: note: expanded from macro 'GTEST_EXPAND_'
tensorflow-2.17.0-r155:   456 | #define GTEST_EXPAND_(arg) arg
tensorflow-2.17.0-r155:       |                            ^~~
tensorflow-2.17.0-r155: external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc:318:32: error: no member named 'StrCat' in namespace 'absl'
tensorflow-2.17.0-r155:   318 |       std::string name = absl::StrCat(
tensorflow-2.17.0-r155:       |                          ~~~~~~^
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:472:32: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
tensorflow-2.17.0-r155:   472 |       auto t = std::make_tuple(__VA_ARGS__);                                 \
tensorflow-2.17.0-r155:       |                                ^~~~~~~~~~~
tensorflow-2.17.0-r155: external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc:318:32: error: no member named 'StrCat' in namespace 'absl'
tensorflow-2.17.0-r155:   318 |       std::string name = absl::StrCat(
tensorflow-2.17.0-r155:       |                          ~~~~~~^
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:477:9: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
tensorflow-2.17.0-r155:   477 |         __VA_ARGS__,                                                         \
tensorflow-2.17.0-r155:       |         ^~~~~~~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:458:47: note: expanded from macro 'GTEST_GET_SECOND_'
tensorflow-2.17.0-r155:   458 | #define GTEST_GET_SECOND_(first, second, ...) second
tensorflow-2.17.0-r155:       |                                               ^~~~~~
tensorflow-2.17.0-r155: external/com_google_googletest/googletest/include/gtest/gtest-param-test.h:456:28: note: expanded from macro 'GTEST_EXPAND_'
tensorflow-2.17.0-r155:   456 | #define GTEST_EXPAND_(arg) arg
tensorflow-2.17.0-r155:       |                            ^~~
tensorflow-2.17.0-r155: 6 errors generated.
INFO: Elapsed time: 289.081s, Critical Path: 124.03s
INFO: 2748 processes: 333 internal, 2415 local.
FAILED: Build did NOT complete successfully
tensorflow-2.17.0-r155:  * ERROR: sci-libs/tensorflow-2.17.0-r155::chromiumos failed (compile phase):
tensorflow-2.17.0-r155:  *   ebazel failed
tensorflow-2.17.0-r155:  *
tensorflow-2.17.0-r155:  * Call stack:
tensorflow-2.17.0-r155:  *               ebuild.sh, line  125:  Called src_compile
tensorflow-2.17.0-r155:  *             environment, line 5775:  Called ebazel 'build' '@org_tensorflow//tensorflow/lite:libtensorflowlite.so' '@cros_tflite//android:libnativewindow.so' '@org_tensorflow//tensorflow/lite/delegates/utils/experimental/stable_delegate:tflite_settings_json_parser' '@org_tensorflow//tensorflow/lite/delegates/utils/experimental/stable_delegate:delegate_loader' '@org_tensorflow//tensorflow/lite/tools/evaluation/tasks/inference_diff:run_eval' '@org_tensorflow//tensorflow/lite/tools/evaluation/tasks/coco_object_detection:run_eval' '@org_tensorflow//tensorflow/lite/tools/benchmark:benchmark_model' '@org_tensorflow//tensorflow/lite/delegates/utils/experimental/stable_delegate:stable_delegate_test_suite' '@cros_tflite//common:async_delegate_test' '@cros_tflite//tool:decompose_model' '@cros_tflite//android:hardware_buffer_test' '@cros_tflite//tool:op_compat_test' '@cros_tflite//tool:tflite_to_json' '@cros_tflite//delegate/sample:tensorflowlite_cros_sample_delegate'
tensorflow-2.17.0-r155:  *             environment, line 3487:  Called die
tensorflow-2.17.0-r155:  * The specific snippet of code:
tensorflow-2.17.0-r155:  *       "${@}" || die "ebazel failed"
tensorflow-2.17.0-r155:  *
tensorflow-2.17.0-r155: /mnt/host/source/src/third_party/chromiumos-overlay/profiles/base/profile.bashrc: line 724: /tmp/FAILED_PACKAGES: Permission denied
tensorflow-2.17.0-r155:  * Build log: /build/hatch/tmp/portage/logs/sci-libs:tensorflow-2.17.0-r155:20241209-214700.log
tensorflow-2.17.0-r155:  * Stable log symlink: /build/hatch/tmp/portage/sci-libs/tensorflow-2.17.0-r155/temp/build.log
tensorflow-2.17.0-r155:  * CWD: /build/hatch/tmp/portage/sci-libs/tensorflow-2.17.0-r155/work/tensorflow-2.17.0
tensorflow-2.17.0-r155:  * S:   /build/hatch/tmp/portage/sci-libs/tensorflow-2.17.0-r155/work/tensorflow-2.17.0

This is probably fixed upstream, but we don't seem to carry many patches for our tensorflow. Will try to find a way forward.

Comments

aj...@google.com <aj...@google.com> #2Dec 9, 2024 10:25PM

Probably this is the fix: https://github.com/tensorflow/tensorflow/commit/9ec6201b4fc4a936210346b8c7b3f631117e4fbf

aj...@google.com <aj...@google.com> #3Dec 10, 2024 12:30AM

Patching this is not as simple, since the chroot paths are actually:

/build/hatch/tmp/portage/sci-libs/tensorflow-2.17.0-r155/work/tensorflow-2.17.0-bazel-base/external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.cc
/build/hatch/tmp/portage/sci-libs/tensorflow-2.17.0-r160/work/tensorflow-2.17.0-bazel-base/external/org_tensorflow/tensorflow/lite/kernels/random_ops_test.c

PATCHES doesn't seem to reach into this, so might need to do manually. Unsure also when this directory is created and where the sources are coming from.

aj...@google.com <aj...@google.com> #4Dec 12, 2024 06:57PM

Got parallels working with a set of slot and install configurations! This does seem to increase on-device size though since we have to install two different abseils.

aj...@google.com <aj...@google.com> #5Jan 11, 2025 12:25AM

Ultimately, upgrading abseil to 20240116.2 was not needed to upgrade the parent bug: upgrading autofdo. We managed to patch the dependency on newer abseil out of autofdo 0.30. That being said, this bug does still exist. Just moving to a lower priority.

Issue 383173946

Description

Issue summary

Comments

aj...@google.com <aj...@google.com> #2Dec 9, 2024 10:25PM

aj...@google.com <aj...@google.com> #3Dec 10, 2024 12:30AM

aj...@google.com <aj...@google.com> #4Dec 12, 2024 06:57PM

aj...@google.com <aj...@google.com> #5Jan 11, 2025 12:25AM

Add comment

Issue metadata