GH-50581: [C++][R] Use bundled simdjson for C++ wrapper builds by Reranko05 · Pull Request #50587 · apache/arrow

Reranko05 · 2026-07-22T01:26:37Z

Rationale

The R build may pick up a system-installed simdjson when simdjson_SOURCE is not explicitly set. This causes build failures in environments where the system version is incompatible or unsuitable for the build.

What changes are included?

This change defaults simdjson_SOURCE to BUNDLED when it is not already set, ensuring C++ wrapper builds invoked through build_arrow_static.sh use the bundled simdjson by default, while still allowing users to override it explicitly.

This addresses the macOS CRAN release and ubuntu-r-only-r (AUTO deps) failures reported in GH-50581.

Related: #50581

GitHub Issue: [R] R package build fails after simdjson introduction due to system simdjson pickup #50581

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-07-22T01:27:02Z

⚠️ GitHub issue #50581 has been automatically assigned in GitHub to PR creator.

kou · 2026-07-22T02:55:51Z

@github-actions crossbow submit -g r

github-actions · 2026-07-22T02:58:49Z

Revision: 9864597

Submitted crossbow builds: ursacomputing/crossbow @ actions-357652d8fe

Task	Status
r-binary-packages
r-recheck-most
test-r-alpine-linux-cran
test-r-arrow-backwards-compatibility
test-r-depsource-system
test-r-dev-duckdb
test-r-devdocs
test-r-extra-packages
test-r-fedora-clang
test-r-gcc-11
test-r-gcc-12
test-r-install-local
test-r-install-local-minsizerel
test-r-linux-as-cran
test-r-linux-rchk
test-r-linux-sanitizers
test-r-linux-valgrind
test-r-m1-san
test-r-macos-as-cran
test-r-offline-maximal
test-r-ubuntu-22.04
test-r-versions
test-r-wasm

Reranko05 · 2026-07-22T16:40:03Z

@kou It looks like the remaining failures are related to the ongoing simdjson version discussion. If we go with requiring simdjson 4.0+ and falling back to the bundled version when the system package is too old, I think that should address this issue.

rok · 2026-07-22T18:45:35Z

I don't think this is isolated to R, e.g. see windows python nightly failure and you can check other nightlies.
I think the approach is correct though and don't worry too much about this, we have until the next release to fix it :D

kou · 2026-07-23T02:23:54Z

Could you try this?

diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index 1677b188b9..dc8629669f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake
+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake
@@ -2828,7 +2828,7 @@ function(build_simdjson)
       TRUE
       PARENT_SCOPE)
 
-  list(APPEND ARROW_BUNDLED_STATIC_LIBS simdjson::simdjson)
+  list(APPEND ARROW_BUNDLED_STATIC_LIBS simdjson)
   set(ARROW_BUNDLED_STATIC_LIBS
       "${ARROW_BUNDLED_STATIC_LIBS}"
       PARENT_SCOPE)
@@ -2845,6 +2845,11 @@ if(ARROW_WITH_SIMDJSON)
                      ${ARROW_SIMDJSON_REQUIRED_VERSION}
                      IS_RUNTIME_DEPENDENCY
                      FALSE)
+  if(SIMDJSON_VENDORED)
+    add_library(arrow::simdjson ALIAS simdjson)
+  else()
+    add_library(arrow::simdjson ALIAS simdjson::simdjson)
+  endif()
 endif()
 
 function(build_rapidjson)
diff --git a/cpp/src/arrow/CMakeLists.txt b/cpp/src/arrow/CMakeLists.txt
index e792574a18..0b21fec605 100644
--- a/cpp/src/arrow/CMakeLists.txt
+++ b/cpp/src/arrow/CMakeLists.txt
@@ -1044,7 +1044,7 @@ if(ARROW_JSON)
                            json/parser.cc
                            json/reader.cc)
   foreach(ARROW_JSON_TARGET ${ARROW_JSON_TARGETS})
-    target_link_libraries(${ARROW_JSON_TARGET} PRIVATE RapidJSON simdjson::simdjson)
+    target_link_libraries(${ARROW_JSON_TARGET} PRIVATE RapidJSON arrow::simdjson)
   endforeach()
 else()
   set(ARROW_JSON_TARGET_SHARED)

This will fix the following case:

https://github.com/ursacomputing/crossbow/actions/runs/29887151358/job/88819918379#step:6:8835

/usr/bin/ar: /usr/lib/x86_64-linux-gnu/libsimdjson.so.9.0.0: file format not recognized

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Reranko05 · 2026-07-23T04:25:23Z

@github-actions crossbow submit -g r

github-actions · 2026-07-23T04:25:58Z

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/29979508366

kou · 2026-07-23T05:02:35Z

@github-actions crossbow submit -g r

Reranko05 · 2026-07-23T08:18:15Z

@kou I suspect the remaining Crossbow failures may be related to changes in the JsonWriter PR that haven't been merged yet (for example, it adds simdjson_SOURCE=BUNDLED to the R Docker test configuration, which seems relevant to the Ubuntu failure).

kou · 2026-07-23T09:30:29Z

test-r-macos-as-cran:

https://github.com/ursacomputing/crossbow/actions/runs/29981142927/job/89122965972#step:9:1373

In file included from /Users/runner/work/crossbow/crossbow/arrow/cpp/src/arrow/json/object_parser.cc:20:
/opt/homebrew/include/simdjson.h:3106:10: error: expected concept name with optional arguments
 3106 |     } -> std::convertible_to<typename std::remove_cvref_t<T>::value_type>;
      |          ^
/opt/homebrew/include/simdjson.h:3127:8: error: no member named 'ranges' in namespace 'std'
 3127 |   std::ranges::input_range<T> && !string_like<T> && !concepts::string_view_keyed_map<T>;
      |   ~~~~~^
/opt/homebrew/include/simdjson.h:3127:28: error: 'T' does not refer to a value
 3127 |   std::ranges::input_range<T> && !string_like<T> && !concepts::string_view_keyed_map<T>;
      |                            ^
/opt/homebrew/include/simdjson.h:3125:20: note: declared here
 3125 | template <typename T>
      |                    ^
/opt/homebrew/include/simdjson.h:3127:34: error: expected identifier
 3127 |   std::ranges::input_range<T> && !string_like<T> && !concepts::string_view_keyed_map<T>;
      |                                  ^
/opt/homebrew/include/simdjson.h:3140:17: error: expected concept name with optional arguments
 3140 |     { c[i] } -> std::convertible_to<typename Container::value_type>;
      |                 ^
In file included from /var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/RtmpjwvCVB/file223d386e52a2/_deps/azure_sdk-src/sdk/core/azure-core/src/http/request_activity_policy.cpp:4:
/var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/RtmpjwvCVB/file223d386e52a2/_deps/azure_sdk-src/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:429:20: warning: virtual method 'ShouldRetryOnTransportFailure' is inside a 'final' class and can never be overridden [-Wunnecessary-virtual-specifier]
  429 |       virtual bool ShouldRetryOnTransportFailure(
      |                    ^
/var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T/RtmpjwvCVB/file223d386e52a2/_deps/azure_sdk-src/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:435:20: warning: virtual method 'ShouldRetryOnResponse' is inside a 'final' class and can never be overridden [-Wunnecessary-virtual-specifier]
  435 |       virtual bool ShouldRetryOnResponse(
      |                    ^
/opt/homebrew/include/simdjson.h:63624:19: error: no member named 'invocable' in namespace 'std'
 63624 |     requires std::invocable<Func, value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:63823:19: error: no member named 'invocable' in namespace 'std'
 63823 |     requires std::invocable<Func, arm64::ondemand::value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:65348:19: error: no member named 'invocable' in namespace 'std'
 65348 |     requires std::invocable<Func, value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:65478:19: error: no member named 'invocable' in namespace 'std'
 65478 |     requires std::invocable<Func, arm64::ondemand::value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:66395:19: error: no member named 'invocable' in namespace 'std'
 66395 |     requires std::invocable<Func, value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:66622:19: error: no member named 'invocable' in namespace 'std'
 66622 |     requires std::invocable<Func, value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:66714:19: error: no member named 'invocable' in namespace 'std'
 66714 |     requires std::invocable<Func, arm64::ondemand::value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:66805:19: error: no member named 'invocable' in namespace 'std'
 66805 |     requires std::invocable<Func, arm64::ondemand::value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:67472:19: error: no member named 'invocable' in namespace 'std'
 67472 |     requires std::invocable<Func, value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:67628:19: error: no member named 'invocable' in namespace 'std'
 67628 |     requires std::invocable<Func, arm64::ondemand::value>
       |                   ^~~~~~~~~
/opt/homebrew/include/simdjson.h:67910:16: error: no type named 'unsigned_integral' in namespace 'std'
 67910 | template <std::unsigned_integral T>
       |           ~~~~~^
/opt/homebrew/include/simdjson.h:67911:51: error: unknown type name 'T'
 67911 | error_code tag_invoke(deserialize_tag, auto &val, T &out) noexcept {
       |                                                   ^
/opt/homebrew/include/simdjson.h:67912:38: error: template argument for template type parameter must be a type
 67912 |   using limits = std::numeric_limits<T>;
       |                                      ^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/limits:442:17: note: template parameter is declared here
  442 | template <class _Tp>
      |                 ^
In file included from /Users/runner/work/crossbow/crossbow/arrow/cpp/src/arrow/json/object_parser.cc:20:
/opt/homebrew/include/simdjson.h:67916:12: error: use of undeclared identifier 'limits'
 67916 |   if (x > (limits::max)()) {
       |            ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

It seems that we can't build simdjson with AppleClang 21:

https://github.com/ursacomputing/crossbow/actions/runs/29981142927/job/89122965972#step:9:56

-- The C compiler identification is AppleClang 21.0.0.21000101
-- The CXX compiler identification is AppleClang 21.0.0.21000101

kou · 2026-07-23T09:39:04Z

test-r-depsource-system:

https://github.com/ursacomputing/crossbow/actions/runs/29981142748/job/89122965384#step:6:3165

simdjson_SOURCE=BUNDLED

https://github.com/ursacomputing/crossbow/actions/runs/29981142748/job/89122965384#step:7:361

CMake Error at cmake_modules/ThirdpartyToolchain.cmake:316 (message):
  Couldn't find simdjson >= 3.0.0

It seems that simdjson_SOURCE=BUNDLED environment variable isn't used.

Could you try this?

diff --git a/r/inst/build_arrow_static.sh b/r/inst/build_arrow_static.sh
index e7f453bb64..8c8d9d6b95 100755
--- a/r/inst/build_arrow_static.sh
+++ b/r/inst/build_arrow_static.sh
@@ -107,8 +107,9 @@ ${CMAKE_WRAPPER} ${CMAKE} -DARROW_BOOST_USE_SHARED=OFF \
     -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON \
     -DCMAKE_UNITY_BUILD=${CMAKE_UNITY_BUILD:-OFF} \
     -DOPENSSL_ROOT_DIR=${OPENSSL_ROOT_DIR} \
-    -Dre2_SOURCE=${re2_SOURCE:-BUNDLED} \
     -Dabsl_SOURCE=${absl_SOURCE:-BUNDLED} \
+    -Dre2_SOURCE=${re2_SOURCE:-BUNDLED} \
+    -Dsimdjson_SOURCE=${simdjson_SOURCE:-} \
     -Dxsimd_SOURCE=${xsimd_SOURCE:-} \
     -Dzstd_SOURCE=${zstd_SOURCE:-} \
     ${EXTRA_CMAKE_FLAGS} \

kou · 2026-07-23T09:49:52Z

It seems that r-recheck-most failures are unrelated.
I've re-run it.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

rok · 2026-07-23T11:13:00Z

-    -Dre2_SOURCE=${re2_SOURCE:-BUNDLED} \
    -Dabsl_SOURCE=${absl_SOURCE:-BUNDLED} \
+    -Dre2_SOURCE=${re2_SOURCE:-BUNDLED} \
+    -Dsimdjson_SOURCE=${simdjson_SOURCE:-} \


@kou should we accept this and remove the change in r/tootls/nixlibs.R?

Good point! Let's try it!

@Reranko05 Could you try this?

rok

This looks good!
As this touches the all the cpp wrappers we should also adjust issue and PR title/text.

rok · 2026-07-23T11:13:00Z

-    -Dre2_SOURCE=${re2_SOURCE:-BUNDLED} \
    -Dabsl_SOURCE=${absl_SOURCE:-BUNDLED} \
+    -Dre2_SOURCE=${re2_SOURCE:-BUNDLED} \
+    -Dsimdjson_SOURCE=${simdjson_SOURCE:-} \


@kou should we accept this and remove the change in r/tootls/nixlibs.R?

Reranko05 · 2026-07-23T11:22:27Z

@rok Updated PR title and description.

kou · 2026-07-23T20:37:15Z

@github-actions crossbow submit -g r

github-actions · 2026-07-23T20:40:31Z

Revision: b265939

Submitted crossbow builds: ursacomputing/crossbow @ actions-aa46ed246c

Task	Status
r-binary-packages
r-recheck-most
test-r-alpine-linux-cran
test-r-arrow-backwards-compatibility
test-r-depsource-system
test-r-dev-duckdb
test-r-devdocs
test-r-extra-packages
test-r-fedora-clang
test-r-gcc-11
test-r-gcc-12
test-r-install-local
test-r-install-local-minsizerel
test-r-linux-as-cran
test-r-linux-rchk
test-r-linux-sanitizers
test-r-linux-valgrind
test-r-m1-san
test-r-macos-as-cran
test-r-offline-maximal
test-r-ubuntu-22.04
test-r-versions
test-r-wasm

kou · 2026-07-24T02:17:45Z

Can we disable ARROW_JSON and ARROW_WITH_SIMDJSON forcibly with AppleClang 21 or older?

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Reranko05 · 2026-07-24T03:55:59Z

@kou I tried implementing the workaround by forcibly disabling ARROW_JSON for AppleClang < 22 in ThirdpartyToolchain.cmake. Since ARROW_WITH_SIMDJSON is only enabled when ARROW_JSON is enabled, this also keeps ARROW_WITH_SIMDJSON disabled.

I used AppleClang < 22 instead of <= 21 because the compiler version reported by the failing job is 21.0.0.21000101, and VERSION_LESS "22" naturally covers all 21.x releases.

Does this look like the right approach, or did you have something else in mind?

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Reranko05 · 2026-07-24T04:20:29Z

@github-actions crossbow submit -g r

@kou can you run this again?

kou · 2026-07-24T04:55:03Z

  set(ARROW_WITH_AZURE_SDK ON)
 endif()

+if(APPLE


APPLE is redundant. AppleClang is available only on Apple.

kou · 2026-07-24T04:55:48Z

+if(APPLE
+   AND CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang"
+   AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS "22")


Could you check this only when ARROW_JSON=ON?

Suggested change

if(APPLE

AND CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang"

AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS "22")

if(ARROW_JSON

AND CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang"

AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS "22")

kou · 2026-07-24T04:56:15Z

+  set(ARROW_JSON
+      OFF
+      CACHE BOOL "" FORCE)


We can treat ARROW_JSON as a normal variable.

Suggested change

set(ARROW_JSON

OFF

CACHE BOOL "" FORCE)

set(ARROW_JSON OFF)

Use bundled simdjson by default

9864597

Reranko05 requested a review from assignUser as a code owner July 22, 2026 01:26

Copilot AI review requested due to automatic review settings July 22, 2026 01:26

Copilot AI reviewed Jul 22, 2026

github-actions Bot added Component: R awaiting review Awaiting review labels Jul 22, 2026

kou added the CI: Extra: R Run extra R CI label Jul 22, 2026

Add simdjson alias target

09faaf9

Copilot AI review requested due to automatic review settings July 23, 2026 03:06

Reranko05 requested a review from pitrou as a code owner July 23, 2026 03:06

Copilot AI reviewed Jul 23, 2026

github-actions Bot added the Component: C++ label Jul 23, 2026

This comment was marked as outdated.

Sign in to view

Forward simdjson_SOURCE to CMake

b265939

Copilot AI review requested due to automatic review settings July 23, 2026 10:18

Reranko05 requested review from jonkeane and thisisnic as code owners July 23, 2026 10:18

Copilot AI reviewed Jul 23, 2026

rok requested a review from Copilot July 23, 2026 10:22

Copilot started reviewing on behalf of rok July 23, 2026 10:22 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

rok approved these changes Jul 23, 2026

View reviewed changes

github-actions Bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Jul 23, 2026

Reranko05 changed the title ~~GH-50581: [R] Use bundled simdjson by default~~ GH-50581: [R] Use bundled simdjson for C++ wrapper builds Jul 23, 2026

rok changed the title ~~GH-50581: [R] Use bundled simdjson for C++ wrapper builds~~ GH-50581: [C++][R] Use bundled simdjson for C++ wrapper builds Jul 23, 2026

github-actions Bot added awaiting changes Awaiting changes and removed awaiting merge Awaiting merge labels Jul 23, 2026

kou mentioned this pull request Jul 24, 2026

GH-50609: [CI][Dev] Fix shellcheck errors in the ci/scripts/r_deps.sh #50610

Open

Copilot AI review requested due to automatic review settings July 24, 2026 03:53

Copilot AI reviewed Jul 24, 2026

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jul 24, 2026

Disable JSON on AppleClang < 22

06732bd

Copilot AI review requested due to automatic review settings July 24, 2026 03:59

Reranko05 force-pushed the R-simdjson-bundle branch from 5841739 to 06732bd Compare July 24, 2026 03:59

Copilot AI reviewed Jul 24, 2026

github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Jul 24, 2026

kou reviewed Jul 24, 2026

View reviewed changes

Uh oh!

Conversation

Reranko05 commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale

What changes are included?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 22, 2026

Uh oh!

kou commented Jul 22, 2026

Uh oh!

github-actions Bot commented Jul 22, 2026

Uh oh!

Reranko05 commented Jul 22, 2026

Uh oh!

rok commented Jul 22, 2026

Uh oh!

kou commented Jul 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reranko05 commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

Uh oh!

kou commented Jul 23, 2026

Uh oh!

This comment was marked as outdated.

Reranko05 commented Jul 23, 2026

Uh oh!

kou commented Jul 23, 2026

Uh oh!

kou commented Jul 23, 2026

Uh oh!

kou commented Jul 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

rok Jul 23, 2026 • edited by kou Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kou Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

kou Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

rok left a comment

Choose a reason for hiding this comment

Uh oh!

rok Jul 23, 2026 • edited by kou Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reranko05 commented Jul 23, 2026

Uh oh!

kou commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

Uh oh!

kou commented Jul 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reranko05 commented Jul 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reranko05 commented Jul 22, 2026 •

edited

Loading

rok Jul 23, 2026 •

edited by kou

Loading

rok Jul 23, 2026 •

edited by kou

Loading