Dependabot PR #92 proposes bumping spark.version 4.0.0 -> 4.2.0. Held — deliberate evaluation, not an auto-merge.
spark.version is pinned to 4.0.0 across all four Maven profiles so a single JAR runs on DBR 17.3 / 18 / 19 (which ship Spark 4.0.x). Building against 4.2.0 risks linking APIs the DBR-shipped Spark doesn't provide (NoSuchMethodError / ClassNotFoundException on-cluster) and jeopardises the 'one JAR on 17.3/18/19' invariant.
Acceptance criteria:
- Confirm the Spark version each supported DBR (17.3/18/19) ships; move only to a version all provide (or drop the ones that don't).
- Build + run the Scala/parity suites against the target Spark; validate the JAR on each DBR.
- Bump
spark.version in all four pom profiles together.
Tracked from #92.
Dependabot PR #92 proposes bumping
spark.version4.0.0 -> 4.2.0. Held — deliberate evaluation, not an auto-merge.spark.versionis pinned to 4.0.0 across all four Maven profiles so a single JAR runs on DBR 17.3 / 18 / 19 (which ship Spark 4.0.x). Building against 4.2.0 risks linking APIs the DBR-shipped Spark doesn't provide (NoSuchMethodError/ClassNotFoundExceptionon-cluster) and jeopardises the 'one JAR on 17.3/18/19' invariant.Acceptance criteria:
spark.versionin all four pom profiles together.Tracked from #92.