Task Summary
Integration-test support is currently hard-wired to the amber module only. Turn it into a reusable mechanism any module can opt into, collected by a single cross-module integration CI job — so integration specs live next to the code they exercise, in whatever module that is.
Trigger (from #6045): the shared VirtualDocumentSpec base and three live-catalog iceberg specs exercise common/workflow-core code, so their specs ideally live there — but common has no integration-test capability today, and pulling common's test scope into amber (WorkflowCore % "test->test") causes a classpath conflict (Hadoop → com.sun.jersey 1.19's JAX-RS 1.x Response shadows amber's javax.ws.rs-api:2.1.1, breaking HuggingFaceModelResourceSpec). #6045 left those specs stranded in amber as a stopgap.
common/workflow-core is just the first module to need this — the same capability should be available to the other Scala modules (*-service, WorkflowOperator, DAO, Auth, …) as they grow integration coverage.
Current state
- The
@IntegrationTest tag lives at amber/src/test/integration/.../amber/tags/IntegrationTest.java — in amber's package and source dir.
- The wiring (
Test / unmanagedSourceDirectories += src/test/integration + the AMBER_TEST_FILTER skip/only switch) is only in amber/build.sbt.
- Only
amber has a src/test/integration source set.
- The
amber-integration CI job runs only WorkflowExecutionService/test (integration-only) + amber Python integration tests — no other module's integration specs.
Suggested direction
Reverse the coupling: instead of dragging a module's test scope into amber (which is what causes the classpath conflict), give each module its own integration source set running on its own classpath.
- Move the
@IntegrationTest tag to a shared location (e.g. common) so any module can reference it.
- Factor the integration source-set +
AMBER_TEST_FILTER wiring into shared, reusable sbt settings (an AutoPlugin or a shared settings sequence) applied to each participating module — not copy-pasted per build.sbt.
- Generalize the
amber-integration CI job into an integration job that runs the integration-tagged specs of every participating module, not just WorkflowExecutionService.
- First adopter: move the iceberg specs +
VirtualDocumentSpec from amber back into common/workflow-core/src/test/integration. Other modules opt in incrementally as they add integration coverage.
Why this avoids the #6045 conflict: each module's integration specs compile and run on that module's own classpath (common already has Hadoop as a legit main dep), so common's jersey 1.19 is never mixed into amber's javax.ws.rs 2.1 classpath.
Follow-up to #6045; related to #6034.
Task Type
Task Summary
Integration-test support is currently hard-wired to the
ambermodule only. Turn it into a reusable mechanism any module can opt into, collected by a single cross-moduleintegrationCI job — so integration specs live next to the code they exercise, in whatever module that is.Trigger (from #6045): the shared
VirtualDocumentSpecbase and three live-catalog iceberg specs exercisecommon/workflow-corecode, so their specs ideally live there — but common has no integration-test capability today, and pulling common's test scope into amber (WorkflowCore % "test->test") causes a classpath conflict (Hadoop →com.sun.jersey1.19's JAX-RS 1.xResponseshadows amber'sjavax.ws.rs-api:2.1.1, breakingHuggingFaceModelResourceSpec). #6045 left those specs stranded inamberas a stopgap.common/workflow-core is just the first module to need this — the same capability should be available to the other Scala modules (
*-service,WorkflowOperator,DAO,Auth, …) as they grow integration coverage.Current state
@IntegrationTesttag lives atamber/src/test/integration/.../amber/tags/IntegrationTest.java— in amber's package and source dir.Test / unmanagedSourceDirectories += src/test/integration+ theAMBER_TEST_FILTERskip/only switch) is only inamber/build.sbt.amberhas asrc/test/integrationsource set.amber-integrationCI job runs onlyWorkflowExecutionService/test(integration-only) + amber Python integration tests — no other module's integration specs.Suggested direction
Reverse the coupling: instead of dragging a module's test scope into
amber(which is what causes the classpath conflict), give each module its own integration source set running on its own classpath.@IntegrationTesttag to a shared location (e.g.common) so any module can reference it.AMBER_TEST_FILTERwiring into shared, reusable sbt settings (an AutoPlugin or a shared settings sequence) applied to each participating module — not copy-pasted perbuild.sbt.amber-integrationCI job into anintegrationjob that runs the integration-tagged specs of every participating module, not justWorkflowExecutionService.VirtualDocumentSpecfromamberback intocommon/workflow-core/src/test/integration. Other modules opt in incrementally as they add integration coverage.Why this avoids the #6045 conflict: each module's integration specs compile and run on that module's own classpath (common already has Hadoop as a legit main dep), so common's
jersey1.19 is never mixed into amber'sjavax.ws.rs2.1 classpath.Follow-up to #6045; related to #6034.
Task Type