[Flink] Drop 1.12–1.16 shims, baseline 1.17+, add 2.x support - #4419
Merged
Conversation
Set Maven release 11, update Spotless compliance, switch Docker and CI workflows to JDK 11, and align module compiler overrides. Closes apache#4409 (part 3/3). Ref apache#4410. Generated-by: Cursor Co-authored-by: Cursor <cursoragent@cursor.com>
Walk the class hierarchy to locate the ucp field on JDK 11+ app classloaders, add JUnit coverage, and enable surefire add-opens on JDK 9+. Closes apache#4409 (part 1/3). Ref apache#4410. Generated-by: Cursor Co-authored-by: Cursor <cursoragent@cursor.com>
Enable add-opens JVM opts, enforce JDK 11+ at startup, fix classpath layout, add javax.annotation-api, and document Console vs job JDK requirements. Closes apache#4409 (part 2/3). Ref apache#4410. Generated-by: Cursor Co-authored-by: Cursor <cursoragent@cursor.com>
Compile MavenWrapperHelper with --release 11 at runtime and stop packaging prebuilt wrapper classes so project builds work in JDK 11 containers. Update basic E2E for Docker 29, local Chrome, Git SSL, and Projects page flows verified on macOS. Generated-by: Cursor Co-authored-by: Cursor <cursoragent@cursor.com>
Migrate all modules from Scala to Java, drop scala-maven-plugin and _2.12 artifact suffixes, and consolidate the JDK 11 platform baseline with a zero-Scala codebase for Flink, Spark, Console, and connectors. Generated-by: Cursor Co-authored-by: Cursor <cursoragent@cursor.com>
Remove legacy Flink 1.12–1.16 shims, keep 1.17–1.20 as the 1.x baseline, and add Java shims-base-v2 with Flink 2.0/2.1/2.2 modules. Update FlinkVersion, shims classloading, console packaging, and UI hints for the new supported version matrix. Generated-by: Cursor Co-authored-by: Cursor <cursoragent@cursor.com>
This was referenced Jul 6, 2026
Contributor
Author
|
Design proposal: #4421 |
Align common and shaded parent POM versions and CLI banner with StreamPark 3.0 preview. Co-authored-by: Cursor <cursoragent@cursor.com>
Update Flink parent and submodule POM versions for StreamPark 3.0 preview. Co-authored-by: Cursor <cursoragent@cursor.com>
![github-advanced-security[bot]](https://avatars.github.com/in/57789?s=60&v=4){kind=small}
Replace printStackTrace with structured logging and document MD5 usage as non-cryptographic file checksums to satisfy SonarCloud security rules. Co-authored-by: Cursor <cursoragent@cursor.com>
Suppress non-cryptographic MD5 checksum usage, remove printStackTrace from ParameterCli, and document safe dynamic ClickHouse batch SQL assembly. Co-authored-by: Cursor <cursoragent@cursor.com>
Add canonical path helpers, resolve jar uploads under trusted base dirs, replace printStackTrace in Flink shims, and suppress non-crypto MD5/SQL checks. Co-authored-by: Cursor <cursoragent@cursor.com>
shangeyao
marked this pull request as draft
July 14, 2026 15:24
Reconcile with dev Java migration (scala-bridge, shims _2.12 suffix for 1.17–1.20) while keeping Flink 2.x shims and extending support to 2.3. Co-authored-by: Cursor <cursoragent@cursor.com>
Revert dev merge's _2.12 suffix and scala-bridge wiring for Flink modules. Restore no-suffix StreamPark artifacts (streampark-common, shims, client) and Java-only 1.17–1.20 shims poms; keep Flink 2.0–2.3 support. Co-authored-by: Cursor <cursoragent@cursor.com>
shangeyao
changed the title
shangeyao
marked this pull request as ready for review
August 3, 2026 07:06
Remove JavaConverters from FlinkVersion and align remaining POM/sources with post-Scala artifact IDs before merging latest dev. Co-authored-by: Cursor <cursoragent@cursor.com>
Integrate apache#4471 (Remove Scala) while preserving Flink 2.x shims and no-scala artifact model from PR apache#4419. Co-authored-by: Cursor <cursoragent@cursor.com>
Reuse shims-base from shims-base-v2 instead of duplicating identical sources, move shared 1.x/2.x context and client classes into base modules, and keep version modules as thin Flink API tier extensions only. Also harden FlinkSqlValidator against null operands and empty parse results. Co-authored-by: Cursor <cursoragent@cursor.com>
Roll back version-module and trait restructuring done for Sonar CPD. Retain only natural dedup: shims-base-v2 depends on shims-base for identical shared sources, plus FlinkSqlValidator null-safe fixes. Co-authored-by: Cursor <cursoragent@cursor.com>
Multi-version shims modules are structurally similar by design; exclude them from CPD so the quality gate reflects maintainable duplication only. Co-authored-by: Cursor <cursoragent@cursor.com>
Address deprecated API annotations, null collections, exception types, and minor refactors across Flink shims, Spark modules, and K8s parsers to reduce new issues reported by SonarCloud. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace java:S* suppressions with real refactors: command handler map for FlinkSqlExecutor, split SqlSplitter/ParameterCli methods, JsonNode-based FlinkRestModels parsers, and simplified RegExpTest patterns. Co-authored-by: Cursor <cursoragent@cursor.com>
Revert accidental Scala-to-Java migration artifacts so Spark modules stay unchanged on this Flink-focused branch. Co-authored-by: Cursor <cursoragent@cursor.com>
Restore k8s Jackson deps, guard null planner type in table initializer, fix 2.x TableExt compile error, complete shims-test dependencies, and update YarnPerJobTestCase for SubmitResponse API. Co-authored-by: Cursor <cursoragent@cursor.com>
Guard SET command operand access to prevent ArrayIndexOutOfBoundsException (javabugs:S6466) and align SqlClient runtime-mode parsing with the same length check. Co-authored-by: Cursor <cursoragent@cursor.com>
Restore scala-bridge module, align Spark/console POM artifact IDs with post-migration naming, and exclude webapp-v2 from RAT checks. Co-authored-by: Cursor <cursoragent@cursor.com>
…ader Drop one-off tools/migrate_connector_phase6.py to fix License header CI. Co-authored-by: Cursor <cursoragent@cursor.com>
…ation Add httpclient test scope dependency and surefire workspace property so MavenToolTest can resolve artifacts in CI and local builds. Co-authored-by: Cursor <cursoragent@cursor.com>
Lazy-initialize the manual YARN harness so CI skips the test instead of failing during static class initialization when FLINK_HOME is absent. Co-authored-by: Cursor <cursoragent@cursor.com>
…ecks Verify deployInternal reflection, SubmitResponse, yarn-per-job CLI parsing, and optionally FLINK_HOME layout when the env var is present. Co-authored-by: Cursor <cursoragent@cursor.com>
Dist packaging no longer ships scala-compiler/library/reflect as third-party artifacts; align known-dependencies with check-LICENSE.sh. Co-authored-by: Cursor <cursoragent@cursor.com>
Console service still loads Scala code at runtime; without these jars the dist package fails to start (ClassNotFoundException: scala.Product). Re-add the scala entries to known-dependencies.txt to match the packaged lib. Co-authored-by: Cursor <cursoragent@cursor.com>
|
GuoNingNing
approved these changes
Aug 4, 2026
GuoNingNing
left a comment
GuoNingNing
left a comment
Contributor
There was a problem hiding this comment.
Locally verified on JDK 11.0.31:
./mvnw -Pfast,flink-2.x-shims -f streampark-flink/streampark-flink-shims/pom.xml compile -DskipTests— all shim modules (1.17–1.20, 2.0–2.3) build clean../mvnw clean install -Pfast -DskipTests— full backend reactor (27 modules) builds clean../mvnw spotless:check— passes across all modules.- CI (38 checks) green, SonarCloud quality gate passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
flink-scala/ scala-bridge deps).streampark-flink-shims-base-v2and thin version shims.streampark-common,streampark-flink-shims_flink-1.20, …) — no_2.12suffix, nostreampark-common-scala-bridgein the Flink build path._2.12layout.Related issues
Closes #4421.
Supported Flink versions after this PR
Test plan
feature/remove-scala-finalonce [Build] Remove Scala overall regression test and post-migration cleanup #4471 mergesAI Disclosure
Made with Cursor