[Flink] Migrate streampark-flink-client from Scala to Java - #4467
Merged
wolfboys merged 10 commits intoAug 3, 2026
Conversation
Convert all 27 Scala sources in client-api and client-core to Java, replace json4s with Jackson in session REST submit, and preserve FlinkShimsProxy reflection contracts for console compatibility. Closes apache#4452 Co-authored-by: Cursor <cursoragent@cursor.com>
6 tasks
Address 31 Sonar findings: serializable request fields (S1948), refactor complex methods (S3776/S1141), replace instanceof checks (S1193), remove printStackTrace (S4507), and improve exception/boolean handling (S112/S5411). Co-authored-by: Cursor <cursoragent@cursor.com>
Use parameter objects (JobClientTarget, SubmitApplicationSpec, etc.) for S107, make BuildResult Serializable for S1948, and standardize client API on FlinkException instead of @SuppressWarnings for S112. Co-authored-by: Cursor <cursoragent@cursor.com>
… Yarn trait Replace throws Exception with FlinkException in FlinkClientEntrypoint and refactor YarnClientTrait to eliminate nested try blocks without suppressions. Co-authored-by: Cursor <cursoragent@cursor.com>
Extract callAsFlinkException helpers, AbstractSavepointClientRequest base class, and generic entrypoint routing to lower new-code duplication. Co-authored-by: Cursor <cursoragent@cursor.com>
Extract shared logging, job-graph submit, and packer build response base classes to bring new-code duplication below the quality gate threshold. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Extract shared deploy request base, hdfs workspace helper, and compose SubmitRequest from SubmitApplicationSpec to eliminate 221 duplicated lines. Co-authored-by: Cursor <cursoragent@cursor.com>
Align constructor parameter indentation with the project formatter so CI spotless:check passes. Co-authored-by: Cursor <cursoragent@cursor.com>
Apply project formatter to client-core implementations and traits so CI spotless:check passes during backend build. Co-authored-by: Cursor <cursoragent@cursor.com>
|
This was referenced Aug 3, 2026
wolfboys
pushed a commit
that referenced
this pull request
Aug 3, 2026
…up (#4471) * [Build] Drop Scala build toolchain from Flink and Console modules Flink modules are fully Java after #4467; remove the common-scala-bridge dependency, enumeratum/scalatest test deps, and scala-maven-plugin from Flink parent, sqlclient, udf, and console-service. Bridge module is retained for Spark until #4453. * [Test][Common][Flink] Add migration regression tests and small cleanups Add unit tests for AutoCloseUtils, ClientBeanUtils, SubmitRequest, and ExitSecurityManager. Remove unused FlinkVersion.flinkLibs Scala bridge, drop unused LoggerSupport from FlinkClient, and minor SubmitRequest tidy. * [Common][Flink] Sonar-oriented fixes and expand migration regression tests Replace printStackTrace and generic RuntimeException wraps with specific exceptions, null-safe SubmitRequest accessors, FlinkShimsProxy URL helper, and TrackId validation cleanup. Add ParameterCliTest, TrackIdTest, and more SqlClient/SubmitRequest coverage. * [Test] Keep unit-test-only coverage; revert production Sonar edits Restore production code from the prior commit and retain focused unit tests for ParameterCli, TrackId, SqlClient, and SubmitRequest. Complex deploy-mode and cluster scenarios are deferred to E2E. * [Common][Flink] Re-apply Sonar Java quality fixes with unit test sync Restore production Sonar improvements (specific exceptions, null-safe accessors, toUrl helper, lambda cleanup) while keeping unit tests focused on pure logic; complex deploy scenarios remain deferred to E2E. * [Flink] Fix shims Sonar issues in code and remove migration Sonar configs Remove @SuppressWarnings and shims-specific sonar-project/.sonarcloud properties added during Scala-to-Java migration; address deprecations and unchecked casts directly in shims source instead. * [Flink] Remove unused imports after TableContext refactor Spotless check requires dropping stale Utils imports from TableContext. * [Flink] Drop invalid @OverRide on FlinkStreamTableTrait.execute execute(String) is not part of StreamTableEnvironment in all Flink versions; keep the method public without @OverRide for compatibility. * [Flink] Address Sonar new-code findings in shims-base Fix unused parameters, replace generic exceptions, reduce cognitive complexity, return empty collections, simplify regex matching, and deduplicate TableExt conversion helpers per SonarCloud dev analysis. * [Flink] Continue Sonar cleanup for shims, client, and SQL splitter Refactor SqlSplitter with QueryExtractor, fix generic type names, ParameterCli stdout emission, SubmitRequest exceptions, and remaining Sonar findings from the dev new-code report. * [Flink] Hoist shared Table API to traits and slim version shims contexts Move common TableEnvironment delegates into FlinkTableTrait and FlinkStreamTableTrait so version-specific TableContext and StreamTableContext only retain legacy or version-gated APIs. Refactor FlinkSqlValidator validation flow for lower complexity. * [Flink] Hoist stream-table bridge APIs and collapse version contexts Move Schema-based stream-table conversions and version-gated Table API delegates into FlinkStreamTableTrait and FlinkTableTrait. Reduce 1.15–1.20 context classes to constructors plus version-specific legacy overrides only where older Flink APIs require them. * [Flink] Remove unchecked proxy casts and tighten error handling Use Class.cast in FlinkShimsProxy.getObject and typed FlinkClient invocations to drop @SuppressWarnings. Fix SqlClient unzip validation, SubmitRequest exception handling, and FlinkSqlValidator caused-by parsing. * [Flink] Clean up packer and kubernetes Sonar findings Replace unchecked casts with Class.cast or typed YAML helpers, use IllegalStateException for operational failures, and document enum constants to remove @SuppressWarnings suppressions. * [Flink] Replace RuntimeException in Yarn client deploy paths * [Console] Replace Flink RuntimeException usages with typed failures Use IllegalStateException with causes for Yarn/K8s deploy and watcher paths, fix FlinkSql rollback error chaining, and remove unchecked JSON casts in FlinkApplication via TypeReference. * [Common] Replace RuntimeException with typed failures in shared utilities Use IllegalStateException/IllegalArgumentException with context across common util classes, refactor StringCastUtils with Class.cast, and align Console shared infrastructure (Shiro, build task, Throws) for Sonar compliance. * [Flink] Fix Sonar duplication and code smell findings for PR #4471 Extract shared Yarn jar upload and K8s Docker build helpers to reduce new-code duplication below the 3% quality gate, rename pipeline enum constants to UPPER_CASE, and address SqlSplitter/StringCastUtils Sonar issues. * [Flink] Extract shared Yarn/K8s build pipeline steps to fix Sonar duplication Hoist Yarn SQL build steps into BuildPipeline, K8s docker/pod-template steps into AbstractK8sApplicationBuildPipeline, and AppControl factory to bring new-code duplication below the 3% quality gate threshold. * [Console] Extract shared application and build pipeline helpers to fix Sonar duplication Consolidate duplicated logic between Flink/Spark entities and build pipeline services so PR #4471 can pass the new-code duplication quality gate. * [Console] Further deduplicate entities, build pipelines, and watchers for Sonar Move shared entity defaults to ApplicationEntitySupport, extract release outcome and watcher init helpers to bring PR #4471 new-code duplication below 3%. * [Common] Fix StringCastUtils generic cast and restore Flink build log level Accept Class<?> in cast() so common_2.12 compiles with ConfigOption callers, and keep Flink dependency-merge failures logged at error while Spark stays at warn.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
streampark-flink-client(api + core) to JavaFlinkSessionSubmitHelper; remove json4s dependency and maven-shade packagingFlinkShimsProxyreflection contracts (FlinkClient,FlinkClientEntrypoint, bean class names) for console compatibilityClusterKey/OptionalAPIs (post [K8s] Migrate streampark-flink-kubernetes from Scala to Java #4465)Closes #4452
Test plan
./mvnw test -pl streampark-flink-client-core -Dtest=ParameterTestCase./mvnw install -pl streampark-console-service -am -DskipTestsMade with Cursor