Skip to content

[Compiler Refactor 3] Use immutable list instead of mutable list in logical plan#1795

Merged
zuozhiw merged 2 commits into
masterfrom
zuozhi-plan-immutable
Jan 18, 2023
Merged

[Compiler Refactor 3] Use immutable list instead of mutable list in logical plan#1795
zuozhiw merged 2 commits into
masterfrom
zuozhi-plan-immutable

Conversation

@zuozhiw

@zuozhiw zuozhiw commented Jan 18, 2023

Copy link
Copy Markdown
Contributor

This PR is a small refactor to replace mutable.MutableList with the corresponding immutable List in scala. This refactor helps the engine to use immutable data structures more often, which is considered a better practice in scala.

Some minor failed tests in WorkflowRewriterSpec are disabled temporarily. These test cases involves minor and unimportant cases in workflow cache feature. They will be fixed and enabled again once the whole refactor is completed.
Update: these case tests are added back in #1800

@zuozhiw zuozhiw requested a review from shengquan-ni January 18, 2023 22:44
@zuozhiw zuozhiw changed the title Use immutable list instead of mutable list in logical plan [Compiler Refactor #3] Use immutable list instead of mutable list in logical plan Jan 18, 2023
@zuozhiw zuozhiw merged commit 7049c39 into master Jan 18, 2023
@zuozhiw zuozhiw deleted the zuozhi-plan-immutable branch January 18, 2023 23:33
@zuozhiw zuozhiw changed the title [Compiler Refactor #3] Use immutable list instead of mutable list in logical plan [Compiler Refactor 3] Use immutable list instead of mutable list in logical plan Jan 19, 2023
zuozhiw added a commit that referenced this pull request Jan 19, 2023
This PR refactors the logical plan layer of the workflow. A new
`LogicalPlan` class is created to replace the old `WorkflowInfo` class.

The `LogicalPlan` class:
  - represents a DAG of logical operators (OperatorDescriptors)
  - provide helper functions on the DAG
- provide schema propagation on the logical DAG (schema propagation code
is moved from `WorkflowCompiler` to `LogicalPlan`
  
Furthermore, the disabled test cases in #1795 are added back in this PR.
yangzhang75 pushed a commit to yangzhang75/texera that referenced this pull request Jun 22, 2026
…st in logical plan (apache#1795)

This PR is a small refactor to replace `mutable.MutableList` with the corresponding immutable `List` in scala. This refactor helps the engine to use immutable data structures more often, which is considered a better practice in scala.

Some minor failed tests in `WorkflowRewriterSpec` are disabled temporarily. These test cases involves minor and unimportant cases in workflow cache feature. They will be fixed and enabled again once the whole refactor is completed.
yangzhang75 pushed a commit to yangzhang75/texera that referenced this pull request Jun 22, 2026
This PR refactors the logical plan layer of the workflow. A new
`LogicalPlan` class is created to replace the old `WorkflowInfo` class.

The `LogicalPlan` class:
  - represents a DAG of logical operators (OperatorDescriptors)
  - provide helper functions on the DAG
- provide schema propagation on the logical DAG (schema propagation code
is moved from `WorkflowCompiler` to `LogicalPlan`
  
Furthermore, the disabled test cases in apache#1795 are added back in this PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants