Block exec until sync is complete - #134
Conversation
| const extensionVersion = require("../../package.json").version; | ||
|
|
||
| export type ConnectionState = "CONNECTED" | "CONNECTING" | "DISCONNECTED"; | ||
| export type SyncStatus = "IN_PROGRESS" | "WATCHING_FOR_CHANGES" | "INACTIVE"; |
There was a problem hiding this comment.
We already have a sync state in code synchroniser. We can extend that with more states.
There was a problem hiding this comment.
Its easier to have this state in connection manager, because this is a object that already being passed into databricks runtime (the vscode runner). To keep it in code synchroniser would require refactoring to have code synchroniser inside the connection manager or somewhere accessible to databricks runtime.
Can do this refactor if recommended
cc: @fjakobs
There was a problem hiding this comment.
I agree with @kartikgupta-db. This should become part of CodeSynchronizer.ts. It gets confusing otherwise.
There was a problem hiding this comment.
It is no longer referenced
| } else { | ||
| this.connection.syncStatus = "IN_PROGRESS"; | ||
| } |
There was a problem hiding this comment.
We will also need an emitter for state change from "watching for changes" -> "in progress". Something like onDidStartSync.
Alternatively, you can have a common emitter, onDidSyncStateChange that fires on every state transition.
There was a problem hiding this comment.
Added an emitter into connection manager. For now it only fires for "in_progress" -> "watching for changes" because that's the only event we care about. Easily modifiable if we need in the future
|
In Progress: separate out the parsing logic into a separate PR and write unit tests |
2b88854 to
16f4488
Compare
| const extensionVersion = require("../../package.json").version; | ||
|
|
||
| export type ConnectionState = "CONNECTED" | "CONNECTING" | "DISCONNECTED"; | ||
| export type SyncStatus = "IN_PROGRESS" | "WATCHING_FOR_CHANGES" | "INACTIVE"; |
There was a problem hiding this comment.
I agree with @kartikgupta-db. This should become part of CodeSynchronizer.ts. It gets confusing otherwise.
3 participants
This PR adds code to parse logs from bricks sync to compute the state of sync completeness. If sync is in progress (ie upload/delete requests are inflight to the workspace), we block execution in the databricks runner
Video for exec:
https://user-images.githubusercontent.com/88374338/197779331-4b560bcf-63b2-4239-a3fb-4ee7a0814b62.mov
Video for exec as workflow
Screen.Recording.2022-10-26.at.2.47.54.PM.mov