Skip to content

Commit 73bf7be

Browse files
authored
Change compiler ID generation logic to use Node.js import specifier (#899)
## Summary This PR changes how the SWC compiler generates IDs for workflows, steps, and classes. Instead of using raw file paths, IDs are now based on **Node.js module specifiers** when the file belongs to a package (either in `node_modules` or a workspace package). ## Motivation Previously, IDs were generated using file paths like `step//src/jobs/order.ts//fetchData`. This caused several issues: 1. **Package exports conditions**: When a package uses conditional exports (e.g., `"workflow"` vs `"default"` conditions in `package.json`), the same import specifier can resolve to different files. Using file paths meant IDs could differ based on which export condition was used. 2. **Cross-bundle consistency**: Classes serialized in one bundle couldn't be deserialized in another if the file paths differed. 3. **Version tracking**: No way to include package versions in IDs for cache invalidation. ## Changes ### New ID Format IDs now use the format `{type}//{modulePath}//{identifier}` where `modulePath` is either: - A **module specifier** like `point@0.0.1` or `@myorg/shared@1.2.3` for package files - A **relative path** prefixed with `./` like `./src/jobs/order` for local app files Examples: - `step//workflow@4.0.1-beta.50//fetch` (SDK step) - `step//./workflows/order//processOrder` (local step) - `class//point@0.0.1//Point` (package class) - `class//./src/models/User//User` (local class) ### New Module Specifier Resolution Added `packages/builders/src/module-specifier.ts` which: - Detects if a file is in `node_modules` or a workspace package - Finds the nearest `package.json` and extracts name/version - Returns the module specifier for the SWC plugin to use ### SWC Plugin Changes - Added `moduleSpecifier` option to plugin config - Updated `naming.rs` to support both module specifiers and relative paths - Added `get_module_path()` helper that uses specifier when available, falls back to `./filename` format ### Special Cases - **Builtin functions** (`__builtin_*`): Continue to use just the function name as the ID for stable, version-independent lookup from the workflow VM runtime. ## Testing - Updated all 125+ SWC plugin test fixtures to use new ID format - Added tests for module specifier resolution - Added tests for Windows path normalization in naming ## Breaking Changes This is technically a breaking change for any persisted workflow runs that reference the old ID format. However, since IDs are internal implementation details and not user-facing, this should not affect end users. ## Files Changed - `packages/builders/src/module-specifier.ts` - **NEW**: Module specifier resolution logic - `packages/builders/src/apply-swc-transform.ts` - Pass module specifier to SWC plugin - `packages/builders/src/base-builder.ts` - Use `getImportPath` for virtual entry imports - `packages/swc-plugin-workflow/transform/src/lib.rs` - Accept and use module specifier - `packages/swc-plugin-workflow/transform/src/naming.rs` - New ID formatting with module paths - `packages/swc-plugin-workflow/spec.md` - Updated documentation - `packages/core/e2e/e2e.test.ts` - Updated test assertions for new ID format
1 parent b895446 commit 73bf7be

145 files changed

Lines changed: 1157 additions & 581 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.changeset/rich-symbols-fold.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
"@workflow/swc-plugin": patch
3+
"@workflow/builders": patch
4+
"@workflow/rollup": patch
5+
"@workflow/next": patch
6+
---
7+
8+
Change compiler ID generation logic to use Node.js import specifiers
9+
10+
IDs for workflows, steps, and classes now use module specifiers:
11+
- Local files use `./path/to/file` format instead of `path/to/file.ext`
12+
- Package files use `packageName@version` format (e.g., `workflow@4.0.1`)
13+
14+
This enables stable IDs across different package.json export conditions.

‎packages/builders/src/apply-swc-transform.ts‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
import { createRequire } from 'node:module';
2-
import { dirname } from 'node:path';
2+
import { dirname, isAbsolute, join } from 'node:path';
33
import { fileURLToPath } from 'node:url';
44
import { transform } from '@swc/core';
55
import { getDecoratorOptionsForDirectory } from './config-helpers.js';
6+
import { resolveModuleSpecifier } from './module-specifier.js';
67

78
const require = createRequire(import.meta.url);
89

@@ -67,6 +68,16 @@ export async function applySwcTransform(
6768
filename.endsWith('.mts') ||
6869
filename.endsWith('.cts');
6970

71+
// Resolve module specifier for packages (node_modules or workspace packages)
72+
const projectRoot = process.cwd();
73+
const absoluteFilename = isAbsolute(filename)
74+
? filename
75+
: join(projectRoot, filename);
76+
const { moduleSpecifier } = resolveModuleSpecifier(
77+
absoluteFilename,
78+
projectRoot
79+
);
80+
7081
// Transform with SWC to support syntax esbuild doesn't
7182
const result = await transform(source, {
7283
filename,
@@ -88,7 +99,7 @@ export async function applySwcTransform(
8899
target: 'es2022',
89100
experimental: mode
90101
? {
91-
plugins: [[swcPluginPath, { mode }]],
102+
plugins: [[swcPluginPath, { mode, moduleSpecifier }]],
92103
}
93104
: undefined,
94105
transform: {

‎packages/builders/src/base-builder.ts‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import {
1313
type WorkflowManifest,
1414
} from './apply-swc-transform.js';
1515
import { createDiscoverEntriesPlugin } from './discover-entries-esbuild-plugin.js';
16+
import { getImportPath } from './module-specifier.js';
1617
import { createNodeModuleErrorPlugin } from './node-module-esbuild-plugin.js';
1718
import { createPseudoPackagePlugin } from './pseudo-package-esbuild-plugin.js';
1819
import { createSwcPlugin } from './swc-esbuild-plugin.js';
@@ -319,7 +320,20 @@ export abstract class BaseBuilder {
319320
});
320321

321322
// Helper to create import statement from file path
323+
// For workspace/node_modules packages, uses the package name so esbuild
324+
// will resolve through package.json exports with the appropriate conditions
322325
const createImport = (file: string) => {
326+
const { importPath, isPackage } = getImportPath(
327+
file,
328+
this.config.workingDir
329+
);
330+
331+
if (isPackage) {
332+
// Use package name - esbuild will resolve via package.json exports
333+
return `import '${importPath}';`;
334+
}
335+
336+
// Local app file - use relative path
323337
// Normalize both paths to forward slashes before calling relative()
324338
// This is critical on Windows where relative() can produce unexpected results with mixed path formats
325339
const normalizedWorkingDir = this.config.workingDir.replace(/\\/g, '/');
@@ -505,7 +519,21 @@ export abstract class BaseBuilder {
505519
await this.writeDebugFile(outfile, { workflowFiles, serdeOnlyFiles });
506520

507521
// Helper to create import statement from file path
522+
// For workspace/node_modules packages, uses the package name so esbuild
523+
// will resolve through package.json exports with conditions: ['workflow']
508524
const createImport = (file: string) => {
525+
const { importPath, isPackage } = getImportPath(
526+
file,
527+
this.config.workingDir
528+
);
529+
530+
if (isPackage) {
531+
// Use package name - esbuild will resolve via package.json exports
532+
// and apply the 'workflow' condition
533+
return `import '${importPath}';`;
534+
}
535+
536+
// Local app file - use relative path
509537
// Normalize both paths to forward slashes before calling relative()
510538
// This is critical on Windows where relative() can produce unexpected results with mixed path formats
511539
const normalizedWorkingDir = this.config.workingDir.replace(/\\/g, '/');

‎packages/builders/src/index.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,13 @@ export {
99
} from './config-helpers.js';
1010
export { STEP_QUEUE_TRIGGER, WORKFLOW_QUEUE_TRIGGER } from './constants.js';
1111
export { createDiscoverEntriesPlugin } from './discover-entries-esbuild-plugin.js';
12+
export {
13+
clearModuleSpecifierCache,
14+
getImportPath,
15+
type ImportPathResult,
16+
type ModuleSpecifierResult,
17+
resolveModuleSpecifier,
18+
} from './module-specifier.js';
1219
export { createNodeModuleErrorPlugin } from './node-module-esbuild-plugin.js';
1320
export {
1421
createPseudoPackagePlugin,
Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
import { existsSync, readFileSync } from 'node:fs';
2+
import { dirname, join, relative, resolve, sep } from 'node:path';
3+
4+
/**
5+
* Result of resolving a module specifier for a file.
6+
*/
7+
export interface ModuleSpecifierResult {
8+
/**
9+
* The module specifier to use for ID generation.
10+
* - For packages: "{name}@{version}" (e.g., "point@1.0.0", "@myorg/shared@2.0.0")
11+
* - For local files: undefined (plugin will use default "./relative/path" format)
12+
*/
13+
moduleSpecifier: string | undefined;
14+
}
15+
16+
/**
17+
* Cache for package.json lookups to avoid repeated filesystem reads.
18+
* Maps directory path to parsed package.json or null if not found.
19+
*/
20+
const packageJsonCache = new Map<
21+
string,
22+
{ name: string; version: string } | null
23+
>();
24+
25+
/**
26+
* Find and read the nearest package.json for a given file path.
27+
* Results are cached for performance.
28+
*/
29+
function findPackageJson(
30+
filePath: string
31+
): { name: string; version: string } | null {
32+
let dir = dirname(filePath);
33+
34+
while (dir !== dirname(dir)) {
35+
// Check cache first
36+
const cached = packageJsonCache.get(dir);
37+
if (cached !== undefined) {
38+
return cached;
39+
}
40+
41+
const packageJsonPath = join(dir, 'package.json');
42+
if (existsSync(packageJsonPath)) {
43+
try {
44+
const content = readFileSync(packageJsonPath, 'utf-8');
45+
const parsed = JSON.parse(content);
46+
if (parsed.name && parsed.version) {
47+
const result = { name: parsed.name, version: parsed.version };
48+
packageJsonCache.set(dir, result);
49+
return result;
50+
}
51+
} catch {
52+
// Invalid JSON or missing fields, continue searching
53+
}
54+
}
55+
56+
packageJsonCache.set(dir, null);
57+
dir = dirname(dir);
58+
}
59+
60+
return null;
61+
}
62+
63+
/**
64+
* Check if a file path is inside node_modules.
65+
*/
66+
function isInNodeModules(filePath: string): boolean {
67+
const normalizedPath = filePath.split(sep).join('/');
68+
return normalizedPath.includes('/node_modules/');
69+
}
70+
71+
/**
72+
* Check if a file path is inside a workspace package.
73+
* This is a heuristic - we check if the file is in a directory with a package.json
74+
* that has a "name" field, but is NOT in node_modules.
75+
*/
76+
function isWorkspacePackage(filePath: string, projectRoot: string): boolean {
77+
if (isInNodeModules(filePath)) {
78+
return false;
79+
}
80+
81+
const pkg = findPackageJson(filePath);
82+
if (!pkg) {
83+
return false;
84+
}
85+
86+
// Check if the package.json is not the root package.json
87+
// Use resolve() to normalize paths for cross-platform comparison
88+
const rootPkgPath = resolve(projectRoot, 'package.json');
89+
90+
// Walk up to find the package.json directory
91+
let dir = dirname(filePath);
92+
while (dir !== dirname(dir)) {
93+
const pkgPath = join(dir, 'package.json');
94+
if (existsSync(pkgPath)) {
95+
// If this is the root package.json, it's not a workspace package
96+
// Use resolve() to normalize both paths before comparison
97+
if (resolve(pkgPath) === rootPkgPath) {
98+
return false;
99+
}
100+
// Found a package.json that's not the root - it's a workspace package
101+
return true;
102+
}
103+
dir = dirname(dir);
104+
}
105+
106+
return false;
107+
}
108+
109+
/**
110+
* Resolve the module specifier for a file.
111+
*
112+
* @param filePath - Absolute path to the file being transformed
113+
* @param projectRoot - Absolute path to the project root (usually process.cwd())
114+
* @returns The module specifier result
115+
*
116+
* @example
117+
* // File in node_modules
118+
* resolveModuleSpecifier('/project/node_modules/point/dist/index.js', '/project')
119+
* // => { moduleSpecifier: 'point@1.0.0' }
120+
*
121+
* @example
122+
* // File in workspace package
123+
* resolveModuleSpecifier('/project/packages/shared/src/utils.ts', '/project')
124+
* // => { moduleSpecifier: '@myorg/shared@0.0.0' }
125+
*
126+
* @example
127+
* // Local app file
128+
* resolveModuleSpecifier('/project/src/workflows/order.ts', '/project')
129+
* // => { moduleSpecifier: undefined }
130+
*/
131+
export function resolveModuleSpecifier(
132+
filePath: string,
133+
projectRoot: string
134+
): ModuleSpecifierResult {
135+
// Check if file is in node_modules or a workspace package
136+
const inNodeModules = isInNodeModules(filePath);
137+
const inWorkspace =
138+
!inNodeModules && isWorkspacePackage(filePath, projectRoot);
139+
140+
if (!inNodeModules && !inWorkspace) {
141+
// Local app file - use default relative path format
142+
return { moduleSpecifier: undefined };
143+
}
144+
145+
// Find the package.json for this file
146+
const pkg = findPackageJson(filePath);
147+
if (!pkg) {
148+
// Couldn't find package.json - fall back to default
149+
return { moduleSpecifier: undefined };
150+
}
151+
152+
// Return the module specifier as "name@version"
153+
return {
154+
moduleSpecifier: `${pkg.name}@${pkg.version}`,
155+
};
156+
}
157+
158+
/**
159+
* Clear the package.json cache. Useful for testing or when package.json files may have changed.
160+
*/
161+
export function clearModuleSpecifierCache(): void {
162+
packageJsonCache.clear();
163+
}
164+
165+
/**
166+
* Result of resolving an import path for a file.
167+
*/
168+
export interface ImportPathResult {
169+
/**
170+
* The import path to use.
171+
* - For workspace packages: the package name (e.g., "@myorg/shared")
172+
* - For node_modules packages: the package name
173+
* - For local files: a relative path (e.g., "./src/workflows/order.ts")
174+
*/
175+
importPath: string;
176+
177+
/**
178+
* Whether this file is from a package (workspace or node_modules).
179+
* When true, the import should go through package resolution which respects export conditions.
180+
*/
181+
isPackage: boolean;
182+
}
183+
184+
/**
185+
* Get the import path to use for a file in a bundle's virtual entry.
186+
*
187+
* For workspace packages and node_modules packages, returns the package name
188+
* so that bundler resolution will respect package.json exports and conditions.
189+
*
190+
* For local app files, returns a relative path.
191+
*
192+
* @param filePath - Absolute path to the file
193+
* @param projectRoot - Absolute path to the project root
194+
* @returns The import path and whether it's a package
195+
*
196+
* @example
197+
* // Workspace package
198+
* getImportPath('/project/packages/shared/src/index.ts', '/project')
199+
* // => { importPath: '@myorg/shared', isPackage: true }
200+
*
201+
* @example
202+
* // Local app file
203+
* getImportPath('/project/src/workflows/order.ts', '/project')
204+
* // => { importPath: './src/workflows/order.ts', isPackage: false }
205+
*/
206+
export function getImportPath(
207+
filePath: string,
208+
projectRoot: string
209+
): ImportPathResult {
210+
// Check if file is in node_modules or a workspace package
211+
const inNodeModules = isInNodeModules(filePath);
212+
const inWorkspace =
213+
!inNodeModules && isWorkspacePackage(filePath, projectRoot);
214+
215+
if (inNodeModules || inWorkspace) {
216+
// Find the package.json for this file
217+
const pkg = findPackageJson(filePath);
218+
if (pkg) {
219+
return {
220+
importPath: pkg.name,
221+
isPackage: true,
222+
};
223+
}
224+
}
225+
226+
// Local app file - use relative path
227+
const normalizedProjectRoot = projectRoot.replace(/\\/g, '/');
228+
const normalizedFilePath = filePath.replace(/\\/g, '/');
229+
230+
let relativePath: string;
231+
if (normalizedFilePath.startsWith(normalizedProjectRoot + '/')) {
232+
relativePath = normalizedFilePath.substring(
233+
normalizedProjectRoot.length + 1
234+
);
235+
} else {
236+
// File is outside project root, use the full path segments after common ancestor
237+
relativePath = relative(projectRoot, filePath).replace(/\\/g, '/');
238+
}
239+
240+
// Ensure relative paths start with ./
241+
if (!relativePath.startsWith('.')) {
242+
relativePath = `./${relativePath}`;
243+
}
244+
245+
return {
246+
importPath: relativePath,
247+
isPackage: false,
248+
};
249+
}

‎packages/core/e2e/e2e.test.ts‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -163,13 +163,17 @@ describe('e2e', () => {
163163
input: [123],
164164
output: 133,
165165
});
166-
// In local vs. vercel backends, the workflow name is different, so we check for either,
167-
// since this test runs against both. Also different workbenches have different directory structures.
168-
expect(json.workflowName).toBeOneOf([
169-
`workflow//example/${workflow.workflowFile}//${workflow.workflowFn}`,
170-
`workflow//${workflow.workflowFile}//${workflow.workflowFn}`,
171-
`workflow//src/${workflow.workflowFile}//${workflow.workflowFn}`,
172-
]);
166+
// Workflow ID format: workflow//./{path-without-extension}//{functionName}
167+
// Different workbenches have different directory structures:
168+
// - workflows/ (standard)
169+
// - src/workflows/ (some frameworks)
170+
// - example/workflows/ (example app)
171+
const fileWithoutExt = workflow.workflowFile.replace(/\.tsx?$/, '');
172+
expect(json.workflowName).toMatch(
173+
new RegExp(
174+
`^workflow//\\./(?:src/|example/)?${fileWithoutExt}//${workflow.workflowFn}$`
175+
)
176+
);
173177
});
174178

175179
const isNext = process.env.APP_NAME?.includes('nextjs');

0 commit comments

Comments
 (0)