Skip to content

Commit 750c529

Browse files
authored
feat: accept source code or path instead of ts-morph SourceFile (#3)
1 parent 0c6824a commit 750c529

File tree

8 files changed

+500
-129
lines changed

8 files changed

+500
-129
lines changed

ARCHITECTURE.md

Lines changed: 65 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,23 @@
11
# TypeBox Code Generation Documentation
22

3-
# Table of Contents
4-
53
- [Overview](#overview)
64
- [Core Component](#core-component)
75
- [Function Flow](#function-flow)
86
- [Import Resolution and Dependency Management](#import-resolution-and-dependency-management)
97
- [DependencyCollector](#dependencycollector)
108
- [Key Features](#key-features)
119
- [Implementation Details](#implementation-details)
12-
- [TSConfig Support](#tsconfig-support)
13-
- [TSConfig Overview](#tsconfig-overview)
14-
- [Usage Examples](#usage-examples)
10+
- [Input Handling System](#input-handling-system)
11+
- [InputOptions Interface](#inputoptions-interface)
12+
- [Input Processing Features](#input-processing-features)
13+
- [Usage Patterns](#usage-patterns)
14+
- [Basic Usage](#basic-usage)
15+
- [With Export Everything](#with-export-everything)
16+
- [Using File Path](#using-file-path)
1517
- [Utility Functions and Modules](#utility-functions-and-modules)
1618
- [Handlers Directory](#handlers-directory)
1719
- [Parsers Directory](#parsers-directory)
18-
- [Performance Considerations](#performance-considerations)
20+
- [Performance Considerations](#performance-considerations)
1921
- [Performance Optimizations](#performance-optimizations)
2022
- [TypeBox Type Handler Optimization](#typebox-type-handler-optimization)
2123
- [Parser Instance Reuse](#parser-instance-reuse)
@@ -26,14 +28,10 @@
2628
- [TypeReferenceExtractor Optimizations](#typereferenceextractor-optimizations)
2729
- [TypeBoxTypeHandlers Optimizations](#typeboxtypehandlers-optimizations)
2830
- [Performance Testing](#performance-testing)
29-
- [Test Categories](#test-categories)
3031
- [Process Overview](#process-overview)
3132
- [Test-Driven Development (TDD) Approach](#test-driven-development-tdd-approach)
3233
- [TDD Cycle](#tdd-cycle)
3334
- [Running Tests](#running-tests)
34-
- [Running All Tests](#running-all-tests)
35-
- [Running Specific Test Files](#running-specific-test-files)
36-
- [Running Tests by Pattern](#running-tests-by-pattern)
3735
- [TDD Workflow for New Features](#tdd-workflow-for-new-features)
3836
- [Test Organization](#test-organization)
3937
- [Best Practices](#best-practices)
@@ -47,25 +45,27 @@ The primary goal of this codebase is to automate the creation of TypeBox schemas
4745

4846
## Core Component
4947

50-
The main logic for code generation resides in the <mcfile name="ts-morph-codegen.ts" path="src/ts-morph-codegen.ts"></mcfile> file. Its primary function, `generateCode`, takes a `SourceFile` object (representing a TypeScript file) as input and returns a string containing the generated TypeBox code.
48+
The main logic for code generation resides in the <mcfile name="ts-morph-codegen.ts" path="src/ts-morph-codegen.ts"></mcfile> file. Its primary function, `generateCode`, takes a `GenerateCodeOptions` object as input and returns a string containing the generated TypeBox code. The input can be either a file path or source code string, with support for relative imports when using existing project contexts.
5149

5250
### Function Flow
5351

54-
1. **Initialization**: A new in-memory `SourceFile` (`temp.ts`) is created to build the generated code. Essential imports for TypeBox (`Type`, `Static`) are added.
52+
1. **Input Processing**: The <mcfile name="input-handler.ts" path="src/input-handler.ts"></mcfile> module processes the input options to create a `SourceFile` object. This supports both file paths and source code strings, with proper validation for relative imports and path resolution.
53+
54+
2. **Initialization**: A new in-memory `SourceFile` (`output.ts`) is created to build the generated code. Essential imports for TypeBox (`Type`, `Static`) are added as separate import declarations for better compatibility.
5555

56-
2. **Parser Instantiation**: Instances of `ImportParser`, `EnumParser`, `TypeAliasParser`, and `FunctionDeclarationParser` are created, each responsible for handling specific types of declarations.
56+
3. **Parser Instantiation**: Instances of `ImportParser`, `EnumParser`, `TypeAliasParser`, and `FunctionDeclarationParser` are created, each responsible for handling specific types of declarations.
5757

58-
3. **Import Processing**: The `ImportParser` is instantiated and processes all import declarations in the input `sourceFile` to resolve imported types from external files. This includes locating corresponding source files for relative module specifiers and processing type aliases from imported files.
58+
4. **Import Processing**: The `ImportParser` is instantiated and processes all import declarations in the input `sourceFile` to resolve imported types from external files. This includes locating corresponding source files for relative module specifiers and processing type aliases from imported files.
5959

60-
4. **Enum Processing**: The `EnumParser` is instantiated and iterates through all `enum` declarations in the input `sourceFile`. For each enum, its original declaration is copied, a TypeBox `Type.Enum` schema is generated, and a corresponding static type alias is added.
60+
5. **Enum Processing**: The `EnumParser` is instantiated and iterates through all `enum` declarations in the input `sourceFile`. For each enum, its original declaration is copied, a TypeBox `Type.Enum` schema is generated, and a corresponding static type alias is added.
6161

62-
5. **Type Alias Processing**: The `TypeAliasParser` is instantiated and iterates through all `type alias` declarations in the input `sourceFile`. For each type alias, its underlying type node is converted into a TypeBox-compatible type representation, a TypeBox schema is generated, and a corresponding static type alias is added.
62+
6. **Type Alias Processing**: The `TypeAliasParser` is instantiated and iterates through all `type alias` declarations in the input `sourceFile`. For each type alias, its underlying type node is converted into a TypeBox-compatible type representation, a TypeBox schema is generated, and a corresponding static type alias is added.
6363

64-
6. **Interface Processing**: The `InterfaceParser` is instantiated and iterates through all `interface` declarations in the input `sourceFile`. For each interface, its properties and methods are converted into TypeBox object schemas with corresponding static type aliases.
64+
7. **Interface Processing**: The `InterfaceParser` is instantiated and iterates through all `interface` declarations in the input `sourceFile`. For each interface, its properties and methods are converted into TypeBox object schemas with corresponding static type aliases.
6565

66-
7. **Function Declaration Processing**: The `FunctionDeclarationParser` is instantiated and iterates through all function declarations in the input `sourceFile`. For each function, its parameters, optional parameters, and return type are converted into TypeBox function schemas with corresponding static type aliases.
66+
8. **Function Declaration Processing**: The `FunctionDeclarationParser` is instantiated and iterates through all function declarations in the input `sourceFile`. For each function, its parameters, optional parameters, and return type are converted into TypeBox function schemas with corresponding static type aliases.
6767

68-
8. **Output**: Finally, the full text content of the newly generated `temp.ts` source file (which now contains all the TypeBox schemas and static types) is returned as a string.
68+
9. **Output**: Finally, the full text content of the newly generated `output.ts` source file (which now contains all the TypeBox schemas and static types) is returned as a string.
6969

7070
### Import Resolution and Dependency Management
7171

@@ -103,25 +103,61 @@ The import resolution process works in two phases:
103103

104104
This approach ensures that complex import scenarios work correctly and generated code compiles without dependency errors.
105105

106-
## TSConfig Support
106+
## Input Handling System
107+
108+
The <mcfile name="input-handler.ts" path="src/input-handler.ts"></mcfile> module provides flexible input processing capabilities for the code generation system. It supports multiple input methods and handles various edge cases related to file resolution and import validation.
109+
110+
### InputOptions Interface
111+
112+
The `InputOptions` interface defines the available input parameters:
113+
114+
```typescript
115+
export interface InputOptions {
116+
filePath?: string // Path to TypeScript file
117+
sourceCode?: string // TypeScript source code as string
118+
callerFile?: string // Context file path for relative import resolution
119+
project?: Project // Existing ts-morph Project instance
120+
}
121+
```
122+
123+
### Input Processing Features
107124

108-
### TSConfig Overview
125+
1. **Dual Input Support**: Accepts either file paths or source code strings
126+
2. **Path Resolution**: Handles both absolute and relative file paths with proper validation
127+
3. **Relative Import Validation**: Prevents relative imports in string-based source code unless a `callerFile` context is provided
128+
4. **Project Context Sharing**: Supports passing existing `ts-morph` Project instances to maintain import resolution context
129+
5. **Error Handling**: Provides clear error messages for invalid inputs and unresolvable paths
109130

110-
The TypeBox code generation system includes automatic support for TypeScript configuration files (tsconfig.json). The system automatically detects and parses the closest tsconfig.json file using `tsconfck.parseNative`, ensuring that generated code respects project-specific TypeScript compiler options, particularly the `verbatimModuleSyntax` setting, which affects how import statements are generated.
131+
### Usage Patterns
111132

112-
### Usage Examples
133+
- **File Path Input**: Automatically resolves and loads TypeScript files from disk
134+
- **Source Code Input**: Processes TypeScript code directly from strings with validation
135+
- **Project Context**: Enables proper relative import resolution when working with in-memory source files
113136

114-
#### Basic Usage
137+
## Basic Usage
115138

116139
```typescript
117-
const result = generateCode(sourceFile)
140+
const result = await generateCode({
141+
sourceCode: sourceFile.getFullText(),
142+
callerFile: sourceFile.getFilePath(),
143+
})
118144
```
119145

120-
#### With Export Everything
146+
### With Export Everything
121147

122148
```typescript
123-
const result = generateCode(sourceFile, {
149+
const result = await generateCode({
150+
sourceCode: sourceFile.getFullText(),
124151
exportEverything: true,
152+
callerFile: sourceFile.getFilePath(),
153+
})
154+
```
155+
156+
### Using File Path
157+
158+
```typescript
159+
const result = await generateCode({
160+
filePath: './types.ts',
125161
})
126162
```
127163

@@ -237,31 +273,7 @@ The optimizations maintain full backward compatibility and test reliability whil
237273

238274
### Performance Testing
239275

240-
To ensure the dependency collection system performs efficiently under various scenarios, comprehensive performance tests have been implemented in <mcfile name="dependency-collector.performance.test.ts" path="tests/ts-morph/dependency-collector.performance.test.ts"></mcfile>. These tests specifically target potential bottlenecks in dependency collection and import processing:
241-
242-
#### Test Categories
243-
244-
1. **Large Dependency Chains**:
245-
- **Deep Import Chains**: Tests performance with 50+ levels of nested imports to verify the system handles deep dependency trees efficiently
246-
- **Wide Import Trees**: Tests scenarios with 100+ parallel imports to ensure the system scales well with broad dependency graphs
247-
248-
2. **Cache Efficiency**:
249-
- **Complex Type Structures**: Validates caching performance with intricate type definitions involving unions, intersections, and nested objects
250-
- **Large Cache Operations**: Tests the system's ability to handle substantial cache sizes without performance degradation
251-
252-
3. **Repeated File Processing**:
253-
- **Diamond Dependency Patterns**: Tests scenarios where multiple import paths converge on the same files, ensuring efficient deduplication
254-
- **Complex Topological Sort**: Validates performance of dependency ordering algorithms with interconnected type relationships
255-
256-
4. **Memory Usage Patterns**:
257-
- **Large Type Definitions**: Tests processing of substantial type definitions to ensure memory efficiency
258-
- **Dependency Map Operations**: Validates performance of core dependency tracking data structures
259-
260-
These performance tests provide baseline measurements and help identify potential bottlenecks before they impact production usage. The tests are designed to complete within reasonable timeframes while exercising the system under stress conditions that could reveal performance issues not apparent in standard unit tests.
261-
262-
- **`ts-morph`**: The `ts-morph` library is heavily utilized for parsing, traversing, and manipulating the TypeScript Abstract Syntax Tree (AST). It provides a programmatic way to interact with TypeScript code.
263-
264-
- **`@sinclair/typebox`**: This is the target library for schema generation. It provides a powerful and performant way to define JSON schemas with TypeScript type inference.
276+
To ensure the dependency collection system performs efficiently under various scenarios, comprehensive performance tests have been implemented in <mcfile name="dependency-collector.performance.test.ts" path="tests/ts-morph/dependency-collector.performance.test.ts"></mcfile>. These tests specifically target potential bottlenecks in dependency collection and import processing.
265277

266278
## Process Overview
267279

@@ -286,33 +298,15 @@ This project follows a Test-Driven Development methodology to ensure code qualit
286298

287299
The project uses Bun as the test runner. Here are the key commands for running tests:
288300

289-
#### Running All Tests
290-
291301
```bash
302+
# Run all tests
292303
bun test
293-
```
294304

295-
#### Running Specific Test Files
296-
297-
```bash
298305
# Run a specific test file
299306
bun test tests/ts-morph/function-types.test.ts
300307

301308
# Run tests in a specific directory
302309
bun test tests/ts-morph/
303-
304-
# Run integration tests
305-
bun test tests/integration/
306-
```
307-
308-
#### Running Tests by Pattern
309-
310-
```bash
311-
# Run tests matching a pattern
312-
bun test --grep "function types"
313-
314-
# Run tests for specific handlers
315-
bun test tests/ts-morph/advanced-types.test.ts
316310
```
317311

318312
### TDD Workflow for New Features

bun.lock

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,6 @@
33
"workspaces": {
44
"": {
55
"name": "new-bun-project",
6-
"dependencies": {
7-
"tsconfck": "^3.1.6",
8-
},
96
"devDependencies": {
107
"@eslint/js": "^9.33.0",
118
"@prettier/sync": "^0.6.1",
@@ -290,8 +287,6 @@
290287

291288
"ts-morph": ["ts-morph@26.0.0", "", { "dependencies": { "@ts-morph/common": "~0.27.0", "code-block-writer": "^13.0.3" } }, "sha512-ztMO++owQnz8c/gIENcM9XfCEzgoGphTv+nKpYNM1bgsdOVC/jRZuEBf6N+mLLDNg68Kl+GgUZfOySaRiG1/Ug=="],
292289

293-
"tsconfck": ["tsconfck@3.1.6", "", { "peerDependencies": { "typescript": "^5.0.0" }, "optionalPeers": ["typescript"], "bin": { "tsconfck": "bin/tsconfck.js" } }, "sha512-ks6Vjr/jEw0P1gmOVwutM3B7fWxoWBL2KRDb1JfqGVawBmO5UsvmWOQFGHBPl5yxYz4eERr19E6L7NMv+Fej4w=="],
294-
295290
"type-check": ["type-check@0.4.0", "", { "dependencies": { "prelude-ls": "^1.2.1" } }, "sha512-XleUoc9uwGXqjWwXaUTZAmzMcFZ5858QA2vvx1Ur5xIcixXIP+8LnFDgRplU30us6teqdlskFfu+ae4K79Ooew=="],
296291

297292
"typescript": ["typescript@5.9.2", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-CWBzXQrc/qOkhidw1OzBTQuYRbfyxDXJMVJ1XNwUHGROVmuaeiEm3OslpZ1RV96d7SKKjZKrSJu3+t/xlw3R9A=="],

package.json

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,12 +27,9 @@
2727
"description": "Codegen for validation schemas",
2828
"private": true,
2929
"scripts": {
30-
"format": "prettier --write .",
30+
"format": "prettier --cache --write .",
3131
"typecheck": "tsc --noEmit",
3232
"lint": "eslint"
3333
},
34-
"type": "module",
35-
"dependencies": {
36-
"tsconfck": "^3.1.6"
37-
}
34+
"type": "module"
3835
}

src/input-handler.ts

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
import { existsSync, statSync } from 'fs'
2+
import { dirname, isAbsolute, resolve } from 'path'
3+
import { Project, SourceFile } from 'ts-morph'
4+
5+
export interface InputOptions {
6+
filePath?: string
7+
sourceCode?: string
8+
callerFile?: string
9+
project?: Project
10+
}
11+
12+
const hasRelativeImports = (sourceFile: SourceFile): boolean => {
13+
const hasRelativeImports = sourceFile
14+
.getImportDeclarations()
15+
.some((importDeclaration) => importDeclaration.isModuleSpecifierRelative())
16+
17+
return hasRelativeImports
18+
}
19+
20+
const resolveFilePath = (input: string, callerFile?: string): string => {
21+
if (isAbsolute(input)) {
22+
if (!existsSync(input)) {
23+
throw new Error(`Absolute path does not exist: ${input}`)
24+
}
25+
return input
26+
}
27+
28+
const possiblePaths: string[] = []
29+
30+
if (callerFile) {
31+
const callerDir = dirname(callerFile)
32+
possiblePaths.push(resolve(callerDir, input))
33+
}
34+
35+
possiblePaths.push(resolve(process.cwd(), input))
36+
37+
const existingPaths = Array.from(new Set(possiblePaths)).filter((path) => existsSync(path))
38+
39+
if (existingPaths.length === 0) {
40+
throw new Error(`Could not resolve path: ${input}. Tried: ${possiblePaths.join(', ')}`)
41+
}
42+
43+
if (existingPaths.length > 1) {
44+
throw new Error(
45+
`Multiple resolutions found for path: ${input}. '` +
46+
`Found: ${existingPaths.join(', ')}. ' +
47+
'Please provide a more specific path.`,
48+
)
49+
}
50+
51+
return existingPaths[0]!
52+
}
53+
54+
const validateInputOptions = (options: InputOptions): void => {
55+
const { filePath, sourceCode } = options
56+
57+
if (!filePath && !sourceCode) {
58+
throw new Error('Either filePath or sourceCode must be provided')
59+
}
60+
61+
if (filePath && sourceCode) {
62+
throw new Error('Only one of filePath or sourceCode can be provided, not both')
63+
}
64+
}
65+
66+
export const createSourceFileFromInput = (options: InputOptions): SourceFile => {
67+
validateInputOptions(options)
68+
69+
const project = options.project || new Project()
70+
const { filePath, sourceCode, callerFile } = options
71+
72+
if (sourceCode) {
73+
// If callerFile is provided, it means this code came from an existing SourceFile
74+
// and relative imports should be allowed
75+
76+
const sourceFile = project.createSourceFile('temp.ts', sourceCode)
77+
78+
if (!callerFile && hasRelativeImports(sourceFile)) {
79+
throw new Error(
80+
'Relative imports are not supported when providing code as string. ' +
81+
'Only package imports from node_modules are allowed. ' +
82+
'Relative imports will be implemented in the future.',
83+
)
84+
}
85+
86+
const virtualPath = callerFile ? resolve(dirname(callerFile), '__virtual__.ts') : 'temp.ts'
87+
88+
return project.createSourceFile(virtualPath, sourceCode, { overwrite: true })
89+
}
90+
91+
if (filePath) {
92+
const resolvedPath = resolveFilePath(filePath, callerFile)
93+
94+
if (!statSync(resolvedPath).isFile()) {
95+
throw new Error(`Path is not a file: ${resolvedPath}`)
96+
}
97+
98+
return project.addSourceFileAtPath(resolvedPath)
99+
}
100+
101+
throw new Error('Invalid input options')
102+
}

0 commit comments

Comments
 (0)