Skip to content

Commit b5f5437

Browse files
committed
Add entrypoin to generate plugin schema
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
1 parent c0c2a45 commit b5f5437

File tree

5 files changed

+373
-9
lines changed

5 files changed

+373
-9
lines changed

adrs/00-template.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# [short title of solved problem and solution]
2+
3+
- Authors: [who wrote the ADR]
4+
- Status: [draft | proposed | rejected | accepted | deprecated | … | superseded by [xxx](xxx.md)]
5+
- Deciders: [list everyone involved in the decision] <!-- optional - to be formalised -->
6+
- Date: [YYYY-MM-DD when the decision was last updated]
7+
- Tags: [space and/or comma separated list of tags]
8+
9+
Technical Story: [description | ticket/issue URL] <!-- optional -->
10+
11+
## Summary
12+
13+
Quick description of the problem and the context. Should not take more than 2-3 lines.
14+
15+
## Problem Statement
16+
17+
Description of the technical problem to solve or to decision to make. This should be concise but provide all
18+
required details and the context related to the technical decision to be taken.
19+
20+
## Goals or Decision Drivers
21+
22+
Depending the context define clearly what are the goals or what are the most important decision drivers.
23+
24+
- [driver 1, e.g., a force, facing concern, …]
25+
- [driver 2, e.g., a force, facing concern, …]
26+
-<!-- numbers of drivers can vary -->
27+
28+
## Non-goals
29+
30+
Define what's out of the scope of this ADR.
31+
32+
## Considered Options <!-- optional -->
33+
34+
- [option 1]
35+
- [option 2]
36+
- [option 3]
37+
-<!-- numbers of options can vary -->
38+
39+
40+
## Pros and Cons of the Options <!-- optional -->
41+
42+
### [option 1]
43+
44+
[example | description | pointer to more information | …] <!-- optional -->
45+
46+
- Good, because [argument a]
47+
- Good, because [argument b]
48+
- Bad, because [argument c]
49+
-<!-- numbers of pros and cons can vary -->
50+
51+
### [option 2]
52+
53+
[example | description | pointer to more information | …] <!-- optional -->
54+
55+
- Good, because [argument a]
56+
- Good, because [argument b]
57+
- Bad, because [argument c]
58+
-<!-- numbers of pros and cons can vary -->
59+
60+
61+
## Solution or decision outcome
62+
63+
Summarize the solution or decision outcome in one-two lines.
64+
65+
## Rationale & discussion
66+
67+
Describe the solution or the decision outcome discussing how decision drivers have been applied
68+
and how it matches the declared goals. This section is expected to be concise though providing comprehensive
69+
description of the technical solution and covering all uncertainty or ambiguous points.
70+
71+
## Links <!-- optional -->
72+
73+
- [Link type](link to adr) <!-- example: Refined by [xxx](yyyymmdd-xxx.md) -->
74+
-<!-- numbers of links can vary -->
75+
76+
## More information
77+
78+
- [What is an ADR and why should you use them](https://github.com/thomvaill/log4brains/tree/master#-what-is-an-adr-and-why-should-you-use-them)
79+
- [ADR GitHub organization](https://adr.github.io/)

adrs/01-plugin-schema.md

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# Plugin Schema
2+
3+
- Authors: Ben Sherman
4+
- Status: accepted
5+
- Deciders: Ben Sherman, Paolo Di Tommaso
6+
- Date: 2025-09-22
7+
- Tags: plugins
8+
9+
## Summary
10+
11+
Provide a way for external systems to understand key information about third-party plugins.
12+
13+
## Problem Statement
14+
15+
Nextflow plugins need a way to statically declare extensions to the Nextflow language so that external systems can extract information about a plugin without loading it in the JVM.
16+
17+
Primary use cases:
18+
19+
- The Nextflow language server needs to know about any config scopes, custom functions, etc, defined by a plugin, in order to recognize them in Nextflow scripts and config files
20+
21+
- The Nextflow plugin registry (or other user interfaces) can use this information to provide API documentation.
22+
23+
## Goals or Decision Drivers
24+
25+
- External systems (e.g. language server) need to be able to understand plugins without having to load them in the JVM.
26+
27+
## Non-goals
28+
29+
- Defining the schema for the core runtime and core plugins: these definitions are handled separately, although they may reuse some of the structures used for plugin schemas.
30+
31+
## Considered Options
32+
33+
### Nextflow plugin system
34+
35+
Require external systems to use Nextflow's plugin system to load plugins at runtime in order to extract information about them.
36+
37+
- **Pro:** Allows any information to be extracted since the entire plugin is loaded
38+
39+
- **Con:** Requires the entire Nextflow plugin system to be reused or reimplemented. Not ideal for Java applications since the plugin system is implemented in Groovy, incompatible with non-JVM applications
40+
41+
- **Con:** Requires plugins to be downloaded, cached, loaded in the JVM, even though there is no need to use the plugin.
42+
43+
### Plugin schema
44+
45+
Define a plugin schema for every plugin release which is stored and served by the plugin registry as JSON.
46+
47+
- **Pro:** Allows any system to access plugin definitions through a standard JSON format. Does not require downloading plugins or loading them into a JVM.
48+
49+
- **Con:** Requires the plugin schema to be generated at build-time and stored in the plugin registry.
50+
51+
- **Con:** Requires a standard JSON format across all versions of Nextflow, the language server, and third-party plugins.
52+
53+
## Solution
54+
55+
Define a plugin schema for every plugin release which is stored and served by the plugin registry as JSON.
56+
57+
- Plugin developers only need to define [extension points](https://nextflow.io/docs/latest/plugins/developing-plugins.html#extension-points) as usual, and the Gradle plugin will extract the plugin schema and store it in the plugin registry as part of each plugin release.
58+
59+
- The language server can retrieve plugin schemas based on the `plugins` block in a config file.
60+
61+
A plugin schema consists of a list of *definitions*. Each definition has a *type* and a *spec*.
62+
63+
Example:
64+
65+
```json
66+
[
67+
{
68+
"type": "ConfigScope",
69+
"spec": {
70+
// ...
71+
}
72+
},
73+
{
74+
"type": "Function",
75+
"spec": {
76+
// ...
77+
}
78+
},
79+
]
80+
```
81+
82+
The following types of definitions are allowed:
83+
84+
**ConfigScope**
85+
86+
Defines a top-level config scope. The spec consists of a *name*, an optional *description*, and *children*.
87+
88+
The children should be a mapping of names to schemas, where each name denotes a nested config scope or option. The following nested schemas are allowed:
89+
90+
- **ConfigOption**: Defines a config option. The spec consists of a *description* and *type*.
91+
92+
- **ConfigScope**: Defines a nested config scope, using the same schema as for top-level scopes.
93+
94+
Example:
95+
96+
```json
97+
{
98+
"type": "ConfigScope",
99+
"spec": {
100+
"name": "hello",
101+
"description": "The `hello` scope controls the behavior of the `nf-hello` plugin.",
102+
"children": {
103+
"message": {
104+
"type": "ConfigOption",
105+
"spec": {
106+
"description": "Message to print to standard output when the plugin is enabled.",
107+
"type": "String"
108+
}
109+
}
110+
}
111+
}
112+
}
113+
```
114+
115+
**Factory**
116+
117+
Defines a channel factory that can be included in Nextflow scripts. The spec is the same as for functions.
118+
119+
**Function**
120+
121+
Defines a function that can be included in Nextflow scripts. The spec consists of a *name*, an optional *description*, a *return type*, and a list of *parameters*. Each parameter consists of a *name* and a *type*.
122+
123+
Example:
124+
125+
```json
126+
{
127+
"type": "Function",
128+
"spec": {
129+
"name": "sayHello",
130+
"description": "Say hello to the given target",
131+
"returnType": "void",
132+
"parameters": [
133+
{
134+
"name": "target",
135+
"type": "String"
136+
}
137+
]
138+
}
139+
}
140+
```
141+
142+
**Operator**
143+
144+
Defines a channel operator that can be included in Nextflow scripts. The spec is the same as for functions.
145+
146+
## Rationale & discussion
147+
148+
Now that there is a Gradle plugin for building Nextflow plugins, as well as a registry to publish and retrieve plugins, the generation and retrieval of plugin schemas can be implemented in a way that is transparent to plugin developers.
149+
150+
The primary challenge is to ensure that the schema of the plugin schema (i.e. the meta-schema) remains consistent across Nextflow versions, or is versioned so that external systems can understand multiple versions over time as needed.

modules/nextflow/src/main/groovy/nextflow/plugin/schema/ConfigSchema.groovy

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -75,21 +75,15 @@ class ConfigSchema {
7575
type: 'ConfigScope',
7676
spec: [
7777
name: scopeName,
78-
description: withLink(scopeName, description),
78+
description: description,
7979
children: children
8080
]
8181
]
8282
}
8383

84-
private static String withLink(String scopeName, String description) {
85-
return scopeName
86-
? "$description\n\n[Read more](https://nextflow.io/docs/latest/reference/config.html#$scopeName)\n"
87-
: description
88-
}
89-
9084
private static Object fromType(ClassNode cn) {
9185
final name = Types.getName(cn.getTypeClass())
92-
if( cn.isUsingGenerics() ) {
86+
if( cn.getGenericsTypes() != null ) {
9387
final typeArguments = cn.getGenericsTypes().collect { gt -> fromType(gt.getType()) }
9488
return [ name: name, typeArguments: typeArguments ]
9589
}

modules/nextflow/src/main/groovy/nextflow/plugin/schema/FunctionSchema.groovy

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ class FunctionSchema {
2727

2828
static Map<String,?> of(Method method, String type) {
2929
final name = method.getName()
30+
final description = method.getAnnotation(Description)?.value()
3031
final returnType = fromType(method.getReturnType())
3132
final parameters = method.getParameters().collect { param ->
3233
[
@@ -39,6 +40,7 @@ class FunctionSchema {
3940
type: type,
4041
spec: [
4142
name: name,
43+
description: description,
4244
returnType: returnType,
4345
parameters: parameters
4446
]
@@ -51,7 +53,7 @@ class FunctionSchema {
5153

5254
private static Object fromType(ClassNode cn) {
5355
final name = Types.getName(cn.getTypeClass())
54-
if( cn.isUsingGenerics() ) {
56+
if( cn.getGenericsTypes() != null ) {
5557
final typeArguments = cn.getGenericsTypes().collect { gt -> fromType(gt.getType()) }
5658
return [ name: name, typeArguments: typeArguments ]
5759
}

0 commit comments

Comments
 (0)