Add subcommand for bulk actions

[jsoriano]

In repositories with multiple packages, and in elastic-package development itself, it is usual to run actions on multiple packages. For this, the developer usually needs to craft some kind of shell pipeline.

It could be useful to introduce in elastic-package a subcommand that can discover packages under certain rules and run some other subcommand for the discovered packages. For example the following command would add a changelog to all the modified packages under the packages directory.

elastic-package foreach --modified --packages ./packages/* changelog add ...

Or to test elastic-package while development with a set of packages:

elastic-package foreach --packages ./test/packages/parallel/* test

In general, its usage would be like this:

elastic-package foreach [discovery flags] [other flags] [subcommand] [subcommand flags]

Discovery flags

The subcommand would contain a set of flags used to discover packages, that can be combined, for example:

• --modified to select only packages with modified files.
• --packages [glob] to select packages matching with a glob pattern.
• --auto to walk the repository looking for compatible manifest files.
• --constraint [constraint] to filter packages based on their constraints (like --constraint kibana.version=8.17.0).

Not all these flags would be needed in a first MVP, they could be added later, as well as other flags.

Technically, the foreach subcommand would call the specified subcommand for each one of the discovered packages, setting for the called subcommands the -C flag.

Other flags

There could be support for other flags that control execution of subcommands, for example --parallel 4 would allow the execution of four subcommands in parallel.

Available subcommands

In principle all subcommands would be supported, we would need to investigate what is the best implementation so the --help makes sense and avoid loops between subcommands.

Additionally, we could add some kind of query subcommand that could help in shell pipelines or just listing info from packages. For example the following to list the name of modified packages:

elastic-package foreach --modified query --path manifest.package.name

[jrmolin]

how would --modified work -- it would have to do some git-fu to figure out what has changed since ... when? i'd rather just take a list of package names, or a glob, or some filter for the package names. i like the idea of the constraint, too.

[jsoriano]

how would --modified work -- it would have to do some git-fu to figure out what has changed since ... when?

This would work on packages with changes in the working copy, though yeah, maybe this flag or a similar one could accept a changeset and the it would find packages modified since this change.

i'd rather just take a list of package names, or a glob, or some filter for the package names

Yep, something like this would be the idea of the --packages flag.