Defaults value for multiple choices of single subcommand in the config file.

[ljw20180420]

I have a subcommand sub. sub can be either sub1 or sub2. The python script all_sub_cmd_with_defaults.py is

import jsonargparse

parser = jsonargparse.ArgumentParser(
    description="I wish to give all option of a required subcommands default values.",
    default_config_files=["all_sub_cmd_with_defaults.yaml"],
)
parser.add_argument("--val")
subs = parser.add_subcommands(required=True, dest="sub")

sub1 = jsonargparse.ArgumentParser()
sub1.add_argument("--sub1val")
subs.add_subcommand("sub1", sub1)

sub2 = jsonargparse.ArgumentParser()
sub2.add_argument("--sub2val")
subs.add_subcommand("sub2", sub2)

args = parser.parse_args()
print(args)

I put default values for both sub1 and sub2 in a single config file. The config file all_sub_cmd_with_defaults.yaml is

val: 0

sub: sub1

sub1:
  sub1val: 1

sub2:
  sub2val: 3

sub1 is the default choice of sub. When I run

python all_sub_cmd_with_defaults.py sub1

or just

python all_sub_cmd_with_defaults.py

it works well, the output is

Namespace(val=0, sub='sub1', sub1=Namespace(sub1val=1, __default_config__=Path_fr(all_sub_cmd_with_defaults.yaml, cwd=D:\z_serious_works\learn_jsonargparse)), __default_config__=Path_fr(all_sub_cmd_with_defaults.yaml, cwd=D:\z_serious_works\learn_jsonargparse))

However, when I run

python all_sub_cmd_with_defaults.py sub2

the default value of the option sub2val for sub2 is not parsed. The output is

Namespace(val=0, sub='sub2', __default_config__=Path_fr(all_sub_cmd_with_defaults.yaml, cwd=D:\z_serious_works\learn_jsonargparse), sub2=Namespace(sub2val=None, __default_config__=Path_fr(all_sub_cmd_with_defaults.yaml, cwd=D:\z_serious_works\learn_jsonargparse)))

So Is it possible to put the default values of all choices of a subcommand in the config file?

I also want to put the default settings in separate config file like those for class. I modify all_sub_cmd_with_defaults.yaml like this.

val: 0

sub: sub1

sub1: sub1.yaml

sub2:
  sub2val: 3

with sub1.yaml as

sub1val: 1

Unfortunately, it does not work. The output is

Traceback (most recent call last):
  File "D:\z_serious_works\learn_jsonargparse\all_sub_cmd_with_defaults.py", line 28, in <module>
    args = parser.parse_args()
           ^^^^^^^^^^^^^^^^^^^
  File "d:\z_serious_works\learn_jsonargparse\.conda\Lib\site-packages\jsonargparse\_deprecated.py", line 130, in patched_parse
    cfg = parse_method(*args, _skip_validation=_skip_validation, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "d:\z_serious_works\learn_jsonargparse\.conda\Lib\site-packages\jsonargparse\_core.py", line 458, in parse_args
    cfg, unk = self.parse_known_args(args=args, namespace=cfg)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "d:\z_serious_works\learn_jsonargparse\.conda\Lib\site-packages\jsonargparse\_core.py", line 309, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "d:\z_serious_works\learn_jsonargparse\.conda\Lib\argparse.py", line 2131, in _parse_known_args
    stop_index = consume_positionals(start_index)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "d:\z_serious_works\learn_jsonargparse\.conda\Lib\argparse.py", line 2087, in consume_positionals
    take_action(action, args)
  File "d:\z_serious_works\learn_jsonargparse\.conda\Lib\argparse.py", line 1983, in take_action
    action(self, namespace, argument_values, option_string)
  File "d:\z_serious_works\learn_jsonargparse\.conda\Lib\site-packages\jsonargparse\_actions.py", line 672, in __call__
    subnamespace = namespace.get(subcommand).clone() if subcommand in namespace else None
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'clone'

If all_sub_cmd_with_defaults.yaml does not specify sub explicitly to sub1 or sub2, i.e.

val: 0

sub1:
  sub1val: 1

sub2:
  sub2val: 3

then even if I specify sub to sub1 explicitly in the command line as

python all_sub_cmd_with_defaults.py sub1

there will be a warning

d:\z_serious_works\learn_jsonargparse\.conda\Lib\site-packages\jsonargparse\_actions.py:716: UserWarning: Multiple subcommand settings provided (sub1, sub2) without an explicit "sub" key. Subcommand "sub1" will be used.
  warnings.warn(
Namespace(val=0, sub='sub1', sub1=Namespace(sub1val=1, __default_config__=Path_fr(all_sub_cmd_with_defaults.yaml, cwd=D:\z_serious_works\learn_jsonargparse)), __default_config__=Path_fr(all_sub_cmd_with_defaults.yaml, cwd=D:\z_serious_works\learn_jsonargparse))

Is it possible to force the user to specify subcommands in the command line, but keep the settings of the subcommands in the config file? More specifically, if the user does not specify the subcommand in the command line, there will be an error, otherwise, if the user specify the subcomman, there is no warning.

[mauvilsa]

Thank you for reporting!

Having the settings of multiple subcommands in a config file is a feature. However, that feature was implemented for config files provided through command line, not in default_config_files. If in the code above you remove the default_config_files part and add parser.add_argument("--config", action="config"), the following commands work correctly:

python all_sub_cmd_with_defaults.py --config all_sub_cmd_with_defaults.yaml --print_config sub1
python all_sub_cmd_with_defaults.py --config all_sub_cmd_with_defaults.yaml --print_config sub2

I hadn't even thought about possible cases for default_config_files and subcommands, which explains why the behavior is weird. Thus, some parts of this issue would be a feature request and others bugs. From first thoughts I would say:

• Providing a subcommand in a default_config_files should fail, since subcommands should be given by the user running the command, not have a default.
• It makes sense to also support settings for multiple subcommands given in default_config_files.
• The "without an explicit sub" warning and working only for sub1 is unintended behavior caused by the support of this for configs given as an argument.
• Support for a subcommand in a separate config file is a valid feature request.

I can't tell how much effort this is, but most likely it is a big task.