Notes on porting GNU Bash to OpenVMS

Welcome to the gnv-bash wiki!

Non-fork-related notes

• In OpenVMS, strcoll() is case-sensitive, like strcmp(), so the test case would define STRCOLL_BROKEN if we ran it. Looking at the code paths modified by this #ifdef, it would be better if we called strcoll() in those situations, then fell back to strcmp(), because then at least the collation will be locale-sensitive when it's supposed to be (even if case-sensitive, which we don't want). So the code path defined by STRCOLL_BROKEN is worse than using the imperfect strcoll().

Working around the lack of fork()

• I've removed the previous VMS "fake fork" hack, which saved/restored the bash state for subshells, rather than calling fork(). Instead, for compatibility and memory isolation, I'm going to vfork()/execve() the subshell and then make sure the environment is set up in the subshell in the same way as it would've been through the current method of doing all the setup work in execute_in_subshell() and then jumping back to main() via setjmp_nosigs (top_level);.
• make_child(cmd, flags) logic (no job control):
  • if (command) free (command);
  • start_pipeline() - sets already_making_children to 1 (tested in execute_cmd.c).
  • possibly call sync_buffered_stream(default_buffered_input);
  • if (interactive_shell) then block SIGTERM, saving the old mask, and call set_signal_handler (SIGTERM, SIG_DFL);
  • call fork(), then in parent, call set_signal_handler (SIGTERM, SIG_IGN); and sigprocmask to the old mask
  • if the parent got an error forking, throw to top level.
  • in the child, unset_bash_input (0);, CLRINTERRUPT;, restore_sigmask(), subshell_environment |= SUBSHELL_IGNTRAP;, default_tty_job_signals ();
  • in the parent, last_made_pid = pid;, if (async_p) last_asynchronous_pid = pid;, add_pid (pid, async_p);
  • return (pid);
• Tracing through main() to see how best to execve() the shell as a subshell:
  • void check_dev_tty() doesn't appear to do anything useful? It was added in bash 2.00, but I don't see what the purpose of it is, since it doesn't have any side effects or return any values. I've #ifdef'd it out for __VMS.
• Subshell implementation strategy:
  • create vfork_subshell.c/h, based on vms_fakefork.c/h, except that it implements functions and a struct to serialize the process state (global variables, local and exported shell variables, aliases, and functions) and then send it to the subshell by writing the flattened state to a pipe whose file descriptor is passed as a command-line argument to the subshell. When the subshell parses argv[] and sees --ssfd n, it reads the global state from file descriptor n and then executes the subshell logic (parsing and executing a string or file), before exiting with the result status (same as UNIX).
  • We can take advantage of bash's behavior of calling setjmp() to set a top-level fallback, to analyze and verify that the subshell can't "throw" any higher than the state that we pass into it to start the child code path.
  • I'm going to re-enable make_child() in nojobs.c but modify it for #ifdef VFORK_SUBSHELL (defined in config_vms.h but could be added as a configure option) to create a new pipe to communicate with the child bash, vfork(), call execve() in the child with --ssfd n as the arguments (subshell file descriptor to read from), then return in the parent to follow the parent code path as normal.
  • For the child code path, I'm going to add an #ifdef VFORK_SUBSHELL section as close to the top of main() as possible to look for the --ssfd option as argv[1] and the file descriptor to read the state from as argv[2]. From there, after calling setlocale() and any other initialization of the C runtime library not already handled, I'm going to jump to as few individual code paths as necessary to handle all of the cases where make_child() is called.

Calls to make_child()

execute_cmd.c

• execute_command_internal( COMMAND *command, int async, int pipe_in, int pipe_out, struct fd_bitmap *fds_to_close )

  • for the cm_subshell and want/force/require subshell cases, make_child() is called with either FORK_ASYNC or 0. The child code path collapses to execute_in_subshell(command, async, pipe_in, pipe_out, fds_to_close)

• execute_coproc( COMMAND *command, int pipe_in, int pipe_out, struct fd_bitmap *fds_to_close )

  • same deal: turns into a call to execute_in_subshell(command, 1, wpipe[0], rpipe[1], fds_to_close). In both of these cases, the child shouldn't have to close any fds if the fcntl() call to set FD_CLOEXEC before calling execve does its job.

• execute_null_command( REDIRECT *redirects, int pipe_in, int pipe_out, int async )

  • if async is set or fork is forced, the child will set interactive and subshell_environment appropriately, then call do_redirections(redirects, RX_ACTIVE) and exit with success or failure.

• execute_simple_command( SIMPLE_COM *simple_command, int pipe_in, int pipe_out, int async, struct fd_bitmap *fds_to_close )

  • This one's pretty complex. The child code path needs to drop into the logic of this function itself, close to the beginning, while the parent code path returns.

• execute_disk_command( WORD_LIST *words, REDIRECT *redirects, char *command_line, int pipe_in, int pipe_out, int async, struct fd_bitmap *fds_to_close, int cmdflags )

  • As with execute_simple_command, the forking code path basically reruns the same function in the child, while exiting in the parent.

subst.c

• process_substitute( char *string, int open_for_read_in_child )

  • For this, the child code path should be executed in the VMS parent process (after vfork()), to set up the redirections, and then the child shell inherits the environment of the parent and calls parse_and_execute(string, "process substitution", (SEVAL_NONINT|SEVAL_NOHIST)) then run_exit_trap() and returns the result as the exit status. Then the parent code path can run to restore the pipeline and record the child PID and the temporary mailbox fds (in place of named FIFOs or /dev/fd support). It may work better to call vfork() directly and not make_child().

• command_substitute( char *string, int quoted, int flags )

  • Same comments as for process_substitute: ultimately the child shell process needs to be able to call parse_and_execute() with the correct subshell context and variables, but my approach for a modified make_child() that only returns on the parent side doesn't work so well for these two uses. And make_child() can't return twice, like fork(), or not without a lot of unportable hackiness, so it's better to call vfork() directly as well as the "send" end of my modified make_child(). I'll put any new code in vfork_subshell.c.

Code paths for child bash process

To summarize the previous section, the child process will need to set up the global environment to appear as if it were forked from the parent (OpenVMS will handle setting up the fds and signal dispositions), and then call one of these five entrypoints:

• execute_in_subshell( COMMAND *command, async, pipe_in, pipe_out, fds_to_close)
• execute_null_command( REDIRECT *redirects, int pipe_in, int pipe_out, int async ) child path
• execute_simple_command( SIMPLE_COM *simple_command, ... ) child path
• execute_disk_command( WORD_LIST *words, REDIRECT *redirects, char *command_line, ... ) child path
• parse_and_execute( char *string, ... ) child path