Skip to content

Use aux-directory as working directory when building PDF #1651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Use aux-directory as working directory when building PDF #1651

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 13 additions & 30 deletions docs/buildsystem.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,20 +109,26 @@ Available default builders are:

To learn, how to create custom builder plugins, refer to [Custom Builder](#custom-builder) section.

Breaking Changes of LaTeXTools v4.5.2:

1. All commands are executed with working directory set to `aux_directory`, which enables support for custom auxiliary directories in all builders and compilers, without the need to explicitly specify `--aux-directory` or `--output-directory` command line arguments.

2. Main TeX document's location is automatically prepended to `$TEXINPUTS`, `$BIBINPUTS` and `$BSTINPUTS`. Hence no special action is required for commands like `bibtex` or `bibtex8`, which don't support `--output-directory` arguments.

## Traditional Builder

The `traditional` builder is designed to work in most circumstances and for most setups. It supports all [builder features](features.md#default-builder) discussed elsewhere in the documentation, including multi-document support, the ability to set LaTeX flags via the [TeX Options](features.md#tex-options) settings, etc.

If available, [latexmk][] is used to generate the document. Otherwise [texify][] is used as fallback.

**Note:** [texify][] doesn't support features, such as specifying output directory, auxiliary directory, or jobname.

Default commands:

```
latexmk -cd -f -%E -interaction=nonstopmode -synctex=1
latexmk -f -%E -interaction=nonstopmode -synctex=1
```

**Note:** `-cd` argument is no longer required as of LaTeXTools v4.5.2

```
texify -b -p --engine=%E --tex-option="--synctex=1"
```
Expand Down Expand Up @@ -253,34 +259,10 @@ Commands are executed in the same path as `$file_path`, i.e. the folder containi

### Output and Auxiliary Directories

If [auxiliary or output directory settings](settings.md#output-directory-settings) are specified, script builder creates them before batch execution. They are provided by [variables](#variables) and need to be passed to relevant commands in `script_commands`.
If [auxiliary or output directory settings](settings.md#output-directory-settings) are specified, script builder creates them before batch execution. They are provided by [variables](#variables) for use in `script_commands`.

`pdflatex` and friends do not create (sub-)directories as needed. If `\include` is used (or anything attempts to `\@openout` a file in a subfolder), they must be created manually by script commands like `"mkdir $output_directory\chapters"` (Windows) or `"mkdir -p $output_directory/chapters"` (Linux/MacOS).

**BibTeX**

Unlike biber, bibtex (and bibtex8) does not support an output directory parameter.
The following workaround can be used to run BibTeX _inside_ the output / auxiliary directory while making the directory containing your main file available to the `BIBINPUTS` environment variable.

```json
{
"builder_settings": {
"linux": {
"script_commands": [
"cd $output_directory; BIBINPUTS=\"$file_path;$BIBINPUTS\" bibtex \"$file_base_name\"",
]
},
"windows": {
"script_commands": [
"cd $output_directory & set BIBINPUTS=\"$file_path:%BIBINPUTS%\" & bibtex \"$file_base_name\""
]
}
}
}
```

**Note:** If a custom style file is used in the same directory, a similar work-around for `BSTINPUTS` environment variable needs to be applied.

### Job Name

If jobname behaviour is used, `$jobname` is to be passed to relevant commands. In particular, a standard build cycle might look something like this:
Expand Down Expand Up @@ -434,9 +416,10 @@ class MySecondBuilder(PdfBuilder):
yield (["pdflatex", f"-synctex={self.sync_id}"], "running pdflatex...")

# By default commands are executed with current working directory (`cwd`)
# set to main tex document's location. To call commands with custom parameters
# set to specified aux_directory if it differs from tex document's location.
# To call commands with custom working directory or parameters,
# use `PdfBuilder.command()` method.
yield (self.command(["bibtex"], cwd=self.aux_directory_full), "running bibtex"...)
yield (self.command(["bibtex"], cwd="/my/custom/working/dir"), "running bibtex"...)

# Prevent aborting workflow if command(s) return with non-zero exit status
self.abort_on_error = False
Expand Down
2 changes: 1 addition & 1 deletion docs/settings.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ This section refers to setting that can be found in a platform-specific block fo

## Output Directory Settings

* `aux_directory` (`""`): specifies the auxiliary directory to store any auxiliary files generated during a LaTeX build. Note that the auxiliary directory option is only useful if you are using MiKTeX. Path can be specified using either an absolute path or a relative path. If `aux_directory` is set from the project file, a relative path will be interpreted as relative to the project file. If it is set in the settings file, it will be interpreted relative to the main tex file. In addition, the following special values are honored:
* `aux_directory` (`".aux"`): specifies the auxiliary directory to store any auxiliary files generated during a LaTeX build. Path can be specified absolute or relative to `tex_root` or, if `aux_directory` set in a project file, relative to project file's location. In addition, the following special values are honored:
* `<<temp>>`: uses a temporary directory in the system temp directory instead of a specified path; this directory will be unique to each main file, but does not persist across restarts.
* `<<cache>>`: uses the ST cache directory (or a suitable directory on ST2) to store the output files; unlike the `<<temp>>` option, this directory can persist across restarts.
* `<<project>>`: uses a sub-directory in the same folder as the main tex file with what should be a unique name; note, this is probably not all that useful and you're better off using one of the other two options or a named relative path
Expand Down
2 changes: 1 addition & 1 deletion latextools/make_pdf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ def worker(self, activity_indicator):

try:
ws = re.compile(r"\s+")
(errors, warnings, badboxes) = parse_tex_log(data, self.caller.builder.tex_dir)
(errors, warnings, badboxes) = parse_tex_log(data, self.caller.builder.aux_directory_full)
content = [""]
if errors:
content.append("Errors:")
Expand Down
41 changes: 9 additions & 32 deletions plugins/builder/basic_builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,6 @@ def commands(self) -> CommandGenerator:
latex = [engine, "-interaction=nonstopmode", "-shell-escape", "-synctex=1"]
biber = ["biber"]

if self.aux_directory:
# No supported engine supports --aux-directory, use --output-directory
# and move final documents later.
biber.append(f"--output-directory={self.aux_directory}")
latex.append(f"--output-directory={self.aux_directory}")

if self.job_name and self.job_name != self.base_name:
latex.append(f"--jobname={self.job_name}")

Expand All @@ -78,7 +72,7 @@ def commands(self) -> CommandGenerator:
# Create required directories and retry
while matches := FILE_WRITE_ERROR_REGEX.findall(self.out):
for path, _ in matches:
abspath = os.path.join(self.aux_directory_full or self.tex_dir, path)
abspath = os.path.join(self.aux_directory_full, path)
os.makedirs(abspath, exist_ok=True)
logger.debug(f"Created directory {abspath}")

Expand All @@ -105,10 +99,14 @@ def commands(self) -> CommandGenerator:

if run_bibtex:
if use_bibtex:
yield (
self.run_bibtex(bibtex),
f"running {bibtex or 'bibtex'}...",
)
# set-up bibtex cmd line
if bibtex is None:
bibtex = [self.builder_settings.get("bibtex", "bibtex")]
elif isinstance(bibtex, str):
bibtex = [bibtex]
bibtex.append(self.job_name)
yield (bibtex, f"running {bibtex[0]}...")

else:
yield (biber + [self.job_name], "running biber...")

Expand All @@ -122,24 +120,3 @@ def commands(self) -> CommandGenerator:
yield (latex, f"running {engine}...")

self.copy_assets_to_output()

def run_bibtex(self, cmd: CommandLine | None=None) -> Command:
# set-up bibtex cmd line
if cmd is None:
cmd = [self.builder_settings.get("bibtex", "bibtex")]
elif isinstance(cmd, str):
cmd = [cmd]
cmd.append(self.job_name)

# return default command line, if build output is tex_dir.
if not self.aux_directory:
return cmd

# to get bibtex to work with the output directory, we change the
# cwd to the output directory and add the main directory to
# BIBINPUTS and BSTINPUTS
env = self.env.copy()
# cwd is, at the point, the path to the main tex file
env["BIBINPUTS"] = self.tex_dir + os.pathsep + env.get("BIBINPUTS", "")
env["BSTINPUTS"] = self.tex_dir + os.pathsep + env.get("BSTINPUTS", "")
return self.command(cmd, cwd=self.aux_directory_full, env=env)
78 changes: 48 additions & 30 deletions plugins/builder/pdf_builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,8 @@ def __init__(
"""

self.display = output
self.tex_root = tex_root
self.tex_dir, self.tex_name = os.path.split(tex_root)
self.tex_root = os.path.normpath(tex_root)
self.tex_dir, self.tex_name = os.path.split(self.tex_root)
self.base_name, self.tex_ext = os.path.splitext(self.tex_name)
self.engine = engine
self.options = options
Expand All @@ -126,29 +126,35 @@ def __init__(
# relative to self.tex_dir, we use that instead of the absolute path
# note that the full path for both is available as
# self.output_directory_full and self.aux_directory_full
self.aux_directory_full = aux_directory
self.aux_directory = (
os.path.relpath(aux_directory, self.tex_dir)
if aux_directory and aux_directory.startswith(self.tex_dir)
else aux_directory
)
if self.aux_directory:
os.makedirs(self.aux_directory_full, exist_ok=True)
if aux_directory:
self.aux_directory_full = os.path.normpath(aux_directory)
try:
gitignore = os.path.join(self.aux_directory_full, ".gitignore")
with open(gitignore, "w+", encoding="utf-8") as fobj:
fobj.write("*\n")
except FileExistsError:
pass

self.output_directory_full = output_directory
self.output_directory = (
os.path.relpath(output_directory, self.tex_dir)
if output_directory and output_directory.startswith(self.tex_dir)
else output_directory
)
if self.output_directory:
os.makedirs(self.output_directory_full, exist_ok=True)
self.aux_directory = os.path.relpath(self.aux_directory_full, self.tex_dir)
except Exception:
self.aux_directory = self.aux_directory_full
if self.aux_directory:
os.makedirs(self.aux_directory_full, exist_ok=True)
try:
gitignore = os.path.join(self.aux_directory_full, ".gitignore")
with open(gitignore, "w+", encoding="utf-8") as fobj:
fobj.write("*\n")
except FileExistsError:
pass
else:
self.aux_directory_full = self.tex_dir
self.aux_directory = ""

if output_directory:
self.output_directory_full = os.path.normpath(output_directory)
try:
self.output_directory = os.path.relpath(self.output_directory_full, self.tex_dir)
except Exception:
self.output_directory = self.output_directory_full
if self.output_directory:
os.makedirs(self.output_directory_full, exist_ok=True)
else:
self.output_directory_full = self.tex_dir
self.output_directory = ""

self.display_log = self.builder_settings.get("display_log", False)
"""
Expand All @@ -157,9 +163,22 @@ def __init__(
Value of `builder_settings: { display_log: ... }` setting
"""

# ensure main TeX document's location is available in environment
# it enables builders such as latexmk or texify to run with different working directory.
for key in ("TEXINPUTS", "BIBINPUTS", "BSTINPUTS"):
env[key] = (
self.tex_dir + os.pathsep + env[key]
if key in env
else self.tex_dir
)

# finally expand variables in custom environment
self.env = {k: self.expandvars(v) for k, v in env.items()} if env else None

logger.debug("tex directory: %s", self.tex_dir)
logger.debug("aux directory: %s", self.aux_directory_full)
logger.debug("out directory: %s", self.output_directory_full)

def set_output(self, out: str) -> None:
"""
Save command output.
Expand Down Expand Up @@ -233,7 +252,7 @@ def command(
Process object representing invoked command.
"""
if cwd is None:
cwd = self.tex_dir
cwd = self.aux_directory_full

if env is None:
env = self.env
Expand Down Expand Up @@ -271,8 +290,8 @@ def expandvars(self, text: str, **custom_vars: str) -> str:
file_name=self.tex_name,
file_ext=self.tex_ext,
file_base_name=self.base_name,
output_directory=self.output_directory_full or self.tex_dir,
aux_directory=self.aux_directory_full or self.tex_dir,
output_directory=self.output_directory_full,
aux_directory=self.aux_directory_full,
jobname=self.job_name,
engine=self.engine,
**custom_vars
Expand All @@ -297,12 +316,11 @@ def copy_assets_to_output(self) -> None:
Original PDF file is kept in place for e.g. latexmk to be able to skip
build if nothing was changed.
"""
dst_dir = self.output_directory_full or self.tex_dir
if self.aux_directory and self.aux_directory_full != dst_dir:
if self.aux_directory_full != self.output_directory_full:
for ext in (".synctex.gz", ".pdf"):
asset_name = self.base_name + ext
src_file = os.path.join(self.aux_directory_full, asset_name)
dst_file = os.path.join(dst_dir, asset_name)
dst_file = os.path.join(self.output_directory_full, asset_name)

src_st = os.stat(src_file)

Expand Down
2 changes: 2 additions & 0 deletions plugins/builder/script_builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,5 @@ def commands(self) -> CommandGenerator:
raise ValueError(f"Invalid command type! '{cmd}' must be a 'str' or 'list'!")

yield (cmd, f"Running '{cmd}'...")

self.copy_assets_to_output()
16 changes: 10 additions & 6 deletions plugins/builder/traditional_builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@

DEFAULT_COMMAND_LATEXMK = [
"latexmk",
"-cd",
"-f",
"-%E",
"-interaction=nonstopmode",
Expand Down Expand Up @@ -81,7 +80,9 @@ def commands(self) -> CommandGenerator:
cmd[i] = c.replace("-%E", flag).replace("%E", engine)

if latexmk:
if self.aux_directory:
# if `-cd` is specified, `-output-directory` is required to prevent
# latexmk changing working directory to tex root. Note, that's slower.
if self.aux_directory and "-cd" in cmd:
# Don't use --aux-directory as the way latexmk moves
# final documents to a possibly defined --output-directory
# prevents files reloading in SumatraPDF or even fails
Expand All @@ -94,11 +95,14 @@ def commands(self) -> CommandGenerator:
cmd += map(lambda o: f"-latexoption={o}", self.options)

elif texify:
if self.job_name != self.base_name:
cmd.append(f'--job-name="{self.job_name}"')

cmd += map(lambda o: f'--tex-option="{o}"', self.options)

# texify wants the .tex extension; latexmk doesn't care either way
yield (cmd + [self.tex_name], f"running {cmd[0]}...")
# texify requires absolute path if aux-directory differs;
# latexmk doesn't care either way
yield (cmd + [self.tex_root], f"running {cmd[0]}...")

# Sync compiled documents with output directory.
if latexmk and self.aux_directory:
self.copy_assets_to_output()
self.copy_assets_to_output()