rustdoc: add fmt_json helper and use it to avoid allocations

[lolbinarycat]

No description provided.

[lolbinarycat]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 294b29d (294b29dff7b5b62164e5b2fef65ad76ae93c0323, parent: c91009892b4cd1776d4c9d85793166612195a03d)

[rust-timer]

Finished benchmarking commit (294b29d): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.6%	[1.6%, 1.6%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary 9.3%, secondary -2.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	9.3%	[9.3%, 9.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.5%	[-2.5%, -2.5%]	1
All ❌✅ (primary)	9.3%	[9.3%, 9.3%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

Results (secondary -0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	1
All ❌✅ (primary)	-	-	0

Bootstrap: 472.486s -> 473.739s (0.27%)
Artifact size: 388.72 MiB -> 388.72 MiB (-0.00%)

[lolbinarycat]

Wow that is an extremely noisy benchmark.... the supposed "regression" doesn't even appear to be in a doc-related test.

now i'm wondering if there's a way we can decrease the noise of the benchmark, as that would let us identify smaller regressions and improves.

[yotamofek]

Wow that is an extremely noisy benchmark.... the supposed "regression" doesn't even appear to be in a doc-related test.

now i'm wondering if there's a way we can decrease the noise of the benchmark, as that would let us identify smaller regressions and improves.

One thing that would probably be pretty good ROI is to allow scheduling perf runs for specific profiles, so i.e. would could benchmark only the "Doc" profile.
That would both reduce some noise for people working on rustdoc, and might also reduce waiting times.
It won't improve the thresholding though, which I think is a science in and of itself.

[yotamofek]

Another thing that might help is adding more test scenarios, i.e. adding crates on which running rustdoc is significantly more time-consuming. (and if we never enable the non-"Doc" profiles for those, they won't make the normal, all-profile perf runs take too much more time)