Add PGO documentation section to crate configuration

[jatinkumarsingh]

Add section explaining Profile Guided Optimization can provide up to 25% performance improvements. Includes three-stage build process instructions and tips for effective PGO usage. References issue #9507.

Which issue does this PR close?

Closes #9561

Rationale for this change

Adds documentation for Profile Guided Optimization (PGO) as requested. PGO can provide up to 25% performance improvements for DataFusion workloads, and users need clear guidance on how to use it.

What changes are included in this PR?

• Added "Profile Guided Optimization (PGO)" section to docs/source/user-guide/crate-configuration.md
• Three-stage build process instructions (instrumentation, profiling, recompilation)
• Tips for effective PGO usage (representative workloads, multiple iterations, combining with other optimizations)
• Links to Rust compiler guide and issue Profile-Guided Optimization (PGO) benchmark results #9507

Are these changes tested?

Yes. Documentation changes are validated by the CI workflow which builds the docs and checks for errors. The markdown syntax is valid and follows existing patterns.

Are there any user-facing changes?

Yes. This adds documentation that will be published on the DataFusion website under "Crate Configuration" > "Optimizing Builds". Users will find guidance on using PGO to improve performance.

[jatinkumarsingh]

@alamb
sir i have raised the pr with the changes required
plzzz have a look on my pr
thnku

[2010YOUY01]

Thank you, this is great.

I'm not familiar with this topic, so I'm adding a question that I think would be helpful to address:

I remember some previous discussions where we decided not to use PGO for release binaries, but I can’t recall the reason. I think @alamb knows it.

[jatinkumarsingh]

@2010YOUY01
Sir, thank you.
Could you please go through the PR?
My merge is currently blocked and it will only open once a reviewer with write access approves it.

[alamb]

Thank you for these instructions @jatinkumarsingh 🙏

i tried them out locally for datafusion-cli:

RUSTFLAGS="-C profile-generate=/tmp/pgo-data" cargo build --release --bin datafusion-cli

then run clickbench

cd datafusion/benchmarks/data
ln -s hits_partitioned hits
../../target/release/datafusion-cli  -f ../queries/clickbench/queries/*.sql

Then recompile:

RUSTFLAGS="-C profile-use=/tmp/pgo-data" cargo build --release --bin datafusion-cli

ANd it worked well

[jatinkumarsingh]

@alamb
tnku so much sir
I have also raised a PR with the changes. Whenever you get a chance, please take a look and merge it

[alamb]

Thanks again @jatinkumarsingh and @2010YOUY01

[zamazan4ik]

I remember some previous discussions where we decided not to use PGO for release binaries, but I can’t recall the reason. I think @alamb knows it.

It's an interesting note! @alamb could you please give a bit more details about this one? I am curious why PGO is not used for Datafusion's release binaries too. Thank you!

[alamb]

It's an interesting note! @alamb could you please give a bit more details about this one? I am curious why PGO is not used for Datafusion's release binaries too. Thank you!

Well one reason is that it is not clear what workload we would use. We could tune it to ClickBench, for example, but I am not sure how well that matches people's actual workloads

I think in general PGO is used when you have a very specific workload you want to tune for

[zamazan4ik]

Yeah, that's what I was expecting. Thanks!