feat: mojo build backend

[sstadick]

This PR adds a pixi-build-mojo backend.

See https://github.com/sstadick/pixi-build-mojo-test as an example project.

It follows the patterns in the other backend impls, for the most part. It adds two more layers to the package.build.configuration with a bins key, which configures an output binary, and a pkg key, which configures an output .mojopkg output.

Since pixi is the recommended build tool for Mojo, this will hopefully make pixi a bit more ergonomic and beginner friendly for creating and using Mojo packages. Any suggestions for improving ergonomics in this vein are very welcome!

I'd recommend starting with the added docs section.

[sstadick]

Related test project: https://github.com/sstadick/pixi-build-mojo-test

@baszalmstra It's so close to working. When I try to build that test project with pixi build if fails to find the ExtraMojo dep, which should be installed from modular-community.

I have no idea why. I can see the installed package in the .pixi dir, but it doesn't end up in any of the paths exported in the .conda_build.sh. Any ideas what is going on there?

[sstadick]

Also note, this is kind of doing two things at once, it does build a conda package, but it's also dropping a binary or mojopkg in a ./target dir for the user. Testing out how well this works as the one-stop-shop build tool for mojo projects / figuring out what works and doesn't work.

[sstadick]

Specifically it seems like my dependencies other than the compiler don't get copied into the bld dir. I see them in the long host_env_placehold_... dir, but not in the bld dir which it seems like the CONDA_PREFIX gets set to.

I did also confirm the same issue with another mojopkg small_time just to sanity check it.

[baszalmstra]

When I check the recipe that is generated with your example and backend these are the dependencies that are installed:

requirements:
  build:
  - max
  host:
  - max
  - extramojo

So I would indeed assume that extramojo is only installed to host_env_placehold_....

@Hofer-Julian Would you happen to know what a proper mojo recipe should look like? I couldnt find an example.

[sstadick]

That was enough of a hint! Failure in reading on my part, I thought dependencies were build-dependencies. https://pixi.sh/latest/reference/pixi_manifest/#build-dependencies

Making extramojo a build-dependency solves it!

[sstadick]

Todo:

• Check how / where env vars are passed through, since it doesn't look like it's in the build scripts for cmake.
• Add more specific tests
• On the example / doc end, figure out what needs to be a host-dep vs a build-dep vs what the vanilla "dependencies" are
• Decide if copying the built artifacts out to a ./target dir is desired or not: decided not by default in favor of pixi install but keeping the option
• Do git refs work outside of "dependencies"
• Add nice defaults so that binary / lib name is the name of the package or something like that. or lib name defaults to the name of the folder defering for now, while it's new I'd rather explicit over implicit

[baszalmstra]

You should not need to copy the artifact out. It should be installable now. You can create an environment in your test example and simply add a dependency on the package with:

[dependencies]
greetings = { path = "." }

If all goes well, the binary should then be installed in the environment and be executable!

what do you mean with:

Do git refs work outside of "dependencies"

[sstadick]

Thanks @baszalmstra! pixi install works great for that! I'm now getting this warning when running it back to back:

WARN failed to determine modification time of input files: failed to parse glob expression. Assuming the package is out-of-date. It seems like I need to be setting globs on generated build but I'm not sure what/where?

Regarding git refs - I'd like to be able to depend on packages like extramojo via extramojo = { git = "..." }. I know that works in the [dependency] block. But does it work in the [package.build-dependency] block? I have an open issue around that regarding using git refs inside rattler-build but wasn't sure how it interacted with things here. prefix-dev/rattler-build#1722 This doesn't block anything here, cool if it works though.

Thanks for your help on this, this is coming together way faster than I thought it would!

[baszalmstra]

And yes pixi supports git refs in all dependency sections, host and build included. You will have to rebase on main to make sure you include this PR in your branch: #241

[sstadick]

I believe this is ready for review. Working example project for it: https://github.com/sstadick/pixi-build-mojo-test

[sstadick]

@baszalmstra I think I've addressed your initial comments and it's ready for a second look!

[hogepodge]

This work looks fantastic, many thanks for putting it together!

[baszalmstra]

One last comment but this look great IMO!

[sstadick]

Thanks @baszalmstra! Comments addressed I think 👍

[sstadick]

@baszalmstra - updated! I think that should fix CI.

[wolfv]

CI is running - let's see! :)

[wolfv]

A few typos and lints 😱

[sstadick]

@wolfv strong chance I got it all fixed! All the lefthook checks are passing now.

Running some of those pixi tasks must have installed it automatically for me, because it wasn't running before for commits or pushes, but now it is 🤷

[baszalmstra]

Very well done @sstadick ! Thank you!

[sstadick]

Thanks for all your help on this @baszalmstra !