MSVC support

[OwenTrokeBillard]

Thank you for all your excellent work. Rust has a bright future with Enzyme for automatic differentiation.

Does Enzyme for Rust support Windows, or is Windows support planned?

[ZuseZ4]

Thanks for the nice words!
LLVM and Rust both support windows, so there is no technical reason on why Enzyme (or Rust-Emzyme)
would not support windows, except of possibly smaller issues like us having messed up some path handling,
did you encounter such bugs?
If yes please feel free to submit smaller bugfixes if you have the capacity, or just create issues.
I don't have a windows machine at hand, but once we got rust-enzyme cleaned up a bit more I will ask a friend or use the Rust CI to find possible windows bugs.

[OwenTrokeBillard]

I tried for several hours to build Rust with Enzyme on Windows. There are some linker errors that are beyond my comprehension. I am likely doing something incorrectly.

Errors

...
LINK : warning LNK4044: unrecognized option '/-gc-sections'; ignored
LLVMTableGenGlobalISel.lib(GIMatchDag.cpp.obj) : error LNK2001: unresolved external symbol __imp__CrtDbgReport
LLVMTableGenGlobalISel.lib(GIMatchTree.cpp.obj) : error LNK2001: unresolved external symbol __imp__CrtDbgReport
LLVMTableGenGlobalISel.lib(CodeExpander.cpp.obj) : error LNK2001: unresolved external symbol __imp__CrtDbgReport
LLVMSupport.lib(JSON.cpp.obj) : error LNK2001: unresolved external symbol __imp__CrtDbgReport
(many more)
...

Steps to reproduce

1. Install Python, CMake, and Ninja.
2. Install Windows Subsystem for Linux (WSL). The configure script requires bash.
3. Install the "C++ Clang tools for Windows" and "Windows 10 SDK" components in Visual Studio Community 2022.
4. Run the configure script from within WSL.

   mkdir build
   cd build
   ../configure --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs
   

   • The flag "--enable-llvm-link-shared" is omitted since it is not supported on Windows.
5. In PowerShell, configure CMake to use the Clang compiler.

   $env:CC="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\x64\bin\clang.exe"
   $env:CXX="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\x64\bin\clang++.exe"
   $env:RC="C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64\rc.exe"
   

   • The paths may be different for you.
6. Configure Clang to use the MSVC linker.

   $env:CFLAGS="-fuse-ld=link"
   $env:CXXFLAGS="-fuse-ld=link"
   

7. Run the build script.

   & python.exe ../x.py build --stage 1 library/std library/proc_macro library/test tools/rustdoc
   

Alternatives to MSVC

On Windows, it sounds like Rust uses LLVM for compilation and MSVC for linking. I tried to configure it as such in the steps above.

LLVM appears to come with its own linker, "lld-link", that aims to be a drop-in replacement for the MSVC linker on Windows. It also comes with its own resource compiler, "llvm-rc". I tried these in place of the MSVC variants to no avail.

All of this is beyond my familiarity. @ZuseZ4 Do any of these steps strike you as obviously incorrect?

[ZuseZ4]

Thanks for the list of things you tried, it looks reasonable to me. I know that someone in the Enzyme core repo added a few cmake changes to support static Enzyme compilation, but I am not sure how rust and windows are supposed to work here.
Unfortunately I will be traveling for the next week, so I can't promise to directly have a look at it, but afterwards I can ask to see if any of the real rustc devs has some insight here,
since we do hope to upstream this soonish.

[OwenTrokeBillard]

@ZuseZ4 I hope you enjoy your travels! Thanks for your quick responses on this issue.

Please let me know if there is anything I can do to help in the future (e.g., testing). I am also open to calling one-on-one.

[OwenTrokeBillard]

@ZuseZ4 Do you know why LLVM is configured to be linked statically (--enable-llvm-link-shared)? This option is not supported on Windows. I was able to build on Linux just fine without this option on your macro2 branch.

[ZuseZ4]

Just to be sure, you meant to write windows, not linux in your last sentence, right? Lorenz and I tried a lot of different settings in our different proof of concept implementations, I would just assume that it fixed an outdated bug and isn't needed then. I'll test it on the windows of a friend, great if the fix is that easy, thanks!

[OwenTrokeBillard]

I did mean Linux. I built on Linux without --enable-llvm-link-shared to make sure that flag is not required. I am still working on the Windows build.

[wsmoses]

I presume the shared library is a holdover from when enzyme was linked in via shared library. If it works without it the rust build state should be put back to its default

…

On Sat, Jan 6, 2024 at 7:36 PM Owen Troke-Billard ***@***.***> wrote: I did mean Linux. I built on Linux without --enable-llvm-link-shared to make sure that flag is not required. I am still working on the Windows build. — Reply to this email directly, view it on GitHub <#54 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJTUXBJDI2R7IMO7X5SCN3YNHUYHAVCNFSM6AAAAAA5HF6XTSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZZHA4TIOJWHA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[OwenTrokeBillard]

Enzyme is not configured to be built with Clang + MSVC. It can be built with Clang + MinGW (see EnzymeAD/Enzyme#1449), and I believe this is how Enzyme.jl works on Windows.

I see a few options for proceeding:

1. Make Enzyme compatible with Clang + MSVC.
2. Force Windows users to install MinGW, and build Enzyme with Clang + MinGW.
3. Use a precompiled Enzyme library instead of building from scratch.
4. Do not support Windows.

I am trying to get Enzyme to build with Clang + MSVC. Please let me know if this approach is not possible or desirable.

As an aside, please ignore my claim about --enable-llvm-link-shared being redundant. I tried another build without it since and it failed.

[ZuseZ4]

Rust does support Windows via MSVC, right? I guess in this case 1) is the best path forward, so it would be great if you find a way to get this to work. @wsmoses Do you know any blockers from the llvm side?

[wsmoses]

Looks like a CMake issue on windows. I don't have a windows machine to reproduce/debug, but I don't see any blockers besides someone finding a machine to test and fix cmake on.

[OwenTrokeBillard]

@wsmoses @ZuseZ4 Here is a log containing the errors that arise when attempting to build Rust with Enzyme on Windows, as you requested: log-2024-01-19.txt. No action is required on your part.

[KhaledHosseini]

@wsmoses @ZuseZ4 Here is a log containing the errors that arise when attempting to build Rust with Enzyme on Windows, as you requested: log-2024-01-19.txt. No action is required on your part.

Did you manage to solve the problem? the exact problem for me. I tried Visual Studio 2019 too, but the same result.
I removed the following line from "src/tools/enzyme/enzyme/CMakeLists.txt":

SET(CMAKE_CXX_FLAGS "-Wall -fno-rtti ${CMAKE_CXX_FLAGS} -Werror=unused-variable -Werror=dangling-else -Werror=unused-but-set-variable -Werror=return-type -Werror=nonnull -Werror=unused-result -Werror=reorder -Werror=switch")

and the 1st series of errors gone (those in your log file), but another error appeared:

fatal error C1189: #error:  The C++ Standard Library forbids macroizing the keyword "private". Enable warning C4005 to find the forbidden define.

I guess I need to add some keywords to the compiler like _ALLOW_KEYWORD_MACROS, but I do not know how.