-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Labels
enhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed
Description
Description
Create similar bindings as with Rust but available in server side js (node, deno, bun, ...). The sdk should closely resemble the rust one, and only deviate when either necessary due to napi limitations, or when it is unidiomatic in JS.
napi.rs seems to be good crate to leverage and is relatively easy to use.
Early Experiment
The branch feat/napi contains a super quick pass at creating the bindings. The experiment only covers these areas
- Create a remote SparkSession
- Create a dataframe with
.sql
- Modify the dataframe with
select
, andfilter
- Perform "action" with
count()
- Perform “action” with
show()
There is a lot of use of clone()
and some not great implementations to create a new empty dataframe to satisfy the napi requirements. The polars js interop is a good example of how the bindings might function.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed