Reusing KDTree weights in PyResample.resample_nearest

[Marston]

Is there a way to use the current PyResample.resample_nearest() version to reuse the weights?

[djhoese]

Could you provide more information and detail about what weights you're referring to?

[djhoese]

You can look at the commit history, pyresample is definitely being developed, just not as rapidly as we'd hope as it works "well enough" for the jobs the maintainers get paid to do.

I recommend Satpy because it is overall high level so you don't have to worry about doing thing yourself. It also uses dask so memory usage can remain relatively low and processing times are faster than pure numpy arrays that stretch the limits of memory. Satpy can be used with in-memory objects as well as long as they are wrapped in an xarray DataArray and a dask array under that. That said, I'll admit that my recommendation of Satpy assumed you were reading data that already had a reader in Satpy. While that isn't a requirement to benefit from Satpy's functionality as you can make your own DataArrays, it does make it less of an obvious win.

Geo2Grid is being actively developed, but just wraps Satpy so not much of a point in using it if you are comfortable with Python and want to customize things.

Regarding GOES2GO, I'm not super familiar with it, but if you can get the S3 URL for the files you are reading then those can be passed to Satpy...oh you have an algorithm on top of that, I see, OK nevermind then.

[Marston]

Understood. I like the GOES2GO philosophy that allows the user to work with Xarray or download to file. But I am disappointed in the pace of PyResample - that coming from a non-paying, non-contributor. I think we are pretty far down the PyResample path to change to Satpy. But I will keep it in my back pocket.

[djhoese]

If it wasn't clear, Satpy uses pyresample under the hood. The features developed as part of Satpy are being (were) migrated to pyresample regarding xarray and dask support.

What exactly are you expecting out of a library that is relatively feature complete to its volunteering (mostly government employees - no money) maintainers? If you want something added then issues and pull requests have always been an option.

Edit: I forgot to mention all of the pyresample 2.0 issues and PRs that I made. I also had a second kid this year so not a lot of time to spare.

[Marston]

You were very clear. What I was saying is that we developed an algorithm that minimizes the IO process and try to keep things in memory. I am trying to reduce the processing time. I like the PyResample philosophy using Xarray and definitions but there are some limitations that affect my use case. Currently we use only the resample_nearest(). I believe the bilinear aspects are still a bit behind with it's integration with Xarray? I brought this up about a year ago and the developer at SMHI disappeared after a few contacts.

I completely understand. It's not fair making demands. My job is more on the commercial side of the house and I am not allotted time to contribute. I have asked and the fundings is just not there. Having a family with toddlers doesn't give me a lot of time on my own either. Congratulations by the way - a family is wonderful achievement :-) Moreover, I appreciate the contributors to a project like this.

I'm open to doing work on my off time. I think I will have more time this year.

[djhoese]

the developer at SMHI disappeared after a few contacts

This can always depend on the developer and how the conversation went.

One thing I forgot to mention when my kid woke up at 11pm last night and I made my previous comment: We do have some Xarray and dask compatible "Resampler" classes in pyresample, but what their final interfaces will be is still not completely stable. Work is being put into the pyresample.future sub-package, but I haven't had time to work on it lately. You are correct that the basic "bilinear" resampler is not where it needs to be for operational usage (very memory hungry), but the newer "gradient search" is very fast and supports bilinear and nearest-neighbor style resampling. It doesn't work for all input types those last I checked.

Overall, if you have opinions about where you'd like to see pyresample go then contributing to github issues and discussions is the best way to let us know. Another place would be the pytroll slack which has a #pyresample channel specific to this package.