Skip to content

Commit 8a92d57

Browse files
jleratjlerat
authored
Feature/gconf (#1)
* chore: setup github actions and converted README to md * fix: fixed cfg * fix: fixed setup * chore: attempt to fix pipeline paths * chore: fixed test path in pipeline * chore: another attempt * chore: so painful * fix: fixed examples
1 parent 6f6bdc8 commit 8a92d57

File tree

9 files changed

+234
-178
lines changed

9 files changed

+234
-178
lines changed

‎azure-pipelines.yml‎ renamed to ‎.azure_devops/azure-pipelines.yml‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ resources:
1414
paths:
1515
exclude:
1616
- azure-pipelines.yml
17-
- env_nriv_v2.yml
17+
- env_mini.yml
1818

1919
trigger:
2020
- master
@@ -23,11 +23,11 @@ pool:
2323
vmImage: ubuntu-latest
2424
strategy:
2525
matrix:
26-
Python38:
27-
python.version: '3.8'
26+
Python310:
27+
python.version: '3.10'
2828

2929
variables:
30-
CONDA_ENV_NAME: env_nriv_v2
30+
CONDA_ENV_NAME: env_mini
3131
CONDA_HOME: /usr/share/miniconda/envs/$(CONDA_ENV_NAME)/
3232
BUILD_DIR: $(System.DefaultWorkingDirectory)
3333

@@ -46,11 +46,11 @@ steps:
4646
- task: Cache@2
4747
displayName: Use cached Anaconda environment
4848
inputs:
49-
key: conda | $(BUILD_DIR)/pygme/env_nriv_v2.yml
49+
key: conda | $(BUILD_DIR)/pygme/env_mini.yml
5050
path: $(CONDA_HOME)
5151
cacheHitVar: CONDA_CACHE_RESTORED
5252

53-
- bash: conda env create --file $(BUILD_DIR)/pygme/env_nriv_v2.yml
53+
- bash: conda env create --file $(BUILD_DIR)/pygme/env_mini.yml
5454
displayName: Create Anaconda environment (if not restored from cache)
5555
condition: eq(variables.CONDA_CACHE_RESTORED, 'false')
5656

‎.github/workflows/python-package-conda.yml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
name: Build pygme
2+
3+
on: [push]
4+
5+
jobs:
6+
build-linux:
7+
runs-on: ubuntu-latest
8+
9+
strategy:
10+
max-parallel: 5
11+
12+
defaults:
13+
run:
14+
shell: bash -l {0}
15+
steps:
16+
- uses: actions/checkout@v2
17+
18+
- name: Set up Python 3.10
19+
uses: actions/setup-python@v3
20+
with:
21+
python-version: '3.10'
22+
23+
- name: Add conda to system path
24+
run: |
25+
# $CONDA is an environment variable pointing to the root of the miniconda directory
26+
echo $CONDA/bin >> $GITHUB_PATH
27+
28+
- name: Create conda environment if needed
29+
run: |
30+
source $CONDA/etc/profile.d/conda.sh
31+
conda env create -f env_mini.yml
32+
33+
- name : Install hydrodiy package in conda env
34+
run: |
35+
source $CONDA/etc/profile.d/conda.sh
36+
conda activate env_mini
37+
cd $GITHUB_WORKSPACE
38+
mkdir -p src
39+
cd src
40+
git clone https://github.com/csiro-hydroinformatics/hydrodiy.git
41+
cd hydrodiy
42+
echo "--- hydrodiy install : located in $PWD ---"
43+
pip install -e .
44+
45+
- name : Install pygme package in conda env
46+
run: |
47+
source $CONDA/etc/profile.d/conda.sh
48+
conda activate env_mini
49+
cd $GITHUB_WORKSPACE
50+
echo "--- pygme install : located in $PWD ---"
51+
pip install -e .
52+
53+
- name: Run tests
54+
run: |
55+
source $CONDA/etc/profile.d/conda.sh
56+
conda activate env_mini
57+
cd $GITHUB_WORKSPACE
58+
echo "--- pygme test : located in $PWD ---"
59+
pytest --cov=pygme --cov-config=.coveragerc --junitxml=junit/test-results.xml --cov-report=xml --cov-report=html tests
60+
61+
- name: Upload pytest test results
62+
uses: actions/upload-artifact@v3
63+
with:
64+
name: pytest-results
65+
path: junit/test-results.xml
66+
67+
# Use always() to always run this step to publish test results when there are test failures
68+
if: ${{ always() }}
69+
70+
- name: Upload to codecov
71+
uses: codecov/codecov-action@v3
72+
73+
- name: Run examples
74+
run: |
75+
source $CONDA/etc/profile.d/conda.sh
76+
conda activate env_mini
77+
cd $GITHUB_WORKSPACE
78+
echo "--- pygme examples : located in $PWD ---"
79+
python examples/run_all_examples.py
80+

‎README.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# pygme
2+
[![CI](https://github.com/csiro-hydroinformatics/pygme/actions/workflows/python-package-conda.yml/badge.svg)](https://github.com/csiro-hydroinformatics/pygme/actions/workflows/python-package-conda.yml) [![codecov](https://codecov.io/gh/csiro-hydroinformatics/pygme/graph/badge.svg?token=ARBFW69TI3)](https://codecov.io/gh/csiro-hydroinformatics/pygme)
3+
4+
Python Generic Modelling Engine (PyGME): A simple python package to design, run and calibrate models used in environmental sciences.
5+
6+
# What is pygme?
7+
- pygme is a set of tools to create simple models and calibrate them via automatic optimizer
8+
- pygme provides interface to classical hydrological models and allows you to create your own models.
9+
10+
# Installation
11+
- Create a suitable python environment. We recommend using [miniconda](https://docs.conda.io/projects/miniconda/en/latest/) combined with the environment specification provided in the [env_mini.yml] (env_mini.yml) file in this repository.
12+
- Git clone this repository and run `pip install .`
13+
14+
# Basic use
15+
To setup a model, change its parameters and run it:
16+
17+
```python
18+
import numpy as np
19+
from pygme.models.gr2m import GR2M
20+
import matplotlib.pyplot as plt
21+
22+
# Get an instance of the GR2M monthly rainfall-runoff model
23+
gr = GR2M()
24+
25+
# Generate random inputs (to be replaced by real data)
26+
inputs = np.random.uniform(0, 10, size=(300, 2))
27+
28+
# Allocate model
29+
# This step allocates all internal variables
30+
# used for any runs of the model.
31+
#
32+
# The number of outputs is set to the maximum
33+
# to generate all GR2M outputs. Default is 1
34+
# which reduces the output variables to
35+
# streamflow only.
36+
gr.allocate(inputs, noutputs=gr.noutputsmax)
37+
38+
# Set parameters
39+
gr.X1 = 500
40+
gr.X2 = 0.8
41+
42+
# Initialise model
43+
# Here we initialise both GR2M stores
44+
gr.initialise([450, 55])
45+
46+
# Run model
47+
gr.run()
48+
49+
# Plot results
50+
# S: production store
51+
# R: routing store
52+
# AE: Actual evapotranspiration
53+
# F: Inter-basin exchange
54+
# Q: Streamflow
55+
df = gr.to_dataframe()
56+
57+
fig = plt.figure(layout="tight")
58+
mosaic = [[s] for s in ["S", "R", "AE", "F", "Q"]]
59+
axs = fig.subplot_mosaic(mosaic)
60+
for varname, ax in axs.items():
61+
df.loc[:, varname].plot(ax=ax)
62+
ax.set_title(varname)
63+
64+
plt.show()
65+
```
66+
A set of examples is provided in the folder [examples](examples).
67+
68+
# License
69+
The source code and documentation of the pygme package is licensed under the
70+
[BSD license](LICENSE.txt).
71+

‎README.rst‎

Lines changed: 0 additions & 81 deletions
This file was deleted.

‎env_mini.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
name: env_mini
2+
channels:
3+
- conda-forge
4+
- defaults
5+
dependencies:
6+
- cython=0.29.32
7+
- matplotlib=3.5.3
8+
- numpy=1.23.2
9+
- pandas=1.4.3
10+
- pip
11+
- pyproj=3.4.0
12+
- pyshp=2.3.1
13+
- pytest=7.2.0
14+
- pytest-cov
15+
- python=3.10.6
16+
- requests=2.28.1
17+
- scipy=1.9.0
18+
- pip:
19+
- versioneer
20+
- pytest-allclose

‎env_nriv_v2.yml‎

Lines changed: 0 additions & 36 deletions
This file was deleted.

‎examples/run_all_examples.py‎

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
##
99
## ------------------------------
1010
import sys, os, re
11+
from pathlib import Path
1112
import subprocess
1213

1314
from hydrodiy.io import csv, iutils
@@ -19,31 +20,33 @@
1920
#----------------------------------------------------------------------
2021
# Folders
2122
#----------------------------------------------------------------------
22-
source_file = os.path.abspath(__file__)
23-
froot = os.path.dirname(source_file)
23+
source_file = Path(__file__).resolve()
24+
froot = source_file.parent
2425

25-
basename = re.sub('\\.py.*', '', os.path.basename(source_file))
26+
#----------------------------------------------------------------------
27+
# Logging
28+
#----------------------------------------------------------------------
29+
basename = re.sub("\\.py.*", "", source_file.stem)
2630
LOGGER = iutils.get_logger(basename)
2731

2832
#----------------------------------------------------------------------
2933
# Get data
3034
#----------------------------------------------------------------------
31-
lf = iutils.find_files(froot, '.*\.py')
35+
lf = froot.glob("*.py")
3236

3337
#----------------------------------------------------------------------
3438
# Process
3539
#----------------------------------------------------------------------
3640

3741
for f in lf:
38-
fname = os.path.basename(f)
39-
if re.search('run_all_examples', f):
40-
LOGGER.info('Skip '+ fname)
42+
if re.search("run_all_examples", f.stem):
43+
LOGGER.info("Skip "+ f.stem)
4144
continue
4245

43-
LOGGER.info('Running {0}'.format(fname))
44-
cmd = 'python ' + f
46+
LOGGER.info(f"Running {f.stem}")
47+
cmd = f"python {f}"
4548
subprocess.check_call(cmd, shell=True)
4649

4750

48-
LOGGER.info('Process completed')
51+
LOGGER.info("Process completed")
4952

‎setup.cfg‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[meta-data]
2-
description-file = README.rst
2+
description-file = README.md
33

44
[nosetests]
55
#with-coverage=1
@@ -9,8 +9,7 @@ with-xunit=1
99
verbose=1
1010

1111
[easy_install]
12-
find_links = https://bitbucket.org/jlerat/pygme
13-
12+
find_links = https://github.com/csiro-hydroinformatics/pygme
1413

1514
# See the docstring in versioneer.py for instructions. Note that you must
1615
# re-run 'versioneer.py setup' after changing this section, and commit the

0 commit comments

Comments
 (0)