gospl recipes¶

Overview¶

What is this?

This online documentation provides some examples of pre- and post- processing techniques used to run different gospl simulations and extract useful information from them.

gospl (short for Global Scalable Paleo Landscape Evolution) is an open source, GPL-licensed library providing a scalable parallelised Python-based numerical model to simulate landscapes and basins reconstruction at global scale.

For an overview of gospl functionalities readers are invited to go through the online documentation:

the technical guide provides in-depth information on the underlying physics,
the getting started pages shows how to install the code itself, but the following sections will provide additional information on that as well,
the complete list of input file parameters is describe in the user guide section,
the API reference describes how methods work and functions have been declared.

Citing

To cite gospl please use following software paper published in JOSS.

Salles et al. (2020) gospl: Global Scalable Paleo Landscape Evolution, Journal of Open Source Software, 5(56), p. 2804. doi: 10.21105/joss.02804.

@article{salles_2020,
    author={Salles, Tristan and Mallard, Claire and Zahirovic, Sabin},
    title={gospl: Global Scalable Paleo Landscape Evolution},
    journal={Journal of Open Source Software},
    year={2020},
    volume={5},
    number={56},
    pages={2804},
    DOI={10.21105/joss.02804}
}

How to use these recipies¶

This documentation is written as an online companion to gospl simulation. The examples and pre- and post- processing methods are actualised on the go, as we develop them.

We are trying to write them in such a way that they are quickly understandable and modifiable to fit with your specific simulation. If you are starting to use gospl, we strongly recommend to go through several of these examples to familiarise yourself with the code capabilities and input/output formats.

The notes are written in Jupyter Notebooks. If you want to run the examples, you can:

run the notebooks online in an interactive window, by clicking on the “Launch Binder” button on the top-right of this page.
download the jupyter notebooks from the recipies repository and run them on your computer.
download some of the dataset from figshare and visualise the model outputs directly.

MyBinder

MyBinder is a service that allows us to provide online computing environments in which the course notebooks can be read and executed. This is a fantastic service that runs on the cloud - each time a user clicks on the top binder link, a so-called “virtual machine” is spun-up and made accessible via the web-browser.

These binder environments are safe, anonymous, and temporary. Temporary means non-persistent: if you close the tab in which the environment was opened, it is lost. If you open the link again, a new environment will be created for you, without your previous modifications. Therefore, if you want to save your work for later (i.e. to be able to upload it in a future binder session), don’t forget to download the files you modified before leaving. Be aware that a Binder environment will be shut down after 10 minutes of inactivity (leaving your tab/window open will count as “activity” only if visible, i.e. leaving the tab open but navigating elsewhere will eventually shut down your session).

MyBinder is a free service provided generously by the Binder and Jupyter communities (FAQ). As such, you can expect some waiting time at launch and relatively limited computer resources. However, these resources should be more than enough to run the course notebooks at home or in class.

Recommended installation¶

The easiest way to set up a full-stack scientific Python deployment is to use a Python distribution. This is an installation of Python with a set of curated packages which are guaranteed to work together.

To start using the gospl recipies presented in this documentation, we recommend the Anaconda Python Distribution. Follow the previous link to obtain a one-click installer for Linux, Mac, or Windows. (Make sure you select the Python 3 installer) In addition to the packages themselves, Anaconda includes a graphical utility to help manage any packages you may want to install which are not already included in the default inclusion list.

Accessing the command line¶

These notes assume you have access to a Unix command line to type in shell commands. For Linux and MacOS users, you can access the command line by opening the terminal program. It’s more difficult for Windows users. For Windows users, you should first install Anaconda (described above) or Miniconda (described below), which gives you access to the “Anaconda Prompt” desktop app. (Instructions for this are given on the Andaconda Website.) From the Anaconda Prompt, you should be able to run conda and other shell commands.

Lightweight alternative: Miniconda¶

If you are installing python on a remote machine via ssh, or you simply don’t want to download a huge file like the Anaconda Python Distribution, there is a lightweight alternative installation method.

Obtain a minimal Python installer We recommend the Miniconda installer; this provides a Python install for your operating system, plus the conda package manager. This way, you can only install the packages you want.
Run the installer You’ll probably need to do this from the command line, e.g.

$ sh Miniconda3-latest-MacOSX-x86_64.sh

Follow the instructions; you can choose where to place the installation ( preferably somewhere you have write access without super-user/root access, like your home directory). At the end of this process, add this path to your rc configuration

$ echo ". $HOME/miniconda3/etc/profile.d/conda.sh" > ~/.bashrc

If you do this, your $PYTHONPATH will be implicitly configured correctly and you will never have to touch it.

Create the gospl environment and install packages. This is described in the next section.

Managing Environments with Conda¶

Python coupled with a package manager provides a way to make isolated, reproducible environments where you have fine-tuned control over all packages and configuration. You should always work within an environment, rather than the “default” environment.

It is strongly recommended to read official Getting Started with Conda guide.

To create a conda environment, you execute the following command:

$ conda create --name my_environment python=3.8 numpy

This will create a special environment in $MINICONDA_HOME/envs/my_environment with only Python and numpy to begin with. Here, we’ve also told conda to install Python version 3.8; you can specify exact versions or minima, and conda will take care of figuring out all the compatibilties between versions for you. To use this environment, simply “activate” it by executing:

$ conda activate my_environment

Regardless of your shell, you should now see the string (my_environment) prepended to your prompt. Now, if you execute any Python-related tool from the command line, it will first search in $MINICONDA_HOME/envs/my_environment/bin to find them. You can deactivate your environment by typing:

$ conda deactivate

To see all the environments on your system:

$ conda info --envs

If you want to permanently remove an environment and delete all the data associated with it:

$ conda remove --name my_environment --all

For extensive documentation on using environments, please see the conda documentation. The most important feature to review here is the ability to share and export your environment; this is the basis for reproducibility in the scientific Python stack. At any time from the shell, you can execute

$ conda list

to get a complete summary of all the packages installed in your environment, the channel they were installed from, and their full version info. Using this info, you can create an environment file in YAML syntax which documents the exact contents of your environment. With that file, a new environment with the exact configuration can be installed by executing

$ conda env create -f my_environment.yml

gospl recipes environment¶

Combining all of the previous sections, we can very easily spin-up a full-featured scientific Python environment with a set of packages curated for the course. Copy and paste the following environment.yml file somewhere on your local hard drive:

name: stellar
channels:
  - conda-forge
  - defaults
  - anaconda
dependencies:
  - python=3.8    # Python version 3.8
  - compilers
  - llvm-openmp
  - sphinx
  - importlib_resources
  - pip           # Python package
  - pandas        # Labeled array library
  - netCDF4       # Wrapper for netcdf4
  - xarray        # N-d labeled array library
  - matplotlib    # 2D plotting library
  - numpy         # N-d array and numerics
  - numpy-indexed # Numerics adds on
  - scipy         # Common math/stats/science functions
  - scikit-image  # Image processing routines
  - scikit-learn  # Machine learning library
  - scikit-fuzzy  # Fuzzy logic library
  - geopandas     # Labeled array library for geosciences
  - shapely       # Geometric library
  - cftime        # Time units decoder
  - cartopy       # Geographic plotting toolkit
  - myst-nb       # Notebook syntax
  - sympy         # Symbolic expression
  - cmocean       # Colormaps for oceanography
  - jupyterlab    # Jupyter Lab environment
  - jupyter-book  # Jupyter book environment
  - basemap       # Mapping package
  - h5py          # Wrapper for HDF5
  - pymannkendall # Mann-Kendall trend analysis
  - seaborn       # Statistical visualizations
  - pyresample    # Geographic resampling tools
  - petsc4py
  - meshio
  - vtk
  - pre-commit
  - ruamel.yaml
  - mpi4py
  - cython
  - pyvista
  - pooch
  - ipywidgets
  - itkwidgets
  - nodejs
  - pyvista
  - pyevtk
  - panel
  - gdown
  - jigsawpy
  - pysheds
  - rioxarray
  - rasterio
  - ipygany
  - fiona
  - pycrs
  - richdem
  - holoviews
  - geoviews
  - hvplot
  - panel
  - wget
  - pip:
    - pyproj      # Geographic projections
    - jupyter-book==0.10.1
    - meshplex
    - gospl
    - stripy
    - zmapio

(Note: Installing this environment will also install many dependencies, including compiled libraries. This is totally fine; even if you have these libraries already installed through your system package manager, conda will install and link for use in the environment a configuration which should be to play nicely and work with all of its components.)

Create this environment through conda

$ conda env create -f environment.yml

Activate this environment

$ source activate stellar

Tip

This environment should be sufficient for all of the presented examples in this documentation.

Installing more packages¶

Once you have a basic Python environment, you can easily add or remove packages using the conda command line utility. Conda was created to help manage the complex dependencies and pre-compiled binary libraries that are necessary in scientific python.

If you set up your python environment using the Anaconda Python Distribution or with Miniconda, you should already have the conda command available on the command line. With it, you can easily install packages from an official, curated set of packages which are built and tested for a number of different system configurations on Linux, Windows, and macOS

$ conda install <package-name>

Additionally, there is a community-maintained collection of packages/recipes called conda forge which is accessible through conda as a special “channel”

$ conda install -c conda-forge <package-name>

While conda allows you to install almost any science-related package, there may be other general-use python packages you wish to you that are not available in via conda. For these, you can use an alternative installation method.

Outside of the scientific python community, the most common way to install packages is to search for them on the official PyPI index. Once you’ve found the package you want to install (you may have also just found it on github or elsewhere), you use the pip command from a the command line:

$ pip install <package-name>

This will fetch the source code, build it, and install it to wherever your $PYTHONPATH is set. This works in the vast majority of cases, particularly when the code you’re installing doesn’t have any compiled dependencies.

If you can’t find a package on either PyPI or conda-forge, you can always install it directly from the source code. If the package is on github, pip already has an alias to do this for you:

$ pip install git+https://github.com/<user>/<package-name>.git

If all else fails, you can always download the source code and install it manually like

$ wget https:/path/to/my/pkg/source.tar.gz
$ tar -xvzf source.tar.gz
$ cd source/
$ python setup.py install

You can also use pip to install code you’ve downloaded:

$ cd source/
$ pip install -e .

This will automatically call setup.py for you. The “-e” flag will install the package in “editable” mode, which means that any change you make to the source code will automatically be recognised when you load the package in Python.

Surface Evolution & Stratigraphy Modelling STELLAR Project