Can not install both gmt and cartopy because of conda dependency conflicts

Q&A PyGMT Q&A
1

I used to be able to install both gmt (v6.1.1) and cartopy (v0.18) using conda without any problems. However, about two days ago, dependency problems started to appear, and the only solution I found is to downgrade cartopy to v0.17. It doesn’t look like the conda-forge feedstocks have changed for either of these packages, so it’s not obvious to me what has changed. Cartopy and (py-)gmt are both popular packages for map projections, so it would be good to ensure that these can co-habitate together. Some python packages install both cartopy and gmt, so this is a major nuisance.

Here is the simplest way to reproduce this problem:

conda create -n test python=3.8 cartopy=0.18 gmt=6.1.1

and if you care, here is the long output describing the dependencies

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package python conflicts for:
cartopy=0.18 -> numpy[version='>=1.16.5,<2.0a0'] -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.9,<3.10.0a0|3.4.*|>=3.5,<3.6.0a0|3.7.*|3.8.*|>=3.6|>=3']
python=3.8
gmt=6.1.1 -> gdal[version='>=3.0.4,<3.1.0a0'] -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
cartopy=0.18 -> python[version='>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']

Package ncurses conflicts for:
python=3.8 -> readline[version='>=8.0,<9.0a0'] -> ncurses[version='5.9.*|>=6.2,<6.3.0a0|>=6.0,<7.0a0|6.0.*']
python=3.8 -> ncurses[version='>=6.1,<6.3.0a0|>=6.2,<7.0a0|>=6.1,<7.0a0']

Package python_abi conflicts for:
cartopy=0.18 -> numpy[version='>=1.16.5,<2.0a0'] -> python_abi[version='2.7.*|3.6|3.9.*',build='*_cp27m|*_cp39|*_pypy36_pp73']
cartopy=0.18 -> python_abi[version='3.6.*|3.7.*|3.8.*',build='*_cp38|*_cp37m|*_cp36m']

Package sqlite conflicts for:
cartopy=0.18 -> proj[version='>=7.1.1,<7.1.2.0a0'] -> sqlite[version='3.13.*|3.20.*|>=3.24.0,<4.0a0|>=3.25.2,<4.0a0|>=3.25.3,<4.0a0|>=3.26.0,<4.0a0|>=3.28.0,<4.0a0|>=3.30.1,<4.0a0|>=3.32.3,<4.0a0|>=3.33.0,<4.0a0|>=3.31.1,<4.0a0|>=3.29.0,<4.0a0|>=3.23.1,<4.0a0|>=3.22.0,<4.0a0|>=3.20.1,<4.0a0|>=3.25.1,<4.0a0|>=3.27.2,<4.0a0|>=3.30.0,<4.0a0']
python=3.8 -> sqlite[version='>=3.30.0,<4.0a0|>=3.30.1,<4.0a0|>=3.32.3,<4.0a0|>=3.33.0,<4.0a0|>=3.31.1,<4.0a0']

Package freetype conflicts for:
gmt=6.1.1 -> ffmpeg -> freetype[version='2.8.1|2.8.1.*|>=2.8.1,<2.8.2.0a0|>=2.8.1,<2.9.0a0|>=2.9.1,<3.0a0|>=2.8,<2.9.0a0']
cartopy=0.18 -> matplotlib-base[version='>=1.5.1'] -> freetype[version='2.5.*|2.6.*|2.7|2.7.*|2.7|2.8.*|2.8.1|2.8.1.*|>=2.10.2,<3.0a0|>=2.10.4,<3.0a0|>=2.9.1,<3.0a0|>=2.8.1,<2.9.0a0|>=2.10.3,<3.0a0|>=2.8,<2.9.0a0']

Package numpy conflicts for:
cartopy=0.18 -> numpy[version='>=1.14.6,<2.0a0|>=1.16.5,<2.0a0']
cartopy=0.18 -> shapely[version='>=1.5.6'] -> numpy[version='1.10.*|1.11.*|1.12.*|1.13.*|>=1.11|>=1.19.2,<2.0a0|>=1.9.3,<2.0a0|>=1.8|>=1.9|>=1.7|1.9.*|1.8.*|>=1.11.3,<2.0a0|>=1.14.1,<2.0a0|>=1.18.5,<2.0a0|>=1.15.4,<2.0a0|>=1.16.6,<2.0a0|>=1.18.1,<2.0a0|>=1.15.1,<2.0a0']
2

It’s possible that we need to rebuild the gmt feedstock as in

3

It looks like Cartopy rebuilt their feedstock for the automatic “Rebuild for gdal31” PR.

Perhaps someone could re-run the tests on the above gmt-PR to see if they now magically pass?

4

Tried to re-run that PR, but the bot deleted this PR automatically. Hopefully it will open another PR in the next few days.

5

It doesn’t look like the bot is going to rebuild this for us. What if we just update gdal to v3.1.3 in the feedstock and see if that works?

6

Do you want to open a PR for it?

7

I could try, but it might take 1 week (busy). I thought that this would just involve changing one version number in a single file, but looking at the old automatic PR that was closed, there were lots of other changes…

8

The bot reopened a new PR at https://github.com/conda-forge/gmt-feedstock/pull/111, but builds are still breaking :sweat_smile:

9

Thanks! Its working now (but not with python 3.9).

10

Cool, great to hear that it works!

About Python 3.9 support, we’re thinking about adding it to PyGMT in https://github.com/GenericMappingTools/pygmt/issues/644, but just curious to know if the rest of the ecosystem (Cartopy, matplotlib, etc) have Python 3.9 support already? If so then we should definitely get this sorted for PyGMT v0.2.1.

Edit: Started a PR for Python 3.9 support at https://github.com/GenericMappingTools/pygmt/pull/689 :smile:

11

This is just to let you know that I am once again having problems installing the most recent version of gmt (6.2.0) and cartopy at the same time using conda.

When using python 3.9, there appears to be an explicit conflict with gdal. GMT requires ‘>=3.3.0,<3.4.0a0’, whereas cartopy requires ‘>=3.1.3,<3.2.0a0’].

$ conda create -n test python=3.9 cartopy gmt=6.2.0

Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package libgdal conflicts for:
gmt=6.2.0 -> gdal[version='>=3.3.0,<3.4.0a0'] -> libgdal[version='3.3.0|3.3.0|3.3.0|3.3.0|3.3.1',build='hd181ca1_1|hd85aa46_2|hd51e85c_3|hd51e85c_1|hd51e85c_0|h9a52621_0']
cartopy -> gdal[version='>=3.1.3,<3.2.0a0'] -> libgdal[version='3.1.2|3.1.2|3.1.2|3.1.2|3.1.2|3.1.3|3.1.3|3.1.4',build='h6ef84d7_0|hd7bf8dc_1|h1f7cb0a_3|h55c8954_6|h0a7a322_9|hc84532a_12|h27e2542_15|hd51e85c_17|hd85aa46_16|hd181ca1_14|h9a52621_13|he0423de_11|h6de91e5_10|h62a8350_8|hbaa4b3a_7|hbdbaf39_5|he1d4179_4|hbb49e5d_2|h7b1ae08_1|hd7bf8dc_0|hd7bf8dc_0|hd7bf8dc_4|h6dfbaa8_3|hea8881a_2|h6dfbaa8_1']

Package python conflicts for:
python=3.9
gmt=6.2.0 -> gdal[version='>=3.3.0,<3.4.0a0'] -> python[version='>=3.6,<3.7.0a0|>=3.9,<3.10.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']

Package sqlite conflicts for:
cartopy -> proj[version='>=7.2.0,<7.2.1.0a0'] -> sqlite[version='3.13.*|3.20.*|>=3.24.0,<4.0a0|>=3.25.2,<4.0a0|>=3.25.3,<4.0a0|>=3.26.0,<4.0a0|>=3.28.0,<4.0a0|>=3.30.1,<4.0a0|>=3.32.3,<4.0a0|>=3.33.0,<4.0a0|>=3.34.0,<4.0a0|>=3.31.1,<4.0a0|>=3.29.0,<4.0a0|>=3.23.1,<4.0a0|>=3.22.0,<4.0a0|>=3.20.1,<4.0a0|>=3.25.1,<4.0a0|>=3.27.2,<4.0a0|>=3.35.4,<4.0a0|>=3.36.0,<4.0a0|>=3.35.5,<4.0a0|>=3.30.0,<4.0a0|3.9.*']
python=3.9 -> sqlite[version='>=3.33.0,<4.0a0|>=3.34.0,<4.0a0|>=3.35.5,<4.0a0|>=3.36.0,<4.0a0|>=3.35.4,<4.0a0']

Package numpy conflicts for:
cartopy -> numpy[version='1.10.*|1.11.*|1.12.*|1.13.*|>=1.11|>=1.14.6,<2.0a0|>=1.16.5,<2.0a0|>=1.16.6,<2.0a0|>=1.17.5,<2.0a0|>=1.18.5,<2.0a0|>=1.19.5,<2.0a0|>=1.19.4,<2.0a0|>=1.19.2,<2.0a0|>=1.9.3,<2.0a0|>=1.8|1.9.*|>=1.11.3,<2.0a0']
cartopy -> matplotlib-base[version='>=1.5.1'] -> numpy[version='1.8.*|>=1.15.1,<2.0a0|>=1.15.4,<2.0a0|>=1.18.1,<2.0a0|>=1.9|>=1.7|>=1.14.1,<2.0a0']

When using python 3.8, the source of the conflict is not obvious to me:

$ conda create -n test python=3.8 cartopy gmt=6.2.0

Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package numpy conflicts for:
cartopy -> numpy[version='1.10.*|1.11.*|1.12.*|1.13.*|>=1.11|>=1.14.6,<2.0a0|>=1.16.5,<2.0a0|>=1.16.6,<2.0a0|>=1.17.5,<2.0a0|>=1.18.5,<2.0a0|>=1.19.5,<2.0a0|>=1.19.4,<2.0a0|>=1.19.2,<2.0a0|>=1.9.3,<2.0a0|>=1.8|1.9.*|>=1.11.3,<2.0a0']
cartopy -> matplotlib-base[version='>=1.5.1'] -> numpy[version='1.8.*|>=1.15.1,<2.0a0|>=1.15.4,<2.0a0|>=1.18.1,<2.0a0|>=1.9|>=1.7|>=1.14.1,<2.0a0']

Package libpng conflicts for:
cartopy -> matplotlib-base[version='>=1.5.1'] -> libpng[version='>=1.6.23,<1.7|>=1.6.35,<1.7.0a0|>=1.6.37,<1.7.0a0|>=1.6.36,<1.7.0a0|>=1.6.34,<1.7.0a0|>=1.6.32,<1.7.0a0']
gmt=6.2.0 -> graphicsmagick -> libpng[version='>=1.6.34,<1.7.0a0|>=1.6.35,<1.7.0a0|>=1.6.37,<1.7.0a0']

Package libcxxabi conflicts for:
python=3.8 -> libcxx[version='>=4.0.1'] -> libcxxabi[version='4.0.1|4.0.1|8.0.0|8.0.0|8.0.0|8.0.0|8.0.1',build='hebd6815_0|3|4|0|2|1|hcfea43d_1']
cartopy -> libcxx[version='>=4.0.1'] -> libcxxabi[version='4.0.1|4.0.1|8.0.0|8.0.0|8.0.0|8.0.0|8.0.1',build='hebd6815_0|3|4|0|2|1|hcfea43d_1']

Package python conflicts for:
gmt=6.2.0 -> gdal[version='>=3.3.0,<3.4.0a0'] -> python[version='>=3.6,<3.7.0a0|>=3.9,<3.10.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
python=3.8

This is not the fault of GMT, however:

  1. I wonder if GMT really requires gdal > 3.3. If gmt will work with >3.1.3, it would be great if they could use this when specifying the dependencies.
  2. Given that gmt and cartopy are common map projection libraries, the developers of both should attempt to ensure that they can both be installed at the same time. Perhaps, a simple test should be included in the tests to check for this.

I will open an issue with Cartopy to let them know about this, but I would hope that the GMT devs would also look into finding a way to resolve this.

Edit: I am also confused as to where the Cartopy gdal dependency comes from as it isn’t explicitly in their dependencies.

12

Here is a response that I got from the Cartopy feedstock maintainers. In essence, Cartopy doesn’t support proj-8 yet, and the GMT has been built with gdal3.3, which requires proj8.

The problem is that cartopy doesn’t support proj=8.x yet (as you’ve gathered in your issue here SciTools/cartopy#1821). But the latest builds of gmt have been build with gdal=3.3, which in turn has been built only with proj=8.x. So I’m afraid until cartopy supports the latest proj, it is going to become increasingly hard to install along with other software like gmt (via gdal) that requires the latest version.

One solution is to simply wait for Cartopy to update their code to work with proj8 (they are working on it). The other solution would be for GMT to use a lower version of gdal that works with a lower version of proj. In any case, I won’t be updating pyshtools to work with gmt 6.2.0 or pygmt 0.4 until this is resolved.

13

Hi @MarkWieczorek, yes, the conda-forge ecosystem has been slowly moving to GDAL 3.3 since May 2021 (see https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/1506) so there are some conda packages like cartopy waiting to catch up. I’ve personally had trouble with another package called cuspatial that is still pinned to GDAL 3.2 too, so you’re not alone in this.

Besides waiting for things to resolve itself at https://github.com/conda-forge/cartopy-feedstock/issues/113, a temporary solution would be to build from source or mix conda packages with PyPI/pip packages. It’s a bit hard for us to downgrade the GMT pinning to GDAL 3.2, much less GDAL 3.1 because while that would allow other users to install cartopy, it will cause conflicts with other geo- packages in the conda-forge ecosystem currently pinned to GDAL 3.3.

14

There is no obligation to build GDAL with proj8, just that it became the default. Also there is no obligation on any GDAL version for GMT. Just that with old GDALs some features will not be available.

15

Thanks. I’ll just wait this out for the time being…