Ways of "parallelizing" tasks

Hi, by default GMT is utilizing a single CPU thread when invoking a command. Are there ways of separating a task manually to increase performance and combine the results on the post side after command processing? In my case I want to divide the command:

gmt grdmath -R-180/180/-89/89 -A0/0/4 -I5m OSM_lines.gmt LDIST = dist_to_coast.nc

into e.g. 4 processes:

gmt grdmath OSM_lines.gmt -R-180/0/-89/0 -A0/0/4 -I5m LDIST = dtc1.nc &
gmt grdmath OSM_lines.gmt -R0/180/-89/0 -A0/0/4 -I5m LDIST = dtc2.nc &
gmt grdmath OSM_lines.gmt -R-180/0/0/89 -A0/0/4 -I5m LDIST = dtc3.nc &
gmt grdmath OSM_lines.gmt -R0/180/0/89 -A0/0/4 -I5m LDIST = dtc4.nc &

and afterwards combine the files dtc1.nc - dtc4.nc to an exact match of the dist_to_coast.nc file as given in the above single line entry. Is this possible using GMT commands or other methods?

There is the isolation mode but in your case you can just run the 4 commands in 4 different shells.

Hi Joaquim,

Yes, running these command lines will start 4 independent calculation processes and consequently 4 resulting netCDF-files (“ideally” containing 1/4 part of the total solution). My question was more related to the possibility afterwards of merging these 4 files into a common netCDF-file which exactly matches the nc-file I would get by running:

gmt grdmath -R-180/180/-89/89 -A0/0/4 -I5m OSM_lines.gmt LDIST = dist_to_coast.nc

Although possible in principle, that would be very case dependent. For example, is an overlap between chunks needed to acunt for border effects? You said in your first post that default GMT is utilizing a single CPU thread. That is not true. Several modules have implemented parallel sections but the parallelization is a very complicated matter and often not even possible (very dependenr on the algorithm in use).

Thanks and when reading my initial post I see your point. I was referring to my case but wrote it like a general issue (which I shouldn’t) - I regret this!

Anyhow, from your answer I see that my case does not seem to have a “simple” solution which implies that I just need to be patient and wait for the routine to finish each time I run it.

The LDISTG is a very slow algorithm. It has to compute geodesic distances all the time. And … it’s already accelerated with OpenMP

You may want to give a look at https://github.com/GenericMappingTools/gmt/pull/2961 but that is very experimental and would require in your case to project the coastlines first into a Cartesian system.

You would use grdblend to stitch all your files together to the final product.