Gmt plot malloc error message

Dear Gurus,

when working with a rather large binary file (~450 MB) I get the following error message:

mbp15-kristof:OSM-Kueste kristof$ ./GMT_OSM_coast_example.sh
### Using GMT version: 6.0.0
### Convert shape file to GMT usable ASCII format ... (ogr2ogr)
Warning 1: The output driver does not seem to natively support Integer64 type for field FID. Converting it to Real instead. -mapFieldType can be used to control field type conversion.
### Convert ASCII file to binary ... (convert)
### Time to plot ...
gmt(13496,0x1178a0dc0) malloc: *** error for object 0x7fa6047a8000: pointer being freed was not allocated
gmt(13496,0x1178a0dc0) malloc: *** set a breakpoint in malloc_error_break to debug
./GMT_OSM_coast_example.sh: line 19: 13496 Abort trap: 6           gmt plot land_polygons_osm.bf2 -bi2f -Wthinnest,black -Ggrey
mbp15-kristof:OSM-Kueste kristof$

I must admit I have no clue what is going on. There is a warning message from GDAL but convert seems not to mind. The plot is shown and looks as expected. I’m quite sure more information is necessary but I do not know what to supply. Please advise what kind of information is needed.

There is no real difference if run with 6.0.0 or latest dev version. The input file land_polygons.shp is the Land Polygon in the Large polygons not split version with WGS84 datum from https://osmdata.openstreetmap.de/data/land-polygons.html (~630 MB download)

The script:

#!/usr/bin/env bash

# function gmt() { /usr/local/opt/gmt_dev/bin/gmt "$@"; }
# export -f gmt

echo "### Using GMT version: $(gmt --version)"

echo "### Convert shape file to GMT usable ASCII format ... (ogr2ogr)"
ogr2ogr -F GMT land_polygons_osm.gmt land_polygons.shp

echo "### Convert ASCII file to binary ... (convert)"
gmt convert land_polygons_osm.gmt -bo2f > land_polygons_osm.bf2

echo "### Time to plot ..."
gmt begin caboverde
  gmt set PS_LINE_CAP round
  gmt set PS_LINE_JOIN round
  gmt coast -JL-24.9/16.55/16.3/16.7/20c  -R-25.14/16.75/-24.8/16.95r -Wred -Ba10mg10m
  gmt plot land_polygons_osm.bf2 -bi2f -Wthinnest,black -Ggrey
  gmt coast -Wred,dashed
gmt end show

The plot:

I don’t get those error messages but id doesn’t even print anything. From what I can tell it failed to allocate a memory chunk but didn’t say that.

No, it’s not an allocation failure but instead somehow the segment 186354 gets freed before using it.

Pardon my ignorance @Joaquim – is there anything I can do to fix that? GMT has 32GB RAM to play with might this be a bottleneck?

Nope, unless you learn (very very fast) how to debug with a debugger, there is nothing you can do. I also have 32 Gb, but I don’t think this is a lack of memory issue.

I think I have that file on my other computer at home. if I can find it and make it crash then I can have a look.

Just find why for segment 186354 plot_geo_polygon_segment() gets called and frees the lon,lat. The crash ocurs because it later tries to use this freed memory.

I think the bug is if t line 3950 of gmt_plot add_pole is false

	if (add_pole) {	/* If we get here then a detour will be needed */
		if ((n = gmt_geo_polarcap_segment (GMT, S, &plon, &plat)) == 0) {	/* Not a global map */

memory is not allocated and hence it should not be freed at the bottom of function.
But even then the plon, plat allocated there would leak the memory in the S struct, no?

… sounds trivial … not.

@pwessel might have the same version of said file as I do. It is updated whenever the OSM planet changes. So @Joaquim might have a different version.

Does not crash for me but it detects a problem:

gmt psxy landpoly_osm.bf2 -bi2f -Wthinnest,black -Ggrey -JL-24.9/16.55/16.3/16.7/20c -R-25.14/16.75/-24.8/16.95r > t.ps
psxy (gmt_io.c:7936(gmt_free_segment)): Wrongly tries to free item
psxy (gmt_io.c:7936(gmt_free_segment)): Wrongly tries to free item

Supposed to work on taxes today…but GMT forensics is more challenging.

But I already gave you the bug. It’s the fix that I’m not sure. That

plon = gmt_M_memory (GMT, NULL, n, double);

when above we had

double *plon = S->data[GMT_X]

has a smell. And freeing it at

	if (free_memory) {	/* Delete what we allocated */
		gmt_M_free (GMT, plon);

is the ignition.

Yes, I agree with you - just needed to test. Will work up a PR.

Your fix works for me in 6.1.0_de11caf_2020.04.05. No more crash. Thank you @pwessel and @Joaquim!

@KristofKoch If you feel deary to test this alternative, with the Julia wrapper one can now do

using GMT
D = gmtread("c:/v/land-polygons-complete-4326/land_polygons.shp", region=[-25.14, -24.8, 16.75, 16.95]);
plot(D, proj=(name=:LambertConic, center=(-24.9,16.55), parallels=(16.3,16.7)), fill=:gray, lw=0.5, figsize=15, Vd=1)
        #psxy  -JL-24.9/16.55/16.3/16.7/15 -Baf -BWSen -R-25.09/-24.85/16.772/16.924 -Ggray -W0.5 -P -K > C:\TMP\GMTjl_tmp.ps

coast!(res=:full, shore=(0.5, :red), fmt=:png, show=true, Vd=1)
        #pscoast  -R -J -Dfull -W0.5,red -K -O >> C:\TMP\GMTjl_tmp.ps

The # lines show the generated GMT command (by effect of the Vd=1).
After warming up Julia (it’s a JIT compiled language) it takes about 30 s to do the plot.

Note that this still dev version in the Julia wrapper so it need to be installed (from the Julia command window) as

]add GMT#master

@Joaquim my knowledge about Julia is very limited. I’m aware that the Julia language exists (thanks to you a couple of years ago) but that’s basically it.

30 s from shape file to plot is impressive! Maybe I have something to toy around with the next weeks.