Dear Gurus,
when working with a rather large binary file (~450 MB) I get the following error message:
mbp15-kristof:OSM-Kueste kristof$ ./GMT_OSM_coast_example.sh
### Using GMT version: 6.0.0
### Convert shape file to GMT usable ASCII format ... (ogr2ogr)
Warning 1: The output driver does not seem to natively support Integer64 type for field FID. Converting it to Real instead. -mapFieldType can be used to control field type conversion.
### Convert ASCII file to binary ... (convert)
### Time to plot ...
gmt(13496,0x1178a0dc0) malloc: *** error for object 0x7fa6047a8000: pointer being freed was not allocated
gmt(13496,0x1178a0dc0) malloc: *** set a breakpoint in malloc_error_break to debug
./GMT_OSM_coast_example.sh: line 19: 13496 Abort trap: 6 gmt plot land_polygons_osm.bf2 -bi2f -Wthinnest,black -Ggrey
mbp15-kristof:OSM-Kueste kristof$
I must admit I have no clue what is going on. There is a warning message from GDAL but convert seems not to mind. The plot is shown and looks as expected. I’m quite sure more information is necessary but I do not know what to supply. Please advise what kind of information is needed.
There is no real difference if run with 6.0.0 or latest dev version. The input file land_polygons.shp is the Land Polygon in the Large polygons not split version with WGS84 datum from Land polygons (~630 MB download)
The script:
#!/usr/bin/env bash
# function gmt() { /usr/local/opt/gmt_dev/bin/gmt "$@"; }
# export -f gmt
echo "### Using GMT version: $(gmt --version)"
echo "### Convert shape file to GMT usable ASCII format ... (ogr2ogr)"
ogr2ogr -F GMT land_polygons_osm.gmt land_polygons.shp
echo "### Convert ASCII file to binary ... (convert)"
gmt convert land_polygons_osm.gmt -bo2f > land_polygons_osm.bf2
echo "### Time to plot ..."
gmt begin caboverde
gmt set PS_LINE_CAP round
gmt set PS_LINE_JOIN round
gmt coast -JL-24.9/16.55/16.3/16.7/20c -R-25.14/16.75/-24.8/16.95r -Wred -Ba10mg10m
gmt plot land_polygons_osm.bf2 -bi2f -Wthinnest,black -Ggrey
gmt coast -Wred,dashed
gmt end show
The plot:
I don’t get those error messages but id doesn’t even print anything. From what I can tell it failed to allocate a memory chunk but didn’t say that.
No, it’s not an allocation failure but instead somehow the segment 186354 gets freed before using it.
Pardon my ignorance @Joaquim – is there anything I can do to fix that? GMT has 32GB RAM to play with might this be a bottleneck?
Nope, unless you learn (very very fast) how to debug with a debugger, there is nothing you can do. I also have 32 Gb, but I don’t think this is a lack of memory issue.
I think I have that file on my other computer at home. if I can find it and make it crash then I can have a look.
Just find why for segment 186354 plot_geo_polygon_segment() gets called and frees the lon,lat. The crash ocurs because it later tries to use this freed memory.
I think the bug is if t line 3950 of gmt_plot add_pole is false
if (add_pole) { /* If we get here then a detour will be needed */
if ((n = gmt_geo_polarcap_segment (GMT, S, &plon, &plat)) == 0) { /* Not a global map */
memory is not allocated and hence it should not be freed at the bottom of function.
But even then the plon, plat allocated there would leak the memory in the S struct, no?
@pwessel might have the same version of said file as I do. It is updated whenever the OSM planet changes. So @Joaquim might have a different version.
Does not crash for me but it detects a problem:
gmt psxy landpoly_osm.bf2 -bi2f -Wthinnest,black -Ggrey -JL-24.9/16.55/16.3/16.7/20c -R-25.14/16.75/-24.8/16.95r > t.ps
psxy (gmt_io.c:7936(gmt_free_segment)): Wrongly tries to free item
psxy (gmt_io.c:7936(gmt_free_segment)): Wrongly tries to free item
Supposed to work on taxes today…but GMT forensics is more challenging.
But I already gave you the bug. It’s the fix that I’m not sure. That
plon = gmt_M_memory (GMT, NULL, n, double);
when above we had
double *plon = S->data[GMT_X]
has a smell. And freeing it at
if (free_memory) { /* Delete what we allocated */
gmt_M_free (GMT, plon);
is the ignition.
Yes, I agree with you - just needed to test. Will work up a PR.
Your fix works for me in 6.1.0_de11caf_2020.04.05. No more crash. Thank you @pwessel and @Joaquim!
@KristofKoch If you feel deary to test this alternative, with the Julia wrapper one can now do
using GMT
D = gmtread("c:/v/land-polygons-complete-4326/land_polygons.shp", region=[-25.14, -24.8, 16.75, 16.95]);
plot(D, proj=(name=:LambertConic, center=(-24.9,16.55), parallels=(16.3,16.7)), fill=:gray, lw=0.5, figsize=15, Vd=1)
#psxy -JL-24.9/16.55/16.3/16.7/15 -Baf -BWSen -R-25.09/-24.85/16.772/16.924 -Ggray -W0.5 -P -K > C:\TMP\GMTjl_tmp.ps
coast!(res=:full, shore=(0.5, :red), fmt=:png, show=true, Vd=1)
#pscoast -R -J -Dfull -W0.5,red -K -O >> C:\TMP\GMTjl_tmp.ps
The # lines show the generated GMT command (by effect of the Vd=1).
After warming up Julia (it’s a JIT compiled language) it takes about 30 s to do the plot.
Note that this still dev version in the Julia wrapper so it need to be installed (from the Julia command window) as
]add GMT#master
@Joaquim my knowledge about Julia is very limited. I’m aware that the Julia language exists (thanks to you a couple of years ago) but that’s basically it.
30 s from shape file to plot is impressive! Maybe I have something to toy around with the next weeks.
