Hello @timhume,
unfortunately that’s a known problem with the GSHHG coastline. In short: What you see is mostly inaccurate data from pre WGS72 times. Additionally some coastline data in GSHHG has a different datum that’s unknown or being ignored leading to position shifts.
Please see this thread for background. The current workaround is what you already did: use OpenStreetMap coastlines if you need high precision. Example 51 in the gallery deals with this as well.