Hello everyone. I’ve been trying to use pygmt.project to project a dataset I have, but I keep running into the same error. My data is in Pandas dataframes with valid values for the fields ‘lon’ and ‘lat’, and the function runs well enough when I use a small number of rows for each dataframe. But as soon as I cross a certain threshold, the process grinds to a halt and I have to close the jupyter notebook to stop it. When I run the function in verbose mode I get this
project [WARNING]: Latitude (2.83351e+190) at line # 63 exceeds -|+ 90! - set to NaN
project [WARNING]: Latitude (8.51889e+223) at line # 64 exceeds -|+ 90! - set to NaN
I guess it’s some kind of overflow based on the magnitude of the numbers. The dataset doesn’t have anything weird on it. Also, I don’t know if the line numbers reference rows on the dataframe, but I can run the function with dataframes of length 120, so it doesn’t make much sense that it would suddenly have a problem with rows 63 and 64 when I run it at length 140. For length 120 I get this output from verbose mode
project [INFORMATION]: Processing input table data
project [INFORMATION]: Reading Data Table from Input memory location via vector
project [INFORMATION]: Writing Data Table to file /tmp/pygmt-vm9gv76b.csv
project [INFORMATION]: 120 read, 120 used
project [INFORMATION]: Processing input table data
project [INFORMATION]: Reading Data Table from Input memory location via vector
project [INFORMATION]: Writing Data Table to file /tmp/pygmt-1f0cynx7.csv
project [INFORMATION]: 120 read, 120 used
This is how I’m calling the function
dataref=data[data['cycle']==cicloref]
datacenter=dataref[['lon','lat']].iloc[0]
datacenter=[datacenter['lon'],datacenter['lat']]
dataend=dataref[['lon','lat']].iloc[-1]
dataend=[dataend['lon'],dataend['lat']]
for ciclo in cycles:
datacycle=data[data['cycle']==ciclo][['lon','lat']].head(120)
dataproj=pygmt.project(data=datacycle,center=datacenter,endpoint=dataend,verbose=True)
I can give the dataset if someone needs it, but I’m guessing it’s a bug and not something specific to my data. Please let me know if it’s a known bug, or if it might be a problem with how I’m approaching the function. Thanks a lot!
Sure, I have a ton of data, but I uploaded the data that was giving me trouble (data.csv) and the track to which I want to project it (ref.csv). I’m using center as the first row of the ref dataframe, and endpoint as the last. When I run just that one projection one time, it does work, but with major issues, as the first values for the data after the projection gives me longitudes of around 1e-310. And when I rerun it, it stops working altogether.
Could you please provide a minimal script to reproduce your issue?
I also see something weird using the following script:
import pygmt
import pandas as pd
import numpy as np
data = pd.read_csv("data.csv")
# works
pygmt.project(data=np.array([data["lon"], data["lat"]]).T, center=(data["lon"][0], data["lat"][0]))
# doesn't work
# pygmt.project(data=np.array([data["lon"], data["lat"]]), center=(data["lon"][0], data["lat"][0]))
I expect [data["lon"], data["lat"]] or np.array([data["lon"], data["lat"]]) should work, but they don’t. I have to use np.array([data["lon"], data["lat"]]).T instead.
I have the data inside my program from previous uses, but trying to reproduce the problem with as minimal a script as I can
import pygmt
import pandas as pd
import numpy as np
data = pd.read_csv("data.csv")
ref=pd.read_csv("ref.csv")
proj=pygmt.project(data=data, center=[ref["lon"].iloc[0], ref["lat"].iloc[0]],
endpoint=[ref["lon"].iloc[-1], ref["lat"].iloc[-1]]
,verbose=True)
print(proj)
I’m getting seven columns instead of six, with the first one being whole, negative numbers, and the projection being clearly wrong (as I guess it’s using that first column as the longitude)
Not the same error but still. I saw your example avoids giving the dataframe as input data, but when I tried to turn it into a numpy array as you did it stopped working. Maybe it’s something to do with the center and endpoint I’m using (from the ref.csv dataframe instead of the data.csv one). Let me know if you can recreate any of the problems I’m getting. Thanks a lot
Sorry, since the first column is the index number of the dataframe, I thought it would be interpreted as so by csv_read. I’ll try again later and let you know. Thanks for your help so far
Hello, sorry for the long wait. I’ve been trying to work out a minimal version of my bug in order to share the data, but I don’t know how to do it. The problems seem to appear when I work with my whole dataset, and not one isolated example. Is there any reason why this shouldn’t work:
import pygmt
import pandas as pd
import numpy as np
data = pd.read_csv("data.csv")
data=data[['lon','lat']]
ref=pd.read_csv("ref.csv")
ref=ref[['lon','lat']]
proj=pygmt.project(data=data, center=[ref["lon"].iloc[0], ref["lat"].iloc[0]],
endpoint=[ref["lon"].iloc[-1], ref["lat"].iloc[-1]]
,verbose=True)
Considering the data I load on my second example is exactly the one I’m saving in the first one? With the first implementation it never stops running, can’t even interrupt the kernel, while with the second one it works perfectly. I see no difference in the calculations I’m making in both cases, so I just assume it’s something to do with how pygmt is using the data when it comes from one source or the other. Also, for some reason, the first example works just the first time I run it, for some reason interpreting the data[‘lon’] column as zero (1e-310) for just the first four lines? And if I try to run it again it just dies?
I’m sorry I can’t give you a more localized example you can play with, I’m loading tons of data on my program and it wouldn’t make much sense to send you all of it, I’ll keep trying to make a simpler example.