Best way to make time file for movie: fill inn date gaps

Q&A
1

I need to prepare my time file for a movie that I want to make.

My starting point is a spreadsheet with dates, and n number of columns/attributes:

1967-01-01 att1 att2 att3
1968-06-08 att1 att2 att3
1971-07-01 att1 att2 att3
1981-01-05 att1 att2 att3
1992-01-05 att1 att2 att3

For my movie to move through time consistently, I need to ‘fill in the gaps’, so I need all dates in between the ones I have in my spreadsheet.

What is the best way to do this?
I was looking at sample1d, but I’m not sure if it suits my need (https://docs.generic-mapping-tools.org/dev/sample1d.html). It seems to require a value that it extrapolates. Tried with a dummy value, but it should not be necessary.

An easy demonstration:

I have this:

1967-01-01 att1 att2 att3
1967-01-04 att1 att2 att3
1967-01-10 att1 att2 att3

But need this:

1967-01-01 att1 att2 att3
1967-01-02
1967-01-03
1967-01-04 att1 att2 att3
1967-01-05
1967-01-06
1967-01-07
1967-01-08
1967-01-09
1967-01-10 att1 att2 att3

My real case includes about 20.000 days, so I cannot do this by hand.

Don’t know how gmt movie reacts to empty fields, but if I can make a file with all dates, much some of the work is done.

2

I think it will break with an empty fields.

Is it possible to repeat the values? In animation 08 Paul did the following. See that the 200 values is repeated.

cat <<- EOF | gmt sample1d -fT --TIME_UNIT=d -I1 -Fa > times.txt
2018-01-01T	160
2018-05-01T	200
2018-06-01T	200
2018-07-01T	200
2018-08-01T	200
2018-12-31T	240
EOF
3

Hi Esteban,

Yes, I tried this, but all columns containing data disappears, e.g.:

$ cat <<eof | gmt sample1d -fT --TIME_UNIT=d -I1 -Fa | head
2020-01-01 1 att1
2020-02-07 1 att2
2020-03-08 2 att3
eof
2020-01-01T00:00:00	1
2020-01-02T00:00:00	1
2020-01-03T00:00:00	1
2020-01-04T00:00:00	1
2020-01-05T00:00:00	1
2020-01-06T00:00:00	1
2020-01-07T00:00:00	1
2020-01-08T00:00:00	1
2020-01-09T00:00:00	1
2020-01-10T00:00:00	1

After creating this file, I then need to join the correct date to the correct row of attributes in my spreadsheet, which is a pita.

Maybe I have to turn to sqlite or something for this.

4

I think it would be possible for me to update sample1d to output the trialing text of an input record if the output time matches it exactly, which it does in your case. Are the attn strings or numerical values?

The second issue would be that movie and batch may need some updates on what to do if MOVIE_WORD2 is empty, for instance. Not sure how you plan to use your attributes in a movie when they are not all there. Please explain.

5

Don’t really matter; they just have to come along. In my case, it’s a mix of both strings and numerical values, but they should not be changed or touched. I just need them for further processing.

And yes, as you say, what to do when MOVIE_WORD2 is empty. I don’t have an answer.

Having sample1d fill in missing dates would at least make it easy to import with spreadsheet software and do further column-stuff. Then I would at least have all dates that form the basis of my time file.

In a way, gmt math is more suitable for me, as I need to generate dates, e.g.

gmt math -o0 -T2020-03-01T/2020-03-07T/1d T =
2020-03-01T00:00:00
2020-03-02T00:00:00
2020-03-03T00:00:00
[...]

and then join the attributes from my csv on the dates (hence maybe use something like `sqlite`).
6

See https://github.com/GenericMappingTools/gmt/pull/6098 for part of the answer. You can fake a dummy input to make sample1d work and then skip it on output to get what you need I think.

7

With latest update you do not need the dummy data column. But, you will need to handle your duplicates first.

8

Thanks Paul!