-
Notifications
You must be signed in to change notification settings - Fork 53
Reading track data from netCDF format files
Craig Arthur edited this page Mar 3, 2019
·
3 revisions
TCRM generates catalogues of synthetic TC tracks, which are stored in netCDF format. We make use of the hierarchical functionality in netCDF-4 to store all tracks from a simulated year in a single file.
Below is the CDL syntax that describes the structure of the track files. Each track is stored in it's own unique group within the file, with the label tracks-XXXX, where XXXX represents the event number for that simulated year (starting from event zero).
netcdf tracks.00000 {
// global attributes:
:_NCProperties = "version=1|netcdflibversion=4.4.1.1|hdf5libversion=1.8.18" ;
:created_on = "2018-09-17 11:37:21" ;
:created_by = "cxa547" ;
group: tracks {
group: tracks-0000 {
types:
compound track_dtype {
int CycloneNumber ;
double Datetime ;
float TimeElapsed ;
double Longitude ;
double Latitude ;
double Speed ;
double Bearing ;
double CentralPressure ;
double EnvPressure ;
double rMax ;
}; // track_dtype
dimensions:
time = UNLIMITED ; // (22 currently)
variables:
double time(time) ;
time:units = "hours since 1900-01-01 00:00" ;
time:calendar = "julian" ;
track_dtype track(time) ;
track:long_name = "Tropical cyclone track data" ;
track:time_units = "hours since 1900-01-01 00:00" ;
track:calendar = "julian" ;
track:lon_units = "degrees east" ;
track:lat_units = "degrees north" ;
track:pressure_units = "hPa" ;
track:speed_units = "m/s" ;
track:length_units = "km" ;
track:trackId = "(0, 0)" ;
} // group tracks-0000
group: tracks-0001 {
types:
compound track_dtype {
int CycloneNumber ;
double Datetime ;
float TimeElapsed ;
double Longitude ;
double Latitude ;
double Speed ;
double Bearing ;
double CentralPressure ;
double EnvPressure ;
double rMax ;
}; // track_dtype
dimensions:
time = UNLIMITED ; // (13 currently)
variables:
double time(time) ;
time:units = "hours since 1900-01-01 00:00" ;
time:calendar = "julian" ;
track_dtype track(time) ;
track:long_name = "Tropical cyclone track data" ;
track:time_units = "hours since 1900-01-01 00:00" ;
track:calendar = "julian" ;
track:lon_units = "degrees east" ;
track:lat_units = "degrees north" ;
track:pressure_units = "hPa" ;
track:speed_units = "m/s" ;
track:length_units = "km" ;
track:trackId = "(1, 0)" ;
} // group tracks-0001
}
Track variables are a one-dimensional, with time the unlimited (and only) dimension. The track details are stored in a compound type, which have different data types for each variable (e.g. integer for the cyclone number, double precision float for latitude, longitude, etc.). See the compound track_dtype section in the CDL syntax above
Track data can be dumped to raw text (in CDL syntax) using ncdump, though this includes a lot of other attributes and is not readily transformed to (for example) csv format.
The Utilities.track module contains the definition of the Track object, as well as functions to read TCRM netcdf track files using Python.
The track data for TC events is passed around TCRM using Track objects. These contain a data attribute, that is a numpy.recarray containing the values of the variables for defining the track.
A Track object contains the following fields:
'Indicator', 'CycloneNumber', 'Year', 'Month', 'Day', 'Hour', 'Minute', 'TimeElapsed', 'Datetime', 'Longitude', 'Latitude', 'Speed', 'Bearing', 'CentralPressure', 'WindSpeed', 'rMax', 'EnvPressure'
Data for a track can be accessed using the Track.data attribute, or the named attribute::
t = Track(data)
print(t.CentralPressure)
will print the central pressure data for the track.
The ncReadTrackData function allows users to read a netcdf track file into a collection of Track objects.
>>> from Utilities.track import ncReadTrackData
>>> trackFile = 'tracks.00000.nc'
>>> tracks = ncReadTrackData(trackFile)
This returns a list of track objects:
>>> tracks
[<Utilities.track.Track object at 0x7fc3aed602d0>, <Utilities.track.Track object at 0x7fc3aed60310>, <Utilities.track.Track object at 0x7fc3aed60350>, <Utilities.track.Track object at 0x7fc3aed60390>, <Utilities.track.Track object at 0x7fc3aed603d0>, <Utilities.track.Track object at 0x7fc3aed60410>, <Utilities.track.Track object at 0x7fc3aed60450>, <Utilities.track.Track object at 0x7fc3aed60490>, <Utilities.track.Track object at 0x7fc3aed604d0>, <Utilities.track.Track object at 0x7fc3aed60510>, <Utilities.track.Track object at 0x7fc3aed60550>, <Utilities.track.Track object at 0x7fc3aed60590>, <Utilities.track.Track object at 0x7fc3aed605d0>, <Utilities.track.Track object at 0x7fc3aed60610>]
The data for each track can be accessed using the data attribute, which returns a numpy.recarray (note the dtype attribute at the bottom)
>>> tracks[0].data
array([ (0, cftime.DatetimeJulian(3811, 11, 14, 15, 0, 0, 0, -1, 1), 0.0, 103.5, -10.800000190734863, 5.599999904632568, 309.0, 1002.0, 1007.52294921875, 79.64189910888672),
(0, cftime.DatetimeJulian(3811, 11, 14, 16, 0, 0, 7, -1, 1), 1.0, 103.46013641357422, -10.76828384399414, 8.154326438903809, 322.1949768066406, 1002.0037841796875, 1007.5339965820312, 86.42660522460938),
(0, cftime.DatetimeJulian(3811, 11, 14, 17, 0, 0, 0, -1, 1), 2.0, 103.41436004638672, -10.71030330657959, 18.085166931152344, 206.7210693359375, 1002.030517578125, 1007.536865234375, 86.46817016601562),
(0, cftime.DatetimeJulian(3811, 11, 14, 18, 0, 0, 0, -1, 1), 3.0, 103.33985137939453, -10.855660438537598, 26.136981964111328, 202.00331115722656, 1001.3370971679688, 1007.4537963867188, 85.18887329101562),
(0, cftime.DatetimeJulian(3811, 11, 14, 19, 0, 0, 7, -1, 1), 4.0, 103.25006103515625, -11.073719024658203, 25.335065841674805, 172.10826110839844, 1000.3381958007812, 1007.3388671875, 83.21622467041016),
(0, cftime.DatetimeJulian(3811, 11, 14, 20, 0, 0, 0, -1, 1), 5.0, 103.281982421875, -11.299545288085938, 27.08563232421875, 191.0495147705078, 999.57275390625, 1007.258056640625, 81.61366271972656),
(0, cftime.DatetimeJulian(3811, 11, 14, 21, 0, 0, 0, -1, 1), 6.0, 103.23430633544922, -11.538763046264648, 25.61430549621582, 192.3207550048828, 999.8688354492188, 1007.261962890625, 82.34835052490234),
(0, cftime.DatetimeJulian(3811, 11, 14, 22, 0, 0, 7, -1, 1), 7.0, 103.18406677246094, -11.763949394226074, 24.350072860717773, 195.83702087402344, 999.335205078125, 1007.2020263671875, 81.22977447509766),
(0, cftime.DatetimeJulian(3811, 11, 14, 23, 0, 0, 0, -1, 1), 8.0, 103.12294006347656, -11.974748611450195, 13.520946502685547, 186.79583740234375, 999.3690185546875, 1007.1919555664062, 81.36389923095703),
(0, cftime.DatetimeJulian(3811, 11, 15, 0, 0, 0, 0, -1, 1), 9.0, 103.10821533203125, -12.095566749572754, 18.450939178466797, 197.3639373779297, 999.078857421875, 1007.1621704101562, 80.73839569091797),
(0, cftime.DatetimeJulian(3811, 11, 15, 1, 0, 0, 7, -1, 1), 10.0, 103.05751037597656, -12.254033088684082, 19.385704040527344, 200.31883239746094, 997.1317138671875, 1006.9984130859375, 76.21783447265625),
(0, cftime.DatetimeJulian(3811, 11, 15, 2, 0, 0, 0, -1, 1), 11.0, 102.9954833984375, -12.417620658874512, 18.134929656982422, 199.478515625, 996.119873046875, 1006.9127197265625, 73.82343292236328),
(0, cftime.DatetimeJulian(3811, 11, 15, 3, 0, 0, 0, -1, 1), 12.0, 102.93972778320312, -12.571468353271484, 20.46228790283203, 186.22499084472656, 997.0133056640625, 1006.9771728515625, 76.0030288696289),
(0, cftime.DatetimeJulian(3811, 11, 15, 4, 0, 0, 7, -1, 1), 13.0, 102.91925811767578, -12.75451946258545, 13.342497825622559, 163.26805114746094, 996.5313110351562, 1006.9364013671875, 74.87490844726562),
(0, cftime.DatetimeJulian(3811, 11, 15, 5, 0, 0, 0, -1, 1), 14.0, 102.9547119140625, -12.869501113891602, 13.784278869628906, 162.74612426757812, 996.14208984375, 1006.904541015625, 73.9548110961914),
(0, cftime.DatetimeJulian(3811, 11, 15, 6, 0, 0, 0, -1, 1), 15.0, 102.99246978759766, -12.987958908081055, 13.017833709716797, 174.6242218017578, 997.730224609375, 1007.0270385742188, 77.7793197631836),
(0, cftime.DatetimeJulian(3811, 11, 15, 7, 0, 0, 7, -1, 1), 16.0, 103.00373840332031, -13.104589462280273, 15.466142654418945, 165.00674438476562, 996.869140625, 1006.955810546875, 75.74639129638672),
(0, cftime.DatetimeJulian(3811, 11, 15, 8, 0, 0, 0, -1, 1), 17.0, 103.04072570800781, -13.239026069641113, 14.768235206604004, 159.97320556640625, 998.4105834960938, 1007.0748901367188, 79.4256820678711),
(0, cftime.DatetimeJulian(3811, 11, 15, 9, 0, 0, 0, -1, 1), 18.0, 103.0875015258789, -13.363883018493652, 12.207679748535156, 159.55685424804688, 999.8299560546875, 1007.1865234375, 82.6702651977539),
(0, cftime.DatetimeJulian(3811, 11, 15, 10, 0, 0, 7, -1, 1), 19.0, 103.126953125, -13.466816902160645, 7.858453273773193, 157.93991088867188, 1000.7838134765625, 1007.260986328125, 84.73249816894531),
(0, cftime.DatetimeJulian(3811, 11, 15, 11, 0, 0, 0, -1, 1), 20.0, 103.1542739868164, -13.532355308532715, 5.656347751617432, 191.00363159179688, 1000.331298828125, 1007.2177734375, 83.80206298828125),
(0, cftime.DatetimeJulian(3811, 11, 15, 12, 0, 0, 0, -1, 1), 21.0, 103.14427947998047, -13.582320213317871, 7.55902099609375, 188.3070831298828, 1000.464111328125, 1007.223388671875, 84.10333251953125)],
dtype=[('CycloneNumber', '<i4'), ('Datetime', 'O'), ('TimeElapsed', '<f4'), ('Longitude', '<f8'), ('Latitude', '<f8'), ('Speed', '<f8'), ('Bearing', '<f8'), ('CentralPressure', '<f8'), ('EnvPressure', '<f8'), ('rMax', '<f8')])
Accessing the data for one of the variables in the track data returns a numpy array:
>>> tracks[0].Datetime
array([cftime.DatetimeJulian(3811, 11, 14, 15, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 16, 0, 0, 7, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 17, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 18, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 19, 0, 0, 7, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 20, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 21, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 22, 0, 0, 7, -1, 1),
cftime.DatetimeJulian(3811, 11, 14, 23, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 0, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 1, 0, 0, 7, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 2, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 3, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 4, 0, 0, 7, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 5, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 6, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 7, 0, 0, 7, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 8, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 9, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 10, 0, 0, 7, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 11, 0, 0, 0, -1, 1),
cftime.DatetimeJulian(3811, 11, 15, 12, 0, 0, 0, -1, 1)], dtype=object)