Developer’s Corner: Improved NetCDF/GRIB support on GeoServer
Dear Readers,
today we want to share some improvements we made in GeoServer in supporting NetCDF and GRIB datasets.
As you know, NetCDF and GRIB are commonly used format in the Meteorological and Oceanographic (MetOc) context for observational data and numerical modeling, being a platform independent format used to represent multidimensional array-oriented scientific data. As an instance, data for air temperature, water current, wind speed computed by mathematical models across multiple dimensions, such as time, depth/elevation or physical entities measured by sensors may be served as NetCDF datasets.
In the last years, we have improved the GeoTools library in such a context, by providing a NetCDF plugin based on Unidata NetCDF java library. It is worth to point out that the same library also allows to access GRIB datasets. As a result, you may configure a NetCDF/GRIB coverage store in GeoServer for a NetCDF/GRIB file and setup different coverages/layers, one for each NetCDF variable/GRIB parameter available in the input file, together with its underlying dimension (time, elevation, …)
Whilst the standalone NetCDF/GRIB provides access to a single NetCDF/GRIB file exposing it as a self contained coverage store, multiple datasets can be served as a single imageMosaic coverage store. This is especially useful when you have to deal with a collection of files representing different runs and forecasts of a meteo model. You may think about a Meteorological agency running a model each day, producing N forecasts by day with 1 hour step. In that case, an ImageMosaic can be configured on top of the folder containing the related NetCDF/GRIB datasets. Moreover, scripts running periodically can automatically add new files to that, in order to update the available data with latest forecasts.
With this approach you can configure a coverageStore based on an ImageMosaic, so that you can send WMS getMap requests in order to depicts, as an instance, wind currents at specific heights above ground or send WCS getCoverage requests to get raw data about precipitations at different times.
The GeoSolutions SpatioTemporal data Training contains some WMS/WCS requests examples involving the DLR’s Polyphemus Air Quality modeling System datasets.
Creating meaningful maps for the user requires proper styling to be applied to raw data. SLD allows to customize the rendering of your NetCDF/GRIB datasets.
As an instance:
- you can show height of waves as dynamic color map
(More details about this capability and SLD to be used can be found here)
- you can combine color maps style and contouring to show air temperature.
(here you can find the SLD for countouring a DEM. Adapting it for the air temperature case only requires to change the levels parameter values from lines 22)
- You can combine wind components and represent them through windbarbs symbology
More details on NetCDF/GRIB styles and other rendering transformations can be found in the Rendering Transformations section of the SpatioTemporal training. In such a context, you can also take a look to this blog post related to wind barbs depicted in the previous example. Full SLD for WindBarbs example is available here.
Whilst WMS allows to create maps/portraits with custom styling to customize the rendering of a specific slice of a NetCDF/GRIB variable, WCS allows to get raw data for an “hypercube” involving multiple values across different dimensions.
In such a context WCS 2.0 defines:
- Trimming subsetting to specify a range across a dimension
- Slicing subsetting to specify a single value for a dimension
Standard output formats such as GeoTIFF, ArcGrid don’t allow to encode multiple “2D slices” of the same coverage related to different time,elevation ranges involved in the request.
Therefore, a NetCDF output format has been developed to store all the requested portions of a coverage into a single multidimensional file.
As an instance, a request like this:
will create a NetCDF file containing all available data for the NO2 (Nitrogen Dioxide) coverage by
- Trimming on latitude [40 -> 50]
- Trimming on longitude [5 -> 20]
- Trimming on elevation [300 -> 1250]
- Trimming on time [2013-03-01T10:00:00.000Z -> 2013-03-01T22:00:00.000Z]
That request is getting all NO2 data within the elevation range [300-1250] for the time period from 2013/03/01 at 10 AM to 2013/03/01 at 10 PM, in the BoundingBox with corners 5°E 40°N – 20°E 50°N.
On Panoply, the output will look like this. You can notice multiple values available across dimensions: 13 time values and 6 elevation values which can be combined together to get 6*13 = 78 different 2D slices of the requested coverage.
An important improvement we recently made in handling NetCDF and GRIB file is related to support for different projections. In the beginning, GT/GS NetCDF plugins were only supporting WGS84 based datasets due to missing logic to parse projection related information. Lately (GeoServer 2.8.x), the NetCDF input format has been improved in order to support different coordinate reference systems which are expressed through GridMapping as per NetCDF-CF conventions so we can support Lambert conformal, Stereographic, Transverse Mercator, Albers Equal Area, Azimuthal Equal Area, Orthographic projections.
The NetCDF-CF GridMapping way, requires to associate a NetCDF variable containing the projection information to a multidimensional Variable containing data defined in that projection. As an instance, your dataset may contain an icing_probability variable declaring a
grid_mapping = “LambertConformal_Projection” attribute as well as that “LambertConformal_Projection” variable containing this definition:
int LambertConformal_Projection;
:grid_mapping_name = “lambert_conformal_conic”;
:latitude_of_projection_origin = 25.0; // double
:longitude_of_central_meridian = -90.0; // double
:standard_parallel = 25.0; // double
:earth_radius = 6371229.0; // double
This information will be internally parsed to setup a Coordinate Reference System.
Moreover, a custom EPSG should be added to the GeoServer’s user_projection definitions matching that CRS in order to have a valid code identifying that custom projection. More information on this topic are available as part of the GeoServer NetCDF community module documentation.
Finally, the NetCDF output format has been improved too, in order to:
- support NetCDF4-Classic format. Long story short, the NetCDF4-Classic format add support for the NetCDF4’s HDF5 compression capability to the NetCDF3 model. This may be helpful to reduce the size of the requested data when performing a GetCoverage requeste involving a wide domain (in terms of bbox, time, elevation, …).
- NetCDF4 output will be automatically created when the getCoverage request specifies application/x-netcdf4 instead of application/x-netcdf.
- NetCDF4 support requires NetCDF native libs are available. More details are available at the related GeoServer page.
- support CF (Climate Forecast) names mapping. Input coverages can be renamed on write, in order to use standard names from the CF convention. Moreover, in case a unit of measure change is involved in this remapping, the data values will be converted to use the canonical unit of measure. Therefore, you may think about configuring the NetCDF output for a “Celsius_Degrees_temp” custom layer to produce an “air_temperature” output expressed in K (Kelvin) to respect the CF Naming convention.
- support DataPacking on write. This feature allows to store, as an instance, a 64 bit Floating point coverage to a 16 bit Integer data type through a scale & offset mechanism.
- support for user defined global attributes to be added to the output coverage.
All these new options can be customized through an additional panel available when configuring a layer.
More information on the NetCDF output features are available as part of the GeoServer NetCDF Output community module documentation. Hope you have enjoy our NetCDF/GRIB support on GeoServer.
The GeoSolutions Team,
