Developer’s Corner: Latest performance optimizations in GeoServer

Blog ‐ Posted on 23.01.2017 by simone giannecchini

Dear Readers,

in this post we would like to describe work that our GeoServer Technical Lead Andrea Aime, has done recently in response the test harness prepared by Camp2Camp to measure and compare performance and scalability of GeoServer, MapServer and QGIS Server against 3 OSM derived maps with points, lines and polygons. If you are interested the presentation (which is in French) you can find it here; if your French is as bad as mine, don’t worry, the important pictures about performance are self explaning:

Before optimizations

Before optimizing GeoServer

After Optimizations

After optimizing GeoServer

Ok, let us now go a little more in detail on what the test harness is about in case you wanted to replicate the experiment.

Test Harness

The overall system is based on a few docker images containing both the data, database and the software. The main repository is here:

https://github.com/camptocamp/ms_perfs

The GeoServer images are pulled from https://github.com/camptocamp/docker-geoserver, similar images are available for QGIS and MapServer. The setup downloads an accurate map of swiss plus extra data around the world and runs the benchmark on it using gatling, driven by some shell scripts, a test class written in Scala, and a python summarization script at the end.

The setup requires to download all the OSM data, download and compile various packages, so a few hours might be needed for it (depending on how fast your network connection is). A test run uses 3 layers, each with two different styles:

  • roads-simple (continous line)
  • roads-dashed (dashed line)
  • points-simple (circle)
  • points-class (3 svg files for different type of locations)
  • buildings-simple (solid fill and stroke)
  • buildsing-dashed (dashed fill and simplestroke)

Each run  works  at random locations at 8 different zoom levels, testing each separately, and run for 1, 2, 5, 10, 20, 40 users, thus 48 runs, each one working for 120 seconds. Each run uses all the layers together, even if the final reports give separate timings layer by layer (thus, 288 results in total, the graphs above average the results for various zoom level into a single number).

A test run works for 1.5 hours long per tested server, so 4.5 hours total. Patience and something else to do in the meantime are thus prerequisites.  The maps at level zoom 7 contain very few items, and for the points layer are sometimes empty, at zoom level 7 instead it becomes a “point cloud”. E.g., zoom level 7:

zoom level 7

zoom level 7

zoom level 0:

zoom level 0

zoom level 0

Findings

The initial configuration of GeoServer had a few drawbacks that we have helped spot and improve:

  • It w as using the default max 10 connections in the pool towards postgresql which was hindering scalability with WMS requests against it, hence we raised it. If you are interested you can find some useful material here, here and here.
  • The styles were not an exact match between the various servers.
  • The SVGs were imported from QGIS as is, and still contained parametric expressions for colors, thicknesses and opacities, causing some exceptions at the beginning of the run.
  • The rasterizer was not setup to be Marlin. More information on the topic in this post and here.
  • The server side geometry simplification using ST_Simplify was turned off as most geometries are very small to start with. This can be done by unchecking the “Support on the fly geometry simplification” checkbox at the end of the Datastore configuration (see Image below).
geometry simplification

geometry simplification

As usual when we do this kind of experiments we find usually nice way to optimize GeoServer by hitting code making use of a code profiler; in this case the following came out and have been improved on GeoServer Master:

Further Potential Improvements Actions (i.e. want to help to make GeoServer better?)

Shipping with Marlin Renderer: we should ship Marlin Renderer among the GeoServer jars by default, enable its usage in the bin/win/osx installers, provide information on how to set it up in other containers in the docs or even as a warning in the GUI (the admin ones) if we detect a JDK < 1.9.

Optimizing data transfer from PostgreSQL: currently we transfer data as WKB, however since PostGIS 2.2 we have TWKB available that seems to significantly reduce the payload transferred. The ST_RemoveRepeatedPoints seems to be also quite interesting, very little overhead, we could use that one by default for postgis >= 2.2 and have a separate flag for the usage of ST_Simplify (which is instead heavy). The second should be enabled on a per dataset basis, maybe considering also scale ranges. Finally, now the code is encoding base64 the binary because old postgresql drivers were limited to text based transfers and they did so by doubling the size of the payload, when base64 only make it 4/3 of the original size). Recent drivers allow to configure pure binary transfer instead, we should explore how it gets enabled (there are parameters), its relationship with prepared statements and see if we can effectively leverage it.

Polygon rendering: polygon filling/stroking in java2d appears to be still significantly slower than in MapServer, this is something we could try to improve at the JDK level.

If you are interested in learning about how we can help you achieving your goals with open source products like GeoServer, Mapstore, GeoNode and GeoNetwork through our Enterprise Support Services and GeoServer Deployment Warranty offerings, feel free to contact us!

The GeoSolutions team,