TileGeneration

Introduction

With this solution, we solve the following issues:

  • Managing millions of files on the file system is difficult.

  • We should be able to update all the generated tiles.

  • We should not have thousands of expired files.

To do so, we need need a tool that can generate the tiles, update some of them contained in given geometries and delete empty tiles.

On-the-fly tiles generation introduces some issues such as having a growing number of tiles that may become unmanageable. For example, when updating the data, it is not possible to figure out what tiles should be updated.

For high usage websites, we want to put the tiles on S3 with the same tool.

One issue we have if we want to generate all the tiles is that the generation time can grow to more than one month, especially if we have a high resolution (low if in m/px) on the last zoom level. Therefore for the last zoom level we should generate the tiles on the fly with a low expiry (4 hours for example). We should use metatiles to reduce the number of requests to postgres. And the tiles should be deleted after the expiry time.

The chosen solution is a combination of two tools:

TileCloud-chain

TileCloud-chain is a TileCloud-based tool that offers a build chain for generating tiles from WMS of Mapnik on a local storage or S3 using a WMTS layout.

It supports the following AWS services for generating tiles: EC2, SQS, SNS.

See the readme.

Configuration

The configuration is done in the file tilegeneration/config.yaml.tmpl.

The main thing to do is to:

  • Set the resolutions we want to generate in the grids. If we want to generate different resolutions per layer, we should create different grids. Sub-level of grids is the grid name.

  • Configure the caches and set the generation/default_cache. Sub-level of caches is the cache name.

  • Configure the layer_default, the layers, and the generation/default_layers. Sub-level of layers is the layer name.

  • We can drop the empty tiles with a hash comparison, tilecloud-chain has a tool to help us:

    docker-compose exec tilecloudchain generate_tiles \
       --get-hash <max-zoom>/0/0 --layer <layer>
    

    We consider that the first tile of the max zoom is empty. Then copy-paste the result in the layer config.

If you generate the tiles locally, you do not need all the configuration variables, because many of them in the generation part are for AWS generation.

Note

Points to check with TileCloud chain:

  • Disabling metatiles should be avoided.

  • Make sure that empty_metatile_detection and empty_tile_detection are configured correctly.

  • Make sure to not generate tiles with a resolution higher than the one in the raster sources.

Tile Generation and management

This package offers two tools, one to generate the tiles locally, see help:

docker-compose exec tilecloudchain generate_tiles --help

one to generate the tiles using AWS, see help:

docker-compose exec tilecloudchain generate_controller --help

Before starting a tile generation on S3, measure the cost:

docker-compose exec tilecloudchain generate_controller --cost

If you setup all the default options, you can generate the tiles by using the command:

docker-compose exec tilecloudchain generate_tiles

AWS credentials

To be able to connect to the S3 service, you should define the following variables in the env.project file:

code:

AWS_ACCESS_KEY_ID=<access_key_id>
AWS_SECRET_ACCESS_KEY=<secret_access_key>

If you do not want to commit these credentials you can add them in your ~/.bashrc file:

code:

export AWS_ACCESS_KEY_ID=<access_key_id>
export AWS_SECRET_ACCESS_KEY=<secret_access_key>

Web Interface

It is possible to run tile generation commands through a web interface located at URL <application main URL>/tiles/admin/. This interface is protected by a password that must be specified in the C2C_SECRET environment variable in the env.project file.

Predefined commands may be set in parameter server > predefined_commands in the tilegeneration/config.yaml.tmpl file.

See also

Upstream documentation

Configure

Configure grids

The grid describes how the tiles are arranged.

Especially on s3 or azure be careful to choose every of the grid settings before generating the tiles. If you change one of them you must regenerate all the tiles.

The resolutions in [px/m] describes all the resolutions available for this layer. For a raster layer, have a look to the maximum resolution of the source files. It is not needed to generate tiles at smaller resolutions than the sources, it is preferable to use the OpenLayers client zoom. Note that you can add a resolution in the end without regenerating all the tiles.

The bbox should match the resolution of the extent. CAREFUL: you will have big issue if you use this parameter to generate the tile on a restricted area: use the bbox on the layer instead.

The srs specifies the code of the projection.

The unit is the unit used by the projection.

The tile_size is the tile size in [px], defaults to 256.

The matrix_identifier is zoom by default and can also be set to resolution. It specifies how the z index is build to store the tiles, for example, for the resolutions [2, 1, 0.5] the used values are [0, 1, 2] based on the zoom and [2, 1, 0_5] based on the resolution. The second has the advantage of allowing to add a new resolution without regenerating all the tiles, but it does not work with MapCache.

Configure caches

The available tile caches are: s3, azure, bsddb, mbtile and filesystem.

The best solution to store the tiles, s3, azure, mbtiles and bsddb, have the advantage of using only one file per layer - style dimensions. To serve the mbtile and the bsddb see Distribute the tiles.

s3 needs a bucket and a folder (defaults to ‘’).

azure needs a container.

mbtiles, bsddb and filesystem just need a folder.

On all the caches we can add some information to generate the URL where the tiles are available. This is needed to generate the capabilities. We can specify:

  • http_url direct url to the tiles root.

  • http_urls (array) urls to the tiles root.

  • http_url and hosts (array), where each value of hosts is used to replace %(host)s in http_url.

In all case http_url or http_urls can include all attributes of this cache as %(attribute)s.

MBTiles vs Berkeley DB (bsddb)

  • Read performance: similar, eventually the MBTiles is 10% faster.

  • Write performance: The Berkeley DB is largely faster, about 10 times.

  • List the tiles: the MBTiles is largely faster, but we usually don’t need it.

Configure layers

First, all the attributes in layer_default are copied in all the layers to define the default values.

We have two type of layer: wms or mapnik.

To start the common attributes are:

min_resolution_seed the minimum resolution that is seeded, other resolutions are served by MapCache.

bbox used to limit the tiles generation.

px_buffer a buffer in px around the object area (geoms or extent).

WMTS layout

To generate the file paths and the WMTS capabilities we need additional information:

The mime_type of the tiles, it’s also used by the WMS GetMap and to upload the tiles.

The wmts_style defaults to ‘default’.

The extension is used to end the filename.

The dimensions (defaults to []) is an array of objects that have a name, a default value specified in the capabilities, a value to generate the tiles (it can be overwritten by an argument), and an array of values that contains all the possible values available in the capabilities.

For example if you generate the tiles and capabilities with the following configuration:

dimensions:
    -   name: DATE
        default: 2012
        value: 2012
        values: [2012]

then with the following configuration:

dimensions:
    -   name: DATE
        default: 2012
        value: 2013
        values: [2012, 2013]

We will have two set of tiles 2012 and 2013, both accessible by the capabilities, and by default we will see the first set of tiles.

Meta tiles

The meta tiles are activated by setting meta to on (by default it’s off).

The meta tiles are used for two things: first to generate multiple tiles with only one WMS query. By setting meta_size to 8 we will generate a square of 8 by 8 tiles in one shot.

The second usage of meta tiles is prevent cut label names: this is solved by getting a bigger image and cutting the borders. The meta_buffer should be set to a bigger value than half the size of the longest label.

Configure hash

We can filter tiles and meta tiles by using an hash.

The configuration of this hash is in the layer like this:

empty_metatile_detection:
    size: 740
    hash: 3237839c217b51b8a9644d596982f342f8041546
empty_tile_detection:
    size: 921
    hash: 1e3da153be87a493c4c71198366485f290cad43c

To easily generate this configuration we can use the following command:

generate-tiles --get-hash <z/x/y> -l <layer_name>

Where <z/x/y> should refer to an empty tile/metatile. Generally it’s a good idea to use z as the maximum zoom, x and y as 0.

Configure geom/sql

We can generate the tiles only on some geometries stored in PostGis.

The configuration is in the layer like this:

geoms:
-   connection: user=www-data password=www-data dbname=<db> host=localhost
    sql: <column> AS geom FROM <table>
    min_resolution: <resolution> # included, optional, last win
    max_resolution: <resolution> # included, optional, last win

Example:

geoms:
-   connection: user=postgres password=postgres dbname=tests host=localhost
    sql: the_geom AS geom FROM tests.polygon
-   connection: user=postgres password=postgres dbname=tests host=localhost
    sql: the_geom AS geom FROM tests.point
    min_resolution: 10
    max_resolution: 20

It’s preferable to use simple geometries, too complex geometries can slow down the generation.

Legends

To be able to generate legends with generate-controller --generate-legend-images you should have legend_mime and legend_extension in the layer configuration.

for example:

legend_mime: image/png
legend_extension: png

Then it will create a legend image per layer and per zoom level named .../1.0.0/{{layer}}/{{wmts_style}}/legend{{zoom}}.{{legend_extension}} only if she is different from the previous zoom level. If we have only one legend image it still stores in the file named legend0.{{legend_extension}}.

When we do generate-controller --generate-wmts-capabilities we will at first parse the legend images to generate a layer configuration like this:

legends:
-   mime_type: image/png
    href: http://host/tiles/layer/style/legend0.png
    min_resolution: 500 # optional, [m/px]
    max_resolution: 2000 # optional, [m/px]
    min_scale: # if define overwrite the min_resolution [m/m]
    max_scale: # if define overwrite the max_resolution [m/m]

If you define a legends array in the layer configuration it is directly used to generate the capabilities.

WMS layers

The additional value needed by the WMS is the URL of the server and the layers.

The previously defined mime_type is also used in the WMS requests.

To customize the request you also have the attributes params, headers and generate_salt. In params you can specify additional parameter of the WMS request, in headers you can modify the request headers. In version, you can change the WMS version. See the Proxy/cache issue for additional information.

Mapnik layers

We need to specify the mapfile path.

With Mapnik we have the possibility to specify a data_buffer then we should set the unneeded meta_buffer to 0.

And the output_format used for the Mapnik renderer, can be png, png256, jpeg, grid (grid_renderer).

Mapnik grid layers

With Mapnik we can generate UTFGrid tiles (JSON format that describes the tiles present on a corresponding tile) by using the output_format ‘grid’, see also: https://github.com/mapnik/mapnik/wiki/MapnikRenderers#grid_renderer.

Specific configuration:

We have a specific way to drop_empty_utfgrid by using the on value.

We should specify the pseudo pixel size [px] with the resolution.

And the layers_fields that we want to get the attributes. Object with the layer name as key and the values in an array as value.

In fact the Mapnik documentation says that’s working only for one layer.

And don’t forget to change the extension to json, and the mime_type to application/utfgrid and the meta to off (not supported).

Configuration example:

grid:
    type: mapnik
    mapfile: style.mapnik
    output_format: grid
    extension: json
    mime_type: application/utfgrid
    drop_empty_utfgrid: on
    resolution: 4
    meta: off
    data_buffer: 128
    layers_fields:
        buildings: [name, street]

Process

We can configure some tile commands to process the tiles. They can be automatically be called in the tile generation it we set the property post_process or pre_hash_post_process in the layer configuration.

The process is a set of names processes, and each one has a list of commands declared like this:

process:  # root process config
    optipng:  # the process command
    -   cmd: optipng %(args)s -q -zc9 -zm8 -zs3 -f5 -o %(out)s %(in)s  # the command line
        need_out: true  # if false the command rewrite the input file, default is false
        arg:  # argument used with the different log switches, in all cases default is ''
            default: '-q' # the argument used by default
            quiet: '-q' # the argument used in quiet mode
            verbose: '-v' # the argument used in verbose mode
            debug: '-log /tmp/optipng.log' # the argument user in debug mode

The cmd can have the following optional argument:

  • args the argument configured in the arg section.

  • in, out the input and output files.

  • x, y, z the tile coordinates.

Logging

Tile logs can be saved to a PostgreSQL database with this configuration:

..code:: yaml

logging:
database:

dbname: my_db host: db port: 5432 table: tilecloud_logs

PostgreSQL authentication can be specified with the PGUSER and PGPASSWORD environment variables. If the database is not reachable, the process will wait until it is.

Tiles error file

If we set a file path in configuration file:

generation:
    error_file: <path>

The tiles that’s in error will be append to the file, ant the tiles can be regenerated with generate-tiles --tiles <path>.

The <path> can be /tmp/error_{layer}_{datetime:%Y-%m-%d_%H:%M:%S} to have one file per layer and per run.

The tiles file looks like:

{.sourceCode .} # [time] some comments z/x/y # [time] the error z/x/y:+m/+m # [time] the error

The first line is just a comment, the second, is for an error on a tile, and the third is for an error on a meta tile.

Proxy/cache issue

In general we shouldn’t generate tiles throw a proxy, to do that you should configure the layers as this:

layers_name:
    url: http://localhost/wms
    headers:
        Host: the_host_name

The idea is to get the WMS server on localhost and use the Host header to select the right Apache VirtualHost.

To don’t have cache we use the as default the headers:

headers:
    Cache-Control: no-cache, no-store
    Pragma: no-cache

And if you steal have issue you can add a SALT random argument by setting the layer parameter generate_salt to true.

Alternate mime type

By default TileCloud support only the image/jpeg and image/png mime type.

Amazon services

Authentication

To be authenticated by Amazon you should set those environments variable before running a command:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...

Configure S3

The cache configuration is like this:

s3:
    type: s3
    # the s3 bucket name
    bucket: tiles
    # the used folder in the bucket [default is '']
    folder: ''
    # for GetCapabilities
    http_url: https://%(host)s/%(bucket)s/%(folder)s/
    cache_control: 'public, max-age=14400'
    hosts:
    - wmts0.<host>

The bucket should already exists. If you don’t use Amazon’s S3, you must specify the host and the tiles_url configuration parameter.

Configure SQS

The configuration in layer is like this:

sqs:
    # The region where the SQS queue is
    region: eu-west-1
    # The SQS queue name, it should already exists
    queue: the_name

The queue should be used only by one layer.

To use the SQS queue we should first fill the queue:

generate-tiles --role master --layer <a_layer>

And then generate the tiles present in the SQS queue:

generate-tiles --role slave --layer <a_layer>

For the slave to keep listening when the queue is empty and be able to support more than one layer, you must enable the daemon mode and must not specify the layer:

generate-tiles --role slave --daemon

Configure SNS

SNS can be used to send a message when the generation ends.

The configuration is like this:

sns:
    topic: arn:aws:sns:eu-west-1:your-account-id:tilecloud
    region: eu-west-1

The topic should already exists.

Amazon tool

Amazon has a command line tool (homepage).

To use it, add in the setup.py:

  • awscli as an install_requires,

  • 'aws = awscli.clidriver:main', in the console_scripts.

Than install it:

pip install awscli

And use it:

aws help

For example to delete many tiles do:

aws s3 rm --recursive s3://your_bucket_name/folder

Configure Azure

The cache configuration is like this:

azure:
    type: azure
    # the Azure container name
    container: tiles
    # the used folder in the container [default is '']
    folder: ''
    # for GetCapabilities
    http_url: https://%(host)s/%(bucket)s/%(folder)s/
    cache_control: 'public, max-age=14400'
    hosts:
    - wmts0.<host>

The container should already exists.

For the authentication you should set those environment variables: AZURE_STORAGE_CONNECTION_STRING on your local environment, or AZURE_STORAGE_ACCOUNT_URL if you run your container on Azure.

Configure the server

The server can be configure as it:

server:
    layers: a_layer # Restrict to serve an certain number of layers [default is all]
    cache: mbtiles # The used cache [default use generation/default_cache]
    # the URL without location to MapCache, [default is http://localhost/]
    geoms_redirect: true # use the geoms to redirect to MapCache [default is false]
    # allowed extension in the static path (default value), not used for s3.
    static_allow_extension: [jpeg, png, xml, js, html, css]

The minimal configuration is to enable it:

server: {}

You should also configure the http_url of the used cache, to something like https://%(host)s/${instanceid}/tiles or like https://%(host)s/${instanceid}/wsgi/tiles if you use the Pyramid view.

Pyramid view

To use the pyramid view use the following configuration:

config.get_settings().update({
    'tilegeneration_configfile': '<the configuration file>',
})
config.add_route('tiles', '/tiles/\*path')
config.add_view('tilecloud_chain.server:PyramidView', route_name='tiles')

Internal WSGI server

in production.ini:

[app:tiles]
use = egg:tilecloud_chain#server
configfile = %(here)s/tilegeneration/config.yaml

with the Apache configuration:

WSGIDaemonProcess tiles:${instanceid} display-name=%{GROUP} user=${modwsgi_user}
WSGIScriptAlias /${instanceid}/tiles ${directory}/apache/wmts.wsgi
<Location /${instanceid}/tiles>
    WSGIProcessGroup tiles:${instanceid}
    WSGIApplicationGroup %{GLOBAL}
</Location>

Commands

Available commands

  • generate-controller generate the annex files like legend.

  • generate-tiles generate the tiles.

  • generate-copy copy the tiles from a cache to an other.

  • generate-process process the tiles using a configured process.

  • generate-cost estimate the cost.

  • import_expiretiles import the osm2pgsql expire-tiles file as geoms in the database.

Each commands have a --help option to give a full arguments help.

Generate tiles

Generate all the tiles:

generate-tiles

Generate a specific layer:

generate-tiles --layer <a_layer>

Generate a specific zoom:

generate-tiles --zoom 5

Generate a specific zoom range:

generate-tiles --zoom 2-8

Generate a specific some zoom levels:

generate-tiles --zoom 2,4,7

Generate tiles from an (error) tiles file:

generate-tiles --layer <a_layer> --tiles <z/x/y>

Generate tiles on a bbox:

generate-tiles --bbox <MINX> <MINY> <MAXX> <MAXY>

Generate a tiles near a tile coordinate (useful for test):

generate-tiles --near <X> <Y>

Generate a tiles in a different cache than the default one:

generate-tiles --cache <a_cache>

Explain cost

Configuration (default values):

cost:
    # [nb/month]
    request_per_layers: 10000000
    cloudfront:
        download: 0.12,
        get: 0.009
    request_per_layers: 10000000
    s3:
        download: 0.12,
        get: 0.01,
        put: 0.01,
        storage: 0.125
    sqs:
        request: 0.01

Layer configuration (default values):

cost:
    metatile_generation_time: 30.0,
    tile_generation_time: 30.0,
    tile_size: 20.0,
    tileonly_generation_time: 60.0

The following commands can be used to know the time and cost to do generation:

generate-controller --cost

Useful options

--quiet or -q: used to display only errors.

--verbose or -v: used to display info messages.

--debug or -d: used to display debug message, please use this option to report issue. With the debug mode we don’t catch exceptions, and we don’t log time messages.

--test <n> or -t <n>: used to generate only <n> tiles, useful for test.

Mutualized

The mutualized mode consist by having multiple project files with the projects related configurations (layers, cache, …) and one main configuration file with the global configuration (number of process, log format, redis, …).

Configuration keys which should be set in the main configuration file are identified in property’s descriptions of the schema.json file.

Important remarks

Especially on S3 the grid name, the layer name, the dimensions, can’t be changed (understand if we want to change them we should regenerate all the tiles).

By default we also can’t insert a zoom level, if you think that you need it we can set the grid property matrix_identifier: resolution, bit it don’t work with MapCache.

Please use the --debug to report issue.

Environment variables

  • TILEGENERATION_CONFIGFILE: Default to /etc/tilegeneration/config.yaml, the all in one configuration file to use.

  • TILEGENERATION_MAIN_CONFIGFILE: Default to /etc/tilegeneration/config.yaml, the main configuration file to use.

  • TILEGENERATION_HOSTSFILE: Default to /etc/tilegeneration/hosts.yaml, the hosts to config file mapping file to use.

  • TILE_NB_THREAD: Default is 2, the number of thread used to generate the tiles (If we use meta tiles)

  • METATILE_NB_THREAD: Default is 25, the number of thread used to generate the meta tiles (If we use meta tiles, also to generate the tiles)

  • SERVER_NB_THREAD: Default to 10, the number of thread used to generate the meta tiles in the server.

  • TILE_QUEUE_SIZE: Default to 2, the queue size just after the Redis queue

  • TILE_CHUNK_SIZE: Default to 1, the chunk size to process the tiles after the meta tiles.

  • TILECLOUD_CHAIN_MAX_OUTPUT_LENGTH: Default to 1000, the maximum number of character of the output to be display in the admin interface.

  • LOG_TYPE: Default to console, can also be json to log in JSON for Logstash.

  • TILECLOUD_CHAIN_LOG_LEVEL Default to INFO, TILECLOUD_LOG_LEVEL Default to INFO, C2CWSGI_LOG_LEVEL Default to WARN, OTHER_LOG_LEVEL Default to WARN, the logging level of deferent components, can be DEBUG, INFO, WARN, ERROR or CRITICAL.

  • TILE_SERVER_LOGLEVEL Default to quiet the log level used in the server part.

  • TILE_MAPCACHE_LOGLEVEL``Default to ``verbose the log level used in the internal mapcache.

  • DEVELOPMENT: Default to 0 set it to 1 to have the Pyramid development options.

  • VISIBLE_ENTRY_POINT Default to /tiles/ the entrypoint path.

Admin and test pages

On the URL <base URL>/admin/ you can see the status of the generation, a tool to generate the tiles, and a link to a test page.

Beware, the test page assumes we have configured only one grid.

TileCloud-chain configuration

https://raw.githubusercontent.com/camptocamp/tilecloud-chain/master/tilecloud_chain/schema.json

type

object

properties

  • defaults

Defaults

Used to put YAML references

type

object

  • grids

Grids

The WMTS grid definitions by grid name, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-grids

type

object

additionalProperties

Grid

  • caches

Caches

The tiles caches definitions by name, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-caches

type

object

additionalProperties

Cache

  • layers

Layers

The layers definitions by name, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-layers

type

object

additionalProperties

Layer

  • process

Process

List of available commands by name, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#process

type

object

additionalProperties

Process command

  • generation

Generation

  • sqs

SQS

The Simple Queue Service configuration

type

object

properties

  • queue

Queue

The queue name, default is ‘tilecloud’

type

string

  • region

AWS region

additionalProperties

False

  • sns

SNS

The Simple Notification Service configuration, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-sns

type

object

properties

  • topic

Topic

The topic

type

string

  • region

AWS region

additionalProperties

False

  • redis

Redis

  • openlayers

OpenLayers

Configuration used to generate the OpenLayers example page

type

object

properties

  • srs

SRS

The projection code

type

string

pattern

^EPSG:[0-9]+$

default

EPSG:2056

  • proj4js_def

The proj4js definition

The proj4js definition, get it from https://epsg.io/

type

string

default

+proj=somerc +lat_0=46.95240555555556 +lon_0=7.439583333333333 +k_0=1 +x_0=2600000 +y_0=1200000 +ellps=bessel +towgs84=674.374,15.056,405.346,0,0,0,0 +units=m +no_defs

  • center_x

Center x

The center easting

type

number

default

2600000

  • center_y

Center y

The center northing

type

number

default

1200000

  • zoom

Map initial zoom

The initial zoom

type

number

default

3

additionalProperties

False

  • server

Server

  • cost

Cost

  • metadata

Metadata

  • provider

Provider

  • logging

`Logging`_

  • authentication

Authentication

The authentication configuration

type

object

properties

  • github_repository

GitHub repository

The GitHub repository name, on witch one we will check the access rights

type

string

  • github_access

GitHub access

The kind of rights the user should have on the repository

type

string

enum

push, pull, admin

default

pull

additionalProperties

False

additionalProperties

False

Headers

The headers that we send to the WMS backend

type

object

additionalProperties

The header value

type

string

Grid

The WMTS grid definition, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-grids

type

object

properties

  • resolution_scale

Resolution scale

The scale used to build a FreeTileGrid typically ‘2’

type

integer

  • resolutions

Resolutions

The resolutions in pixel per meter

type

array

items

type

number

  • bbox

Bounding box

The bounding box in meter

type

array

maxLength

4

minLength

4

items

type

number

  • srs

SRS

The projection reference

type

string

pattern

^EPSG:[0-9]+$

  • proj4_literal

Proj4 literal

The Proj4 definition

type

string

  • unit

Unit

The projection unit, default is ‘m’

type

string

default

m

  • tile_size

Tile size

The tile size in pixel, default is 256

type

integer

default

256

  • matrix_identifier

Matrix identifier

The identifier to use in the tiles URL, recommend to be resolution (default)

type

string

enum

zoom, resolution

default

zoom

additionalProperties

False

Cache WMST capabilities file

The generated WMTS capabilities file name (by ‘default 1.0.0/WMTSCapabilities.xml’)

type

string

Cache HTTP URL

The HTTP URL %host will be replaces by one of the hosts

type

string

Cache host

The host used to build the HTTP URLs

type

array

items

type

string

Cache HTTP URLs

type

array

items

type

string

Cache folder

The root folder of the cache, default is ‘’

type

string

default

Cache filesystem

type

object

properties

  • type

const

filesystem

  • wmtscapabilities_file

Cache WMST capabilities file

  • http_url

Cache HTTP URL

  • hosts

Cache host

  • http_urls

Cache HTTP URLs

  • folder

Cache folder

additionalProperties

type

string

Cache S3

type

object

properties

  • type

const

s3

  • wmtscapabilities_file

Cache WMST capabilities file

  • http_url

Cache HTTP URL

  • hosts

Cache host

  • http_urls

Cache HTTP URLs

  • tiles_url

Tiles URL

The template tiles URL on S3, the argument can be region, bucket and folder (default is ‘http://s3-{region}.amazonaws.com/{bucket}/{folder}’)

type

string

  • host

Host

The S3 host, default is ‘s3-eu-west-1.amazonaws.com’

type

string

  • bucket

Bucket

The S3 bucker name

type

string

  • region

AWS region

  • cache_control

Cache control

The Cache-Control used to store tiles on S3

type

string

  • folder

Cache folder

additionalProperties

type

string

Cache Azure

Azure Blob Storage

type

object

properties

  • type

const

azure

  • wmtscapabilities_file

Cache WMST capabilities file

  • http_url

Cache HTTP URL

  • hosts

Cache host

  • http_urls

Cache HTTP URLs

  • folder

Cache folder

  • container

Container

The Azure container name

type

string

  • cache_control

Cache control

The Cache-Control used to store tiles on Azure

type

string

additionalProperties

type

string

Cache MBtiles

type

object

properties

  • type

const

mbtiles

  • wmtscapabilities_file

Cache WMST capabilities file

  • http_url

Cache HTTP URL

  • hosts

Cache host

  • http_urls

Cache HTTP URLs

  • folder

Cache folder

additionalProperties

type

string

Cache BSDDB

type

object

properties

  • type

const

bsddb

  • wmtscapabilities_file

Cache WMST capabilities file

  • http_url

Cache HTTP URL

  • hosts

Cache host

  • http_urls

Cache HTTP URLs

  • folder

Cache folder

additionalProperties

type

string

Cache

The tiles cache definition, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-caches

anyOf

Cache filesystem

Cache S3

Cache Azure

Cache MBtiles

Cache BSDDB

Layer title

The title, use to generate the capabilities

type

string

Layer grid

The used grid name

type

string

Layer bounding box

The bounding box where we will generate the tiles

type

array

maxLength

4

minLength

4

items

type

number

layer min resolution seed

The minimum resolutions to pre-generate

type

number

Layer pixel buffer

The buffer in pixel used to calculate geometry intersection, default is 0

type

integer

default

0

Layer meta

Use meta-tiles, default is False, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#meta-tiles

type

boolean

default

False

Layer meta size

The meta-tile size in tiles, default is 5

type

integer

Layer meta buffer

The meta-tiles buffer in pixels, default is 128

type

integer

Layer layers

The WMS layers

type

string

Layer WMTS style

The WMTS style

type

string

pattern

^[a-zA-Z0-9_\-\+~\.]+$

Layer MIME type

The MIME type of the tiles

type

string

pattern

^[a-zA-Z0-9!#$%^&\*_\-\+{}\|’.`~]+/[a-zA-Z0-9!#$%^&\*_\-\+{}\|’.`~]+$

Layer extension

The layer extension

type

string

Layer dimension name

The dimension name

type

string

pattern

^(?!(?i)(SERVICE|VERSION|REQUEST|LAYERS|STYLES|SRS|CRS|BBOX|WIDTH|HEIGHT|FORMAT|BGCOLOR|TRANSPARENT|SLD|EXCEPTIONS|SALT))[a-z0-9_\-~\.]+$

layer dimensions

The WMTS dimensions

type

array

items

type

object

properties

  • name

Layer dimension name

  • generate

Generate

The values that should be generate

type

array

items

type

string

pattern

^[a-zA-Z0-9_\-\+~\.]+$

  • values

Values

The values present in the capabilities

type

array

items

type

string

pattern

^[a-zA-Z0-9_\-\+~\.]+$

  • default

Default

The default value present in the capabilities

type

string

pattern

^[a-zA-Z0-9_\-\+~\.]+$

additionalProperties

False

Layer legends

The provided legend

type

array

items

type

object

properties

  • mime_type

MIME type

The mime type used in the WMS request

type

string

pattern

^[a-zA-Z0-9!#$%^&\*_\-\+{}\|’.`~]+/[a-zA-Z0-9!#$%^&\*_\-\+{}\|’.`~]+$

  • href

Href

The URL of the legend image

type

string

  • width

Width

The width of the legend image

type

string

  • height

Height

The height of the legend image

type

string

  • min_scale

Min scale

The max scale of the legend image

type

string

  • max_scale

Max scale

The max scale of the legend image

type

string

  • min_resolution

Min resolution

The max resolution of the legend image

type

string

  • max_resolution

Max resolution

The max resolution of the legend image

type

string

additionalProperties

False

Layer legend MIME

The mime type used to store the generated legend

type

string

pattern

^[a-zA-Z0-9!#$%^&\*_\-\+{}\|’.`~]+/[a-zA-Z0-9!#$%^&\*_\-\+{}\|’.`~]+$

Layer legend extension

The extension used to store the generated legend

type

string

pattern

^[a-zA-Z0-9]+$

Layer pre hash post process

Do an image post process before the empty hash check

type

string

Layer post process

Do an image post process after the empty hash check

type

string

Layer geometries

The geometries used to determine where we should create the tiles, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-geomsql

type

array

items

type

object

properties

  • connection

Connection

The PostgreSQL connection string

type

string

  • sql

SQL

The SQL query that get the geometry in geom e.g. ‘the_geom AS geom FROM my_table’

type

string

  • min_resolution

Min resolution

The min resolution where the query is valid

type

number

  • max_resolution

Max resolution

The max resolution where the query is valid

type

number

additionalProperties

False

Layer empty tile detection

The rules used to detect the empty tiles, use generate-tiles –get-hash to get what we can use, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-hash

type

object

properties

  • size

Title

The tile size

type

integer

  • hash

Hash

The tile hash

type

string

additionalProperties

False

Layer empty meta-tile detection

The rules used to detect the empty meta-tiles, use generate-tiles –get-hash to get what we can use, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-hash

type

object

properties

  • size

Size

The meta-tile size

type

integer

  • hash

Hash

The meta-tile hash

type

string

additionalProperties

False

Layer cost

The rules used to calculate the cost

type

object

properties

  • tileonly_generation_time

tile only generation time

The time to generate a tile without meta-tile

type

number

default

40

  • tile_generation_time

tile generation time

The time to generate a tile from the meta-tile

type

number

default

30

  • metatile_generation_time

Meta tile generation time

The time to generate a meta-tile

type

number

default

30

  • tile_size

Tile size

The tile mean size in bytes

type

number

default

20

additionalProperties

False

Layer WMS

type

object

properties

  • type

const

wms

  • title

Layer title

  • grid

Layer grid

  • bbox

Layer bounding box

  • min_resolution_seed

layer min resolution seed

  • px_buffer

Layer pixel buffer

  • meta

Layer meta

  • meta_size

default

5

Layer meta size

  • meta_buffer

default

128

Layer meta buffer

  • layers

Layer layers

  • wmts_style

Layer WMTS style

  • mime_type

Layer MIME type

  • extension

Layer extension

  • dimensions

layer dimensions

  • legends

Layer legends

  • legend_mime

Layer legend MIME

  • legend_extension

Layer legend extension

  • pre_hash_post_process

Layer pre hash post process

  • post_process

Layer post process

  • geoms

Layer geometries

  • empty_tile_detection

Layer empty tile detection

  • empty_metatile_detection

Layer empty meta-tile detection

  • cost

Layer cost

  • url

URL

The WMS service URL

type

string

  • generate_salt

Generate salt

Should generate a salt to drop the cache, default is False

type

boolean

  • query_layers

Query layers

The layers use for query (To be used with the server)

type

string

  • info_formats

Info formats

The query info format

type

array

items

type

string

  • params

Parameters

Additional parameters to the WMS query (like dimension)

type

object

additionalProperties

The parameter value

type

string

  • headers

Headers

  • version

Version

The used WMS version (default is ‘1.1.1’)

type

string

additionalProperties

False

Layer Mapnik

type

object

properties

  • type

const

mapnik

  • title

Layer title

  • grid

Layer grid

  • bbox

Layer bounding box

  • min_resolution_seed

layer min resolution seed

  • px_buffer

Layer pixel buffer

  • meta

Layer meta

  • meta_size

default

1

Layer meta size

  • meta_buffer

default

0

Layer meta buffer

  • layers

default

__all__

Layer layers

  • wmts_style

Layer WMTS style

  • mime_type

Layer MIME type

  • extension

Layer extension

  • dimensions

layer dimensions

  • legends

Layer legends

  • legend_mime

Layer legend MIME

  • legend_extension

Layer legend extension

  • pre_hash_post_process

Layer pre hash post process

  • post_process

Layer post process

  • geoms

Layer geometries

  • empty_tile_detection

Layer empty tile detection

  • empty_metatile_detection

Layer empty meta-tile detection

  • cost

Layer cost

  • mapfile

MapFile

The Mapnik map file

type

string

  • data_buffer

Data buffer

The data buffer, default is 128

type

integer

  • output_format

Output format

The Mapnik output format, default is ‘png’

type

string

enum

png, png256, jpeg, grid

  • wms_url

WMS URL

A WMS fallback URL (deprecated)

type

string

  • resolution

Resolution

The resolution, default is 4

type

integer

  • layers_fields

Layers fields

The Mapnik layers fields

type

object

additionalProperties

The Mapnik layer fields

type

array

items

type

string

  • drop_empty_utfgrid

Drop empty UTFGrid

Drop if the tile is empty, default is False

type

boolean

additionalProperties

False

Layer

The layer definition, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-layers

anyOf

layer_wms

layer_mapnik

Process command

A command

type

array

items

type

object

properties

  • cmd

Command

The shell command, available parameters: %(in)s, %(out)s, %(args)s, %(x)s, %(y)s, %(z)s.

type

string

  • need_out

Need out

The command will generate an output in a file, default is False

type

boolean

default

False

  • arg

Argument

Used to build the %(args)

type

object

properties

  • default

Properties

The arguments used by default

type

string

  • verbose

Verbose

The arguments used on verbose mode

type

string

  • debug

Debug

The arguments used on debug mode

type

string

  • quiet

Quiet

The arguments used on quiet mode

type

string

additionalProperties

False

additionalProperties

False

Generation

The configuration used for the generation

type

object

properties

  • default_cache

Default cache

The default cache name to be used, default do ‘default’

type

string

default

default

  • default_layers

Default layers

The default layers to be generated

type

array

items

type

string

  • authorised_user

Authorised user

The authorised user to generate the tiles (used to avoid permission issue on generated tiles) (main configuration)

type

string

  • maxconsecutive_errors

Max consecutive errors

The maximum number of consecutive errors (main configuration), default is 10

type

integer

default

10

  • error_file

Error file

File name generated with the tiles in error, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#tiles-error-file (main configuration)

type

string

  • number_process

Number process

Number of process used to generate the tiles (main configuration), default is 1

type

integer

default

1

additionalProperties

False

AWS region

The region, default is ‘eu-west-1’

type

string

pattern

^(eu|us|ap|sa)-(north|south|east|west|central)(east|west)?-[1-3]$

Redis

The Redis configuration (main configuration)

type

object

properties

  • url

URL

The server URL (main configuration)

type

string

pattern

^redis://[^:]+:[^:]+$

  • sentinels

Sentinels

The sentinels (main configuration)

type

array

items

A sentinel (main configuration)

type

array

items

Sentinel host

The sentinel host name (main configuration)

type

string

Sentinel port

The sentinel port (main configuration)

anyOf

type

string

type

integer

additionalItems

False

  • connection_kwargs

The Redis connection arguments (main configuration)

type

object

  • sentinel_kwargs

The Redis sentinel arguments (main configuration)

type

object

  • service_name

Service name

The service name, default is ‘mymaster’ (main configuration)

type

string

default

mymaster

  • socket_timeout

Socket timeout

The socket timeout (main configuration)

type

integer

  • db

Database

type

integer

  • queue

Queue

The queue name (main configuration)

type

string

default

tilecloud

  • timeout

Timeout

The timeout (main configuration), default is 5

type

integer

default

5

  • pending_timeout

Pending timeout

The pending timeout (main configuration), default is 300

type

integer

default

300

  • max_retries

Max retries

The max retries (main configuration), default is 5

type

integer

default

5

  • max_errors_age

Max errors age

The max error age (main configuration), default is 86400 (1 day)

type

integer

default

86400

  • max_errors_nb

Max errors number

The max error number (main configuration), default is 100

type

integer

default

100

  • prefix

Prefix

The prefix (main configuration), default is ‘tilecloud_cache’

type

string

default

tilecloud_cache

  • expiration

Expiration

The meta-tile in queue expiration (main configuration), default is 28800 (8 hours)

type

integer

default

28800

additionalProperties

False

Server

Configuration used by the tile server, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#distribute-the-tiles

type

object

properties

  • cache

Cache

The used cache name

type

string

  • layers

WMS Layers

Layers available in the server, default is all layers

type

array

items

type

string

  • geoms_redirect

Geometries redirect

Take care on the geometries, default is False

type

boolean

default

False

  • static_allow_extension

Static allow extension

The allowed extension of static files, defaults to [jpeg, png, xml, js, html, css]

type

array

items

type

string

  • wmts_path

WMTS path

The sub-path for the WMTS (main configuration), default is ‘wmts’

type

string

default

wmts

  • static_path

Static path

The sub-path for the static files (main configuration), default is ‘static’

type

string

default

static

  • admin_path

Admin path

The sub-path for the admin (main configuration), default is ‘admin’

type

string

default

admin

  • expires

Expires

The browser cache expiration, default is 8 (hours)

type

integer

default

8

  • predefined_commands

Predefined commands

The predefined commands used to generate the tiles

type

array

items

type

object

properties

  • command

Command

The command to run

type

string

  • name

Name

The name used in the admin interface

type

string

additionalProperties

False

  • allowed_commands

Allowed commands

The allowed commands (main configuration)

type

array

items

type

string

  • allowed_arguments

Allowed arguments

The allowed arguments (main configuration)

type

array

items

type

string

additionalProperties

False

Cost

The configuration use to calculate the cast (unmaintained)

type

object

properties

  • request_per_layers

Request per layers

Tile request per hours, default is 10 000 000

type

integer

default

10000000

  • s3

S3 cost

The S3 cost (main configuration)

type

object

properties

  • storage

Storage

The storage cost in $ / Gio / month (main configuration)

type

number

default

0.125

  • put

Put

The cost of put in $ per 10 000 requests (main configuration)

type

number

default

0.01

  • get

Get

The cost of get in $ per 10 000 requests (main configuration)

type

number

default

0.01

  • download

Download

The cost of download in $ per Gio (main configuration)

type

number

default

0.12

additionalProperties

False

  • cloudfront

CloudFront cost

The CloudFront cost (main configuration)

type

object

properties

  • get

Get

The cost of get in $ per 10 000 requests (main configuration)

type

number

default

0.009

  • download

Download

The cost of download in $ per Gio (main configuration)

type

number

default

0.12

additionalProperties

False

  • sqs

SQS cost

The SQS cost, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#configure-sqs (main configuration)

type

object

properties

  • request

Request

The cost of request in $ per 1 000 000 requests (main configuration)

type

number

default

0.01

additionalProperties

False

additionalProperties

False

Metadata

The configuration of the WMTS capabilities metadata

type

object

properties

  • title

Title

The title

type

string

  • abstract

Abstract

The abstract

type

string

  • servicetype

Service type

The service type, default is ‘OGC WMTS’

type

string

default

OGC WMTS

  • keywords

Keywords

The keywords

type

array

items

type

string

  • fees

Fees

The fees

type

string

  • access_constraints

Access constraints

The access constraints

type

string

additionalProperties

False

Provider

The provider

type

object

properties

  • name

Name

type

string

  • url

URL

The public URL

type

string

  • contact

Contact

The contact

type

object

properties

  • name

Name

type

string

  • position

Position

type

string

  • info

Info

The information

type

object

properties

  • phone

Phone

The phone

type

object

properties

  • voice

Voice

The voice number

type

string

  • fax

Fax

The fax number

type

string

additionalProperties

False

  • address

Address

The address

type

object

properties

  • delivery

Delivery

The delivery

type

string

  • city

City

The city

type

string

  • area

Area

The area

type

string

  • postal_code

Postal code

The postal code

type

integer

  • country

Country

The country

type

string

  • email

Email

The email

type

string

additionalProperties

False

additionalProperties

False

additionalProperties

False

additionalProperties

False

Logging

The logging configuration to database, see https://github.com/camptocamp/tilecloud-chain/blob/master/tilecloud_chain/USAGE.rst#logging (main configuration)

type

object

properties

  • database

Database

The database (main configuration)

type

object

properties

  • host

Host

The host (main configuration)

type

string

  • port

Port

The port (main configuration), default is 5432

type

integer

default

5432

  • dbname

Database

The database name (main configuration)

type

string

  • table

Table

The table name (main configuration)

type

string

  • user

User

The user name (main configuration)

type

string

  • password

Password

The password (main configuration)

type

string

additionalProperties

False

additionalProperties

False