Server-side development

Create development environment in a project

c2cgeoportal developers often need to test c2cgeoportal changes in the context of an existing c2cgeoportal application. Here is how:

Build the new containers:

git clone git@github.com:camptocamp/c2cgeoportal.git
cd c2cgeoportal
make docker-build

Now, the new containers are ready to use in the application on the same host.

Tests

Running tests

Prerequisite: to be able to run c2cgeoportal tests, you need to have the c2cgeoportal source code, and a make environment for building the library.

c2cgeoportal has two types of tests: unit tests and functional tests. The unit tests are self-contained, and do not require any specific setup. The functional tests require to run with docker-compose-run.

docker-run will:

  • Run the given command in the docker build image.
  • Mount the current directory in /src.
  • Use the current user and home directory.
  • Use a named volume related to the current folder in /build.

docker-compose will also:

  • Create a test database and a test MapServer.

Docker image dependencies:

../_images/docker.png

Image nomenclature:

  • camptocamp/geomapfish-* for all images generated by/for c2cgeoportal.
  • camptocamp/geomapfish-test-* for all images used only for the CI (not pushed on a Docker repository).
  • camptocamp/testgeomapfish-* for all images used by the CI to
    test applications generated using scaffolds.

Unit tests

Before running the tests, install and build all dependencies:

./docker-run make prepare-tests

Run the tests:

./docker-compose-run make tests

To run only a specific test, use the -k switch. For example:

./docker-compose-run py.test -k test_catalogue geoportal/tests

Profiling

At the end of the file apache/application.wsgi, add:

from wsgi_lineprof.middleware import LineProfilerMiddleware
from wsgi_lineprof.filters import FilenameFilter, TotalTimeSorter

filters = [
    FilenameFilter("entry.py"),
    TotalTimeSorter(),
]
application = LineProfilerMiddleware(application, stream=open('/tmp/pro', 'w'), filters=filters)

sudo apache2ctl graceful

Do your request(s).

The profile result will be in the file /tmp/pro.

Upgrade dependencies

When we start a new version of c2cgeoportal, or just before a new development phase, it is a good idea to update the dependencies.

Eggs

All the c2cgeoportal (and tilecloud-chain) dependencies are present in the c2cgeoportal/scaffolds/update/CONST_versions.mako file.

To update them, you can simply get them from a travis build in the ./docker-run pip freeze task.

Submodules

Go to the OpenLayers folder:

cd c2cgeoportal/static/lib/openlayers/

Get the new revision of OpenLayers:

git fetch
git checkout release-<version>

Then you can commit it:

cd -
git add c2cgeoportal/static/lib/openlayers/
git commit -m "update OpenLayers to <version>"

Database

Object model

../_images/database.png

TreeItem and TreeGroup are abstract (cannot be created) classes used to create the tree.

FullTextSearch references a first level LayerGroup, but without any constraints.

It is not visible on this schema, but the User of a child schema has a link (parent_role) to the Role of the parent schema.

metadata vs functionality

The same functionality can be reused by more than one element.

functionalities are designed to configure and customize various parts of the application. For instance, with a functionality, you can make the application change the default basemap when a new theme is loaded.

The metadata contains attributes that are directly related to an element, for example the layer disclaimer.

Migration

We use the alembic module for database migration. alembic works with a so-called migration repository, which is a simple directory /opt/alembic in the docker image. So developers who modify the c2cgeoportal database schema should add migration scripts.

Add a new script call from the application’s root directory:

./docker-compose-run alembic --name=[main|static] revision --message "<Explicit name>"

Or in c2cgeoportal root directory:

./docker-compose-run alembic \
    --config geoportal/tests/functional/alembic.ini --name=[main|static] \
    revision --message "<Explicit name>"

This will generate the migration script in commons/c2cgeoportal/commons/alembic/[main|static]/xxx_<Explicite_name>.py.

To get the project schema, use: schema = context.get_context().config.get_main_option('schema')

The scripts should not fail if they are run more than once. See: http://alembic.readthedocs.org/en/latest/cookbook.html#conditional-migration-elements

Then, customize the migration to suit your needs, test it:

./docker-compose-run alembic upgrade head
More information at:

Subdomain

All the static resources use subdomains by using the configuration variables: subdomain_url_template and subdomains.

To be able to use a subdomain in a view, we should configure the route as follows:

from c2cgeoportal_geoportal.lib import MultiDomainPregenerator
config.add_route(
    '<name>', '<path>',
    pregenerator=MultiDomainPregenerator())

And use the route_url with an additional argument subdomain:

request.route_url('<name>', subdomain='<subdomain>')}",

Code

Coding style

Please read http://www.python.org/dev/peps/pep-0008/.

And run validation:

./docker-run make checks