Populating the database

Data install or update

Populate your database with command:

./manage.py cities_light

By default, this command attempts to do the least work possible, update what is necessary only. If you want to disable all these optimisations/skips, use –force-all.

This command is well documented, consult the help with:

./manage.py help cities_light

Signals

Signals for this application.

cities_light.signals.city_items_pre_import

Emited by city_import() in the cities_light command for each row parsed in the data file. If a signal reciever raises InvalidItems then it will be skipped.

An example is worth 1000 words: if you want to import only cities from France, USA and Belgium you could do as such:

import cities_light

def filter_city_import(sender, items, **kwargs):
    if items[8] not in ('FR', 'US', 'BE'):
        raise cities_light.InvalidItems()

cities_light.signals.city_items_pre_import.connect(filter_city_import)

Note: this signal gets a list rather than a City instance for performance reasons.

cities_light.signals.region_items_pre_import

Same as city_items_pre_import, for example:

def filter_region_import(sender, items, **kwargs):
    if items[0].split('.')[0] not in ('FR', 'US', 'BE'):
        raise cities_light.InvalidItems()
cities_light.signals.region_items_pre_import.connect(
    filter_region_import)

cities_light.signals.country_items_pre_import

Same as region_items_pre_import and cities_light.signals.city_items_pre_import, for example:

def filter_country_import(sender, items, **args):
    if items[0].split('.')[0] not in ('FR', 'US', 'BE'):
        raise cities_light.InvalidItems()

cities_light.signals.country_items_pre_import.connect(
    filter_country_import)

cities_light.signals.filter_non_cities(sender, items, **kwargs)

Reports non populated places as invalid.

By default, this reciever is connected to city_items_pre_import, it raises InvalidItems if the row doesn’t have PPL in its features (it’s not a populated place).

exception cities_light.exceptions.InvalidItems

The cities_light command will skip item if a city_items_pre_import signal reciever raises this exception.

Configure logging

This command is made to be compatible with background usage like from cron, to keep the database fresh. So it doesn’t do direct output. To get output from this command, simply configure a handler and formatter for cities_light logger. For example:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'handlers': {
        'console':{
            'level':'DEBUG',
            'class':'logging.StreamHandler',
            'formatter': 'simple'
        },
    },
    'loggers': {
        'cities_light': {
            'handlers':['console'],
            'propagate': True,
            'level':'DEBUG',
        },
        # also use this one to see SQL queries
        'django': {
            'handlers':['console'],
            'propagate': True,
            'level':'DEBUG',
        },
    }
}