Consular¶
Consular is a micro-service that relays information between Marathon and Consul. It registers itself for HTTP event callbacks with Marathon and makes the appropriate API calls to register applications that Marathon runs as services in Consul. De-registrations of applications happens in the same way.
Marathon is always considered the source of truth.
If Marathon application definitions contain labels (application metadata) they will be added to the Consul key/value store. This can be especially useful when Consul is combined with Consul Template to automatically generate configuration files for proxies such as HAProxy and Nginx.
Usage¶
$ pip install consular
$ consular --help
Installing for local dev¶
$ git clone https://github.com/universalcore/consular.git
$ cd consular
$ virtualenv ve
$ source ve/bin/activate
(ve)$ pip install -e .
(ve)$ pip install -r requirements-dev.txt
Running tests¶
$ source ve/bin/activate
(ve)$ py.test consular
Consular Class Documentation¶
-
class
consular.main.
Consular
(consul_endpoint, marathon_endpoint, enable_fallback, registration_id)[source]¶ Parameters: - consul_endpoint (str) – The HTTP endpoint for Consul (often http://example.org:8500).
- marathon_endpoint (str) – The HTTP endpoint for Marathon (often http://example.org:8080).
- enable_fallback (bool) – Fallback to the main Consul endpoint for registrations if unable to reach Consul running on the machine running a specific Marathon task.
- registration_id (str) – A unique parameter for this Consul server. It is used for house-keeping purposes such as purging tasks that are no longer running in Marathon.
-
check_apps_namespace_clash
(apps)[source]¶ Checks if app names in Marathon will cause a namespace clash in Consul. Throws an exception if there is a collision, else returns the apps.
Param: apps: The JSON list of apps from Marathon’s API.
-
clean_consul_app_labels
(*args, **kwargs)[source]¶ Delete app labels stored in the Consul k/v store under the given app name that aren’t present in the given set of labels.
-
deregister_consul_service
(agent_endpoint, service_id)[source]¶ Deregister a service from a Consul agent.
Parameters:
-
deregister_task_service
(task_id, host)[source]¶ Deregister a Marathon task’s service from Consul.
Parameters:
-
events
(request)[source]¶ Listens to incoming events from Marathon on
/events
.Parameters: request (klein.app.KleinRequest) – The Klein HTTP request
Get the app ID from the app ID tag in the given tags, or None if the tag could not be found.
-
get_consul_consular_keys
()[source]¶ Get the next level of Consul k/v keys at ‘consular/’, i.e. will return ‘consular/my-app’ but not ‘consular/my-app/my-label’.
-
handle_status_update_event
(request, event)[source]¶ Handles status updates from Marathon.
The various task stages are handled as follows:
TASK_STAGING: ignored TASK_STARTING: ignored TASK_RUNNING: task data updated on Consul TASK_FINISHED: task data removed from Consul TASK_FAILED: task data removed from Consul TASK_KILLED: task data removed from Consul TASK_LOST: task data removed from Consul
Parameters: - request (klein.app.KleinRequest) – The Klein HTTP request
- event (dict) – The Marathon event
-
purge_dead_app_labels
(*args, **kwargs)[source]¶ Delete any keys stored in the Consul k/v store that belong to apps that no longer exist.
Param: apps: The list of apps as returned by the Marathon API.
-
put_consul_app_labels
(app_name, labels)[source]¶ Store the given set of labels under the given app name in the Consul k/v store.
-
register_marathon_event_callback
(*args, **kwargs)[source]¶ Register Consular with Marathon to receive HTTP event callbacks. To use this ensure that Marathon is configured to send HTTP event callbacks for state changes in tasks.
Parameters: events_url (str) – The HTTP endpoint to register with Marathon for event callbacks.
-
register_task_service
(app_id, task_id, host, ports)[source]¶ Register a Marathon task as a service in Consul.
Parameters:
-
schedule_sync
(interval, purge=False)[source]¶ Schedule a recurring sync of apps, starting after this method is called.
Parameters: Returns: A tuple of the LoopingCall object and the deferred created when it was started.
-
sync_app_labels
(*args, **kwargs)[source]¶ Sync the app labels for the given app by pushing its labels to the Consul k/v store and cleaning any labels there that are no longer present.
Param: app: The app JSON as return by the Marathon HTTP API.
-
sync_apps
(*args, **kwargs)[source]¶ Ensure all the apps in Marathon are registered as services in Consul.
Set
purge
toTrue
if you automatically want services in Consul that aren’t registered in Marathon to be purged. Consular only purges services that have been registered with the sameregistration-id
.Parameters: purge (bool) – To purge or not to purge.
-
consular.main.
get_app_name
(app_id)[source]¶ Get the app name from the marathon app ID. Separators in the ID (‘/’) are replaced with ‘-‘s while the leading separator is removed.
-
consular.main.
handle_not_found_error
(*args, **kwargs)[source]¶ Perform a request and catch the not found (404) error if one occurs.
Param: f: The function to call to perform the request. The function may return a deferred. Param: args: The arguments to call the function with. Param: kwargs: The keyword arguments to call the function with. Returns: The return value of the function call or None if there was a 404 response code.