Base Configuration¶

The docker-compose.yml file contains base settings for use in both the Production Environment and Development Environment. The file includes the following services:

Cyphon services
PostgreSQL
RabbitMQ
Nginx
GeoIP
Elastic stack
MongoDB
Logspout

Cyphon services¶

Cyphon¶

This service is based on the Cyphon production image. It links to PostgreSQL, which it uses as its Django backend database service for storing Cyphon admin settings and alerts. It also links to Elasticsearch and (optionally) MongoDB for storing data.

The service creates a volume for Cyphon fixtures, which can be used to load settings. It also uses the GeoLite2 database from the GeoIP service to assign geolocations to IP addresses.

Extends	Cyphon production image
Links	PostgreSQL, Elasticsearch, (optionally) MongoDB
Dependency	PostgreSQL
Shared Volume	GeoIP

YAML:

cyphon:
  extends:
    file: common-services.yml
    service: cyphon-prod
  volumes_from:
    - geoip
  links:
    - elasticsearch
    # - mongo
    - postgres
  depends_on:
    - postgres

    serialized_results = [AlertDetailSerializer(alert) for alert in queryset]

Celerybeat¶

This service runs a celery beat scheduler for periodic tasks, such as checking and processing email. It is similar to the Cyphon service, but it runs a different command on start up, and it has a link to RabbitMQ, where it sends the scheduled tasks.

Extends	Cyphon production image
Command	run_celerybeat.sh
Links	PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB
Dependencies	PostgreSQL, RabbitMQ
Shared Volume	GeoIP

YAML:

celerybeat:
  extends:
    file: common-services.yml
    service: cyphon-prod
  restart: always
  command: ../entrypoints/run_celerybeat.sh
  volumes_from:
    - geoip
  links:
    - elasticsearch
    # - mongo
    - postgres
    - rabbit
  depends_on:
    - cyphon
    - rabbit

Celeryworker¶

This service runs a celery worker to handle periodic tasks sent to RabbitMQ by Celerybeat. It is similar to the Celerybeat service, but it runs a different command on start up.

Extends	Cyphon production image
Command	run_celeryworker.sh
Links	PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB
Dependencies	PostgreSQL, RabbitMQ
Shared Volume	GeoIP

YAML:

celeryworker:
  extends:
    file: common-services.yml
    service: cyphon-prod
  restart: always
  command: ../entrypoints/run_celeryworker.sh
  volumes_from:
    - geoip
  links:
    - elasticsearch
    # - mongo
    - postgres
    - rabbit
  depends_on:
    - cyphon
    - rabbit

LogChutes¶

This service creates a queue consumer for log messages sent to RabbitMQ from Logstash. The messages are sent to Cyphon’s LogChutes for processing.

Image	Cyphon production image
Command	run_receiver.sh `logchutes`
Links	PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB
Dependencies	PostgreSQL, RabbitMQ
Shared Volume	GeoIP

YAML:

logchutes:
  extends:
    file: common-services.yml
    service: cyphon-prod
  restart: always
  command: ../entrypoints/run_receiver.sh logchutes
  volumes_from:
    - geoip
  links:
    - elasticsearch
    # - mongo
    - postgres
    - rabbit
  depends_on:
    - cyphon
    - rabbit

Monitors¶

This service creates a queue consumer for JSON messages sent to RabbitMQ from Logstash. These messages are inspected by Cyphon’s Monitors.

Image	Cyphon production image
Command	run_receiver.sh `monitors`
Links	PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB
Dependencies	PostgreSQL, RabbitMQ
Shared Volume	GeoIP

YAML:

monitors:
  extends:
    file: common-services.yml
    service: cyphon-prod
  restart: always
  command: ../entrypoints/run_receiver.sh monitors
  volumes_from:
    - geoip
  links:
    - elasticsearch
    # - mongo
    - postgres
    - rabbit
  depends_on:
    - cyphon
    - rabbit

Watchdogs¶

This service creates a queue consumer for JSON messages sent to RabbitMQ from Logstash. These messages are inspected by Cyphon’s Watchdogs.

Image	Cyphon production image
Command	run_receiver.sh `watchdogs`
Links	PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB
Dependencies	PostgreSQL, RabbitMQ
Shared Volume	GeoIP

YAML:

watchdogs:
  extends:
    file: common-services.yml
    service: cyphon-prod
  restart: always
  command: ../entrypoints/run_receiver.sh watchdogs
  volumes_from:
    - geoip
  links:
    - elasticsearch
    # - mongo
    - postgres
    - rabbit
  depends_on:
    - cyphon
    - rabbit

PostgreSQL¶

This service creates a PostGIS database for saving Cyphon configurations and alerts. It is used as the database backend for GeoDjango. The host name, database name, username, and passowrd are determined by settings in the Environment Variables.

Image	mdillon/postgis
Env File	cyphon.env

YAML:

postgres:
  image: mdillon/postgis:${POSTGRES_VER}
  restart: always
  env_file:
    - ./config/env/cyphon.env

RabbitMQ¶

This service creates a RabbitMQ message broker for Logstash and Cyphon services. Defaults for the host name, virtual host, username, and password are determined by settings in the Environment Variables.

Image	rabbitmq
Env File	cyphon.env

YAML:

rabbit:
  image: rabbitmq:${RABBITMQ_VER}
  restart: always
  env_file:
    - ./config/env/cyphon.env

Nginx¶

This service creates a web service for Cyphon. It shares volumes from Cyphon, including directories for static assets and media files.

Image	nginx
Links	Cyphon, RabbitMQ, Kibana
Dependencies	Cyphon
Shared Volume	Cyphon

YAML:

nginx:
  image: nginx:${NGINX_VER}
  restart: always
  volumes:
    - ./config/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    - ./config/nginx/ssl.crt:/etc/nginx/ssl.crt:ro
    - ./config/nginx/ssl.key:/etc/nginx/ssl.key:ro
    - /www/static
  volumes_from:
    - cyphon
  links:
    - cyphon
    - kibana
    - rabbit
  depends_on:
    - cyphon

GeoIP¶

This service provides GeoLite2 databases for Cyphon’s GeoIP package.

Image

geoip

YAML:

geoip:
  image: dunbar/geoip
  restart: always

Elastic stack¶

Elasticsearch¶

This service provides an Elasticsearch backend for Cyphon’s Warehouses. It’s also used to store data from Logstash.

The host name and port are determined by settings in the Environment Variables.

Image	nginx
Environment	`http.host=0.0.0.0` `transport.host=127.0.0.1`
Links	Cyphon, RabbitMQ, Kibana
Volumes	elasticsearch.yml, jvm.options, log4j2.properties

YAML:

elasticsearch:
  image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VER}
  restart: always
  environment:
    - http.host=0.0.0.0
    - transport.host=127.0.0.1
  volumes:
    - ./config/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
    - ./config/elasticsearch/jvm.options:/usr/share/elasticsearch/config/jvm.options:ro
    - ./config/elasticsearch/log4j2.properties:/usr/share/elasticsearch/config/log4j2.properties:ro

Logstash¶

This service ingests and parses logs, and sends them to Elasticsearch and RabbitMQ.

Image	docker.elastic.co/logstash/logstash
Command	`logstash -f /usr/share/logstash/pipeline --config.reload.automatic`
Links	Elasticsearch, RabbitMQ
Dependencies	Elasticsearch, RabbitMQ
Volumes	config, patterns, pipeline

YAML:

logstash:
  image: docker.elastic.co/logstash/logstash:${ELASTIC_VER}
  restart: always
  command: logstash -f /usr/share/logstash/pipeline --config.reload.automatic
  volumes:
    - ./config/logstash/config:/usr/share/logstash/config:ro
    - ./config/logstash/patterns:/usr/share/logstash/patterns:ro
    - ./config/logstash/pipeline:/usr/share/logstash/pipeline:ro
  links:
    - elasticsearch
    - rabbit
  depends_on:
    - elasticsearch
    - rabbit

Filebeat¶

This optional service can be used to monitor logs and send them to Logstash. You can use it for local testing of your Filebeat Configurations.

Image	docker.elastic.co/beats/filebeat
Links	Logstash
Dependencies	Logstash
Volumes	filebeat.yml, `./log`

YAML:

# filebeat:
#   image: docker.elastic.co/beats/filebeat:${ELASTIC_VER}
#   restart: always
#   links:
#     - logstash
#   depends_on:
#     - logstash
#   volumes:
#     - ./config/beats/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:rw
#     - ./log:/var/log

Kibana¶

This service provides a dashboard for viewing Elasticsearch data.

Image	docker.elastic.co/kibana/kibana
Environment	`LOGSPOUT: ignore`
Links	Elasticsearch
Dependencies	Elasticsearch
Volumes	kibana.yml

YAML:

kibana:
  image: docker.elastic.co/kibana/kibana:${ELASTIC_VER}
  restart: always
  environment:
    LOGSPOUT: ignore  # don't send Kibana's logs to Logspout
  links:
    - elasticsearch
  depends_on:
    - elasticsearch
  volumes:
    - ./config/kibana/kibana.yml:/usr/share/kibana/config/kibana.yml:ro

MongoDB¶

This optional service provides a MongoDB backend for Cyphon’s Warehouses. It can also be used to store data from Logstash when used with Logstash’s MongoDB output plugin.

The host name and port are determined by settings in the Environment Variables.

Image

mongo

YAML:

# mongo:
#   image: mongo:${MONGODB_VER}
#   restart: always

Logspout¶

This service collects logs from the other Docker containers and sends them to Logstash. From there, they can be stored in Elasticsearch and viewed in Kibana.

Image	gliderlabs/logspout
Command	`syslog://logstash:5000`
Expose	5000/udp
Links	Logstash
Dependencies	Logstash
Volumes	`/var/run/docker.sock`

YAML:

logspout:
  image: gliderlabs/logspout:${LOGSPOUT_VER}
  restart: always
  expose:
    - "5000/udp"
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock:ro
  command: syslog://logstash:5000
  links:
    - logstash
  depends_on:
    - logstash