Base Configuration¶
The docker-compose.yml file contains base settings for use in both the Production Environment and Development Environment. The file includes the following services:
Cyphon services¶
Cyphon¶
This service is based on the Cyphon production image. It links to PostgreSQL, which it uses as its Django backend database service for storing Cyphon admin settings and alerts. It also links to Elasticsearch and (optionally) MongoDB for storing data.
The service creates a volume for Cyphon fixtures, which can be used to load settings. It also uses the GeoLite2 database from the GeoIP service to assign geolocations to IP addresses.
Extends | Cyphon production image |
Links | PostgreSQL, Elasticsearch, (optionally) MongoDB |
Dependency | PostgreSQL |
Shared Volume | GeoIP |
YAML:
cyphon:
extends:
file: common-services.yml
service: cyphon-prod
volumes_from:
- geoip
links:
- elasticsearch
# - mongo
- postgres
depends_on:
- postgres
serialized_results = [AlertDetailSerializer(alert) for alert in queryset]
Celerybeat¶
This service runs a celery beat scheduler for periodic tasks, such as checking and processing email. It is similar to the Cyphon service, but it runs a different command on start up, and it has a link to RabbitMQ, where it sends the scheduled tasks.
Extends | Cyphon production image |
Command | run_celerybeat.sh |
Links | PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB |
Dependencies | PostgreSQL, RabbitMQ |
Shared Volume | GeoIP |
YAML:
celerybeat:
extends:
file: common-services.yml
service: cyphon-prod
restart: always
command: ../entrypoints/run_celerybeat.sh
volumes_from:
- geoip
links:
- elasticsearch
# - mongo
- postgres
- rabbit
depends_on:
- cyphon
- rabbit
Celeryworker¶
This service runs a celery worker to handle periodic tasks sent to RabbitMQ by Celerybeat. It is similar to the Celerybeat service, but it runs a different command on start up.
Extends | Cyphon production image |
Command | run_celeryworker.sh |
Links | PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB |
Dependencies | PostgreSQL, RabbitMQ |
Shared Volume | GeoIP |
YAML:
celeryworker:
extends:
file: common-services.yml
service: cyphon-prod
restart: always
command: ../entrypoints/run_celeryworker.sh
volumes_from:
- geoip
links:
- elasticsearch
# - mongo
- postgres
- rabbit
depends_on:
- cyphon
- rabbit
LogChutes¶
This service creates a queue consumer for log messages sent to RabbitMQ from Logstash. The messages are sent to Cyphon’s LogChutes for processing.
Image | Cyphon production image |
Command | run_receiver.sh logchutes |
Links | PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB |
Dependencies | PostgreSQL, RabbitMQ |
Shared Volume | GeoIP |
YAML:
logchutes:
extends:
file: common-services.yml
service: cyphon-prod
restart: always
command: ../entrypoints/run_receiver.sh logchutes
volumes_from:
- geoip
links:
- elasticsearch
# - mongo
- postgres
- rabbit
depends_on:
- cyphon
- rabbit
Monitors¶
This service creates a queue consumer for JSON messages sent to RabbitMQ from Logstash. These messages are inspected by Cyphon’s Monitors.
Image | Cyphon production image |
Command | run_receiver.sh monitors |
Links | PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB |
Dependencies | PostgreSQL, RabbitMQ |
Shared Volume | GeoIP |
YAML:
monitors:
extends:
file: common-services.yml
service: cyphon-prod
restart: always
command: ../entrypoints/run_receiver.sh monitors
volumes_from:
- geoip
links:
- elasticsearch
# - mongo
- postgres
- rabbit
depends_on:
- cyphon
- rabbit
Watchdogs¶
This service creates a queue consumer for JSON messages sent to RabbitMQ from Logstash. These messages are inspected by Cyphon’s Watchdogs.
Image | Cyphon production image |
Command | run_receiver.sh watchdogs |
Links | PostgreSQL, RabbitMQ, Elasticsearch, (optionally) MongoDB |
Dependencies | PostgreSQL, RabbitMQ |
Shared Volume | GeoIP |
YAML:
watchdogs:
extends:
file: common-services.yml
service: cyphon-prod
restart: always
command: ../entrypoints/run_receiver.sh watchdogs
volumes_from:
- geoip
links:
- elasticsearch
# - mongo
- postgres
- rabbit
depends_on:
- cyphon
- rabbit
PostgreSQL¶
This service creates a PostGIS database for saving Cyphon configurations and alerts. It is used as the database backend for GeoDjango. The host name, database name, username, and passowrd are determined by settings in the Environment Variables.
Image | mdillon/postgis |
Env File | cyphon.env |
YAML:
postgres:
image: mdillon/postgis:${POSTGRES_VER}
restart: always
env_file:
- ./config/env/cyphon.env
RabbitMQ¶
This service creates a RabbitMQ message broker for Logstash and Cyphon services. Defaults for the host name, virtual host, username, and password are determined by settings in the Environment Variables.
Image | rabbitmq |
Env File | cyphon.env |
YAML:
rabbit:
image: rabbitmq:${RABBITMQ_VER}
restart: always
env_file:
- ./config/env/cyphon.env
Nginx¶
This service creates a web service for Cyphon. It shares volumes from Cyphon, including directories for static assets and media files.
Image | nginx |
Links | Cyphon, RabbitMQ, Kibana |
Dependencies | Cyphon |
Shared Volume | Cyphon |
YAML:
nginx:
image: nginx:${NGINX_VER}
restart: always
volumes:
- ./config/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./config/nginx/ssl.crt:/etc/nginx/ssl.crt:ro
- ./config/nginx/ssl.key:/etc/nginx/ssl.key:ro
- /www/static
volumes_from:
- cyphon
links:
- cyphon
- kibana
- rabbit
depends_on:
- cyphon
GeoIP¶
This service provides GeoLite2 databases for Cyphon’s GeoIP package.
Image | geoip |
YAML:
geoip:
image: dunbar/geoip
restart: always
Elastic stack¶
Elasticsearch¶
This service provides an Elasticsearch backend for Cyphon’s Warehouses. It’s also used to store data from Logstash.
The host name and port are determined by settings in the Environment Variables.
Image | nginx |
Environment | http.host=0.0.0.0 transport.host=127.0.0.1 |
Links | Cyphon, RabbitMQ, Kibana |
Volumes | elasticsearch.yml, jvm.options, log4j2.properties |
YAML:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VER}
restart: always
environment:
- http.host=0.0.0.0
- transport.host=127.0.0.1
volumes:
- ./config/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
- ./config/elasticsearch/jvm.options:/usr/share/elasticsearch/config/jvm.options:ro
- ./config/elasticsearch/log4j2.properties:/usr/share/elasticsearch/config/log4j2.properties:ro
Logstash¶
This service ingests and parses logs, and sends them to Elasticsearch and RabbitMQ.
Image | docker.elastic.co/logstash/logstash |
Command | logstash -f /usr/share/logstash/pipeline --config.reload.automatic |
Links | Elasticsearch, RabbitMQ |
Dependencies | Elasticsearch, RabbitMQ |
Volumes | config, patterns, pipeline |
YAML:
logstash:
image: docker.elastic.co/logstash/logstash:${ELASTIC_VER}
restart: always
command: logstash -f /usr/share/logstash/pipeline --config.reload.automatic
volumes:
- ./config/logstash/config:/usr/share/logstash/config:ro
- ./config/logstash/patterns:/usr/share/logstash/patterns:ro
- ./config/logstash/pipeline:/usr/share/logstash/pipeline:ro
links:
- elasticsearch
- rabbit
depends_on:
- elasticsearch
- rabbit
Filebeat¶
This optional service can be used to monitor logs and send them to Logstash. You can use it for local testing of your Filebeat Configurations.
Image | docker.elastic.co/beats/filebeat |
Links | Logstash |
Dependencies | Logstash |
Volumes | filebeat.yml, ./log |
YAML:
# filebeat:
# image: docker.elastic.co/beats/filebeat:${ELASTIC_VER}
# restart: always
# links:
# - logstash
# depends_on:
# - logstash
# volumes:
# - ./config/beats/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:rw
# - ./log:/var/log
Kibana¶
This service provides a dashboard for viewing Elasticsearch data.
Image | docker.elastic.co/kibana/kibana |
Environment | LOGSPOUT: ignore |
Links | Elasticsearch |
Dependencies | Elasticsearch |
Volumes | kibana.yml |
YAML:
kibana:
image: docker.elastic.co/kibana/kibana:${ELASTIC_VER}
restart: always
environment:
LOGSPOUT: ignore # don't send Kibana's logs to Logspout
links:
- elasticsearch
depends_on:
- elasticsearch
volumes:
- ./config/kibana/kibana.yml:/usr/share/kibana/config/kibana.yml:ro
MongoDB¶
This optional service provides a MongoDB backend for Cyphon’s Warehouses. It can also be used to store data from Logstash when used with Logstash’s MongoDB output plugin.
The host name and port are determined by settings in the Environment Variables.
Image | mongo |
YAML:
# mongo:
# image: mongo:${MONGODB_VER}
# restart: always
Logspout¶
This service collects logs from the other Docker containers and sends them to Logstash. From there, they can be stored in Elasticsearch and viewed in Kibana.
Image | gliderlabs/logspout |
Command | syslog://logstash:5000 |
Expose | 5000/udp |
Links | Logstash |
Dependencies | Logstash |
Volumes | /var/run/docker.sock |
YAML:
logspout:
image: gliderlabs/logspout:${LOGSPOUT_VER}
restart: always
expose:
- "5000/udp"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
command: syslog://logstash:5000
links:
- logstash
depends_on:
- logstash