Commit 2305b1e2 authored by Jens Langhammer's avatar Jens Langhammer

cleanup

parent 196c829c
Pipeline #3482 failed with stages
in 4 minutes and 29 seconds
This diff is collapsed.
# Contributing
When contributing to this repository, please first discuss the change you wish to make via issue,
email, or any other method with the owners of this repository before making a change.
Please note we have a code of conduct, please follow it in all your interactions with the project.
## Pull Request Process
1. Ensure any install or build dependencies are removed before the end of the layer when doing a
build.
2. Update the README.md with details of changes to the interface, this includes new environment
variables, exposed ports, useful file locations and container parameters.
3. Increase the version numbers in any examples files and the README.md to the new version that this
Pull Request would represent. The versioning scheme we use is [SemVer](http://semver.org/).
4. You may merge the Pull Request in once you have the sign-off of two other developers, or if you
do not have permission to do that, you may request the second reviewer to merge it for you.
## Code Style
All Code should adhere to the pep8 standard. We use a maximum line-length of 100 characters.
All imports should be absolute.
Python code should be indented with 4 spaces, HTML and JavaScript should be indented with 2 spaces.
To test if code follows these standards, run `invoke env.env.test-complete` to run all tests.
## Commit Style
All Commits should be tagged with a tag like `[tag] test`. This tag identifies the part of supervisr
which work was done on. If you worked on multiple parts, chain tags together in an alphabetical order
like so: `[core] [dns] test`. There are also some special tags which are defined as such:
- `[ci]`: Use this tag if you modified CI relevant files, like `.gitlab-ci.yml`.
- `[debian]`: Use this tag if you modified files in the `debian/` subdirectory.
- `[docs]`: Use this tag if you modified files in the `docs/` subdirectory.
- `[tasks]`: Use this tag if you modified files in `supervisr/cli/tasks`.
- `[assets]`: Use this tag if you modified files in the `assets/` subdirectory.
- `[<app>]`: Use this tag if you modified files for `<app>`.
- `[ui]`: Use this tag if you modified files in `supervisr-ui/`.
- `[other]`: Use this tag as fallback, if none of the above apply.
## Code of Conduct
### Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
nationality, personal appearance, race, religion, or sexual identity and
orientation.
### Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
### Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
### Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
### Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at supervisr@beryju.org. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
### Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at [http://contributor-covenant.org/version/1/4][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/
supervisr (0.02) stable; urgency=medium
* [docs] fix path typo
* [ci] automatically build and upload docs in CI
* [ci] create ssh dir before adding known_host
* [ci] disable services for docs, fix requirements
* [ci] install rsync for doc uploading
* [ci] switch doc images back to python:3.6
* [ci] run apt update before installing rsync
* [ci] fix host_key verification issue
* [ci] import SSH key before running rsync
* [ci] initialise SSH-Agent before importing key...
* [core] [cli] cleanup
* [core] fix Provider Base Class declaration
* [*] unify test and tests module into tests for every app
* [*] finish cleanup
* [core] more style cleanup
* [mod/web_proxy] add web_proxy as a module, using django-revproxy
* [core] add missing dependency on colorlog
* [core] cleanup provider creation, enable isort and prospector in CI again
* [mod/stats/influx] migrate settings to GenericSettingsView
* [ci] show python version
* [*] fix import order, again
* [core] rewrite Admin views to CBV
* [assets] fix sidebar nav-link highlight
* [core] better handle Provider Results and exceptions
* [core] set dependency on prospector to <1.0 for pylint-django
* [puppet] rewrote ReleaseBuilder and ForgeImporter to Celery tasks
* [core] fix wrong view for admin product index
* [mod/provider/libcloud] add basic libcloud provider for DNS
* [mod/provider/libcloud] add missing provider files
* [core] ask for current password change_password view
* [core] show red asterisk on forms when field is required
* [core] add view to list all tasks
* [core] API now returns 401 if user is not authenticated
* [core] apply red asterik to every label on forms
* [core] change template to template_name on GenericViews to match Django (template still works but throws deprecation warning)
* [puppet] fix linting errors
* [core] fix validation in ChangePassword Form
* [dns] rewrite DNS Zone Model with SOA Data integrated
* [core] add http_basic_auth API endpoint
-- Jens Langhammer <jens.langhammer@beryju.org> Fri, 27 Jul 2018 10:54:52 +0000
supervisr (0.01) stable; urgency=medium
* [pack] add basic packer template and debian preseed
* Revert "Merge branch 'master' into 728-dns-resourceset"
* [other] Add contribution guide
* [other] fix wrong CONTRIBUITING path
* [tasks] add url_prefix to flower task
* [mod/beacon] no longer create a new pulse instance, simply update existing one
* [core] add sentry to celery, save connection user -> task in db, require invoker for certain actions
* [*] fix all unittests/style issues
* [ci] replace mysql:latest with mariadb:latest
* [ci] update mysql hostname as well
* [core] update decorators to use django's wraps
* [core] fix import order
* [ci] fix yaml syntax
* [ci] don't run deactivate first on pip_install
* [ci] install virtualenv in pip_install
* [ci] pip install from correct branch
* [core] add mod _seed again
* [seed] move _seed to seed so it's seen by setup.py
* [ci] fix pip_install copy path, cleanup
* [core] fix encryption not working with pycryptodome
* [core] fix email_missing_middleware redirect looping
* [mod/auth/oauth/client] add reddit provider
* [core] fix whitespace error
* [core] add footer block to sidebar, with custom links from SETTINGS
* [static] add FilePageBootstrapper, re-add filepages
* [core] update requirements
* [assets] fix light theme for footer-card
* [*] cleanup imports
* [core] fix unittests
* [tasks] fix tasks.py styling
* [tasks] fix celery output when running with invoke
* [mod/auth/tfa] disable autocomplete on TFA Verify Form
* [core] [puppet] Clean up after unittests
* [*] we open source now!
* [ci] temporarily disable isort
* Initial release.
-- Jens Langhammer <jens.langhammer@beryju.org> Sat, 23 Jun 2018 14:03:08 +0000
#!/bin/sh
# config maintainer script for supervisr
# source debconf stuff
. /usr/share/debconf/confmodule
dbc_first_version=1.0.0
dbc_dbuser=supervisr
dbc_dbname=supervisr
# source dbconfig-common shell library, and call the hook function
if [ -f /usr/share/dbconfig-common/dpkg/config.mysql ]; then
. /usr/share/dbconfig-common/dpkg/config.mysql
dbc_go supervisr "$@"
fi
#DEBHELPER#
exit 0
Source: supervisr
Section: admin
Priority: optional
Maintainer: BeryJu.org <support@beryju.org>
Uploaders: Jens Langhammer <jens@beryju.org>, BeryJu.org <support@beryju.org>
Build-Depends: debhelper (>= 9), dh-exec, wget, dh-virtualenv, python3 (>= 3.5), dh-exec
Standards-Version: 3.9.6
Package: supervisr
Architecture: all
Recommends: redis-server
Depends: python3 (>= 3.5), dbconfig-mysql | dbconfig-no-thanks, ${misc:Depends}
Description: Supervise your IT
MIT License
Copyright (c) 2018 BeryJu.org
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
etc/supervisr/
etc/supervisr/config.d/
var/log/supervisr/
usr/share/supervisr/
usr/doc/supervisr/
debug: false
admins:
- admin <admin@domain.tld>
email:
host: mx1.domain.tld
from: Supervisr <supervisr@domain.tld>
http:
default:
listen: 0.0.0.0
port: 8000
threads: 30
secret_key_file: /etc/supervisr/secret_key
log:
level:
console: INFO
file: DEBUG
file:
web: /var/log/supervisr/supervisr.log
task-runner: /var/log/supervisr/supervisr-tasks.log
task-scheduler: /var/log/supervisr/supervisr-tasks-scheduler.log
task-monitor: /var/log/supervisr/supervisr-tasks-monitor.log
# Logs can be sent to a syslog server by enabling this setting
# syslog:
# host: 1.2.3.4
# port: 514
# Specify redis connection string
# Format: <user>:<password>@<hostname>:<port>/<db_number>
redis: localhost
# Error reporting
# disabled by default
# sentry: https://c5f3fa4e642d4dbfaa5db684bd0f6a13@sentry.services.beryju.org/6
supervisr /usr/share/supervisr/
sv /usr/share/supervisr/
tasks.py /usr/share/supervisr/
ATTRIBUTIONS.md /usr/share/supervisr/
debian/etc/supervisr /etc/
debian/templates/database.yml /usr/share/supervisr/
/usr/share/supervisr/sv /usr/bin/sv
/usr/share/doc/supervisr/ /usr/share/supervisr/docs
#!/bin/bash
set -e
. /usr/share/debconf/confmodule
. /usr/share/dbconfig-common/dpkg/postinst.mysql
# you can set the default database encoding to something else
dbc_mysql_createdb_encoding="UTF8"
dbc_generate_include=template:/etc/supervisr/config.d/database.yml
dbc_generate_include_args="-o template_infile=/usr/share/supervisr/database.yml"
dbc_go supervisr "$@"
declare -a services=("supervisr-task-runner.service" "supervisr.service" "supervisr-task-scheduler.service" "supervisr-task-monitor.service")
if [ -z "`getent group supervisr`" ]; then
addgroup --quiet --system supervisr
fi
if [ -z "`getent passwd supervisr`" ]; then
echo " * Creating user and group supervisr..."
adduser --quiet --system --home /usr/share/supervisr --shell /bin/false --ingroup supervisr --disabled-password --disabled-login --gecos "Supervisr User" supervisr
fi
chown -R supervisr: /var/log/supervisr/
chown -R supervisr: /usr/share/supervisr/
chown -R supervisr: /etc/supervisr/
if [ ! -f '/etc/supervisr/secret_key' ]; then
echo " * Generating Secret Key"
sv utils.generate-secret-key > /etc/supervisr/secret_key 2> /dev/null
chown -R supervisr: /etc/supervisr/
echo " * Running initial migrations..."
sv manage migrate sessions > /dev/null
fi
chmod 440 /etc/supervisr/secret_key
echo " * (Re-)starting Services..."
for service in "${services[@]}"
do
systemctl enable $service
systemctl restart $service
echo " * (Re-)started $service..."
done
#DEBHELPER#
#!/bin/sh
set -e
if [ -f /usr/share/debconf/confmodule ]; then
. /usr/share/debconf/confmodule
fi
if [ -f /usr/share/dbconfig-common/dpkg/postrm.mysql ]; then
. /usr/share/dbconfig-common/dpkg/postrm.mysql
dbc_go supervisr "$@"
fi
if [ "$1" = "purge" ]; then
if which ucf >/dev/null 2>&1; then
ucf --purge /etc/supervisr/config.d/database.yml
ucfr --purge supervisr /etc/supervisr/config.d/database.yml
fi
rm -rf /etc/supervisr/
fi
#DEBHELPER#
#!/bin/sh
set -e
. /usr/share/debconf/confmodule
. /usr/share/dbconfig-common/dpkg/prerm.mysql
dbc_go supervisr "$@"
#DEBHELPER#
#!/usr/bin/make -f
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
export DH_VIRTUALENV_INSTALL_ROOT=/usr/share/supervisr/
export DH_VIRTUALENV_INSTALL_SUFFIX=env
export DH_VIRTUALENV_ARGUMENTS="-p python3"
%:
dh $@ --with python-virtualenv #--buildsystem=dh_virtualenv
override_dh_virtualenv:
dh_virtualenv --python=python3 --install-suffix=env
override_dh_installinit:
dh_installinit --name=supervisr
dh_installinit --name=supervisr-task-runner
dh_installinit --name=supervisr-task-scheduler
dh_installinit --name=supervisr-task-monitor
# override_dh_usrlocal to do nothing
override_dh_usrlocal:
tar-ignore = "supervisr/environments/local.yml"
[Unit]
Description=supervisr Task Monitor
After=syslog.target network.target
Requires=network.target supervisr-task-runner.service
[Service]
User=supervisr
Group=supervisr
WorkingDirectory=/usr/share/supervisr
Type=simple
ExecStart=/usr/bin/sv run.worker-monitor
[Install]
WantedBy=multi-user.target
[Unit]
Description=supervisr Task Runner
After=syslog.target network.target
Requires=network.target
[Service]
User=supervisr
Group=supervisr
WorkingDirectory=/usr/share/supervisr
Type=simple
ExecStart=/usr/bin/sv run.worker
[Install]
WantedBy=multi-user.target
[Unit]
Description=supervisr Task Scheduler
After=syslog.target network.target
Requires=network.target supervisr-task-runner.service
[Service]
User=supervisr
Group=supervisr
WorkingDirectory=/usr/share/supervisr
Type=simple
ExecStart=/usr/bin/sv run.worker-scheduler
[Install]
WantedBy=multi-user.target
[Unit]
Description=supervisr
After=syslog.target network.target
Requires=network.target supervisr-task-runner.service
[Service]
User=supervisr
Group=supervisr
WorkingDirectory=/usr/share/supervisr
Type=simple
ExecStart=/usr/bin/sv run.web --production
[Install]
WantedBy=multi-user.target
databases:
default:
engine: django.db.backends.mysql
name: _DBC_DBNAME_
user: _DBC_DBUSER_
password: _DBC_DBPASS_
host: _DBC_DBSERVER_
port: _DBC_DBPORT_
options:
charset: UTF8MB4
sql_mode: STRICT_TRANS_TABLES,NO_ZERO_DATE,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO
# Register interest in Python interpreter changes and
# don't make the Python package dependent on the virtualenv package
# processing (noawait)
interest-noawait /usr/bin/python3.5
interest-noawait /usr/bin/python3.6
interest-noawait /usr/bin/python3.7
# Also provide a symbolic trigger for all dh-virtualenv packages
interest dh-virtualenv-interpreter-update
version: '3'
services:
db:
image: mariadb:latest
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=yes
- MYSQL_DATABASE=supervisr
- MYSQL_USER=supervisr
- MYSQL_PASSWORD=EK-5jnKfjrGRm<77
redis:
image: redis:latest
supervisr:
image: beryjuorg/supervisr:latest
command: sv run.web
restart: always
ports:
- "8000:8000"
depends_on:
- db
- redis
supervisr-task-monitor: