Are you happy with your logging solution? Would you help us out by taking a 30-second survey? Click here

sovereign

A set of Ansible playbooks to build and maintain your own private cloud: email, calendar, contacts, file sync, IRC bouncer, VPN, and more.

Subscribe to updates I use sovereign


Statistics on sovereign

Number of watchers on Github 6866
Number of open issues 64
Average time to close an issue about 1 month
Main language PHP
Average time to merge a PR 7 days
Open pull requests 44+
Closed pull requests 24+
Last commit almost 2 years ago
Repo Created about 6 years ago
Repo Last Updated over 1 year ago
Size 1010 KB
Organization / Authorsovereign
Contributors66
Page Updated
Do you use sovereign? Leave a review!
View open issues (64)
View sovereign activity
View on github
Fresh, new opensource launches πŸš€πŸš€πŸš€
Trendy new open source projects in your inbox! View examples

Subscribe to our mailing list

Evaluating sovereign for your project? Score Explanation
Commits Score (?)
Issues & PR Score (?)

Build Status CII Best Practices

Introduction

Sovereign is a set of Ansible playbooks that you can use to build and maintain your own personal cloud based entirely on open source software, so youre in control.

If youve never used Ansible before, you might find these playbooks useful to learn from, since they show off a fair bit of what the tool can do.

The original author's background and motivations might be of interest. tl;dr: frustrations with Google Apps and concerns about privacy and long-term support.

Sovereign offers useful cloud services while being reasonably secure and low-maintenance. Use it to set up your server, SSH in every couple weeks, but mostly forget about it.

Services Provided

What do you get if you point Sovereign at a server? All kinds of good stuff!

  • IMAP over SSL via Dovecot, complete with full text search provided by Solr.
  • POP3 over SSL, also via Dovecot
  • SMTP over SSL via Postfix, including a nice set of DNSBLs to discard spam before it ever hits your filters.
  • Virtual domains for your email, backed by PostgreSQL.
  • Spam fighting via Rspamd.
  • Mail server verification using DKIM and DMARC so the Internet knows your mailserver is legit.
  • Secure on-disk storage for email and more via EncFS.
  • Webmail via Roundcube.
  • Mobile push notifications via Z-Push.
  • Email client automatic configuration.
  • Jabber/XMPP instant messaging via Prosody.
  • An RSS Reader via Selfoss.
  • CalDAV and CardDAV to keep your calendars and contacts in sync, via ownCloud.
  • Your own private storage cloud via ownCloud.
  • Your own VPN server via OpenVPN.
  • An IRC bouncer via ZNC.
  • Monit to keep everything running smoothly (and alert you when its not).
  • collectd to collect system statistics.
  • Web hosting (ex: for your blog) via Apache.
  • Firewall management via Uncomplicated Firewall (ufw).
  • Intrusion prevention via fail2ban and rootkit detection via rkhunter.
  • SSH configuration preventing root login and insecure password authentication
  • RFC6238 two-factor authentication compatible with Google Authenticator and various hardware tokens
  • Nightly backups to Tarsnap.
  • Git hosting via cgit and gitolite.
  • Read-it-later via Wallabag
  • A bunch of nice-to-have tools like mosh and htop that make life with a server a little easier.

Dont want one or more of the above services? Comment out the relevant role in site.yml. Or get more granular and comment out the associated include: directive in one of the playbooks.

Usage

What Youll Need

  1. A VPS (or bare-metal server if you wanna ball hard). My VPS is hosted at Linode. Youll probably want at least 512 MB of RAM between Apache, Solr, and PostgreSQL. Mine has 1024.
  2. 64-bit Debian 8.3 or an equivalent Linux distribution. (You can use whatever distro you want, but deviating from Debian will require more tweaks to the playbooks. See Ansibles different packaging modules.)
  3. A Tarsnap account with some credit in it. You could comment this out if you want to use a different backup service. Consider paying your hosting provider for backups or using an additional backup service for redundancy.

You do not need to acquire an SSL certificate. The SSL certificates you need will be obtained from Let's Encrypt automatically when you deploy your server.

Installation

On the remote server

The following steps are done on the remote server by sshing into it and running these commands.

1. Install required packages e.g aptitude is required on Debian

apt-get install sudo python

2. Get a Tarsnap machine key

If you havent already, download and install Tarsnap, or use brew install tarsnap if you use Homebrew.

Create a new machine key for your server:

tarsnap-keygen --keyfile roles/tarsnap/files/decrypted_tarsnap.key --user me@example.com --machine example.com

3. Prep the server

For goodness sake, change the root password:

passwd

Create a user account for Ansible to do its thing through:

useradd deploy
passwd deploy
mkdir /home/deploy

Authorize your ssh key if you want passwordless ssh login (optional):

mkdir /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
nano /home/deploy/.ssh/authorized_keys
chmod 400 /home/deploy/.ssh/authorized_keys
chown deploy:deploy /home/deploy -R
echo 'deploy ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/deploy

Your new account will be automatically set up for passwordless sudo. Or you can just add your deploy user to the sudo group.

adduser deploy sudo

On your local machine

Ansible (the tool setting up your server) runs locally on your computer and sends commands to the remote server. Download this repository somewhere on your machine, either through Clone or Download > Download ZIP above, wget, or git as below

git clone https://github.com/sovereign/sovereign.git

4. Configure your installation

Modify the settings in the group_vars/sovereign folder to your liking. If you want to see how theyre used in context, just search for the corresponding string. All of the variables in group_vars/sovereign must be set for sovereign to function.

For Git hosting, copy your public key into place:

cp ~/.ssh/id_rsa.pub roles/git/files/gitolite.pub

Finally, replace the host.example.net in the file hosts. If your SSH daemon listens on a non-standard port, add a colon and the port number after the IP address. In that case you also need to add your custom port to the task Set firewall rules for web traffic and SSH in the file roles/common/tasks/ufw.yml.

5. Set up DNS

If youve just bought a new domain name, point it at Linodes DNS Manager or similar. Most VPS services (and even some domain registrars) offer a managed DNS service that you can use for this at no charge. If youre using an existing domain thats already managed elsewhere, you can probably just modify a few records.

Create A or CNAME records which point to your server's IP address:

  • example.com
  • mail.example.com
  • www.example.com (for Web hosting)
  • autoconfig.example.com (for email client automatic configuration)
  • read.example.com (for Wallabag)
  • news.example.com (for Selfoss)
  • cloud.example.com (for ownCloud)
  • git.example.com (for cgit)

6. Run the Ansible Playbooks

First, make sure youve got Ansible 1.9.3+ installed.

To run the whole dang thing:

ansible-playbook -i ./hosts --ask-sudo-pass site.yml

If you chose to make a passwordless sudo deploy user, you can omit the --ask-sudo-pass argument.

To run just one or more piece, use tags. I try to tag all my includes for easy isolated development. For example, to focus in on your firewall setup:

ansible-playbook -i ./hosts --tags=ufw site.yml

You might find that it fails at one point or another. This is probably because something needs to be done manually, usually because theres no good way of automating it. Fortunately, all the tasks are clearly named so you should be able to find out where it stopped. Ive tried to add comments where manual intervention is necessary.

The dependencies tag just installs dependencies, performing no other operations. The tasks associated with the dependencies tag do not rely on the user-provided settings that live in group_vars/sovereign. Running the playbook with the dependencies tag is particularly convenient for working with Docker images.

7. Finish DNS set-up

Create an MX record for example.com which assigns mail.example.com as the domains mail server.

To ensure your emails pass DKIM checks you need to add a txt record. The name field will be default._domainkey.EXAMPLE.COM. The value field contains the public key used by DKIM. The exact value needed can be found in the file /var/lib/rspamd/dkim/EXAMPLE.COM.default.txt. It will look something like this:

v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDKKAQfMwKVx+oJripQI+Ag4uTwYnsXKjgBGtl7Tk6UMTUwhMqnitqbR/ZQEZjcNolTkNDtyKZY2Z6LqvM4KsrITpiMbkV1eX6GKczT8Lws5KXn+6BHCKULGdireTAUr3Id7mtjLrbi/E3248Pq0Zs39hkDxsDcve12WccjafJVwIDAQAB

For DMARC you'll also need to add a txt record. The name field should be _dmarc.EXAMPLE.COM and the value should be v=DMARC1; p=none. More info on DMARC can be found here.

Set up SPF and reverse DNS as per this post. Make sure to validate that its all working, for example, by sending an email to check-auth@verifier.port25.com and reviewing the report that will be emailed back to you.

8. Miscellaneous Configuration

Sign in to the ZNC web interface and set things up to your liking. It isnt exposed through the firewall, so you must first set up an SSH tunnel:

ssh deploy@example.com -L 6643:localhost:6643

Then proceed to http://localhost:6643 in your web browser.

Similarly, to access the server monitoring page, use another SSH tunnel:

ssh deploy@example.com -L 2812:localhost:2812

Again proceeding to http://localhost:2812 in your web browser.

Finally, sign into ownCloud with a new administrator account to set it up. You should select PostgreSQL as the configuration backend. Use owncloud as the database user and the database name. For the database password ansible has created a set of random passwords for each service and stores them in your local folder secret, use the one in the file owncloud_db_password.

How To Use Your New Personal Cloud

Were collecting known-good client setups on our wiki.

Troubleshooting

If you run into an errors, please check the wiki page. If the problem you encountered, is not listed, please go ahead and create an issue. If you already have a bugfix and/or workaround, just put them in the issue and the wiki page.

Reboots

You will need to manually enter the password for any encrypted volumes on reboot. This is not Sovereign-specific, but rather a function of how EncFS works. This will necessitate SSHing into your machine after reboot, or accessing it via a console interface if one is available to you. Once you're in, run this:

encfs /encrypted /decrypted --public

It is possible that some daemons may need to be restarted after you enter your password for the encrypted volume(s). Some services may stall out while looking for resources that will only be available once the /decrypted volume is available and visible to daemon user accounts.

IRC

Ask questions and provide feedback in #sovereign on Freenode.

sovereign open issues Ask a question     (View All Issues)
  • about 3 years How do I train rspamd?
  • about 3 years Task list for postfix is a bit brittle
  • about 3 years Sieve configuration
  • about 3 years Get CII "Best practices" badge
  • about 3 years Webmail role fails due to composer running as root
  • about 3 years Multiple configuration errors
  • about 3 years Use RainLoop instead of Roundcube
  • over 3 years Correct Travis build failures
  • over 3 years Use z-push from packages and update to version 2.3.1
  • over 3 years Use EFF's packages for LE support
  • over 3 years Track security status of packages provided by distributions
  • over 3 years Improve OpenVPN security on Ubuntu Xenial
  • over 3 years Upgrade to Ansible 2.0
  • over 3 years Set machine hostname
  • over 3 years Remove OpenDKIM milter from postfix configuration
  • over 3 years How is dspam configured for junk mail handling?
  • over 3 years Redirect HTTP/S requests from {{domain}} to www.{{domain}}
  • over 3 years Break up roles in monitoring
  • over 3 years Remove librato references in the collectd role
  • over 3 years Permit use of wildcard certs instead of always using Lets Encrypt
  • over 3 years Don't bounce Apache on certificate renewal
  • over 3 years Run opendmarc reporting with postgres as the back end
  • almost 4 years Document mirror autoselect behavior in ubuntu
  • almost 4 years Pin Let's Encrypt's certificate using HPKP
  • almost 4 years Strengthen SSH DH groups
  • over 4 years Rewrite OpenVPN role
  • over 4 years Add configuration profile for iOS and OS X
  • over 4 years Generate 2048+ bit DHparam file to mitigate logjam attack
  • over 4 years fqdn.conf ServerName matching ServerName in VirtualHost can mess up HTTP (not HTTPS)
  • over 5 years Incorrect permissions on mail directories
sovereign open pull requests (View All Pulls)
  • Fix travis build by fixing ansible-lint version
  • Become
  • Radicale CalDAV and CardDAV as an alternative to owncloud.
  • Pass {auth_type} to milters, fixing OpenDKIM signing of authenticated SMTP messages.
  • Make OpenDMARC cron job log everything instead of mailing some output to root.
  • Reinstate webmail role
  • Add non-interactive to LE client options
  • Modularize sovereign
  • openvpn: only CN needed for openssl certificate
  • Remove unnecessary ownCloud configuration
  • Eliminate need for access_compat module in Apache
  • VPN - enforce TLS min version on the client
  • Use submission port for client outgoing email
  • Remove duplicate when statement in Let's Encrypt task
  • enabled password plugin and sql function for updating passwords
  • changed mailserver role to not clobber users/passwords on every run
  • Private variables, service teardown
  • lint cleanup and one issue fix
  • update z-push to new repo-backed version.
  • activate imap on trusty.
  • Roundcube from source
  • fix for debian bug in opendmarc report scripts. closes #497
  • Reinstate webmail role (take 2)
  • use wallabag v2
  • Add antispam plugin config for dovecot to trigger rspamd learning whe…
  • Add BaΓ―kal role
  • Set default locales in order to start postgresql
  • Add group name ssl-cert for SSL certificates
  • Restart prosody after LE cert renewal
  • Use postmaster@ for DMARC reports
  • fix pgsql connection issues from missing quotes in selfoss config file
  • Creating VPN directory behind Basic auth for an easy openvpn configuration retrieval
  • Implementing password hashing inside password_hash filter plugin
  • Autoconfigure ownCloud
  • Minor fixes to OpenVPN set-up
  • Use wget to add apt-key for OwnCloud rather that apt_key module.
  • Use php* packages rather than php5* for packages on Ubuntu
  • Updated scripts to use raw mysql command as the Ansible mysql module …
  • Update the defaults file for OpenDKIM to include the SOCKET so that t…
  • Updated the name of the fail2ban jails
  • Updated blog to use www, and provide a 404 page.
  • Upgrade Selfoss to version 2.17
  • Added changes for Ubuntu 16.04.2 LTS
  • Change PHP package names on Xenial and Stretch
sovereign list of languages used
Other projects in PHP