Serving Secrets Securely

Bri Hatch Personal Work

Onsight, Inc
bri@ifokr.org ExtraHop Networks
bri@extrahop.com

Copyright 2016, Bri Hatch, Creative Commons BY-NC-SA License

Bri Hatch	Personal	Work
Onsight, Inc bri@ifokr.org	ExtraHop Networks bri@extrahop.com

Audience

Anyone who values security
Some Linux/Unix experience
Don't mind a bit of code and command line

Secrets?

Examples

Passwords (databases, ssh, fileservers...)
Tokens (api, oauth keys...)
Crypto Keys (ssl pems, gpg keys)

Where are secrets traditionally stored?

Command line paramaters
Configuration Files
Environment variables
Memory

Can you protect from Root?

Can you protect from the Root Account?

No.
Well maybe.
But assume "No".

Root is rather powerful...

Root can

Read any file

$ sudo wc -l /etc/shadow
35 /etc/shadow

Trace/debug any process

$ sudo strace -p 1757
close(3)                                = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe9d0b78000
write(4, "Myv0iceIsmyPassport,Verifym3?", 29) = 29

Read any environment

$ sudo perl -pe 's/\0/\n/g' /proc/1757/environ
LANG=en_US.UTF-8
DB_HOST=172.30.28.2
DB_PW=Myv0iceIsmyPassport,Verifym3?

Levels of vulnerability

Extended access
Moment-of access

Extended access

Extended Access Examples

passphrases/passwords
passphraseless keys
gpg/ssh/pem passphrase + key files

Only way to protect after attack is to change or stop using them.

Moment-of access

Moment-of Access Examples

gpg/ssh agents
Time-limited tokens/certificates
OTP codes

Allow access for a limited window of vulnerability.

How do you know it's encrypted?

It's not really encrypted if

...it's easy
...it works without a human
Security, price, ease of use - choose two.

Encrypted Filesystems

Mountable encrypted filesystems
Full Disk Encryption (FDE)

Prevent access to sensitive data

from physical attackers
from IaaS provider

Cryptsetup / LUKS partition

Encrypted partition setup

$ sudo cryptsetup -y luksFormat /dev/xvdc
Are you sure? (Type uppercase yes): YES
Enter passphrase: *******
 
$ sudo cryptsetup luksOpen /dev/xvdc data_crypt
Enter passphrase for /dev/xvdc: *******
 
$ sudo mkfs.ext4 /dev/mapper/data_crypt
$ sudo cryptsetup luksClose data_crypt

Cryptsetup / LUKS partition (cont)

Encrypted partition mounting

$ sudo mkdir /mnt/data_crypt 

$ sudo cryptsetup luksOpen /dev/xvdc data_crypt
Enter passphrase for /dev/xvdc: *******

$ sudo mount /mnt/data_crypt /dev/mapper/data_crypt

$ ls -l /mnt/data_crypt
drwx------ 2 root root 12288 Apr 19 04:43 lost+found

Full Disk Encryption

Most distros let you choose this at install time:

Full Disk Encryption Pitfalls

Problems with FDE:

Rebooting an encrypted machine remotely

Remote Console (iDrac, lish, etc)
Remote SSH unlock

Forgetting the unlock passphrase

Password Safes
Multiple keys, ala "sudo cryptsetup luksAddKey /dev/sdb"
Sticky Notes

FDE Remote Unlock

Remote Unlock?

Great info at /usr/share/doc/cryptsetup/README.remote.gz (debian)
SSH server (dropbear) in initrd
Unlock disk
Continue boot

Cryptsetup / LUKS partition (redux)

If you've got an FDE machine w/ a second drive crypto disk:

$ sudo dd if=/dev/urandom of=/root/.luks/sdb.bin bs=1024 count=4
$ sudo cryptsetup luksFormat /dev/sdb /root/.luks/sdb.bin
$ sudo cryptsetup luksOpen /dev/sdb data_crypt \
       --key-file /root/.luks/sdb.bin
$ sudo mkfs.ext4 /dev/mapper/data_crypt
 
$ sudo cryptsetup luksDump /dev/sdb | grep ^UUID
UUID:           da4d10ba-a886-11e6-8e5d-382c4ab5a4c6 
$ sudoedit /etc/crypttab ; tail -1 /etc/crypttab
UUID=da4d10ba-a886-11e6-8e5d-382c4ab5a4c6 /root/.luks/sdb.bin luks
 
$ sudo mkdir -p /mnt/data_crypt
$ sudoedit /etc/fstab ; tail -f /etc/fstab
/dev/mapper/data_crypt /mnt/data_crypt ext4 defaults 0 2
 
$ sudo reboot

EncFS

EncFS - Encyrpted FUSE Filesystem

Security Notes: https://defuse.ca/audits/encfs.htm
Does not require a dedicated partition
Does not need to be mounted as root
Can be restricted to just invoking user
Creates new cleartext mountpoint, stores in encrypted format
Name mangles by default
Creates one encrypted file per source file
Can work in "reverse" mode

EncFS (cont)

Forward Encryption

$ cd /tmp
$ mkdir encrypted clear
$ encfs /tmp/encrypted /tmp/clear 
Creating new encrypted volume.
...
New Encfs Password: *******
 
$ echo "Hello World" > clear/hello.txt
$ find encrypted clear  -type f
-rw-rw-r--  me  me     12 Apr 19 04:43 clear/hello.txt
-rw-rw-r--  me  me   1092 Apr 19 04:43 encrypted/.encfs6.xml
-rw-rw-r--  me  me   1092 Apr 19 04:43 encrypted/OmXQVCFFzGA2hYn6rhgwqdbq
 
$ mount | grep clear
encfs on /tmp/clear type fuse.encfs (rw,nosuid,nodev,relatime,...)
$ fusermount -u /tmp/clear

EncFS (cont)

Reverse Encryption

$ mkdir /mnt/encrypted-home 
$ encfs --reverse /home/me /mnt/encrypted-home 
Creating new encrypted volume.
...
New Encfs Password: *******
 
$ find /mnt/encrypted-home/me
drwxr-xr-x  me  me     12 Apr 19 04:43 H6oOfxaZFFt0YxYn5JJYzJv8
-rwxr-xr-x  me  me     12 Apr 19 04:43 H6oOfxaZFFt0YxYn5JJYzJv8/gLFJsn2ol3YS9Fm7jxSq8Ijn
...
 
$ ls -la /home/me/.encfs6.xml
-rw-------  me  me     12 Apr 19 04:43 /home/me/.encfs6.xml

EncFS Alternatives

Alternatives

eCryptFS
VeraCrypt
CryFS

SSH-Agent

SSH Agent - passwordless keys are not ok!

$ ssh-agent -a /var/tmp/backups.sock
SSH_AUTH_SOCK=/var/tmp/backups.sock; export SSH_AUTH_SOCK;
...
 
$ SSH_AUTH_SOCK=/var/tmp/backups.sock ssh-add /path/to/backups_key
Enter passphrase for /path/to/backups_key: *******

$ SSH_AUTH_SOCK=/var/tmp/backups.sock ssh-add -l
2048 SHA256:Gn5D1Hk4lE1ZzuUjxZ1NoS9EZSFymEG81A9gOV4gxdc id_rsa (RSA)

Additional ssh-add options include ttl, lock/unlock, confirmation.

SSH-Agent (cont)

Using the agent

$ less /opt/bin/backups.sh
#!/bin/bash
SSH_AUTH_SOCK=/var/tmp/backups.sock
export SSH_AUTH_SOCK

rsync -a /etc/ backupserver:/backups/$(hostname)/etc/
...

GnuPG Agent

Many changes between GnuPG versions
Many different ways to 'preseed' the agent
Just use 2.1 or later

GnuPG Agent (cont)

Always puts the socket in gnupghome
Can have addl socket at your direction
Preseed by doing an encryption/decryption

GnuPG Agent (cont)

# Start it
$ /usr/bin/gpg-agent \
	--daemon \
	--allow-loopback-pinentry \
	--options /dev/null \
	--default-cache-ttl 315360000 \
	--max-cache-ttl 315360000

# Seed it
$ /usr/bin/gpg2 --pinentry-mode loopback <encrypted.asc >/dev/null

Hashicorp Vault

Encrypted secret store
API-based, w/ command line tool
Multiple storage and authentication backends
Audit log
Heirarchical Tokens and ACLs
Dynamic secrets, e.g. AWS resources
Multiple 'unseal' keys to start

Hashicorp Vault (cont)

$ vault auth bcad4ce4-a8b0-11e6-8550-c7f441a4e8ae
Successfully authenticated!
 
$ vault write secret/mysecret value=wasabi
$ vault read secret/mysecret
Key           	Value
lease_duration	2592000
value         	wasabi

Hashicorp Vault Policies

Policies - written in HCL (JSON-compatible)

$ cat acl.hcl
path "proddb/*" {
  policy = "read"
}
 
$ vault policy-write proddb acl.hcl
$ vault token-create -policy=proddb
Key             Value
token           c55cd250-a8b1-11e6-9f30-8fc0b1553eeb
token_accessor  c55cea88-a8b1-11e6-9de8-dbbd2a550339
...

Hashicorp Vault (cont)

AWS Magic

$  vault write aws/config/root \
    access_key=8XCOH3HISEZZQRCIGOSX \
    secret_key=8ih55dkNt6Ral+8OhuA2HAyfKjRlolZ5hVOqr+eB 
Success! Data written to: aws/config/root
 
$ vault write aws/roles/deploy policy=@policy.json
Success! Data written to: aws/roles/deploy
 
$ vault read aws/creds/deploy
Key         Value
lease_id    aws/creds/deploy/09e11cb8-a8a5-11e6-b3fd-cf58d0aeb98f
access_key  YXEIZPVXYYBJHUWZVEXV 
secret_key  EH5djvuY7C9R+6Xz8tmHV1Dvp5uRzsTTAiOpGPgU

Simple Socket Secret Provider

Python daemon written at ExtraHop
Good for simple scripts, or bootstrapping other services
GPG key/values stored in memory
Uses unix file perms for ACLs
Also has named pipe support
Python Library and command line tool

$ sudo pip install sssp

Simple Socket Secret Provider (cont)

SSSP initialization

$ sssp --init
Enter desired passphrase: ********************
Generating gnupg keys...
Generating empty secrets file...
$ ls -la
drwx------  me me  4096 Apr 19 04:43 .gnupg/
-rw-------  me me  1028 Apr 19 04:43 secrets.asc
$ sssp --edit
db-prod00_ro=CorrectHorseBatteryStaple
db-prod00_rw=CollectHouseButterySteeple
...
 
$ sssp --name=dbaccess --group db-team --secrets secrets.asc
Enter passphrase: ********************
 
$ ls -la /var/tmp/dbaccess
srw-rw----  1 me  db-team 0 Apr 19 04:43 /var/tmp/sssp.dbaccess.sock

Simple Socket Secret Provider (cont)

Getting values

$ sssp --name dbaccess --get db-prod00_rw
CorrectHorseBatteryStaple


$ less somecode.py
#!/usr/bin/env python
import sssp
...

   pw = sssp.get_value('dbaccess', 'db-prod00_rw')

   conn = mysql.connector.connect(
	password=pw, host='proddb', user='....
...

Shamir's Secret Sharing Scheme

A way to require multiple parties to access a secret.

$ ssss-split  -t 3 -n 15 -w vaultpw
Generating shares using a (3,15) scheme with dynamic security level.
Enter the secret, at most 128 ASCII characters: ********************
vaultpw-01-6c7b791d613ec5f11adb6e5c60a6ee2df45d5a54612200b26bc613
vaultpw-02-1affabcc4ac3cf3d90cf7d2600caa8c31b7cc8747cf0f50c255718
vaultpw-03-edfd0e823a2c5a1e027aa5cfa55ad0686db5154a50c77ef2519f9d
vaultpw-04-2679c7634cc2b9ac4e04c89075bdd64060abccdf88f6d13f249dec
vaultpw-05-d17b622d3c2d2c8fdcb11079d02daeeb166211e1a4c15ac150557b
vaultpw-06-a7ffb0fc17d0264356a50303b041e805f94383c1b913af7f1ec454
vaultpw-07-50fd15b2673fb360c410dbea15d190ae8f8a5eff952424816a0cc5
vaultpw-08-194a3a08072dd8acd81deef649ece63c015546275e75a46754a9a9
vaultpw-09-ee489f4677c24d8f4aa8361fec7c9e97779c9b1972422f99206162
vaultpw-10-98cc4d975c3f4743c0bc25658c10d87998bd09396f90da276ef081
vaultpw-11-6fcee8d92cd0d2605209fd8c2980a0d2ee74d40743a751d91a384c
vaultpw-12-a44a21385a3e31d21e7790d3f967a6fae36a0d929b96fe146f3b55
...

Shamir's Secret Sharing Scheme (cont)

Reconstructing

$ ssss-combine -t 3
Enter 3 shares separated by newlines:
Share [1/3]: vaultpw-14-25cc56a7012cae3d06d65b403c9b98bf7a82428caa73805455624d
Share [2/3]: vaultpw-06-a7ffb0fc17d0264356a50303b041e805f94383c1b913af7f1ec454
Share [3/3]: vaultpw-03-edfd0e823a2c5a1e027aa5cfa55ad0686db5154a50c77ef2519f9d
Resulting secret: Affirm*Barbarian!Moratorium

SeaGL, 2016-11-12