1. Secret Logo
  2. Blog

How does Secrets store your data?

Update: Secrets 4.0 introduced a new storage format so this post no longer applies. Please see the new security design for up to date information.

 

When choosing a password manager it's expected that one of the main concerns relates to information security. Keeping your information secure takes many forms but the most notable is how your data is stored on disk. Namely, how is it encrypted and protected.

Cryptography is hard... we know. Although the concepts behind different encryption strategies and integrity protection algorithms are easy to understand, putting it all together can easily introduce vulnerabilities. This is why we decided to use the most recent version of the open and battle proven standard called OpenPGP.

OpenPGP defines how data should be stored. It builds on top of the familiar encryption building blocks like AES and the SHA family. It's been around for many years and has been widely used -- from securing e-mail to authenticating a developer's contributions to a source code repository. And of course, it has also seen some revisions along the way.

We understand OpenPGP is a pretty complex standard. It covers symmetric and asymmetric cryptography (supported by different algorithms), authentication and integrity protection, key storage... and having different revisions makes for an even more complex implementation. So why choose it versus some other simpler standard?

Here is the rationale behind the decision:

  1. Although complex it is also very flexible. It will allow us to evolve Secrets with new features while still using a familiar and tested format.

  2. It's been tested and used by many, giving us confidence on the security it provides.

  3. There are open third-party tools that can read it, providing an easy way for our users to verify our claims.

The remainder of this post will elaborate on that last point. Explaining how Secrets stores its data and how you can verify it for yourself. This is going to get a bit technical so if you're not into that here's the gist of it:

Secrets uses OpenPGP format with AES-128 for symmetric encryption, RSA for asymmetric encryption and signatures, and SHA-256 as the hashing algorithm.

File Format

Secret's on disk file format is actually composed of many different components bundled together in what's called a "file package".

A file package perceived by the user as a single file but it's actually a directory containing files and possibly other directories.

Package structure

Secrets.secrets		
├── index		①
├── keys		②
├── master		③
└── store		④

① index

Each item the user saves in the store file contains an identifier. The index file maps words used on those items to the respective identifiers. This is then used to perform fuzzy searching.

It's encrypted and signed according to the OpenPGP format. The symmetric-key, also called a session key, is generated at the time of writing. This symmetric-key is used to drive an AES-128 cipher in CFB mode encrypting this file's payload. The session key is encrypted with the RSA encryption public-key present in the master file. Finally the file is signed with the signature private-key present in the master file.

This can be verified using the pgpdump tool as follows:

$ pgpdump Secrets.secrets/index
New: One-Pass Signature Packet(tag 4)(13 bytes)
	New version(3)
	Sig type - Signature of a binary document(0x00).
	Hash alg - SHA256(hash 8)
	Pub alg - RSA Encrypt or Sign(pub 1)
	Key ID - 0xF1832FA88B9C5A66
	Next packet - other than one pass signature
New: Public-Key Encrypted Session Key Packet(tag 1)(140 bytes)
	New version(3)
	Key ID - 0xF3D47B4CF542BA5B
	Pub alg - RSA Encrypt or Sign(pub 1)
	RSA m^e mod n(1024 bits) - ...
		-> m = sym alg(1 byte) + checksum(2 bytes) + PKCS-1 block type 02
New: Symmetrically Encrypted and MDC Packet(tag 18)(10263 bytes)
	Ver 1
	Encrypted data \[sym alg is specified in pub-key encrypted session key]
		(plain text + MDC SHA1(20 bytes))
New: Signature Packet(tag 2)(156 bytes)
	Ver 4 - new
	Sig type - Signature of a binary document(0x00).
	Pub alg - RSA Encrypt or Sign(pub 1)
	Hash alg - SHA256(hash 8)
	Hashed Sub: signature creation time(sub 2)(4 bytes)
		Time - Sun Mar 29 10:35:48 WEST 2015
	Hashed Sub: issuer key ID(sub 16)(8 bytes)
		Key ID - 0xF1832FA88B9C5A66
	Hash left 2 bytes - ab 70 
	RSA m^d mod n(1023 bits) - ...
		-> PKCS-1

② keys

The keys, also called a keyring, stores a series of OpenPGP public keys that the user has trusted. This is reserved for future features and is currently not in use.

③ master

Contains the RSA key pairs for this file package. It consists of three RSA key pairs: a 4096 bit RSA master key, and two 2048 bit sub-keys for encrypting and signing. All private keys are encrypted with an encryption key based on the master passphrase the user has chosen. An iterated and salted key derivation method is used. This method is documented in the OpenPGP specification.

This file can be verified with the pgpdump tool as follows:

$ pgpdump Secrets.secrets/master
New: Secret Key Packet(tag 5)(1862 bytes)
	Ver 4 - new
	Public key creation time - Mon Mar 30 21:44:41 WEST 2015
	Pub alg - RSA Encrypt or Sign(pub 1)
	RSA n(4096 bits) - ...
	RSA e(17 bits) - ...
	Sym alg - AES with 128-bit key(sym 7)
	Iterated and salted string-to-key(s2k 3):
		Hash alg - SHA256(hash 8)
		Salt - 68 99 58 eb 66 fb 77 c3 
		Count - 11534336(coded count 214)
	IV - 7b ca 80 9d 24 af 9d 42 06 fc 5d c3 f1 77 6f 52 
	Encrypted RSA d
	Encrypted RSA p
	Encrypted RSA q
	Encrypted RSA u
	Encrypted SHA1 hash
New: User ID Packet(tag 13)(68 bytes)
	User ID - 74988A20-B556-4E93-9199-59CEA24683EE@datastore.secrets.outercorner.com
New: Signature Packet(tag 2)(543 bytes)
	Ver 4 - new
	Sig type - Generic certification of a User ID and Public Key packet(0x10).
	Pub alg - RSA Encrypt or Sign(pub 1)
	Hash alg - SHA256(hash 8)
	Hashed Sub: issuer key ID(sub 16)(8 bytes)
		Key ID - 0x2A836F62575F7752
	Hashed Sub: signature creation time(sub 2)(4 bytes)
		Time - Mon Mar 30 21:44:41 WEST 2015
	Hashed Sub: primary User ID(sub 25)(1 bytes)
		Primary - Yes
	Hash left 2 bytes - 8d c4 
	RSA m^d mod n(4093 bits) - ...
		-> PKCS-1
New: Secret Subkey Packet(tag 7)(966 bytes)
	Ver 4 - new
	Public key creation time - Mon Mar 30 21:44:31 WEST 2015
	Pub alg - RSA Encrypt or Sign(pub 1)
	RSA n(2048 bits) - ...
	RSA e(17 bits) - ...
	Sym alg - AES with 128-bit key(sym 7)
	Iterated and salted string-to-key(s2k 3):
		Hash alg - SHA256(hash 8)
		Salt - 59 90 3c 76 67 e3 c8 f0 
		Count - 11534336(coded count 214)
	IV - 15 e0 1d 1b 06 e5 3a f2 ce 32 2b e1 c0 7e af cc 
	Encrypted RSA d
	Encrypted RSA p
	Encrypted RSA q
	Encrypted RSA u
	Encrypted SHA1 hash
New: Signature Packet(tag 2)(833 bytes)
	Ver 4 - new
	Sig type - Subkey Binding Signature(0x18).
	Pub alg - RSA Encrypt or Sign(pub 1)
	Hash alg - SHA256(hash 8)
	Hashed Sub: issuer key ID(sub 16)(8 bytes)
		Key ID - 0x2A836F62575F7752
	Hashed Sub: signature creation time(sub 2)(4 bytes)
		Time - Mon Mar 30 21:44:41 WEST 2015
	Hashed Sub: key flags(sub 27)(1 bytes)
		Flag - This key may be used to sign data
		Flag - This key may be used for authentication
	Hashed Sub: embedded signature(sub 32)(284 bytes)
	Ver 4 - new
	Sig type - Primary Key Binding Signature(0x19).
	Pub alg - RSA Encrypt or Sign(pub 1)
	Hash alg - SHA256(hash 8)
	Hashed Sub: issuer key ID(sub 16)(8 bytes)
		Key ID - 0x9C436901DE8A3416
	Hashed Sub: signature creation time(sub 2)(4 bytes)
		Time - Mon Mar 30 21:44:41 WEST 2015
	Hash left 2 bytes - dc a8 
	RSA m^d mod n(2047 bits) - ...
		-> PKCS-1
	Hash left 2 bytes - 0d b5 
	RSA m^d mod n(4094 bits) - ...
		-> PKCS-1
New: Secret Subkey Packet(tag 7)(966 bytes)
	Ver 4 - new
	Public key creation time - Mon Mar 30 21:44:31 WEST 2015
	Pub alg - RSA Encrypt or Sign(pub 1)
	RSA n(2048 bits) - ...
	RSA e(17 bits) - ...
	Sym alg - AES with 128-bit key(sym 7)
	Iterated and salted string-to-key(s2k 3):
		Hash alg - SHA256(hash 8)
		Salt - 23 ed 5a b1 6b 5b e7 0e 
		Count - 11534336(coded count 214)
	IV - 59 18 5b 65 20 9f 9b 8f c9 84 5a 75 7a a0 13 a7 
	Encrypted RSA d
	Encrypted RSA p
	Encrypted RSA q
	Encrypted RSA u
	Encrypted SHA1 hash
New: Signature Packet(tag 2)(543 bytes)
	Ver 4 - new
	Sig type - Subkey Binding Signature(0x18).
	Pub alg - RSA Encrypt or Sign(pub 1)
	Hash alg - SHA256(hash 8)
	Hashed Sub: issuer key ID(sub 16)(8 bytes)
		Key ID - 0x2A836F62575F7752
	Hashed Sub: signature creation time(sub 2)(4 bytes)
		Time - Mon Mar 30 21:44:41 WEST 2015
	Hashed Sub: key flags(sub 27)(1 bytes)
		Flag - This key may be used to encrypt communications
		Flag - This key may be used to encrypt storage
	Hash left 2 bytes - 3f 54 
	RSA m^d mod n(4096 bits) - ...
		-> PKCS-1

④ store

This is where all user data is stored. It's actually a TAR file composed of two files: metadata and objects. The metadata file is in plain text and contains information such as the store id and model version. The objects file is encrypted and signed in the same fashion as the index file described above.

The metadata file is an NSKeyedArchiver file and can be extracted with the PlistExplorer tool:

$ tar -xvf Secrets.secrets/store metadata
x metadata
$ ~/bin/PlistExplorer metadata
NSPersistenceFrameworkVersion
  526
NSStoreModelVersionHashes
    BankAccount
        NSData with: 32 Bytes
    Credential
        NSData with: 32 Bytes
    CreditCard
        NSData with: 32 Bytes
    Note
        NSData with: 32 Bytes
    PairedPeer
        NSData with: 32 Bytes
    Secret
        NSData with: 32 Bytes
    Service
        NSData with: 32 Bytes
    Setting
        NSData with: 32 Bytes
    SoftwareLicense
        NSData with: 32 Bytes
    SyncedObject
        NSData with: 32 Bytes
    SyncedRelationship
        NSData with: 32 Bytes
    SyncedStore
        NSData with: 32 Bytes
    Tag
        NSData with: 32 Bytes
NSStoreModelVersionHashesVersion
  3
NSStoreModelVersionIdentifiers        
NSStoreType
  PGPAtomicStore
NSStoreUUID
  CE35A8CC-759E-4504-B719-F7E257E39767
lastReferenceObject
  399
sbx_identifier
  183E6D35-B74D-48EE-A9C5-1ADFAC955E02@datastore.secrets.outercorner.com

The objects file can be verified as with the pgpdump tool as follows:

$ tar -xOf Secrets.secrets/store objects | pgpdump
New: One-Pass Signature Packet(tag 4)(13 bytes)
	New version(3)
	Sig type - Signature of a binary document(0x00).
	Hash alg - SHA256(hash 8)
	Pub alg - RSA Encrypt or Sign(pub 1)
	Key ID - 0xF1832FA88B9C5A66
	Next packet - other than one pass signature
New: Public-Key Encrypted Session Key Packet(tag 1)(140 bytes)
	New version(3)
	Key ID - 0xF3D47B4CF542BA5B
	Pub alg - RSA Encrypt or Sign(pub 1)
	RSA m^e mod n(1023 bits) - ...
		-> m = sym alg(1 byte) + checksum(2 bytes) + PKCS-1 block type 02
New: Symmetrically Encrypted and MDC Packet(tag 18)(133256 bytes)
	Ver 1
	Encrypted data [sym alg is specified in pub-key encrypted session key]
		(plain text + MDC SHA1(20 bytes))
New: Signature Packet(tag 2)(156 bytes)
	Ver 4 - new
	Sig type - Signature of a binary document(0x00).
	Pub alg - RSA Encrypt or Sign(pub 1)
	Hash alg - SHA256(hash 8)
	Hashed Sub: signature creation time(sub 2)(4 bytes)
		Time - Sun Mar 29 10:35:48 WEST 2015
	Hashed Sub: issuer key ID(sub 16)(8 bytes)
		Key ID - 0xF1832FA88B9C5A66
	Hash left 2 bytes - c7 e1 
	RSA m^d mod n(1021 bits) - ...
		-> PKCS-1

Conclusion

We do our best to use proven and open standards everywhere we can. Although complex, OpenPGP is flexible enough to be used in many different scenarios and it will serve as the foundation for many new features we have planed. Stay tuned!