Clean up ahead of PR.

This commit is contained in:
Matthew Chen 2018-01-25 10:22:08 -05:00
parent a57b0c1adb
commit fcc56870be
3 changed files with 129 additions and 34 deletions

View File

@ -4,6 +4,8 @@
NS_ASSUME_NONNULL_BEGIN
#ifdef SQLITE_HAS_CODEC
extern const NSUInteger kSqliteHeaderLength;
extern const NSUInteger kSQLCipherSaltLength;
extern const NSUInteger kSQLCipherDerivedKeyLength;
@ -12,24 +14,136 @@ extern const NSUInteger kSQLCipherKeySpecLength;
typedef void (^YapDatabaseSaltBlock)(NSData *saltData);
typedef void (^YapDatabaseKeySpecBlock)(NSData *keySpecData);
// Used to convert YapDatabase/SQLCipher databases whose header is encrypted
// to databases whose first 32 bytes are unencrypted so that iOS can determine
// that this is a SQLite database using WAL and therefore not terminate the app
// when it is suspended.
// This class contains utility methods for use with SQLCipher encrypted
// databases, specifically to address an issue around database files that
// reside in the "shared data container" used to share files between
// iOS main apps and their app extensions.
//
// See comments in YapDatabaseOptions for more details.
//
// The Issue
//
// iOS will terminate suspended apps which hold a file lock on files in the shared
// container. An exception is made for certain kinds of Sqlite files, so that iOS apps
// can share databases with their app extensions. Unfortunately, this exception does
// not apply for SQLCipher databases which have encrypted the Sqlite file header,
// which is the default behavior of SQLCipher. Therefore apps which try to share an
// SQLCipher database with their app extensions and use WAL (write-ahead logging) will
// be terminated whenever they are sent to the background (0x10deadcc terminations).
//
// * YapDatabase always uses WAL.
// * This issue seems to affect all versions of iOS and all device models.
// * iOS only terminates apps for this reason when app transition from the `background`
// to `suspended` states. iOS main apps can delay being suspended by creating a
// "background task", but this only defers the issue briefly as there are strict
// limits on the duration of "background tasks".
// * `0xdead10cc` terminations don't occur in the simulator and won't occur on devices
// if the debugger is attached.
// * These `0xdead10cc` terminations usually don't yield crash logs on the device, but
// always show up in the device console logs.
//
// See:
//
// * https://developer.apple.com/library/content/technotes/tn2408/_index.html
// * References to 0x10deadcc in https://developer.apple.com/library/content/technotes/tn2151/_index.html
//
//
// Solution
//
// The solution is to have SQLCipher encrypt everything _except_ the first 32 bytes of
// the Sqlite file, which corresponds to the first part of the Sqlite header. This is
// accomplished using the cipher_plaintext_header_size PRAGMA.
//
// The header does not contain any user data. See:
// https://www.sqlite.org/fileformat.html#the_database_header
//
// However, Sqlite normally uses the first 16 bytes of the Sqlite header to store
// a salt value. Therefore when using unencrypted headers, it is also necessary
// to explicitly specify a salt value.
//
// It is possible to convert SQLCipher databases with encrypted headers to use
// unencrypted headers. However, during this conversion, the salt must be extracted
// and preserved by reading the first 16 bytes of the unconverted file.
//
//
// Implementation
//
// To open (a new or existing) YapDatabase using unencrypted headers, you have two
// options:
//
// Option A:
//
// * Use cipherKeyBlock as usual to specify the database password.
// * Use cipherSaltBlock to specify the database salt. It should be kSQLCipherSaltLength long.
// * Use cipherUnencryptedHeaderLength to specify how many bytes to leave unencrypted.
// This should be kSqliteHeaderLength.
// * Do not use a cipherKeySpecBlock.
//
// Option B:
//
// * Use cipherKeySpecBlock to specify the database key spec. It should be kSQLCipherKeySpecLength long.
// * Use cipherUnencryptedHeaderLength to specify how many bytes to leave unencrypted.
// This should be kSqliteHeaderLength.
// * The "key spec" includes the key derived from the database password and the salt,
// so do not use a cipherKeyBlock or cipherSaltBlock.
//
// Option B is more performant than Option A and is therefore recommended.
//
//
// Upgrading legacy databases to use unencrypted headers:
//
// * Call the convertDatabaseIfNecessary method of this class _before_
// trying to open any YapDatabase that may need to be converted.
// * This method will have no effect if the YapDatabase has already been converted.
// * This method should always be pretty fast, and should be safe to
// call from within [UIApplicationDelegate application: didFinishLaunchingWithOptions:].
// * If convertDatabaseIfNecessary converts the database, it will use its
// saltBlock and keySpecBlock parameters to inform you of the salt
// and keyspec for this database. These values will be needed when
// opening the database, so they should presumably stored in the
// keychain (like the database password).
//
//
// Creating new databases with unencrypted headers:
//
// * Randomly generate a database password and salt, presumably using SecRandomCopyBytes().
// * Derive a keyspec using databaseKeySpecForPassword:.
// * You probably should store these values in the keychain.
//
//
// Note and Disclaimer
//
// There is no authoritative documentation from Apple about iOS' usage of the Sqlite
// file header to make an exception for suspended apps with a file lock on database
// files in the shared container. Our usage of the first 32 bytes as being sufficient
// is only empirical.
@interface YapDatabaseCryptoUtils : NSObject
- (instancetype)init NS_UNAVAILABLE;
// Returns YES IFF the database appears to have encrypted headers.
+ (BOOL)doesDatabaseNeedToBeConverted:(NSString *)databaseFilePath;
// * Call the convertDatabaseIfNecessary method of this class _before_
// trying to open any YapDatabase that may need to be converted.
// * This method will have no effect if the YapDatabase has already been converted.
// * This method should always be pretty fast, and should be safe to
// call from within [UIApplicationDelegate application: didFinishLaunchingWithOptions:].
// * If convertDatabaseIfNecessary converts the database, it will use its
// saltBlock and keySpecBlock parameters to inform you of the salt
// and keyspec for this database. These values will be needed when
// opening the database, so they should presumably stored in the
// keychain (like the database password).
+ (nullable NSError *)convertDatabaseIfNecessary:(NSString *)databaseFilePath
databasePassword:(NSData *)databasePassword
saltBlock:(YapDatabaseSaltBlock)saltBlock
keySpecBlock:(YapDatabaseKeySpecBlock)keySpecBlock;
+ (nullable NSData *)deriveDatabaseKeyForPassword:(NSData *)passwordData saltData:(NSData *)saltData;
// This method can be used to derive a SQLCipher "key spec" from a
// database password and salt. Key spec derivation is somewhat costly.
// The key spec is needed every time the database file is opened
// (including every time YapDatabse makes a new database connection),
// So it benefits performance to pass a pre-derived key spec to
// YapDatabase.
+ (nullable NSData *)databaseKeySpecForPassword:(NSData *)passwordData saltData:(NSData *)saltData;
#pragma mark - Utils
@ -38,4 +152,6 @@ typedef void (^YapDatabaseKeySpecBlock)(NSData *keySpecData);
@end
#endif
NS_ASSUME_NONNULL_END

View File

@ -10,6 +10,8 @@
NS_ASSUME_NONNULL_BEGIN
#ifdef SQLITE_HAS_CODEC
#if DEBUG
static const int ydbLogLevel = YDB_LOG_LEVEL_INFO;
#else
@ -489,4 +491,6 @@ NSError *YDBErrorWithDescription(NSString *description)
@end
#endif
NS_ASSUME_NONNULL_END

View File

@ -237,8 +237,7 @@ typedef NSData *_Nonnull (^YapDatabaseCipherKeyBlock)(void);
*
* cipherSaltBlock will be ignored if you don't also set cipherKeyBlock and cipherUnencryptedHeaderLength.
*
* See comments on cipherUnencryptedHeaderLength. These two properties are
* intended to be used in conjunction.
* For more information, see comments in YapDatabaseCryptoUtils.h.
**/
@property (nonatomic, copy, readwrite) YapDatabaseCipherKeyBlock cipherSaltBlock;
@ -255,8 +254,7 @@ typedef NSData *_Nonnull (^YapDatabaseCipherKeyBlock)(void);
*
* cipherKeySpecBlock will be ignored if you don't also set cipherUnencryptedHeaderLength.
*
* See comments on cipherUnencryptedHeaderLength. These two properties are
* intended to be used in conjunction.
* For more information, see comments in YapDatabaseCryptoUtils.h.
**/
@property (nonatomic, copy, readwrite) YapDatabaseCipherKeyBlock cipherKeySpecBlock;
@ -266,36 +264,13 @@ typedef NSData *_Nonnull (^YapDatabaseCipherKeyBlock)(void);
*
* This value will be passed to SQLCipher via `PRAGMA cipher_plaintext_header_size`.
*
* iOS will terminate suspended apps which hold a file lock on files in the shared
* container. An exception is made for certain kinds of Sqlite files, so that iOS apps
* can share databases with their app extensions. Unfortunately, this exception does
* not apply for SQLCipher databases which have encrypted the Sqlite file header,
* which is the default behavior. Therefore apps which try to share an SQLCipher
* database with their app extensions and use WAL (write-ahead logging) will be
* terminated whenever they are sent to the background (0x10deadcc terminations).
*
* The solution is to have SQLCipher encrypt everything _except_ the first 32 bytes
* of the Sqlite file. This is accomplished using the cipher_plaintext_header_size
* PRAGMA. The header does not contain any user data.
*
* However, Sqlite normally uses the first 16 bytes of the Sqlite header to store
* a salt value. Therefore when using unencrypted headers, it is also necessary
* to explicitly specify a salt value.
*
* NOTE: When converting SQLCipher databases from using encrypted headers to using
* unencrypted headers, the salt can extracted and preserved by reading the first
* 16 bytes of the file.
*
* See: https://developer.apple.com/library/content/technotes/tn2151/_index.html
*
* You must use the 'YapDatabase/SQLCipher' subspec
* in your Podfile for this option to take effect.
*
* cipherUnencryptedHeaderLength will be ignored if you don't also set
* ((cipherKeyBlock AND cipherSaltBlock) OR cipherKeySpecBlock).
*
* See comments on cipherSaltBlock and cipherKeySpecBlock. These properties are
* intended to be used in conjunction.
* For more information, see comments in YapDatabaseCryptoUtils.h.
**/
@property (nonatomic, assign, readwrite) NSUInteger cipherUnencryptedHeaderLength;