Skip to content

Commit

Permalink
Resolve #29 Support additional sqlite open flags
Browse files Browse the repository at this point in the history
  • Loading branch information
@npurushe
npurushe committed Jan 6, 2017
1 parent 92e8bab commit 6f2070b
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 85 deletions.
8 changes: 3 additions & 5 deletions sqlite-android/src/main/java/io/requery/android/database/sqlite/SQLiteConnection.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,9 @@ void close() {
}

private void open() {
mConnectionPtr = nativeOpen(mConfiguration.path, mConfiguration.openFlags,
mConnectionPtr = nativeOpen(mConfiguration.path,
// remove the wal flag as its a custom flag not supported by sqlite3_open_v2
mConfiguration.openFlags & ~SQLiteDatabase.ENABLE_WRITE_AHEAD_LOGGING,
mConfiguration.label,
SQLiteDebug.DEBUG_SQL_STATEMENTS, SQLiteDebug.DEBUG_SQL_TIME);

Expand Down Expand Up @@ -378,10 +380,6 @@ private void setJournalMode(String newValue) {
}

private void setLocaleFromConfiguration() {
if ((mConfiguration.openFlags & SQLiteDatabase.NO_LOCALIZED_COLLATORS) != 0) {
return;
}

// Register the localized collators.
final String newLocale = mConfiguration.locale.toString();
nativeRegisterLocalizedCollators(mConnectionPtr, newLocale);
Expand Down
94 changes: 34 additions & 60 deletions sqlite-android/src/main/java/io/requery/android/database/sqlite/SQLiteDatabase.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -206,70 +206,48 @@ protected SQLiteSession initialValue() {
private static final String[] CONFLICT_VALUES = new String[]
{"", " OR ROLLBACK ", " OR ABORT ", " OR FAIL ", " OR IGNORE ", " OR REPLACE "};

/**
* Maximum Length Of A LIKE Or GLOB Pattern
* The pattern matching algorithm used in the default LIKE and GLOB implementation
* of SQLite can exhibit O(N^2) performance (where N is the number of characters in
* the pattern) for certain pathological cases. To avoid denial-of-service attacks
* the length of the LIKE or GLOB pattern is limited to SQLITE_MAX_LIKE_PATTERN_LENGTH bytes.
* The default value of this limit is 50000. A modern workstation can evaluate
* even a pathological LIKE or GLOB pattern of 50000 bytes relatively quickly.
* The denial of service problem only comes into play when the pattern length gets
* into millions of bytes. Nevertheless, since most useful LIKE or GLOB patterns
* are at most a few dozen bytes in length, paranoid application developers may
* want to reduce this parameter to something in the range of a few hundred
* if they know that external users are able to generate arbitrary patterns.
*/
public static final int SQLITE_MAX_LIKE_PATTERN_LENGTH = 50000;
/** Open flag to open in the database in read only mode */
public static final int OPEN_READONLY = 0x00000001;

/**
* Open flag: Flag for {@link #openDatabase} to open the database for reading and writing.
* If the disk is full, this may fail even before you actually write anything.
*
* {@more} Note that the value of this flag is 0, so it is the default.
*/
public static final int OPEN_READWRITE = 0x00000000; // update native code if changing
/** Open flag to open in the database in read/write mode */
public static final int OPEN_READWRITE = 0x00000002;

/**
* Open flag: Flag for {@link #openDatabase} to open the database for reading only.
* This is the only reliable way to open a database if the disk may be full.
*/
public static final int OPEN_READONLY = 0x00000001; // update native code if changing
/** Open flag to create the database if it does not exist */
public static final int OPEN_CREATE = 0x00000004;

/**
* Open flag: Flag for {@link #openDatabase} to open the database without support for
* localized collators.
*
* {@more} This causes the collator <code>LOCALIZED</code> not to be created.
* You must be consistent when using this flag to use the setting the database was
* created with. If this is set, {@link #setLocale} will do nothing.
*/
public static final int NO_LOCALIZED_COLLATORS = 0x00000010; // update native code if changing
/** Open flag to support URI filenames */
public static final int OPEN_URI = 0x00000040;

/**
* Open flag: Flag for {@link #openDatabase} to create the database file if it does not
* already exist.
*/
public static final int CREATE_IF_NECESSARY = 0x10000000; // update native code if changing
/** Open flag opens the database in multi-thread threading mode */
public static final int OPEN_NOMUTEX = 0x00008000;

/**
* Open flag: Flag for {@link #openDatabase} to open the database file with
* write-ahead logging enabled by default. Using this flag is more efficient
* than calling {@link #enableWriteAheadLogging}.
*
* Write-ahead logging cannot be used with read-only databases so the value of
* this flag is ignored if the database is opened read-only.
*
* @see #enableWriteAheadLogging
*/
/** Open flag opens the database in serialized threading mode */
public static final int OPEN_FULLMUTEX = 0x00010000;

/** Open flag opens the database in shared cache mode */
public static final int OPEN_SHAREDCACHE = 0x00020000;

/** Open flag opens the database in private cache mode */
public static final int OPEN_PRIVATECACHE = 0x00040000;

/** Open flag equivalent to {@link #OPEN_READWRITE} | {@link #OPEN_CREATE} */
public static final int CREATE_IF_NECESSARY = OPEN_READWRITE | OPEN_CREATE;

/** Open flag to enable write-ahead logging */ // custom flag remove for sqlite3_open_v2
public static final int ENABLE_WRITE_AHEAD_LOGGING = 0x20000000;

/** Integer flag definition for the database open options */
@SuppressLint("UniqueConstants") // duplicate values provided for compatibility
@IntDef(flag = true, value = {
OPEN_READONLY,
OPEN_READWRITE,
OPEN_CREATE,
OPEN_URI,
OPEN_NOMUTEX,
OPEN_FULLMUTEX,
OPEN_SHAREDCACHE,
OPEN_PRIVATECACHE,
CREATE_IF_NECESSARY,
NO_LOCALIZED_COLLATORS,
ENABLE_WRITE_AHEAD_LOGGING})
@Retention(RetentionPolicy.SOURCE)
public @interface OpenFlags {
Expand Down Expand Up @@ -651,8 +629,7 @@ private boolean yieldIfContendedHelper(boolean throwIfUnsafe, long sleepAfterYie
}

/**
* Open the database according to the flags {@link #OPEN_READWRITE}
* {@link #OPEN_READONLY} {@link #CREATE_IF_NECESSARY} and/or {@link #NO_LOCALIZED_COLLATORS}.
* Open the database according to the flags {@link OpenFlags}
*
* <p>Sets the locale of the database to the the system's current locale.
* Call {@link #setLocale} if you would like something else.</p>
Expand All @@ -671,8 +648,7 @@ public static SQLiteDatabase openDatabase(String path,
}

/**
* Open the database according to the flags {@link #OPEN_READWRITE}
* {@link #OPEN_READONLY} {@link #CREATE_IF_NECESSARY} and/or {@link #NO_LOCALIZED_COLLATORS}.
* Open the database according to the flags {@link OpenFlags}
*
* <p>Sets the locale of the database to the the system's current locale.
* Call {@link #setLocale} if you would like something else.</p>
Expand Down Expand Up @@ -700,8 +676,7 @@ public static SQLiteDatabase openDatabase(String path,
}

/**
* Open the database according to the flags {@link #OPEN_READWRITE}
* {@link #OPEN_READONLY} {@link #CREATE_IF_NECESSARY} and/or {@link #NO_LOCALIZED_COLLATORS}.
* Open the database according to the given configuration.
*
* <p>Sets the locale of the database to the the system's current locale.
* Call {@link #setLocale} if you would like something else.</p>
Expand Down Expand Up @@ -1742,8 +1717,7 @@ public final String getPath() {
}

/**
* Sets the locale for this database. Does nothing if this database has
* the {@link #NO_LOCALIZED_COLLATORS} flag set or was opened read only.
* Sets the locale for this database.
*
* @param locale The new locale.
*
Expand Down
22 changes: 2 additions & 20 deletions sqlite-android/src/main/jni/sqlite/android_database_SQLiteConnection.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,16 +63,6 @@ static struct {
} gStringClassInfo;

struct SQLiteConnection {
// Open flags.
// Must be kept in sync with the constants defined in SQLiteDatabase.java.
enum {
OPEN_READWRITE = 0x00000000,
OPEN_READONLY = 0x00000001,
OPEN_READ_MASK = 0x00000001,
NO_LOCALIZED_COLLATORS = 0x00000010,
CREATE_IF_NECESSARY = 0x10000000,
};

sqlite3* const db;
const int openFlags;
std::string path;
Expand Down Expand Up @@ -131,14 +121,6 @@ static int coll_localized(

static jlong nativeOpen(JNIEnv* env, jclass clazz, jstring pathStr, jint openFlags,
jstring labelStr, jboolean enableTrace, jboolean enableProfile) {
int sqliteFlags;
if (openFlags & SQLiteConnection::CREATE_IF_NECESSARY) {
sqliteFlags = SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE;
} else if (openFlags & SQLiteConnection::OPEN_READONLY) {
sqliteFlags = SQLITE_OPEN_READONLY;
} else {
sqliteFlags = SQLITE_OPEN_READWRITE;
}

const char* pathChars = env->GetStringUTFChars(pathStr, NULL);
std::string path(pathChars);
Expand All @@ -149,7 +131,7 @@ static jlong nativeOpen(JNIEnv* env, jclass clazz, jstring pathStr, jint openFla
env->ReleaseStringUTFChars(labelStr, labelChars);

sqlite3* db;
int err = sqlite3_open_v2(path.c_str(), &db, sqliteFlags, NULL);
int err = sqlite3_open_v2(path.c_str(), &db, openFlags, NULL);
if (err != SQLITE_OK) {
throw_sqlite3_exception_errcode(env, err, "Could not open database");
return 0;
Expand All @@ -162,7 +144,7 @@ static jlong nativeOpen(JNIEnv* env, jclass clazz, jstring pathStr, jint openFla
}

// Check that the database is really read/write when that is what we asked for.
if ((sqliteFlags & SQLITE_OPEN_READWRITE) && sqlite3_db_readonly(db, NULL)) {
if ((openFlags & SQLITE_OPEN_READWRITE) && sqlite3_db_readonly(db, NULL)) {
throw_sqlite3_exception(env, db, "Could not open the database in read/write mode.");
sqlite3_close(db);
return 0;
Expand Down

3 comments on commit 6f2070b

@mikehardy
Copy link
Contributor

@mikehardy mikehardy commented on 6f2070b Jul 3, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@npurushe - this is a very late comment and I may be misunderstanding something, but as I work through upgrading AnkiDroid dependencies, I'm stumped taking requery/sqlite-android past 3.16 because this commit removes NO_LOCALIZED_COLLATORS, which is still present in the system implementation and is used by our project:

https://developer.android.com/reference/android/database/sqlite/SQLiteDatabase#NO_LOCALIZED_COLLATORS

If I were to stop opening our database with this flag, would that imply that databases created prior to the change with the flag will potentially have different locale settings than are being used to query it? Is that inconsistency problematic or is it safe to remove the flag and carry on?

I saw no discussion anywhere with searching so I'm guessing it's not a problem but we have a huge installed base with lots of languages and versions...

Thanks!

Update:
I tried simply stripping the flag from the open call and on existing databases, I get this:

07-03 07:56:59.001 4420-4452/com.ichi2.anki I/CollectionHelper: openCollection: /storage/emulated/0/AnkiDroid/collection.anki2
07-03 07:56:59.024 4420-4452/com.ichi2.anki E/SQLiteLog: (21) misuse at line 142191 of [ada05cfa86]
07-03 07:56:59.025 4420-4452/com.ichi2.anki E/SQLiteDatabase: Failed to open database '/storage/emulated/0/AnkiDroid/collection.anki2'.
android.database.sqlite.SQLiteMisuseException: unknown error (code 21): Could not open database

I suppose I'm missing a generic way to inject underlying flags into the requery-level open database call and I'll research that...

@npurushe
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mikehardy Can you please open an issue with this information. Thanks!

@mikehardy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing

Please sign in to comment.