MigrationTestHelper rejects a migration that AppDatabase_Impl's onValidateSchema accepts (creation of FTS table) [145858914]

Fixed

Bug

androidx-bug-bounty

Status Update

No update yet.

Description

ah...@gmail.com

created issue #1

Dec 8, 2019 08:13PM

Component used:
Version used: 2.2.2
Devices/Android versions reproduced on: Android emulator running API 28

I've included a sample project with a MigrationTest class that fails like so when invoking MigrationTestHelper:

java.lang.IllegalStateException: Migration didn't properly handle: user_fts
Expected: FtsTableInfo{name='user_fts', columns=[name, rowid], options=[]}
Found: FtsTableInfo{name='user_fts', columns=[name], options=[]}

Yet the AppDatabase_Impl sees the schema as being correct.

The root cause seems to be that they treat the implicit `rowid` column differently: the generated schema json file includes the `rowid` field causing MigrationTestHelper to expect its presence, but AppDatabase_Impl's onValidateSchema method doesn't check for rowid.

I believe AppDatabase_Impl has the correct behaviour in this case: MigrationTestHelper uses FtsTableInfo which uses `PRAGMA table_info()` to discover the columns of the fts table, which *doesn't* return `rowid` as being one of the columns, but MigrationTestHelper is comparing that FtsTableInfo with the json data which *does* include `rowid`, so it thinks the schema doesn't match.

RoomFtsMigrationFailRepro.zip

1.4 MB

Download

Comments

yb...@google.com <yb...@google.com> #2Jul 30, 2020 08:33PM

i think we should probably make validation ignore rowid for fts tables. an alternative might be to change the generated schema to include it but that might have the side effect of changing the validation hashes etc :/.

da...@google.com <da...@google.com> #3Jul 30, 2020 08:40PM

I actually have a CL to fix this, but never followed through on review: https://android-review.googlesource.com/c/platform/frameworks/support/+/1250152

ap...@google.com <ap...@google.com> #4Aug 4, 2020 04:11PM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit d62ebcd6f4a555e25596db7bdc46e6b339391045
Author: Daniel Santiago Rivera <danysantiago@google.com>
Date: Wed Mar 04 09:48:24 2020

Don't export hidden FTS columns in schema bundles.

FTS tables always have a hidden rowid table and an optional
languageid. When exporting the schema to a bundle that gets
serialized to a JSON, we shouldn't include such columns
because during verification they will not be available when
executing PRAGMA table_info.

This fixes users of the testing library that had migrations
to an FTS table with either a named rowid column or a languageid
column.

Bug: 145858914
Test: FtsMigrationTest
Change-Id: I3ff09337c861fdca6056c2bce29f2873dd8166c1

M room/compiler/src/main/kotlin/androidx/room/vo/FtsEntity.kt
A room/integration-tests/testapp/schemas/androidx.room.integration.testapp.migration.FtsMigrationTest.FtsMigrationDb/5.json
A room/integration-tests/testapp/schemas/androidx.room.integration.testapp.migration.FtsMigrationTest.FtsMigrationDb/6.json
M room/integration-tests/testapp/src/androidTest/java/androidx/room/integration/testapp/migration/FtsMigrationTest.java

https://android-review.googlesource.com/1250152

da...@google.com <da...@google.com> Aug 4, 2020 04:13PM

Marked as fixed.

to...@gmail.com <to...@gmail.com> #5Nov 13, 2020 09:17AM

Version used: 2.3.0-alpha03
Devices/Android versions reproduced on: oneplus 5t and Huawei honer8

E/TestRunner: java.lang.IllegalStateException: Migration failed. expected:TableInfo{name='messages_fts4', columns={message_id=Column{name='message_id', type='TEXT', affinity='2', notNull=true, primaryKeyPosition=0, defaultValue='null'}, content=Column{name='content', type='TEXT', affinity='2', notNull=false, primaryKeyPosition=0, defaultValue='null'}}, foreignKeys=[], indices=[]} , found:TableInfo{name='messages_fts4', columns={message_id=Column{name='message_id', type='', affinity='1', notNull=false, primaryKeyPosition=0, defaultValue='null'}, content=Column{name='content', type='', affinity='1', notNull=false, primaryKeyPosition=0, defaultValue='null'}}, foreignKeys=[], indices=[]}

Database_Impl.java

_db.execSQL("CREATE VIRTUAL TABLE IF NOT EXISTS `messages_fts4` USING FTS4(`message_id` TEXT NOT NULL, `content` TEXT, tokenize=unicode61, notindexed=`message_id`)");

10.json

{
"ftsVersion": "FTS4",
"ftsOptions": {
"tokenizer": "unicode61",
"tokenizerArgs": [],
"contentTable": "",
"languageIdColumnName": "",
"matchInfo": "FTS4",
"notIndexedColumns": [
"message_id"
],
"prefixSizes": [],
"preferredOrder": "ASC"
},
"contentSyncTriggers": [],
"tableName": "messages_fts4",
"createSql": "CREATE VIRTUAL TABLE IF NOT EXISTS `${TABLE_NAME}` USING FTS4(`message_id` TEXT NOT NULL, `content` TEXT, tokenize=unicode61, notindexed=`message_id`)",
"fields": [
{
"fieldPath": "messageId",
"columnName": "message_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "content",
"columnName": "content",
"affinity": "TEXT",
"notNull": false
}
],
"primaryKey": {
"columnNames": [],
"autoGenerate": false
},
"indices": [],
"foreignKeys": []
}

Maybe it's a same issue, I don't know the [found:TableInfo] come from.

Issue 145858914