IssueTracker

Project: platform/frameworks/support
Branch: androidx-master-dev

commit b90079595f33f58fece04026a97faa0d243acdb1
Author: Yuichi Araki <yaraki@google.com>
Date:   Wed Sep 18 16:55:49 2019

    Change the way to detect mismatch between POJO and query
   
    This fixes cursor mismatch warnings with expandProjection.
   
    Bug: 140759491
    Test: QueryMethodProcessorTest
    Change-Id: I7659002e5e0d1ef60fc1af2a625c0c36da0664d8

M       room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt
M       room/compiler/src/test/kotlin/androidx/room/processor/QueryMethodProcessorTest.kt
M       room/compiler/src/test/kotlin/androidx/room/testing/TestProcessor.kt

https://android-review.googlesource.com/1123258
https://goto.google.com/android-sha1/b90079595f33f58fece04026a97faa0d243acdb1

  The following changes were cherrypicked through Coastguard (https://android-build.googleplex.com/coastguard/)
  Release Track: (https://android-build.googleplex.com/coastguard/dashboard/6093116828680192/#/cherrypickHistory/all)
  Created: 2019-10-14 17:23:42.610596+00:00
  Changes: aosp/1123258
 

Project: platform/frameworks/support
Branch: androidx-master-dev

commit bdde5a1a970ddc9007b28de4aa29d60ffa588f08
Author: Yigit Boyar <yboyar@google.com>
Date:   Thu Apr 16 16:47:05 2020

    Re-factor how errors are dismissed when query is re-written
   
    This CL changes how we handle errors/warnings if query is
    re-written.
    There was a bug in expandProjection where we would report warnings
    for things that Room already fixes automatically (b/140759491).
    The solution to that problem (I7659002e5e0d1ef60fc1af2a625c0c36da0664d8)
    solved it by deferring validating of columns until after re-write
    decision is made. Unfortunately, this required changing PojoRowAdapter
    to have a dummy mapping until it is validating, make it hard to use
    as it does have a non-null mapping which is not useful.
   
    This CL partially reverts that change and instead rely on the log
    deferring logic we have in Context. This way, we don't need to break
    the stability of PojoRowAdapter while still having the ability to
    drop warnings that room fixes. This will also play nicer when we
    have different query re-writing options that can use more information
    about the query results.
   
    Bug: 153387066
    Bug: 140759491
    Test: existing tests pass
    Change-Id: I2ec967c763d33d7a3ff02c1a13c6953b460d1e5f

M       room/compiler/src/main/kotlin/androidx/room/log/RLog.kt
M       room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt

https://android-review.googlesource.com/1288456

if you have an easy way to create a repro that would be the fastest because order of calls seems to be important here :/.

also we are about to ship room-beta1 and paging-alpha7, i don't anticipate it to fix this problem but it might allow you update to latest.

Sure thing, I'll try to repro it and send you guys a link.

Can't seem to reproduce it in a clean codebase, despite it happening constantly in my real codebase... I don't really see any holes in the logic that RoomSQLiteQuery is using to lock the pool and recycle queries, either. I don't have enough time to screw around with it for now, but I'll come back to this and try to reproduce ASAP.

this is very troubling yet I cannot understand how this happens. I mean, we might have a bug in recycling queries, yet for a Single query, the instance we give to it does not get recycled until the query returns.
below is some sample code generated from a test that returns Single<List>.

@Override
  public Single<List<QueryTransactionTest.Entity1>> single() {
    final String _sql = "select * from Entity1";
    final RoomSQLiteQuery _statement = RoomSQLiteQuery.acquire(_sql, 0);
    return Single.fromCallable(new Callable<List<QueryTransactionTest.Entity1>>() {
      @Override
      public List<QueryTransactionTest.Entity1> call() throws Exception {
        final Cursor _cursor = __db.query(_statement);
        try {
          final int _cursorIndexOfId = _cursor.getColumnIndexOrThrow("id");
          final int _cursorIndexOfValue = _cursor.getColumnIndexOrThrow("value");
          final java.util.List<android.arch.persistence.room.integration.testapp.test.QueryTransactionTest.Entity1> _result = new java.util.ArrayList<android.arch.persistence.room.integration.testapp.test.QueryTransactionTest.Entity1>(_cursor.getCount());
          while(_cursor.moveToNext()) {
            final android.arch.persistence.room.integration.testapp.test.QueryTransactionTest.Entity1 _item;
            final int _tmpId;
            _tmpId = _cursor.getInt(_cursorIndexOfId);
            final java.lang.String _tmpValue;
            _tmpValue = _cursor.getString(_cursorIndexOfValue);
            _item = new android.arch.persistence.room.integration.testapp.test.QueryTransactionTest.Entity1(_tmpId,_tmpValue);
            _result.add(_item);
          }
          if(_result == null) {
            throw new EmptyResultSetException("Query returned empty result set: " + _statement.getSql());
          }
          return _result;
        } finally {
          _cursor.close();
          _statement.release();
        }
      }
    });
  }

An option could be that limit offset data source releasing queries that it wants to re-use. But if the count is coming from there, here is the generated count code from a query.

public int countItems() {
        final RoomSQLiteQuery sqLiteQuery = RoomSQLiteQuery.acquire(mCountQuery,
                mSourceQuery.getArgCount());
        sqLiteQuery.copyArgumentsFrom(mSourceQuery);
        Cursor cursor = mDb.query(sqLiteQuery);
        try {
            if (cursor.moveToFirst()) {
                return cursor.getInt(0);
            }
            return 0;
        } finally {
            cursor.close();
            sqLiteQuery.release();
        }
    }

it again obtains and releases the query in succession so I can't see how it happened to leak it.

I looked over the source that Room generated for my specific case and it all looked totally fine to me, too... I couldn't think of a way this would occur. However, the screenshot is pretty telling and it lines up with the exception I get. My temp "fix" involves just re-issuing the same query and it works fine (because the first failed attempt corrects the condition, despite eventually failing itself).

I'm swamped and under a major deadline for the next week or so, but I'll take a look at this again after that.

oh stupid me. the bug is actually right there in #8.

You are holding onto the Single so first time you call, it executes and releases the statement, second time you call, it does not allocate a new one :/

Basically this should never release a statement (unless we override finalize in that callable).
Thanks for the report. I'll figure out a way to audit all of these calls.

repro test:

@Test
    public void singleUser_keepSingleReference() throws InterruptedException {
        User[] users = TestUtil.createUsersArray(1, 2);
        mUserDao.insertAll(users);
        TestObserver<User> testObserver1 = new TestObserver<>();
        Single<User> userSingle1 = mUserDao.singleUserById(1);
        Disposable disposable1 = userSingle1.observeOn(mTestScheduler)
                .subscribeWith(testObserver1);
        drain();
        testObserver1.assertComplete();
        testObserver1.assertValue(users[0]);
        disposable1.dispose();

        // how get single for 2
        TestObserver<User> testObserver2 = new TestObserver<>();
        Single<User> userSingle2 = mUserDao.singleUserById(2);
        Disposable disposable2 = userSingle1.observeOn(mTestScheduler)
                .subscribeWith(testObserver2);
        drain();
        testObserver2.assertComplete();
        testObserver2.assertValue(users[1]);
        disposable2.dispose();

        // now re-use the first single
        TestObserver<User> testObserver3 = new TestObserver<>();
        Disposable disposable3 = userSingle1.observeOn(mTestScheduler)
                .subscribeWith(testObserver3);
        drain();
        testObserver3.assertComplete();
        testObserver3.assertValue(users[0]);
        disposable3.dispose();
    }

Solid, thanks!