IssueTracker

Project: platform/frameworks/support
Branch: androidx-master-dev

commit b90079595f33f58fece04026a97faa0d243acdb1
Author: Yuichi Araki <yaraki@google.com>
Date:   Wed Sep 18 16:55:49 2019

    Change the way to detect mismatch between POJO and query
   
    This fixes cursor mismatch warnings with expandProjection.
   
    Bug: 140759491
    Test: QueryMethodProcessorTest
    Change-Id: I7659002e5e0d1ef60fc1af2a625c0c36da0664d8

M       room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt
M       room/compiler/src/test/kotlin/androidx/room/processor/QueryMethodProcessorTest.kt
M       room/compiler/src/test/kotlin/androidx/room/testing/TestProcessor.kt

https://android-review.googlesource.com/1123258
https://goto.google.com/android-sha1/b90079595f33f58fece04026a97faa0d243acdb1

  The following changes were cherrypicked through Coastguard (https://android-build.googleplex.com/coastguard/)
  Release Track: (https://android-build.googleplex.com/coastguard/dashboard/6093116828680192/#/cherrypickHistory/all)
  Created: 2019-10-14 17:23:42.610596+00:00
  Changes: aosp/1123258
 

Project: platform/frameworks/support
Branch: androidx-master-dev

commit bdde5a1a970ddc9007b28de4aa29d60ffa588f08
Author: Yigit Boyar <yboyar@google.com>
Date:   Thu Apr 16 16:47:05 2020

    Re-factor how errors are dismissed when query is re-written
   
    This CL changes how we handle errors/warnings if query is
    re-written.
    There was a bug in expandProjection where we would report warnings
    for things that Room already fixes automatically (b/140759491).
    The solution to that problem (I7659002e5e0d1ef60fc1af2a625c0c36da0664d8)
    solved it by deferring validating of columns until after re-write
    decision is made. Unfortunately, this required changing PojoRowAdapter
    to have a dummy mapping until it is validating, make it hard to use
    as it does have a non-null mapping which is not useful.
   
    This CL partially reverts that change and instead rely on the log
    deferring logic we have in Context. This way, we don't need to break
    the stability of PojoRowAdapter while still having the ability to
    drop warnings that room fixes. This will also play nicer when we
    have different query re-writing options that can use more information
    about the query results.
   
    Bug: 153387066
    Bug: 140759491
    Test: existing tests pass
    Change-Id: I2ec967c763d33d7a3ff02c1a13c6953b460d1e5f

M       room/compiler/src/main/kotlin/androidx/room/log/RLog.kt
M       room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt

https://android-review.googlesource.com/1288456

This is a welcome addition, but is there a way to get it to emit in a paged, chunked [1] or windowed fashion?


The Flow<T> return type will hopefully alleviate the need to either load the entire query result in memory or doing some home-baked paging scheme when dealing with items one-by-one, but as I mentioned in the issue linked below [2], I would very often like to process a [large] query result in a paged/windowed/chunked fashion, for example uploading data to a server in batches.

In the past I've used `Cursor`s and done the buffering myself, but it kind of defeats the purpose of using Room (at least for that one aspect).

I could probably jury-rig something myself using `collectIndexed` and an array buffer, but I can't see how I could be the only one needing to do this.

1. https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/chunked.html
2. https://issuetracker.google.com/issues/109711034

Am I missing something? I just tested it out and it seems as if the resultant function only returns the first item in the result set, and then --waits. Is this the intended behaviour? I'm struggling to see how it would prove useful.

// DAO
@Query("SELECT * FROM my_items WHERE synced_time is NULL ORDER BY captured_on")
fun unsyncedItemsFlow(): Flow<Item>

// usage
dao.unsyncedItemsFlow().collect {
  // log it
}

Is there an example of proper usage I can look at?



I've used Kotlin flow in my own code to dump a result set to a text file, and it worked just fine.

Here is what I used, and expected to be able to replace:

 @FlowPreview
    private fun Cursor.export() = flow {

        use { c ->
            val columnCount = c.columnCount
            val separator = "\t"
            emit(c.columnNames.joinToString(separator))
            while (c.moveToNext()) {
                emit((0 until columnCount).joinToString(separator) {
                    when (c.getType(it)) {
                        Cursor.FIELD_TYPE_NULL -> "null"
                        Cursor.FIELD_TYPE_INTEGER -> c.getInt(it).toString()
                        Cursor.FIELD_TYPE_FLOAT -> c.getFloat(it).toString()
                        Cursor.FIELD_TYPE_STRING -> "\"${c.getString(it)}\""
                        Cursor.FIELD_TYPE_BLOB -> c.getBlob(it).toString()
                        else -> c.getString(it)
                    }
                })
            }
        }
    }

// in DAO
    @Query("SELECT * FROM items WHERE synced_time is NULL")
    fun unsyncedItems(): Cursor

    // in Repository
        @FlowPreview
    override suspend fun unsyncedItems() =  dao.unsyncedItems().export()

and then  
flow: Flow<String> = unsyncedItems()
flow.collect { appendLn(it) }

@Query("SELECT * FROM my_items WHERE synced_time is NULL ORDER BY captured_on")
fun unsyncedItemsFlow(): Flow<Item>

This method should return Flow<List<Item>>. That's the reason you are only getting the first item.
The Cursor underneath is returning all lines in your query result, but you are only getting the first in your DAO because you are returning a single element.

I don't see the point why your example should work with Flow<Item>, the result is already loaded into memory anyway, why should it emit one element at a time through the Flow?

comment #7.

I clearly misunderstood what this (returning Flow) is trying to achieve, I definitely do not want to load the whole query result into memory. I thought Flow would be used to emit single values (or preferably pages) as I do in my own usages of Flow.

For some of my use cases being able to iterate the entire result set is useful, with SQLiteCursor's internal buffering (CursorWindow) and setFillWindowForwardOnly it can be done fairly quickly even for large sets (that do not fit into memory as a whole). Wrapping the result in a Sequence or, as shown above, Flow, to abstract it away from Cursor seems to work well enough. I was hoping that this addition to Room meant I would not have to roll these by hand (the example above is just a simple version that emits strings).

comment #8:

I just plugged this into TryKotlin, `f` is of type `Flow<String>`, and the whole array is printed, just as I want.

fun main() {
        runBlocking {
                val f = arrayOf("one", "two").asFlow()
                f.collect {
                println(it)
                }    
    }  
}


I see there is a modification made to Flow<T> results in Room 2.2.0-beta01, I'll check it out, but otherwise, if it's intended to return an entire result set at a time itis not what I'm looking for.

Just tried out Room 2.2.0-beta01, it's use of Flow<T> is unfortunately not what I need, although it could prove useful as a scaffolding tool.

Flow<Item> only returns the first while Flow<List<Item>> generates `final List<Reading> _result = new ArrayList<Reading>(_cursor.getCount());`, which is exactly what I'm trying to avoid.

Its intended to return the result all at once.

I would be careful in using a Flow<T> where T is a single item representing a row and you get N row emissions. It is not wise to keep a Cursor open for a long time, specifically in a way such that consuming every emission takes a while and another database operation modifies the same table, possibly adding rows or deleting rows. If the query result does not fit in a single window then other operations will cause the CursorWindow to throw due to inconsistencies with the original count and the result of filling a new window. This also bring the question of what is your expected behavior if you have not finished consuming the Flow but new operations have modified the underlying data, did you not expect the Flow to react to that?

Room consumes the cursor completely and loads the data into memory to avoid issues with the CursorWindow. If the data is to big then the recommendation is to paginate it.