IssueTracker

Project: platform/frameworks/support
Branch: androidx-master-dev

commit 3c819141d3be0a6194a6d02e5642942ac15d86af
Author: Daniel Santiago Rivera <danysantiago@google.com>
Date:   Mon Feb 04 10:39:54 2019

    Use JetBrains's Kotlin metadata library.
   
    Use kotlinx-metadata-jvm to read Kotlin's Metadata annotation
    information. This library has less of an API surface than kotlin-metdata
    meaning KotlinClassMetadataUtils only exposes what Room needs. However
    it does allows us to fix the JVM descriptors (JvmDescriptorUtils) that
    are needed to match annotation processing elements to kotlin metadata
    elements.
   
    Bug: 123767877
    Test: ./gradlew room:room-compiler:test \
                    room:integration-tests:room-testapp-kotlin:cC \
                    room:integration-tests:room-testapp:cC
   
    Change-Id: Iddbe3968865b32168144294e55765a77320a40da

M       buildSrc/src/main/kotlin/androidx/build/dependencies/Dependencies.kt
M       room/compiler/build.gradle
D       room/compiler/src/main/kotlin/androidx/room/ext/KotlinMetadataElement.kt
M       room/compiler/src/main/kotlin/androidx/room/ext/element_ext.kt
A       room/compiler/src/main/kotlin/androidx/room/kotlin/JvmDescriptorUtils.kt
A       room/compiler/src/main/kotlin/androidx/room/kotlin/KotlinClassMetadataUtils.kt
A       room/compiler/src/main/kotlin/androidx/room/kotlin/KotlinMetadataElement.kt
M       room/compiler/src/main/kotlin/androidx/room/processor/MethodProcessorDelegate.kt
M       room/compiler/src/main/kotlin/androidx/room/processor/PojoProcessor.kt
M       room/compiler/src/main/kotlin/androidx/room/writer/DaoWriter.kt
A       room/compiler/src/test/kotlin/androidx/room/kotlin/JvmDescriptorUtilsTest.kt
A       room/compiler/src/test/kotlin/androidx/room/kotlin/KotlinMetadataElementTest.kt
M       room/integration-tests/kotlintestapp/src/androidTest/java/androidx/room/integration/kotlintestapp/dao/BooksDao.kt
A       room/integration-tests/kotlintestapp/src/androidTest/java/androidx/room/integration/kotlintestapp/vo/AnswerConverter.kt

https://android-review.googlesource.com/894734
https://goto.google.com/android-sha1/3c819141d3be0a6194a6d02e5642942ac15d86af

sorry we need to have a stable 2.1 release and it has been postponed a lot due to complicated coroutines integration.
And we don't yet have a way to add experimental features in AndroidX.
We will branch out 2.1 once it is beta (very soon) and will ship a 2.2 alpha w/ this later on. Sorry for the delay.

Project: platform/frameworks/support
Branch: androidx-master-dev

commit b4c4a0d60152500ec6485624bf74f769ec6d0fca
Author: Daniel Santiago Rivera <danysantiago@google.com>
Date:   Mon Apr 15 09:41:13 2019

    Support Kotlin Coroutines Flow as query return types.
   
    This CL adds QueryResultBinders and Providers Flow. Similar to Rx the
    strategy is to generate code that calls into a runtime helper method to
    setup Flow given a Callable that actually executes the query.
   
    Channels are not directly supported and instead we show an error
    suggesting to change the DAO function return type to Flow and then use
    the neighboring functions to create a Channel. The reason is that we
    can't reasonably produce a channel without a CoroutineScope and it is
    a weird API to ask users to pass a CoroutineScope in their DAO method.
   
    Suspend DAO function that returns Flow is an error.
   
    Bug: 127328278
    Bug: 130428884
    Test: CoroutinesRoomTest, FlowQueryTest
    Change-Id: If0fc6fdb741735f2e4b6b42015730bbf76e8dc18

M       buildSrc/src/main/kotlin/androidx/build/dependencies/Dependencies.kt
M       room/compiler/src/main/kotlin/androidx/room/ext/javapoet_ext.kt
M       room/compiler/src/main/kotlin/androidx/room/processor/ProcessorErrors.kt
M       room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
A       room/compiler/src/main/kotlin/androidx/room/solver/binderprovider/CoroutineFlowResultBinderProvider.kt
A       room/compiler/src/main/kotlin/androidx/room/solver/query/result/CoroutineFlowResultBinder.kt
A       room/compiler/src/test/data/common/input/coroutines/Channel.java
A       room/compiler/src/test/data/common/input/coroutines/ReceiveChannel.java
A       room/compiler/src/test/data/common/input/coroutines/SendChannel.java
M       room/compiler/src/test/kotlin/androidx/room/processor/QueryMethodProcessorTest.kt
M       room/compiler/src/test/kotlin/androidx/room/testing/test_util.kt
M       room/integration-tests/kotlintestapp/build.gradle
M       room/integration-tests/kotlintestapp/src/androidTest/java/androidx/room/integration/kotlintestapp/dao/BooksDao.kt
A       room/integration-tests/kotlintestapp/src/androidTest/java/androidx/room/integration/kotlintestapp/test/FlowQueryTest.kt
M       room/ktx/api/restricted_2.2.0-alpha01.txt
M       room/ktx/api/restricted_2.2.0-alpha02.txt
M       room/ktx/api/restricted_current.txt
M       room/ktx/build.gradle
M       room/ktx/src/main/java/androidx/room/CoroutinesRoom.kt
A       room/ktx/src/test/java/androidx/room/CoroutinesRoomTest.kt

https://android-review.googlesource.com/944458
https://goto.google.com/android-sha1/b4c4a0d60152500ec6485624bf74f769ec6d0fca

This is a welcome addition, but is there a way to get it to emit in a paged, chunked [1] or windowed fashion?


The Flow<T> return type will hopefully alleviate the need to either load the entire query result in memory or doing some home-baked paging scheme when dealing with items one-by-one, but as I mentioned in the issue linked below [2], I would very often like to process a [large] query result in a paged/windowed/chunked fashion, for example uploading data to a server in batches.

In the past I've used `Cursor`s and done the buffering myself, but it kind of defeats the purpose of using Room (at least for that one aspect).

I could probably jury-rig something myself using `collectIndexed` and an array buffer, but I can't see how I could be the only one needing to do this.

1. https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/chunked.html
2. https://issuetracker.google.com/issues/109711034

Am I missing something? I just tested it out and it seems as if the resultant function only returns the first item in the result set, and then --waits. Is this the intended behaviour? I'm struggling to see how it would prove useful.

// DAO
@Query("SELECT * FROM my_items WHERE synced_time is NULL ORDER BY captured_on")
fun unsyncedItemsFlow(): Flow<Item>

// usage
dao.unsyncedItemsFlow().collect {
  // log it
}

Is there an example of proper usage I can look at?



I've used Kotlin flow in my own code to dump a result set to a text file, and it worked just fine.

Here is what I used, and expected to be able to replace:

 @FlowPreview
    private fun Cursor.export() = flow {

        use { c ->
            val columnCount = c.columnCount
            val separator = "\t"
            emit(c.columnNames.joinToString(separator))
            while (c.moveToNext()) {
                emit((0 until columnCount).joinToString(separator) {
                    when (c.getType(it)) {
                        Cursor.FIELD_TYPE_NULL -> "null"
                        Cursor.FIELD_TYPE_INTEGER -> c.getInt(it).toString()
                        Cursor.FIELD_TYPE_FLOAT -> c.getFloat(it).toString()
                        Cursor.FIELD_TYPE_STRING -> "\"${c.getString(it)}\""
                        Cursor.FIELD_TYPE_BLOB -> c.getBlob(it).toString()
                        else -> c.getString(it)
                    }
                })
            }
        }
    }

// in DAO
    @Query("SELECT * FROM items WHERE synced_time is NULL")
    fun unsyncedItems(): Cursor

    // in Repository
        @FlowPreview
    override suspend fun unsyncedItems() =  dao.unsyncedItems().export()

and then  
flow: Flow<String> = unsyncedItems()
flow.collect { appendLn(it) }

@Query("SELECT * FROM my_items WHERE synced_time is NULL ORDER BY captured_on")
fun unsyncedItemsFlow(): Flow<Item>

This method should return Flow<List<Item>>. That's the reason you are only getting the first item.
The Cursor underneath is returning all lines in your query result, but you are only getting the first in your DAO because you are returning a single element.

I don't see the point why your example should work with Flow<Item>, the result is already loaded into memory anyway, why should it emit one element at a time through the Flow?

comment #7.

I clearly misunderstood what this (returning Flow) is trying to achieve, I definitely do not want to load the whole query result into memory. I thought Flow would be used to emit single values (or preferably pages) as I do in my own usages of Flow.

For some of my use cases being able to iterate the entire result set is useful, with SQLiteCursor's internal buffering (CursorWindow) and setFillWindowForwardOnly it can be done fairly quickly even for large sets (that do not fit into memory as a whole). Wrapping the result in a Sequence or, as shown above, Flow, to abstract it away from Cursor seems to work well enough. I was hoping that this addition to Room meant I would not have to roll these by hand (the example above is just a simple version that emits strings).

comment #8:

I just plugged this into TryKotlin, `f` is of type `Flow<String>`, and the whole array is printed, just as I want.

fun main() {
        runBlocking {
                val f = arrayOf("one", "two").asFlow()
                f.collect {
                println(it)
                }    
    }  
}


I see there is a modification made to Flow<T> results in Room 2.2.0-beta01, I'll check it out, but otherwise, if it's intended to return an entire result set at a time itis not what I'm looking for.

Just tried out Room 2.2.0-beta01, it's use of Flow<T> is unfortunately not what I need, although it could prove useful as a scaffolding tool.

Flow<Item> only returns the first while Flow<List<Item>> generates `final List<Reading> _result = new ArrayList<Reading>(_cursor.getCount());`, which is exactly what I'm trying to avoid.

Its intended to return the result all at once.

I would be careful in using a Flow<T> where T is a single item representing a row and you get N row emissions. It is not wise to keep a Cursor open for a long time, specifically in a way such that consuming every emission takes a while and another database operation modifies the same table, possibly adding rows or deleting rows. If the query result does not fit in a single window then other operations will cause the CursorWindow to throw due to inconsistencies with the original count and the result of filling a new window. This also bring the question of what is your expected behavior if you have not finished consuming the Flow but new operations have modified the underlying data, did you not expect the Flow to react to that?

Room consumes the cursor completely and loads the data into memory to avoid issues with the CursorWindow. If the data is to big then the recommendation is to paginate it.