IssueTracker

The SAF as currently implemented does not provide any mechanism to coordinate retrieving/processing "pages" of data from the target provider. While it does provide a mechanism to signal the framework to make repeated calls back in to a DocumentsProvider implementation via the DocumentsContract.EXTRA_LOADING flag, what this is typically used for is to enable a DocumentsProvider implementation to incorporate threading to continuously gather document information and load the cursor with it. While this is a great mechanism to "batch load" a cursor, it still means that the cursor will end up with an entire set of data.

What is ALSO needed, is a way to process large data sets visually in pages. For example, assume a particular directory has 1000's of files in it. Whether we try and load all items into the cursor at once (bad idea), or we use the EXTRA_LOADING flag to indicate batches of 25-50 items at a time and also display a loading message - in the end we are loading 1000's of entries into that cursor.

Instead, we need a way to load the first "batch" (say 100), display those, then let the user know there are more to be had if they swipe up/down, select a new button in the chooser "next", "previous" or what have you.

This works well on the DocumentsProvider side if the targeted resource gives that type of information, which many sources support. For example, Facebook/Instagram REST APIs all provide the concept of pagination - if you use their APIs in a DocumentsProvider implementation to query for a user's media and limit the query to a batch size of 50 for example, you will get back the 50 items, plus a json string that identifies the "next" url, and the "previous" url. Back in the DocumentsProvider, it could hang on to a reference to those, then when the "next" or "previous" indication from the SAF framework came through, the provider would use those URLs to gather the next/previous batch and load that into the cursor, and so on.

In this way, we could process directories with more files than can reasonably fit in a cursor much more effectively from virtually any resource that itself supports pagination.

One way I could see this working on the provider side, is much like we have the EXTRA_LOADING flag today, we use the cursor.getExtras() to put a new flag EXTRA_PAGINATION, to signal to the framework that this provider recognizes it needs to process a large group of items as pages.

The values for EXTRA_PAGINATION could be:

PREVIOUS_ONLY       Indicates to the chooser to only show the "previous" button (the provider has
                    returned a cursor with a batch that is the "bottom set" of the long list.
NEXT_ONLY           Indicates to the chooser to only show the "next" button (the provider has
                    returned a cursor with a batch that is the "first set" of the long list.
PREVIOUS_AND_NEXT   Indicates to the chooser to show both "Next" and "Previous" buttons (the
                    provider has returned a cursor with a batch somewhere in the middle of the
                    long list, and the user can page up or down.

If EXTRA_PAGINATION is not passed, then the chooser behaves like it does today.

If it is passed, then the file chooser UI is augmented to include a new visual component (could be "next"/"previous" buttons, could be a new gesture, what have you), that allows the user to indicate they want to move "next" and "previous" through the data within this same subdirectory.

Then when the framework calls back into the queryChildDocuments() API, it uses a new optional fourth parameter. This parameter would indicate whether the request is for "previous batch" or "next batch", within the same directory (same parentDocumentId) based on what the user action was in the file chooser ui. Something like "REQUEST_NEXT" and "REQUEST_PREVIOUS". If this 4th parameter is not passed, then we are not in batch mode - if the parentDocumentId is the same, it means we do whatever we did before. If it is different, then we load the directory per the way we do it today.

So to summarize the requested changes:

UI: Update the File Chooser to allow a user to request paging up/down within a directory when a DocumentsProvider has indicated EXTRA_PAGINATION. Have that request be passed as a parameter into the DocumentsProvider API via an optional parameter to queryChildDocuments().

DocumentsProvider: Provide a new cursor.getExtras() flag EXTRA_PAGINATION, with associated values PREVIOUS_ONLY, NEXT_ONLY, PREVIOUS_AND_NEXT that indicate what UI choices should be available to the user to request page up/page down within the current subdirectory.