IssueTracker

A short description of the issue:

When I page the search results, I get duplicate results. My query is:

    params = {
        'fields': 'mediaItems(id,baseUrl,filename,mimeType,productUrl),nextPageToken',
    }
    search_json = {
        "pageSize": 100,
        "filters": {
            "includeArchivedMedia": False,
            "contentFilter": {
                "excludedContentCategories": [
                    "DOCUMENTS",
                    "RECEIPTS",
                    "SCREENSHOTS",
                    "UTILITY",
                    "WHITEBOARDS",
                ]
            },
            "mediaTypeFilter": {
                "mediaTypes": [
                    "PHOTO",
                ],
            },
        },
    }

using the Google Python API to execute it:

        rsp = session.post(
            'https://photoslibrary.googleapis.com/v1/mediaItems:search',
            params=params,
            json=search_json,
        ).json()

where session is of type AuthorizedSession:

    from google.auth.transport.requests import AuthorizedSession

Paging this API queries ~18k items in total, about 5-10 of which are duplicates. Which photos IDs are duplicated does not seem to be deterministic.

A small code sample that reliably reproduces the issue. The sample should run as-is or with minimal setup.

You can see all the code here: https://github.com/dhalperi/mosaic/blob/65cf3c7c16de7d9df484da48a47a719a25787b7d/download_images.py

You can run it if you have app credentials + user token in JSON that already works stored locally. You'll need your own app credentials regardless, but you can get your own user credentials dynamically (using the out-of-band method) by commenting out the first two lines of the __main__ block, and uncomment the third.

The calls to the API that lead to the error. Include the sequence of calls, including request headers and body.
Do not include any personal information, authentication secrets, media item or album IDs.

What steps will reproduce the problem?

Run the linked script on a reasonable large user's album, after setting up the app credentials needed.

What is the expected output? What do you see instead?  If you see error messages, please provide them.

Expected: A list of media_item entries, all of which are unique.
Actual: A list of media_item entries, with some duplicates.

Please provide any additional information below.

I'm happy to provide any information that's useful!

Here's an example from my own log:

There were four duplicated IDs out of 18K + items, spread across probably over 200 calls to the search API (100 items per, often fewer than 100 returned).

The duplicated IDs were returned in sequence, but I did not log the accompanying nextPageTokens.

One media item ID that was returned twice: AGj1epXH-iAAeNQQ_uk9BPAFjF1YIBc3HqsfZ88X6xN94-qG4uML8fJQk07naI_WnppXejtav2Eg_-c.

--

I am happy to go back and add better logging of the requests in order to debug the dupes if that helps!