IssueTracker

Problem you have encountered:

When requesting grounding, I am unable to also specify structured output of JSON format. My present workaround is to make two calls to Gemini: the first to retrieve grounded information, the second to take that grounded information and massage it into an expected JSON schema. Furthermore, if the system prompt contains anything that looks like JSON, such as {}, the request will no longer be grounded.

What you expected to happen:

Being able to request grounding and structured output in the same call. Furthermore, being able to control grounding to always happen, such as setting a dynamicThreshold to 0 (which is supported in Gemini 1.5), would be fantastic. See Example curl below of an ideal API call:

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=secret" \
     -H 'Content-Type: application/json' \
     -X POST \
     -d '{
  "contents": [
    {
      "parts": [
        {
          "text": "Can you give me the original 151 pokemon?"
        }
      ]
    }
  ],
  "generationConfig": {
    "responseMimeType": "application/json",
    "responseSchema": {
      "type": "OBJECT",
      "properties": [
        {
          "key": "response",
          "value": {
            "type": "STRING"
          }
        }
      ]
    }
  },
  "tools": [
    {
      "googleSearchRetrieval": {
        "dynamicRetrievalConfig": {
          "mode": "MODE_DYNAMIC",
          "dynamicThreshold": 0
        }
      }
    }
  ]
}'

Steps to reproduce:

1. Request Grounding
2. Request Structured Output
3. Make Request
4. Receive HTTP 400 Bad Request Response
5. Feel sad

Other information (workarounds you have tried, documentation consulted, etc):

This can be mitigated by making two calls to Gemini: the first for grounding and then the second for structured output based on the response from the first call. This is less than ideal, considering that LLM calls are not the fastest and this could be handled with one request.