IssueTracker

Issue description (short summary)

With a Generative Agent and set up to use a Function Tool Playbook when the prompt matches the description for this Function Tool. When I send a prompt that matched the Function Tool via the REST API, I receive a response that contains an empty object instead of the expected response message within the "responseMessages" array.

Steps to reproduce the issue (detailed description)

1. Create a new Agent

2. Create a Function Tool using the weather example from the documentation

3. Add the Tool within the instructions of the Agent which tells it to use this Function Tool when there is a question about the weather.

4. Send a POST to "https://REGION_ID-dialogflow.googleapis.com/v3/projects/PROJECT_ID/locations/LOCATION_ID/agents/AGENT_ID/sessions/SESSION_ID:detectIntent" with a payload that contains a queryInput with a text of "Hello!"

5. The Agent will respond with a properly formatted object within the "responseMessages" array greeting the user as a response

6. Send a POST to "https://REGION_ID-dialogflow.googleapis.com/v3/projects/PROJECT_ID/locations/LOCATION_ID/agents/AGENT_ID/sessions/SESSION_ID:detectIntent" with a payload that contains a queryInput with a text of "What is the weather in Miami today?"

7. The Agent will respond with an empty object within the "responseMessages" array instead of giving information about the Function Tool and the output params it is expecting to receive as a response.

8. If I try to send a new prompt message, the API will return an error because it is waiting to receive the output param for the Function Tool which I cannot do because it did not send me the information about the tool it used and the data it needs.

Expected behavior

According to the documentation, the response should look like this:

{
  "queryResult": {
    "text": "what is the weather in Mountain View",
    "languageCode": "en",
    "responseMessages": [
      {
        "source": "VIRTUAL_AGENT",
        "toolCall": {
          "tool": "<tool-resource-name>",
          "action": "get-weather-tool",
          "inputParameters": {
            "location": "Mountain View"
          }
        }
      }
    ]
  }
}

Actual behavior

But what I actually get is:

{
  "queryResult": {
    "text": "what is the weather in Mountain View",
    "languageCode": "en",
    "responseMessages": [
      {}
    ]
  }
}

I also get the following error if I send another prompt without sending the tool's output params it needs:

{
   "error": {
       "code": 400,
       "message": "com.google.apps.framework.request.BadRequestException: Session is waiting for tool call result of tool, c67f9b26-7ce7-470c-abbe-19c13511665b, and action, Get Weather.",
       "status": "INVALID_ARGUMENT"
   }
}

This makes sense, but I cannot send the data it is waiting for without telling the client what it needs.

Workarounds (if any)

None.

Documentation consulted

https://cloud.google.com/dialogflow/cx/docs/concept/playbook/tool#function

Other information