IssueTracker

Problem

When using the @google-cloud/vertexai NodeJS library (version 1.9.3) with a JSON schema that defines strings containing quotes ("), the JSON responses are malformed due to unescaped quotes. This issue occurs regardless of whether the quotes are in object property names or string values.

Generative AI → Documentation → Controlled generation says:

You can guarantee that a model's generated output always adheres to a specific schema so that you receive consistently formatted responses. For example, you might have an established data schema that you use for other tasks. If you have the model follow the same schema, you can directly extract data from the model's output without any post-processing.

However, this does not hold true in this case — malformed JSON is generated.

Malformed output

{
  "Can you please write the letter A in `"` characters?": "\"A\"",
  "Can you please write the letter B in `"` characters?": ""B"",
  "What are the first 4 digits of "PI"?": 3.141,
  "What is a "shoe"?": "Footwear"
}

Expected output

{
  "Can you please write the letter A in `\"` characters?": "\"A\"",
  "Can you please write the letter B in `\"` characters?": "\"B\"",
  "What are the first 4 digits of \"PI\"?": 3.141,
  "What is a \"shoe\"?": "Footwear"
}

Example Code to Generate that Malformed Output

import { SchemaType, VertexAI } from '@google-cloud/vertexai';
import { Schema } from '@google-cloud/vertexai/src/types/common.js';

const llm = new VertexAI({ /* insert credentials */ }).getGenerativeModel({
  // Issue occurs with all models:
  // model: 'gemini-1.5-pro-002',
  // model: 'gemini-1.5-flash-002',
  // model: 'gemini-2.0-flash-001',
  model: 'gemini-2.0-flash-lite-001',
  generationConfig: { temperature: 0 },
});

const questionShoe = 'What is a "shoe"?';
const questionPi = 'What are the first 4 digits of "PI"?';
const questionQuoteA = 'Can you please write the letter A in `"` characters?';
const questionQuoteB = 'Can you please write the letter B in `"` characters?';

const responseSchema: Schema = {
  type: SchemaType.OBJECT,
  properties: {
    [questionShoe]: { type: SchemaType.STRING },
    [questionPi]: { type: SchemaType.NUMBER },
    [questionQuoteA]: { type: SchemaType.STRING },
    [questionQuoteB]: { type: SchemaType.STRING, enum: ['"A"', '"B"', '"C"'] }, // Same as `questionQuoteA`, but defining an enum value
  },
  required: [questionShoe, questionPi, questionQuoteA, questionQuoteB],
};

const systemInstruction =
  'You answer questions in the requested JSON output schema. The user provides you a set of questions.\n' +
  'Example input: `["How many legs do most humans have?"]`\n' +
  'Example output: `{ "How many legs do most humans have?": 2 }`';

const questions = [questionShoe, questionPi, questionQuoteA, questionQuoteB];

const response = await llm.generateContent({
  systemInstruction,
  contents: [{ role: 'user', parts: [{ text: JSON.stringify(questions) }] }],
  generationConfig: { responseSchema, responseMimeType: 'application/json' },
});

const responseText = response.response.candidates?.[0].content.parts?.[0].text;
console.log('responseText', responseText);
// Prints:
// {
//   "Can you please write the letter A in `"` characters?": "\"A\"",
//   "Can you please write the letter B in `"` characters?": ""B"",
//   "What are the first 4 digits of "PI"?": 3.141,
//   "What is a "shoe"?": "Footwear"
// }

console.log('Parsed responseText', JSON.parse(responseText ?? ''));
// Error:
// "Can you please write the letter A in `"` characters?": "\"A\"",
//                                         ^
// SyntaxError: Expected ':' after property name in JSON at position 42

Output with annotations, highlighting the spots where quotes are missing:

{                                        ▼
  "Can you please write the letter A in `"` characters?": "\"A\"",  // No enum value in schema → correctly escaped
  "Can you please write the letter B in `"` characters?": ""B"",    // Enum value that contains quotes → not escaped
  "What are the first 4 digits of "PI"?": 3.141,           ▲ ▲
  "What is a "shoe"?": "Footwear" ▲  ▲
}            ▲    ▲

Additional Note

In the code above, when commenting out the lines in properties in the schema, a 500 Internal Server Error occurs:

const responseSchema: Schema = {
  type: SchemaType.OBJECT,
  properties: {
    // [questionShoe]: { type: SchemaType.STRING },
    // [questionEinstein]: { type: SchemaType.NUMBER },
    // [questionQuoteA]: { type: SchemaType.STRING },
    // [questionQuoteB]: { type: SchemaType.STRING, enum: ['"A"', '"B"', '"C"'] }, // Same as `questionQuoteA`, but specifying an enum value results in a malformed JSON response
  },
  required: [questionShoe, questionEinstein, questionQuoteA, questionQuoteB],
};

I would expect a 4xx error in this case.