with model=latest_short and single_uttterance=false I can get an end of utterance event and then nothing else [275157236]

Assigned

Bug

Status Update

No update yet.

Description

da...@drachtio.org

created issue #1

Mar 26, 2023 12:46AM

I'm using the v1p1beta1 speech SDK from a C++ grpc client with set_single_utterance(false). I have tested with the 'latest_long' model and am finding a strange result. If after connecting and send audio I (as the caller, and source of audio) cough or mumble I am able to get an 'end of utterance' event back from google. If at that point I start speaking I do not get any transcripts back. I simply got the end of utterance and then my transcription session seems essentially dead.

If I run the same test with model 'command_and-search' it behaves differently. No matter how I cough or mumble I can't get an initial end of utterance event to come. I then start speaking and I get transcripts.

I recently changed from command_and_search to latest_short because I thought that was recommended, but it seems this behavior has broken some of my use cases.

Has anyone experienced this? The docs (

https://cloud.google.com/speech-to-text/docs/reference/rpc/google.cloud.speech.v1p1beta1#google.cloud.speech.v1p1beta1.StreamingRecognitionConfig) seem to indicate the single_utterance isnt even support for latest_short, so this seems like a bug to me.

Would love to hear thoughts or experience related to this. Thanks

Comments

da...@drachtio.org <da...@drachtio.org> #2Mar 26, 2023 12:47AM

I have informed our engineering team of this feature request. There is currently no ETA for its implementation.

A current workaround would be to check the returned "boundingPoly" [1] "vertices" for the returned "textAnnotations". If the calculated rectangle's heights > widths, than your image is sideways.

[1]

https://cloud.google.com/vision/reference/rest/v1/images/annotate#boundingpoly

nr...@google.com <nr...@google.com> Mar 28, 2023 06:12AM

Assigned to nr...@google.com.

nr...@google.com <nr...@google.com> #3Mar 29, 2023 10:35AM

I also need this problem solved :)

da...@drachtio.org <da...@drachtio.org> #4Mar 29, 2023 11:57AM

same :D

nr...@google.com <nr...@google.com> #5Apr 5, 2023 11:29PM

Reassigned to gc...@google.com.

pb...@google.com <pb...@google.com> #6Jan 4, 2024 04:08AM

Comment has been deleted.

Message last modified on Jan 4, 2024 04:09AM

rl...@gmail.com <rl...@gmail.com> #7May 15, 2024 02:50PM

This needs more attention. It's not just a display issue as described in the report. The co-ordinates returned in 'boundingPoly' are incorrect if the image was taken on a phone. All the x points should be y and vice versa.

The workaround does not make sense as "boundingPoly" [1] "vertices" for "textAnnotations" does not indicate the image dimensions - it indicates the dimensions of the relevant text block inside the image.