Google Cloud Speech streaming in the Unity Game Engine

I would like to know if anyone has been succesful with implementing Google Cloud Speech (streaming) into the Unity Game Engine? It would be wonderful if Google's Speech technology could be used for games, especially VR games!

Note: I am interested in Streaming since it allows real-time responses which is important in games.

Found projects

So far I have tried Unity's deprecated project: "https://bitbucket.org/Unity-Technologies/speech-to-text", unfortunately I didn't get it to work and it has now been removed from the Untiy Asset Store.

Secondly, I found a Google Speech asset on the Unity asset store: "https://www.assetstore.unity3d.com/en/#!/content/72625". Unfortunately it has no streaming support. But they are the only developers that I found so far that are trying to make it work.

Own implementation

I have been succesful in creating a working Unity project (with streaming) where the user can talk in real-time with an NPC. However I am not happy with the way I have built it. The way it's currently setup only works for Windows platforms.

My setup: In a seperate C# project I build a slightly modified Google's C# example project. Then I copy the assemblies (DLL's) and the executable (Recognize.exe) to my Unity project. In Unity I run the executable as a background process with additional commandline parameters.

Reason for setup: I am currently a student that has yet to learn a lot of technological aspects. I have tried to properly integrate the C# example project directly into Unity. Unfortunately with no success, a lot of issues came up that I was simply not able to resolve.

My questions

  • Is anyone working on Google Cloud Speech streaming in Unity? Or has someone already been succesful that I can contact?
  • Does anyone know if Google themselves are creating an Unity example? I know that other services such as Watson do provide an example project for Unity.
Jon Skeet
people
quotationmark

No, streaming in Google Cloud Speech API requires gRPC, and gRPC isn't supported in Unity at the moment. Once gRPC is supported, I'd expect the client library to port very easily - or just work out of the box, potentially. Supporting gRPC in Unity is far from trivial, partly because it relies on a later version of .NET than Unity supports1, and partly because it relies on a native component which will need to be compiled for each of the architectures your app will target. (It's also unclear whether it will ever be feasible for web player, but that's not supported any more anyway.)

I've done some prototyping of gRPC on Unity, and it's "on the radar" in that there are teams in Google who would definitely like it to be supported - but there's nothing to announce at the moment, I'm afraid.


1 I'm aware that Unity is moving on, but ideally we'd like to support old versions as well.

people

See more on this question at Stackoverflow