Home  ›  Dialog Flow Why Cant I Upload Zip for Restore the Zip

Dialog Flow Why Cant I Upload Zip for Restore the Zip

Written By Monday, April 11, 2022 Add Comment Edit

Virtual Assistant (with Google Dialogflow)

Contents

Virtual Assistant (with Google Dialogflow)¶

This Virtual Assistant (with Google Dialogflow) sample application demonstrates the integration of Google Dialogflow and Riva Spoken language Services in the course of a conditions chatbot spider web application.

In this sample, we utilise Riva for ASR and TTS and Google Dialogflow for NLP and Dialog Direction (DM).

Demo Video¶

To see how the Riva and Google Dialogflow weather chatbot service works, the demo video can be institute here.

Implementation¶

At a high-level, the integration takes reward of the native API support of Google Dialogflow and gRPC back up in Riva. The Weatherbot Customer coordinates the workflow with Riva Services and Dialogflow, and then interacts with the finish-user via a spider web UI. There are 3 main parts to this solution; Riva AI Services, Dialogflow Weatherbot, and the Weatherbot Client awarding.

Here is the implementation at a loftier-level:

  • Riva AI Services

    • Exposes Speech communication Services (ASR/NLP/TTS) over gRPC endpoints.

    • Needs a GPU.

  • Riva and Dialogflow Chatbot

    • Dialogflow Weatherbot

      • Exposes API endpoints to communicate with the chatbot.

      • Takes user text as input and returns a response.

      • Responsible for fulfillment, when needed.

      • Runs on GCP.

    • Weatherbot Customer application

      • Includes the Riva Client Python library.

      • Communicates with Riva AI Services and Dialogflow Weatherbot over gRPC and REST API endpoints respectively.

      • Pipelines ASR, NLP, TTS, and dialog manager functionalities.

      • Contains the Weatherbot Customer application (web UI and web service).

      • Does not demand GPUs.

Architecture¶

Riva and Dialogflow virtual assistant architecture

The above diagram shows the architecture of the Riva and Dialogflow Weatherbot. Audio input from the user is collected through the microphone past the web UI of the Weatherbot Client application. The input audio from the user is sent to Riva AI Services for ASR, by the client application. Riva AI Services returns the transcribed text back to the Client awarding. The transcribed text from Riva AI Services is then sent to the Dialogflow Weatherbot (running on GCP). The Dialogflow Weatherbot returns the appropriate response for the text. The response text is then sent to Riva AI Services for TTS. A vox response is returned back to the customer application, which is and so played on the user's speakers by the web UI.

Code Structure¶

This department shows the high-level code structure of the Weatherbot Customer application (in Riva and Dialogflow Chatbot).

  • asr.py

    • This file contains the functionality to make the gRPC call to Riva ASR, using the Riva Python Client libraries with the audio snippet and returns the text transcript.

    • ASR is used in streaming mode

  • dialogflow.py

    • This file contains the functionality to make an API call to Dialogflow, with the user input and sender ID and returns a text response obtained past Dialogflow.

  • tts.py and tts_stream.py

    • These files comprise the functionality to make the gRPC call to Riva TTS, using the Riva Python Client libraries, with a text snippet, and returns the corresponding audio spoken communication.

    • TTS tin exist used in either Batch or Streaming style, depending on whether tts.py or tts_stream.py is used. This can be set by changing the import statements in lines 12 and xiii in the virtual-banana-dialogflow/dialogflow-riva-weatherbot-webapp/riva/chatbot/chatbot.py script.

  • chatbot.py

    • This file contains the Chatbot course which is responsible for pipelining all the ASR, TTS and Dialogflow operations.

    • Creates one case of the Chatbot course per conversation.

Running the Demo¶

  1. Start Riva Speech Services per Riva Quick Outset Guide.

  2. Run Riva and the Dialogflow Virtual Assistant.

    1. Run the Riva Sample container.

      1. Create a directory to hold the code for Riva and the Dialogflow Virtual Assistant.

                                    mkdir riva-dialogflow-va-temp

      2. Pull the Riva Sample container.

                                                                docker                            pull                            nvcr                            .                            io                            /                            nvidia                            /                            riva                            /                            riva                            -                            speech                            -                            client                            :                            2.0.0                            -                            samples

      3. Run the Riva Sample container.

                                    docker run -it --rm -p                            6006:6006 -v <Path to riva-dialogflow-va-temp>:/riva-dialogflow-va-temp nvcr.io/nvidia/riva/riva-speech-client:ii.0.0-samples /bin/bash

      4. Navigate to the samples/virtual-banana-dialogflow directory.

                                                                cd                            samples/virtual-assistant-dialogflow

      5. Modify the API endpoint setting per the Network Configuration section.

    2. Prepare up Google Dialogflow. The entire set up process for Dialogflow can take some time to complete. We have tried to consummate equally much of the set up as possible in the Docker container, however, the following steps must exist completed.

      1. Read through Dialogflow Basics and Nearly the Google Deject Console to amend empathize the basics of Dialogflow.

      2. Follow the steps in Create a Projection and Enable the API from the Dialogflow setup. Enable Billing and Enable audit logs are not needed for this demo.

      3. Follow the Gear up Authentication instructions. When done, run the command in the Utilize the service business relationship key file in your environment step in the Riva Samples container.

                                                                export                            GOOGLE_APPLICATION_CREDENTIALS                            =                            "<Path to key json file>"

      4. Install and initialize the Deject SDK in the container, except for initilializing gcloud CLI. In the Riva Samples container, run:

        During this command, y'all volition need to provide your Project ID. To notice your Project ID, perform the following steps:

        1. In the Google Cloud Platform (GCP) Dashboard, select your projection from the top-left driblet-downward, found on the right side of the GCP banner.

        2. Under the DASHBOARD tab, the Project ID can be found in the Project Info section.

      5. Consummate Examination the SDK and authentication.

                                    gcloud auth application-default impress-access-token

      6. Skip Install the Dialogflow client library. This pace has been completed in the Riva Samples container, therefore, no activity is needed.

      7. In config.py , update PROJECT_ID parameter with your project ID. To find your Projection ID, perform the post-obit steps:

        1. In the Google Deject Platform (GCP) Dashboard, select your project from the elevation-left drop-down, found on the correct side of the GCP banner.

        2. Nether the DASHBOARD tab, the Project ID tin can be found in the Projection Info section.

    3. Initialize and beginning the Dialogflow Weatherbot.

      1. Copy the code from the Samples container to the host system.

                                    cp -r /workspace/samples/virtual-assistant-dialogflow/* /riva-dialogflow-va-temp/

      2. Follow the steps hither to create an agent.

      3. Click the Setting button next to the amanuensis proper name in the Dialogflow console. Under the Export and Import tab, choose Restore From ZIP and upload the zipped folder from your host at <Path to riva-dialogflow-va-temp>/dialogflow-weatherbot/dialogflow-weatherbot.zero .

      4. Add fullfillment.

        1. Open up the Fulfillment section and enable the Inline Editor in the Dialogflow console.

        2. Copy and paste the contents of the <Path to riva-dialogflow-va-temp>/dialogflow-weatherbot/fulfillment/alphabetize.js into alphabetize.js nether the Inline Editor.

        3. Re-create and paste the contents of the <Path to riva-dialogflow-va-temp>/dialogflow-weatherbot/fulfillment/packet.json into package.json under the Inline Editor.

      5. In index.js , at line iv, update the weatherstack_APIkey with your Weatherstack API key. A new Weatherstack API fundamental tin be obtained from here.

    4. Start Riva and the Dialogflow chatbot client.

      1. Activate the chatbot client Python environs.

                                    . /pythonenvs/dialogflow/bin/actuate

      2. Navigate to the chatbot customer binder.

                                                                cd                            dialogflow-riva-weatherbot-webapp

      3. Start the chatbot web server.

    5. Open up the interface in a web browser https://<riva chatbot server host IP>:6006/rivaWeather , e.chiliad. https://0.0.0.0:6006/rivaWeather.

Network Configuration¶

Update the config.py script for inter-service communication.

                                riva_config                =                {                "RIVA_SPEECH_API_URL"                :                "[riva spoken language service host IP]:50051"                ,                "..."                :                "..."                }

In the case where i is using _localhost_ , the "RIVA_SPEECH_API_URL" must exist set to the local auto's IP, instead of _localhost_ .

Sample Conversations¶

Sample ane¶

Virtual Banana: Hullo, welcome to RIVA weather service. How can I help y'all?

User: How's the weather?

Virtual Assistant: For which location?

User: In Seattle.

Virtual Assistant: Information technology is Partly cloudy in Seattle at the moment. The temperature is 16 degrees, the humidity is 97 percent and the wind speed is 0 miles per hour.

Sample 2¶

Virtual Assistant: Hi, welcome to RIVA weather service. How tin can I help you?

User: How's the weather in Paris?

Virtual Assistant: Information technology is Clear in Paris at the moment. The temperature is twenty degrees, the humidity is 73 per centum and the current of air speed is 9 miles per hour.

Sample 3¶

Virtual Assistant: Hi, welcome to RIVA weather condition service. How can I help you?

User: Hullo

Virtual Assistant: Hey! How can I assistance you?

User: What is your proper name?

Virtual Assistant: My friends call me Ja-Di.

Sample 4¶

Virtual Assistant: Howdy, welcome to RIVA weather service. How tin I help you?

User: Good forenoon

Virtual Assistant: Hey! How can I assist you lot?

User: See you around

Virtual Assistant: Bye

Limitations¶

  • The provided samples are not complete virtual assistants; but are intended as unproblematic examples of how to build basic task-oriented chatbots with Riva. Consequently, the intent classifier and slot filling models have been trained with pocket-sized amounts of data and are non expected to be highly authentic.

  • The Dialogflow Weatherbot sample supports intents for cloudy, humidity, rainfall, snow, sunny, temperature, weather and windy checks. It does not support general conversational queries or other domains.

  • The Dialogflow Weatherbot supports just 1 slot for city . Information technology doesn't take into business relationship the day associated with the query while providing the response.

  • This sample supports upwardly to four concurrent users. This restriction is not considering of Riva, merely because of the web framework (Flask and Flask-ScoketIO) that is beingness used by the client web application. The socket connection to stream sound to (TTS) and from (ASR) the user is unable to sustain more than 4 concurrent socket connections.

License¶

For applicative licenses, refer to the License section.

mangoldpands1942.blogspot.com

Source: https://docs.nvidia.com/deeplearning/riva/user-guide/docs/samples/dialogflow.html

0 Response to "Dialog Flow Why Cant I Upload Zip for Restore the Zip"

Post a Comment

Subscribe to: Post Comments (Atom)

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel