Intelligent summary frontend application example

​

Important notes

1. API key security: Do not hard-code your API key or commit it to a code repository. Manage your API key using an environment variable.
2. Browser permissions: The real-time recording feature requires users to grant microphone permissions in the browser.
3. Network requirements: The project needs to access the Alibaba Cloud Model Studio service. Ensure your network connection is stable.
4. Important: The dependency packages in this project are not published to the public npm repository. To perform custom development or publish the packages, rename them yourself.

​

Features

​

Quick Start

​

1. Prerequisites

1. Log on to the Alibaba Cloud Model Studio console.
2. In the navigation pane on the left, choose API key.
3. Click Create New API Key.
4. You can copy the generated API key.

1. Log on to the Alibaba Cloud Model Studio console.
2. Click Create Application to generate your application.
3. You can find and copy the App ID from the application card.
   1. 

​

2. Project configuration and startup

​

Environment requirements

• Node.js: Version 18.0.0 or later
• Package manager: npm

​

Startup steps

1. Download the project: You can download the offline file to obtain the project code.

1. Install dependencies

cd tingwu-web
npm install

2. Configure the API key (Important!) You can edit the .env file and replace the placeholders (xxxxx) with your actual values. Ensure you add the .env file to your project's .gitignore file to prevent key leakage.

#TODO: Replace with your API_KEY and APP_ID
REACT_APP_API_KEY=xxxxx
REACT_APP_APP_ID=xxxxx

3. Start the development server

npm run start

4. View the result You can open http://localhost:3000 in your browser. The project automatically opens and displays the frontend interface in your browser.

​

API usage guide

​

Feature usage examples

​

Audio and video file upload

• Supports single-channel or dual-channel audio files in MP3, WAV, M4A, WMA, AAC, OGG, AMR, FLAC, or AIFF format. Supports video files in MP4, WMV, M4V, FLV, RMVB, DAT, MOV, MKV, WEBM, AVI, MPEG, 3GP, or OGG format.
• The file size cannot exceed 6 GB.
• The audio duration cannot exceed 6 hours.
• Supported audio sampling rates are 8 kHz, 16 kHz, 24 kHz, and 48 kHz.

1. Click the Select File button to select a local audio or video file.
2. You can upload the file to your own business server or OSS.
3. You can call the createTask API to create a task. You can replace the appId in the code with your actual value and customize the parameters and analysis parameters as needed.
4. You can call the getTask API to query the task parsing status. The dataId parameter is the return value from the createTask API call in the previous step.

const { output, code } = await createTaskAPI({
input: {
 task: "createTask",
 type: "offline",
 fileUrl, // The URL of the file after uploading it to your business OSS or server
 appId: "Your APP_ID",
},
parameters: {
 transcription: {
   diarizationEnabled: true,
   diarizationSpeakerCount: 0,
   translationEnabled: false,
 },
 analysis: {},
},
});

​

Real-time recording

• Supported input formats: PCM (uncompressed PCM or WAV), OPUS, SPEEX, MP3, and AAC. The audio bit depth must be 16-bit and the sound channel must be mono.
• Supported audio sampling rates: 16000 Hz and 8000 Hz.
• Maximum duration for a single recording: 24 hours.

1. Before you start recording, verify that the browser's microphone permission is enabled.
2. You can call the createTask API to create a real-time task. You can replace the appId and configure the parameters as needed. Code example:

const requestParams = {
input: {
 task: "createTask",
 type: "realtime",
 format: "pcm",
 sampleRate: 16000,
 appId: "Your APP_ID",
},
parameters: {
 transcription: {
   model: language,
   diarizationEnabled: true,
   diarizationSpeakerCount: 0,
   translationTargetLang: "zh",
   translationEnabled: true,
 },
 analysis: {},
},
};

3. You can establish a WebSocket connection. This example includes a heartbeat and a reconnection mechanism, with up to 3 retries.
4. There are two methods to obtain the WebSocket Secure (WSS) URL: First, the server-side returns the complete URL. Second, the frontend constructs the URL, and the server-side returns a temporary token. This example uses the second method. Because the temporary token has a short validity period, you must obtain a new one for each reconnection attempt.
5. You can call the getTask API to query the task parsing status. The dataId parameter is the return value from the createTask API call.

​

Get and display results

​

Intelligent summary content

• Transcribed text: The complete speech-to-text result.
• Speaker diarization: Differentiates the speech of different speakers.
• Intelligent summary: Automatically generates a summary of the meeting content.
• Q&A review: Extracts key questions and answers.
• Chapter overview: Browse the summary by time or content segments.
• Translation: Supports multi-language translation. This feature must be enabled when the task is created.

​

Developer reference

​

System architecture

• For more information, see the project file at ./tingwu-web/docs/ARCHITECTURE.md.

​

Project structure

tingwu-web/
├── packages/
│   ├── tingwu-example/     # Example application (main startup entry)
│   ├── tingwu/service/     # API service layer
│   ├── tingwu-components/  # Component library
│   ├── tingwu/core/       # Core logic
│   ├── tingwu/summary/       # Tingwu intelligent summary details page
│   ├── tingwu/home/       # Configuration page to enable intelligent summary
│   └── ...                # Other packages

​

Main commands

• npm run start: Starts the development server.
• npm run build: Builds the production version.
• npm run lint: Runs code linting.
• npm run test: Runs tests.

​

Important notes

1. API key security: Do not hard-code your API key or commit it to a code repository. Manage your API key using an environment variable.
2. Browser permissions: The real-time recording feature requires users to grant microphone permissions in the browser.
3. Network requirements: The project needs to access the Alibaba Cloud Model Studio service. Ensure your network connection is stable.