Skip to main content

Paraformer non-real-time speech recognition HarmonyOS SDK

This guide explains how to use the Paraformer non-real-time speech recognition HarmonyOS SDK to convert speech to text.

User guide: Non-real-time speech recognition

Quick start

  1. Obtain an API key: Obtain and configure an API key. For security, we recommend configuring the API key as an environment variable.
    To grant temporary access to third-party applications or users, or to strictly control high-risk operations such as accessing or deleting sensitive data, use a temporary API key. A temporary API key is valid for 60 seconds by default. Obtain a new one after it expires.
  2. Download the SDK and run the sample code:
    • Download the latest SDK package.
    • Extract the TAR package. Obtain the HAR-format SDK from the neonui directory and add it to your project dependencies. For C++ integration, obtain the dynamic libraries and header files from native/libs and native/include in the TAR package.
    • Open the project in DevEco Studio. The sample code is located in DashParaformerFileTranscriberPage.ets. Replace the API key to try the feature.

Invocation steps

Synchronous mode

  1. Initialize the SDK.
  2. Configure parameters based on your business requirements.
  3. Call startFileTranscriber to start a recognition task with async_request set to false.
  4. In the onFileTransEventCallback interface, listen for the EVENT_FILE_TRANS_RESULT event to obtain the final recognition result.
  5. Call release to release SDK resources.

Asynchronous mode

  1. Initialize the SDK.
  2. Configure parameters based on your business requirements.
  3. Call startFileTranscriber to start a recognition task with async_request set to true.
  4. Call queryFileTranscriber to query the recognition progress or result.
  5. In the onFileTransEventCallback interface, listen for the EVENT_FILE_TRANS_QUERY_RESULT event to obtain the current query result.
  6. In the onFileTransEventCallback interface, listen for the EVENT_FILE_TRANS_RESULT event to obtain the final recognition result.
  7. Call release to release SDK resources.

Request parameters

Connection and control parameters

Pass a JSON string to the parameters parameter of initializeFileTrans to configure the following parameters. Example: The following JSON string is an example and does not include all parameters. Add parameters as needed: 
{
    "url": "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/audio/asr/transcription",
    "apikey": "st-****",
    "device_id": "my_device_id",
    "service_mode": "1"
}
  • Parameter descriptions
ParameterTypeRequiredDescription
urlstringYesThe endpoint. This is fixed at wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/audio/asr/transcription. Replace {WorkspaceId} with your actual workspace ID.
apikeystringYesThe API key. We recommend using a more secure temporary API key with a short validity period to reduce the risk of leaking a long-term key.
service_modestringYesThe operating mode. For non-real-time speech recognition, this is fixed at "1".
device_idstringYesA unique string that identifies the end user. You can set it to an in-app user ID or a unique device identifier generated by the client. This ID is mainly used for log tracking and troubleshooting.
debug_pathstringNoThe storage path for log files. This parameter takes effect only if you set save_log to true when you call initializeFileTrans. In this case, you must specify a log file path. Otherwise, an error occurs. A maximum of two log files are retained locally.
max_log_file_sizenumberNoSets the maximum size of a log file in bytes. This parameter takes effect only if you set save_log to true when you call initializeFileTrans. Default value: 104857600 (100 * 1024 * 1024 bytes, or 100 MiB).

Speech recognition effect parameters

Configure the nls_config parameter by using setParams, or configure all speech recognition effect parameters by using startFileTranscriber. Example: The following JSON string is an example and does not include all parameters. Add parameters as needed: 
{
    "file_urls": [
        "{YOUR_AUDIO_URL}"
    ],
    "async_request": false,
    "nls_config": {
        "model":"paraformer-v2",
        "disfluency_removal_enabled":false,
        "timestamp_alignment_enabled": false
    }
}
  • Parameter descriptions
ParameterTypeRequiredDescription
file_urlsarray[string]Yes

A list of URLs for the audio or video files to transcribe. HTTP and HTTPS are supported. A single request supports only 1 URL. If an audio file is stored in Alibaba Cloud OSS, the SDK does not support a temporary URL that uses the oss:// prefix. - Audio formats: aac, amr, avi, flac, flv, m4a, mkv, mov, mp3, mp4, mpeg, ogg, opus, wav, webm, wma, and wmv.

Because audio and video formats have many variants, it is technically impossible to test all of them. The API cannot guarantee that every format can be correctly recognized. Test your file to verify that it produces a valid recognition result.

- Audio sample rates vary by model: - paraformer-v2 supports any sample rate. - paraformer-8k-v2 supports only 8 kHz. - Audio file size and duration: The file cannot exceed 2 GB, and its duration cannot exceed 12 hours. To process a file that exceeds these limits, preprocess it to reduce the file size. For best practices, see Preprocess video files to improve transcription efficiency for audio file recognition.

async_requestbooleanNoSpecifies whether to use an asynchronous request. Default value: false. Valid values: - true: asynchronous request - false: synchronous request.
apikeystringNoIf apikey in Connection and control parameters is a temporary API key, update it here before it expires.
nls_configobjectYesThe core speech recognition configuration object, which contains key parameters for model selection and recognition effect control.
nls_config.modelstringYesThe speech recognition model.
nls_config.language_hintsarray[string]NoSpecifies language codes for the audio to recognize. This parameter applies only to paraformer-v2. Default value: ["zh", "en"]. Supported language codes: - zh: Chinese - en: English - ja: Japanese - yue: Cantonese - ko: Korean - de: German - fr: French - ru: Russian.
nls_config.disfluency_removal_enabledbooleanNoSpecifies whether to remove disfluencies such as filler words. Default value: false. Valid values: - true: removes disfluencies - false: retains disfluencies.
nls_config.timestamp_alignment_enabledbooleanNoSpecifies whether to enable timestamp alignment. Default value: false. Valid values: - true: enabled - false: disabled.
nls_config.special_word_filterobjectNoSpecifies sensitive words to process during speech recognition and supports different processing methods for different words. If this parameter is omitted, the built-in sensitive-word filtering logic is enabled. Words that match the Alibaba Cloud Model Studio sensitive word list are replaced with the same number of * characters. If this parameter is specified, matched words can be replaced with * characters or removed. The value is a JSON object in the following format: { "filter_with_signed": { "word_list": ["test"] }, "filter_with_empty": { "word_list": ["start", "happen"] }, "system_reserved_filter": true }. JSON field descriptions: - filter_with_signed: An optional object that specifies words to replace with the same number of * characters. In the preceding example, the result for “Help me test this piece of code” is “Help me ** this piece of code”. Its word_list field is an array of strings to replace. - filter_with_empty: An optional object that specifies words to remove from the result. In the preceding example, the result for “Is the game about to start?” is “Is the game about to?”. Its word_list field is an array of strings to remove. - system_reserved_filter: An optional Boolean. Default value: true. Specifies whether to enable built-in sensitive-word rules. If set to true, words matching the Alibaba Cloud Model Studio sensitive word list are replaced with the same number of * characters.
nls_config.channel_idarray[integer]No

Specifies the indexes of audio tracks to recognize in a multitrack audio file. Indexes start from 0. For example, [0] recognizes the first track, and [0, 1] recognizes the first and second tracks. If omitted, the first track is processed.

Each specified track is billed separately. For example, [0, 1] for one file incurs two separate charges.

Default value: [0].

nls_config.diarization_enabledbooleanNo

Specifies whether to enable speaker diarization. This feature is disabled by default. It applies only to mono audio. Multichannel audio does not support speaker diarization. When enabled, the recognition result contains speaker_id to distinguish speakers.

If speaker diarization is enabled, we recommend limiting the audio duration to 2 hours to avoid recognition failure or timeout.

For an example of speaker_id, see Recognition result descriptions.

nls_config.speaker_countintegerNoA reference value for the number of speakers. To use this feature, set diarization_enabled to true. By default, the number of speakers is automatically determined. If this parameter is set, it only guides the algorithm to output the specified number and does not guarantee that number. Valid range: [2, 100]. The minimum is 2 because this feature distinguishes multiple speakers.
nls_config.vocabulary_idstringNoThe hotword vocabulary ID, which improves the recognition accuracy of specific words. This parameter applies to v2 and later models. For more information, see Custom hotwords.

Key interfaces

NativeNui

initializeFileTrans

Initializes a speech transcription SDK instance. Do not initialize the instance again before you call release.
Unlike real-time speech recognition, non-real-time audio file transcription must use initializeFileTrans and pass an INativeFileTransCallback callback instead of using initialize.
This interface blocks the calling thread. Call it from a non-UI thread.
  • Method signature
public initializeFileTrans(callback: INativeFileTransCallback,
                           parameters: string,
                           level: number,
                           save_log: boolean = false): number
  • Parameter descriptions
ParameterTypeDescription
callbackINativeFileTransCallbackAn implementation of the file transcription event and data callback interface.
parametersstringA JSON string that contains authentication, connection, and debugging parameters. See Connection and control parameters.
levelnumberControls the SDK log level. Valid values are defined by the Constants.LogLevel enumeration.
save_logbooleanSpecifies whether to save local logs. If this parameter is true, use debug_path in Connection and control parameters to specify a path. You can also use max_log_file_size to set the file size.
  • Return value

setParams

Sets or updates only the nls_config parameter. If all parameters are provided in startFileTranscriber, you do not need to call this method.
  • Method signature
public setParams(params: string): number
  • Parameter descriptions
ParameterTypeDescription
paramsstringThe nls_config parameter in Speech recognition effect parameters. Parameters outside nls_config cannot be set by using this method. Example: { "nls_config": { "model":"paraformer-v2", "disfluency_removal_enabled":false, "timestamp_alignment_enabled": false } }
  • Return value

startFileTranscriber

Starts recognition.
  • Method signature
public startFileTranscriber(params: string, task_id: ArrayBuffer): number
  • Parameter descriptions
ParameterTypeDescription
paramsstringThe speech recognition effect parameters. Example: { "file_urls": [ "{YOUR_AUDIO_URL}" ], "async_request": false, "nls_config": { "model":"paraformer-v2", "disfluency_removal_enabled":false, "timestamp_alignment_enabled": false } }
task_idArrayBufferThe task ID buffer. The SDK writes an internally generated random task ID string to this buffer. The buffer must be at least 33 bytes long. The example uses new ArrayBuffer(64). After a successful call, decode the buffer to obtain the task's task_id.
  • Return value

queryFileTranscriber

Queries the current status and result of an asynchronous task. After a successful call, the result is returned by the EVENT_FILE_TRANS_QUERY_RESULT event in the onFileTransEventCallback callback.
  • Method signature
public queryFileTranscriber(task_id: string): number
  • Parameter descriptions
ParameterTypeDescription
task_idstringThe ID of the task to query, obtained from the buffer written by startFileTranscriber.
  • Return value

cancelFileTranscriber

Immediately cancels the current task.
  • Method signature
public cancelFileTranscriber(task_id: string): number
  • Parameter descriptions
ParameterTypeDescription
task_idstringThe ID of the task to cancel.
  • Return value

release

Releases all internal SDK resources. After this method is called, the SDK instance becomes unavailable. To use it again, call initializeFileTrans to reinitialize it.
  • Method signature
public release(): number
  • Return value

GetVersion

Obtains the current SDK version information.
  • Method signature
public GetVersion(): string
  • Return value
The current SDK version information.

INativeFileTransCallback

Defines callbacks for file transcription events and recognition results.

onFileTransEventCallback

Listens for file transcription events and obtains speech recognition results.
  • Method signature
onFileTransEventCallback: (event: Constants.NuiEvent, resultCode: number, finish: number,
                           asrResult: AsrResult, taskId: string) => void;
  • Parameter descriptions
ParameterTypeDescription
eventConstants.NuiEventThe callback event.
resultCodenumberValid only when the EVENT_ASR_ERROR event occurs.
finishnumberIndicates whether the task has ended.
asrResultAsrResultThe speech recognition result.
taskIdstringThe task ID.

Constants.NuiEvent

HarmonyOS SDK event types are defined by the Constants.NuiEvent enumeration. The following events apply to audio file transcription:
EventDescription
EVENT_FILE_TRANS_CONNECTEDThe service is connected.
EVENT_FILE_TRANS_UPLOADEDThe audio file to recognize is uploaded.
EVENT_FILE_TRANS_QUERY_RESULTThe task query result.
EVENT_FILE_TRANS_RESULTThe final recognition result.
EVENT_ASR_ERRORAn error occurs during speech recognition.

Auxiliary types

Constants.LogLevel

The enumeration values for the level parameter are as follows:
ValueDescription
LOG_LEVEL_VERBOSEThe most detailed logs.
LOG_LEVEL_DEBUGDebug logs.
LOG_LEVEL_INFOInformational logs (default).
LOG_LEVEL_WARNINGWarning logs.
LOG_LEVEL_ERRORError logs.
LOG_LEVEL_NONEDisables logging.

Download the result

Non-real-time speech recognition results are generated asynchronously. The response returned by the EVENT_FILE_TRANS_RESULT event contains transcription_url. Download the recognition text in JSON format from this URL. The URL has a validity period. Download the result promptly.
API Usage
Text Generation
Image Generation
Video Generation
World models
3D Model Generation
Audio
Real-time Multimodal
Realtime API
Text Embedding
Translation models
Decision Model
More Models
Toolkit/Framework
TokenPlan
Model Production
File Management
Billing
Model Management
More