Skip to main content

Best practices for Xiyan GBI virtual data sources

For data security reasons, some customers do not grant Alibaba Cloud services access to their databases. Xiyan GBI offers an unmanaged database connection solution that uses virtual data sources. This solution ensures that database operations are performed locally, while still allowing customers to use the SQL generation capabilities of the cloud service.

Prerequisites

Introduction to database connection methods

image.png
  • The standard managed database connection method offers easy integration with the Xiyan GBI cloud service. You can simply authorize and associate your data source on the product page. Xiyan GBI can then directly access your business database to retrieve SQL query results.
  • The unmanaged database connection method provides enhanced data security. However, it requires you to execute SQL statements and retrieve query results on your local client. The virtual data source is the Xiyan GBI solution that simplifies this process and reduces associated costs.

Introduction to the virtual data source solution

Xiyan GBI offers the virtual data source solution for users who cannot use a managed database connection because of data security concerns. You can use the Xiyan GBI SDK to create a virtual data source that mirrors the structure of your business database. Then, you can associate this virtual data source with a workspace in Xiyan GBI. This process lets you generate SQL queries without directly accessing your business database.
image.png

Create, associate, and authorize a virtual data source

Step 1: Create a virtual data source instance

Use the createVirtualDatasourceInstance API to create up to ten virtual data source instances per workspace. Each instance is an empty database instance, similar to an ApsaraDB RDS instance. 
/**
     * Create a virtual data source instance
     *
     * @param client
     */
    public void create(AsyncClient client) {
        CreateVirtualDatasourceInstanceRequest createVirtualDatasourceInstanceRequest = CreateVirtualDatasourceInstanceRequest.builder()
                // The ID of the workspace where the virtual data source instance will be created.
                .workspaceId("workspaceId")
                // The name of the virtual data source instance.
                .name("name")
                // The description of the virtual data source instance.
                .description("description")
                // The dialect type of the virtual data source instance. 51: MySQL. 52: PostgreSQL.
                .type(51)
                .build();
        CompletableFuture<CreateVirtualDatasourceInstanceResponse> virtualDatasourceInstance = client.createVirtualDatasourceInstance(createVirtualDatasourceInstanceRequest);
        CreateVirtualDatasourceInstanceResponse createVirtualDatasourceInstanceResponse = virtualDatasourceInstance.join();
        System.out.println(createVirtualDatasourceInstanceResponse.getBody().getData());
    }

Step 2: View the list of virtual data source instances

Use the listVirtualDatasourceInstanceRequest API to view all virtual data source instances in the current workspace. This lets you obtain the unique identifier (vdbId) for each virtual data source. 
    /**
     * Get the list of virtual data source instances
     *
     * @param client
     */
    public void list(AsyncClient client) {
        ListVirtualDatasourceInstanceRequest listVirtualDatasourceInstanceRequest = ListVirtualDatasourceInstanceRequest.builder()
                // The ID of the workspace whose data source instance list you want to retrieve.
                .workspaceId("")
                .build();
        ListVirtualDatasourceInstanceResponse listVirtualDatasourceInstanceResponse = client.listVirtualDatasourceInstance(listVirtualDatasourceInstanceRequest).join();
        Gson gson = new Gson();
        String jsonString = gson.toJson(listVirtualDatasourceInstanceResponse.getBody().getData());
        System.out.println(jsonString);
    }

Step 3: Add DDL statements to the virtual data source instance

Use the saveVirtualDatasourceDdl API to add DDL statements to a specified virtual data source instance (vdbId). The Xiyan GBI cloud service parses these DDL statements to generate and maintain structured information for the corresponding data tables and columns. 
/**
     * Add DDL statements to the virtual data source
     * @param client
     */
    public void addDdlForVirtualDatasource(AsyncClient client){
        SaveVirtualDatasourceDdlRequest saveVirtualDatasourceDdlRequest = SaveVirtualDatasourceDdlRequest.builder()
                // The workspace to which the virtual data source instance belongs.
                .workspaceId("workspaceId")
                // The unique identifier (ID) of the virtual data source instance.
                .vdbId("vdb-id")
                // The DDL statements to add. You can add multiple statements separated by semicolons (;).
                .ddl("CREATE TABLE user (id int NOT NULL AUTO_INCREMENT comment 'ID',name varchar(10) NOT NULL comment 'Name', dept_id int NOT NULL comment 'Department ID', salary bigint NULL DEFAULT 0 comment 'Salary',  PRIMARY KEY (`id`),   KEY `idx_name` (`name`),   KEY `d_id` (`dept_id`),    CONSTRAINT `d_id` FOREIGN KEY (`dept_id`) REFERENCES `department` (`id`)) COMMENT='User table' ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;")
                .build();
        SaveVirtualDatasourceDdlResponse  saveVirtualDatasourceDdlResponse = client.saveVirtualDatasourceDdl(saveVirtualDatasourceDdlRequest).join();
        System.out.println(saveVirtualDatasourceDdlResponse.getBody().getData());
    }

Step 4: Attach the virtual data source instance to your workspace

Use the createDatasourceAuthorization API to associate a specified virtual data source instance (vdbId) with a workspace. 
/**
     * Authorize and associate the data source instance with the workspace
     * @param client
     */
    public void createAuthorize(Client client){
        CreateDatasourceAuthorizationRequest createAuthorizeRequest = new CreateDatasourceAuthorizationRequest();
        // The type of the associated data source instance. For details, see the SDK documentation. When associating a virtual data source instance, the type specified during creation is used by default.
        createAuthorizeRequest.setType(51);
        // The ID of the associated workspace.
        createAuthorizeRequest.setWorkspaceId("workspaceId");
        // If you are associating a virtual data source, add the virtual data source instance ID.
        createAuthorizeRequest.setVdbId("vdb-id");
        try {
            CreateDatasourceAuthorizationResponse createDatasourceAuthorizationResponse = client.createDatasourceAuthorization(createAuthorizeRequest);
            System.out.println(createDatasourceAuthorizationResponse.getBody().getData());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

Step 5: Select the required data tables in your virtual data source

Use the syncRemoteTables API to synchronize information about specified data tables with the Xiyan GBI cloud service. Subsequent intelligent Q&A and SQL generation operations will be based on the data tables you select. 
/**
     * Associate data tables from the data source with the specified workspace. Intelligent Q&A will be based on the associated data tables.
     * @param client
     */
    public void syncRemoteTables(AsyncClient client){
        ArrayList<String> tableNames = new ArrayList<>();
        tableNames.add("user");
        SyncRemoteTablesRequest syncRemoteTablesRequest = SyncRemoteTablesRequest
                .builder()
                // The workspace for the data tables to be associated.
                .workspaceId("workspaceId")
                .tableNames(tableNames)
                .build();
        SyncRemoteTablesResponse syncRemoteTablesResponse = client.syncRemoteTables(syncRemoteTablesRequest).join();
        System.out.println(syncRemoteTablesResponse.getBody().getData());
    }

Code example for using a virtual data source instance

This code example uses the Xiyan GBI SDK to demonstrate an end-to-end scenario that includes creating, associating, and authorizing a virtual data source. This example is designed to help you become familiar with the API. Before you run the code, replace the accessKeyId, accessKeySecret, and workspaceId placeholders with your actual values. The following code example demonstrates how to manage a virtual data source instance, including operations such as creating, modifying, deleting, and viewing the instance. The example also shows how to authorize and associate the instance with a workspace and link specific tables from the instance to Xiyan GBI. 
package com.alibaba.iic.llmsolution.gbi.util;

import com.aliyun.auth.credentials.Credential;
import com.aliyun.auth.credentials.provider.StaticCredentialProvider;
import com.aliyun.sdk.gateway.pop.Configuration;
import com.aliyun.sdk.gateway.pop.auth.SignatureVersion;
import com.aliyun.sdk.service.dataanalysisgbi20240823.AsyncClient;
import com.aliyun.sdk.service.dataanalysisgbi20240823.models.*;
import com.google.gson.Gson;
import darabonba.core.client.ClientOverrideConfiguration;

import java.util.ArrayList;
import java.util.concurrent.CompletableFuture;

public class VirtualDatasourceTest {
    public static void main(String[] args) {
        StaticCredentialProvider provider = StaticCredentialProvider.create(
                Credential.builder()
                        .accessKeyId("")
                        .accessKeySecret("")
                        .build()
        );

        AsyncClient client = AsyncClient.builder()
                .region("cn-beijing")
                .credentialsProvider(provider)
                .serviceConfiguration(Configuration.create()
                        .setSignatureVersion(SignatureVersion.V3)
                )
                .overrideConfiguration(
                        ClientOverrideConfiguration.create()
                                .setProtocol("HTTPS")
                                .setEndpointOverride("dataanalysisgbi.cn-beijing.aliyuncs.com")
                )
                .build();
    }

    /**
     * Create a virtual data source instance
     *
     * @param client
     */
    public void create(AsyncClient client) {
        CreateVirtualDatasourceInstanceRequest createVirtualDatasourceInstanceRequest = CreateVirtualDatasourceInstanceRequest.builder()
                // The ID of the workspace where the virtual data source instance will be created.
                .workspaceId("workspaceId")
                // The name of the virtual data source instance.
                .name("name")
                // The description of the virtual data source instance.
                .description("description")
                // The dialect type of the virtual data source instance. 51: MySQL. 52: PostgreSQL.
                .type(51)
                .build();
        CompletableFuture<CreateVirtualDatasourceInstanceResponse> virtualDatasourceInstance = client.createVirtualDatasourceInstance(createVirtualDatasourceInstanceRequest);
        CreateVirtualDatasourceInstanceResponse createVirtualDatasourceInstanceResponse = virtualDatasourceInstance.join();
        System.out.println(createVirtualDatasourceInstanceResponse.getBody().getData());
    }

    /**
     * Get the list of virtual data source instances
     *
     * @param client
     */
    public void list(AsyncClient client) {
        ListVirtualDatasourceInstanceRequest listVirtualDatasourceInstanceRequest = ListVirtualDatasourceInstanceRequest.builder()
                // The ID of the workspace whose data source instance list you want to retrieve.
                .workspaceId("")
                .build();
        ListVirtualDatasourceInstanceResponse listVirtualDatasourceInstanceResponse = client.listVirtualDatasourceInstance(listVirtualDatasourceInstanceRequest).join();
        Gson gson = new Gson();
        String jsonString = gson.toJson(listVirtualDatasourceInstanceResponse.getBody().getData());
        System.out.println(jsonString);
    }

    /**
     * Modify the information of a virtual data source instance
     * @param client
     */
    public void update(AsyncClient client) {
        UpdateVirtualDatasourceInstanceRequest virtualDatasourceInstanceRequest = UpdateVirtualDatasourceInstanceRequest.builder()
                // The ID of the workspace where the virtual data source instance was created.
                .workspaceId("")
                // The name of the virtual data source instance.
                .name("name")
                // The description of the virtual data source instance.
                .description("description")
                // The unique identifier (ID) of the virtual data source instance. You can obtain this from the virtual data source instance list.
                .vdbId("vdb-id")
                .build();
        UpdateVirtualDatasourceInstanceResponse updateVirtualDatasourceInstanceResponse = client.updateVirtualDatasourceInstance(virtualDatasourceInstanceRequest).join();
        System.out.println(updateVirtualDatasourceInstanceResponse.getBody().getData());
    }

    /**
     * Delete a virtual data source instance
     * @param client
     */
    public void delete(AsyncClient client){
        DeleteVirtualDatasourceInstanceRequest deleteVirtualDatasourceInstanceRequest = DeleteVirtualDatasourceInstanceRequest.builder()
                // The workspace to which the virtual data source instance to be deleted belongs.
                .workspaceId("workspaceId")
                // The unique identifier (ID) of the virtual data source instance to be deleted.
                .vdbId("vdb-id")
                .build();
        DeleteVirtualDatasourceInstanceResponse deleteVirtualDatasourceInstanceResponse = client.deleteVirtualDatasourceInstance(deleteVirtualDatasourceInstanceRequest).join();
        System.out.println(deleteVirtualDatasourceInstanceResponse.getBody().getData());
    }

    /**
     * Add DDL statements to the virtual data source
     * @param client
     */
    public void addDdlForVirtualDatasource(AsyncClient client){
        SaveVirtualDatasourceDdlRequest saveVirtualDatasourceDdlRequest = SaveVirtualDatasourceDdlRequest.builder()
                // The workspace to which the virtual data source instance belongs.
                .workspaceId("workspaceId")
                // The unique identifier (ID) of the virtual data source instance.
                .vdbId("vdb-id")
                // The DDL statements to add. You can add multiple statements separated by semicolons (;).
                .ddl("CREATE TABLE user (id int NOT NULL AUTO_INCREMENT comment 'ID',name varchar(10) NOT NULL comment 'Name', dept_id int NOT NULL comment 'Department ID', salary bigint NULL DEFAULT 0 comment 'Salary',  PRIMARY KEY (`id`),   KEY `idx_name` (`name`),   KEY `d_id` (`dept_id`),    CONSTRAINT `d_id` FOREIGN KEY (`dept_id`) REFERENCES `department` (`id`)) COMMENT='User table' ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;")
                .build();
        SaveVirtualDatasourceDdlResponse  saveVirtualDatasourceDdlResponse = client.saveVirtualDatasourceDdl(saveVirtualDatasourceDdlRequest).join();
        System.out.println(saveVirtualDatasourceDdlResponse.getBody().getData());
    }

    /**
     * Authorize and associate the data source instance with the workspace
     * @param client
     */
    public void createAuthorize(Client client){
        CreateDatasourceAuthorizationRequest createAuthorizeRequest = new CreateDatasourceAuthorizationRequest();
        // The type of the associated data source instance. For details, see the SDK documentation. When associating a virtual data source instance, the type specified during creation is used by default.
        createAuthorizeRequest.setType(51);
        // The ID of the associated workspace.
        createAuthorizeRequest.setWorkspaceId("workspaceId");
        // If you are associating a virtual data source, add the virtual data source instance ID.
        createAuthorizeRequest.setVdbId("vdb-id");
        try {
            CreateDatasourceAuthorizationResponse createDatasourceAuthorizationResponse = client.createDatasourceAuthorization(createAuthorizeRequest);
            System.out.println(createDatasourceAuthorizationResponse.getBody().getData());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /**
     * Associate data tables from the data source with the specified workspace. Intelligent Q&A will be based on the associated data tables.
     * @param client
     */
    public void syncRemoteTables(AsyncClient client){
        ArrayList<String> tableNames = new ArrayList<>();
        tableNames.add("user");
        SyncRemoteTablesRequest syncRemoteTablesRequest = SyncRemoteTablesRequest
                .builder()
                // The workspace for the data tables to be associated.
                .workspaceId("workspaceId")
                .tableNames(tableNames)
                .build();
        SyncRemoteTablesResponse syncRemoteTablesResponse = client.syncRemoteTables(syncRemoteTablesRequest).join();
        System.out.println(syncRemoteTablesResponse.getBody().getData());
    }

}
Agent Development
Agent Context
Agent Governance & Evolution
Store
Application Marketplace
Use Cases
Support