Data Enrichment using ChatGPT

Dataset Finder powered by ChatGPT API




Zoho DataPrep helps you enrich your data by finding new datasets based on your requirements using GenAI, powered by OpenAI or Zoho LLM. You can use these datasets for data enrichment purposes. To enable this feature in your DataPrep organization, click Find new dataset with ChatGPT during import, GenAI configuration tab will open when you try to enable it for the first time or go to the GenAI Integration tab in the Settings page. Click here to know more about enabling GenAI integrated features in the settings page.  With this connector, you can streamline your ETL workflows, enable smooth data movement, and simplify data integration.
Here's a quick video on ChatGPT API integration with Zoho DataPrep. 



Here's a quick demo on the features powered by OpenAI in Zoho DataPrep. 



Notes
  • The prompt and required contextual data will be shared with the selected AI vendor. Personally identifiable information (PII) is never shared with the selected AI Vendor.
  • The GenAI features, when enabled, can be used by the Account admin and Organization admins in your DataPrep organization.
  • The API key is specific to the DataPrep organization it is configured in and will not be shared across organizations or with any other Zoho service.

    • To configure and enable OpenAI's ChatGPT integration

    Info
    OpenAI in Zoho DataPrep works in a BYOK (Bring Your Own Key) model, in which users bring their own ChatGPT API Key and OpenAI Organization ID. If you are already an OpenAI user, you can use your API key and organization ID to unlock AI features in DataPrep. 

    Info Data Privacy Notice: Zoho DataPrep utilizes OpenAI's ChatGPT API, an external service based in the USA, subject to OpenAI's Terms of Use. Please be aware that the information sent to OpenAI depends on the access provided by you, the user. The data shared with ChatGPT is based on the features you enable. Details such as column names and/or selected cell data will be shared based on each feature. You can choose to enable or disable each feature based on the level of data you want to share. Therefore, exercise caution while using this service and ensure that the data you provide does not contain confidential or personal information. 


    Notes
    Notes: 
    1. The generative responses are coming in from ChatGPT's training data. Accuracy, consistency, and relevance of the response are subjective to the type of prompts and the data available in its training set. 
    2. ChatGPT doesn't respond to prompts about personal information.

    1.  During import, choose the Zoho Apps category > Find new dataset with ChatGPT. The GenAI Configuration tab will open. Click Authenticate Now to authenticate with the selected AI vendor.

    The GenAI Configuration tab opens only for the first time. From then on, you can edit the configuration from the GenAI Configuration tab on the Settings page.





    Alert
    Note: The GenAI configuration on DataPrep's Settings page will be visible only to the Account admin.
    Only the Account admin can enable features for your DataPrep organization.

    2. Click Authenticate now next to OpenAI. 



    3. A consent pop-up will be shown. Please read the data privacy notice carefully before you proceed. By proceeding, you acknowledge and accept the risks and OpenAI's terms of use policy. 



    4. Provide the API Key and Organization ID of your OpenAI organization to configure OpenAI integration in your DataPrep organization. 
    Notes Note: The key will be encrypted and securely stored in Zoho DataPrep. Learn how to get the Organization ID and the API key from OpenAI


    5. Once saved, go to the GenAI Skills tab. Toggle on Dataset Finder, and select OpenAI from the vendor dropdown.



    Organization ID and API key in OpenAI

    1. Go to your settings page in OpenAI to get the Organization name and ID. Here's a snapshot for reference. Read more from OpenAI



    2. Go to the API keys tab under the Organization section to get your API Key. Here's a snapshot for reference. Read more from OpenAI


    To configure and enable Zoho LLM

    Zoho LLM is Zoho's own large language model. You can enable it directly from the GenAI configuration settings without needing an external API key. Once enabled, it powers AI features in Zoho DataPrep.
    NotesNote: This setting is visible and accessible only to the Account admin.

    1. During import, choose the Zoho Apps category > Find new dataset with ChatGPT. The GenAI Configuration tab will open. Click Authenticate Now to authenticate with the selected AI vendor.

    The GenAI Configuration tab opens only for the first time. From then on, you can edit the configuration from the GenAI Configuration tab on the Settings page.





    2. Locate Zoho LLM in the AI Vendor list and click Enable configuration.



    3. A consent pop-up will be shown. Please read the data privacy notice carefully before you proceed.



    4. In the Authentication panel, confirm that the Zoho LLM toggle is enabled and click Save.



    5. Once saved, go to the GenAI Skills tab. Toggle on Dataset Finder, and select Zoho LLM from the vendor dropdown.


    Data Enrichment powered by GenAI API

    1. Create a pipeline or open an existing pipeline from the Home Page, Pipelines tab or Workspaces tab and click the Add data option. You can also click the Import data option under the Workspaces tab to import data.  
    Info
    Info: You can also click the Import data  icon at the top of the pipeline builder and bring data from multiple sources into the pipeline.




    2. Choose the Zoho Apps category from the left pane and click Find new dataset with ChatGPT. You can also directly search the option in the search box.



    3. Provide a hint about the type of data you are looking for in the Instruction prompt. For example, create a dataset for salesforce CRM instagram "Liked" list. You can also insert one of the prompts from the Insert hints section.



    4. Click Fetch data and you will see a preview of the data generated by ChatGPT.



    5. Click the Import button to import data. The Pipeline builder page opens and you can start applying transforms to the ETL pipeline. You can also right-click the stage and choose the Prepare data option to prepare your data using various transforms in the DataPrep Studio page. Click here to know more about the transforms. 


    7. Once you are done creating your data flow and applying necessary transforms in your stages, you can right-click a stage and add a destination to complete your data flow.
    Note:  After adding a destination to the ETL pipeline, you can try executing your pipeline using a manual run at first. Once you make sure manual run works, you can then set up schedule to automate the pipeline and data movement. Learn about the different types of runs here.
    While configuring the Schedule, Backfill, Manual reload, Webhooks, or Zoho Flow, the import configuration needs to be mandatorily setup for all the sources. Without setting up the import configuration, the run cannot be saved. Click here to know more about how to set up import configuration.

    8. After configuring a run, a pipeline job will be created at the run time. You can view the status of a job with the granular details in the Job summary. Click here to know more about the job summary.

    Here are some sample prompts for you to try out

    a. Create a dataset of sales tax rates by state in the US.



    b. Create a dataset for places with the highest average temperature in the US.