Incremental data import from local databases

Import Incremental data from local databases




Incremental data fetch is a method used to import new or modified data from a source. Zoho DataPrep helps you import incremental data from the following local databases using Zoho Databridge:
  1. MySQL
  2. MS SQL Server
  3. Oracle
  4. PostgreSQL
  5. Maria DB
  6. Pervasive SQL
  7. Sybase
  8. DB2
  9. Exasol
  10. Sqlite
  11. Actian Vector
  12. Greenplum
  13. Denodo
  14. Progress OpenEdge
  15. YugabyteDB
  16. Microsoft Access
  17. SAP Hana
  18. Connect using JDBC URL

To start with import

1. Open an existing pipeline or create a pipeline from the Home Page, Pipelines tab or Workspaces tab and click the Add data option.
InfoInfo: You can also click the Import data  icon at the top of the pipeline builder to bring data from multiple sources into the pipeline.


2. In the next screen, choose the required database or click the Databases category from the left pane. 



3. Select New connection from the Connection drop down. If you have existing connections, you can choose the required connection from the Connection drop down. 

4. Give your connection a name under the Connection name section.

5. Zoho Databridge is a tool that facilitates importing data from local databases. Databridge is mandatory to import data from local network.
Notes
Note: If this is the first time you are downloading Databridge, see how to install it here.



6. Once you have installed Databridge on your machine, select your Databridge from the Databridge drop-down. 
Notes
Note: Select the Databridge which is installed in the same network as the database you want to import the data from.

7. Select your Database type and enter the Database server host name and Port number.

8. Enter your Database name and provide the username and password if authentication is required.



9. Save your database configuration and connect to the database using Connect.
Notes
Note: The connection configuration will be automatically saved for importing from the database in the future. Credentials are securely encrypted and stored.


10. Select the table that need to be imported and click the Import button.
Warning
The incremental fetch option is not available when the data is imported using a query from databases.

11Once you have completed importing data, Pipeline builder page will open from where you can start applying transforms. You can also right-click the stage and choose the Prepare data option to prepare your data in the DataPrep Studio pageClick here to know more about the transforms.



12. Once you are done creating your data flow and applying necessary transforms in your stages, you can right-click a stage and add a destination to complete your data flow.
Notes
Note:  After adding a destination to the pipeline, you can try executing your pipeline using a manual run at first. Once you make sure manual run works, you can then set up schedule to automate the pipeline. Learn about the different types of runs here.

Import configuration for local database


If you import data from local database using the Select tables option, you can configure how to import and fetch incremental data from your local database using the below Import configuration options.

Below is a snapshot from the schedule configuration.



How to import data from source? Select the way you would like to import your data from the drop-down - Import all dataIncremental file fetchDo not import data.

Import all data

This option will import all available data for every run.



Incremental data fetch

Incremental data import is a method used to import new or modified records in a specific data interval.
Warning
Important: This incremental fetch option is not available when the data is imported using a query from databases.

 




Only modified and new data

To import the modified and new data incrementally from the last imported time, select Only modified and new data option from the drop-down.

Fetch based on: You can enter the date-time column name based on which the data must be sorted and imported.

Use the previously imported file if no new file is available: 

When there are no new data in the source during incremental import,

If the checkbox is checked: The last fetched data will be imported again.

If the checkbox is unchecked: The import will be skipped and no data will be imported.



Do not import data

The data is imported only once. The second time, the rules get applied to the same data and get exported.



      Create. Review. Publish.

      Write, edit, collaborate on, and publish documents to different content management platforms.

      Get Started Now


        Access your files securely from anywhere

          Zoho CRM Training Programs

          Learn how to use the best tools for sales force automation and better customer engagement from Zoho's implementation specialists.

          Zoho CRM Training
            Redefine the way you work
            with Zoho Workplace

              Zoho DataPrep Personalized Demo

              If you'd like a personalized walk-through of our data preparation tool, please request a demo and we'll be happy to show you how to get the best out of Zoho DataPrep.

              Zoho CRM Training

                Create, share, and deliver

                beautiful slides from anywhere.

                Get Started Now


                  Zoho Sign now offers specialized one-on-one training for both administrators and developers.

                  BOOK A SESSION







                              Quick LinksWorkflow AutomationData Collection
                              Web FormsRetailOnline Data Collection Tool
                              Embeddable FormsBankingBegin Data Collection
                              Interactive FormsWorkplaceData Collection App
                              CRM FormsCustomer ServiceForms for Solopreneurs
                              Digital FormsMarketingForms for Small Business
                              HTML FormsEducationForms for Enterprise
                              Contact FormsE-commerceForms for any business
                              Lead Generation FormsHealthcareForms for Startups
                              Wordpress FormsCustomer onboardingForms for Small Business
                              No Code FormsConstructionRSVP tool for holidays
                              Free FormsTravelFeatures for Order Forms
                              Prefill FormsNon-Profit
                              Forms for Government
                              Intake FormsLegal
                              Mobile App
                              Form DesignerHR
                              Mobile Forms
                              Card FormsFoodOffline Forms
                              Assign FormsPhotographyMobile Forms Features
                              Translate FormsReal EstateKiosk in Mobile Forms
                              Electronic FormsInsurance
                              Drag & drop form builder

                              Notification Emails for FormsAlternativesSecurity & Compliance
                              Holiday FormsGoogle Forms alternative GDPR
                              Form to PDFJotform alternativeHIPAA Forms
                              Email FormsWufoo alternativeEncrypted Forms
                              Accessible FormsTypeform alternativeSecure Forms

                              WCAG

                                          Create. Review. Publish.

                                          Write, edit, collaborate on, and publish documents to different content management platforms.

                                          Get Started Now






                                                            You are currently viewing the help pages of Qntrl’s earlier version. Click here to view our latest version—Qntrl 3.0's help articles.




                                                                Manage your brands on social media

                                                                  Use cases

                                                                  Make the most of Zoho Desk with the use cases.

                                                                   
                                                                    

                                                                  eBooks

                                                                  Download free eBooks and access a range of topics to get deeper insight on successfully using Zoho Desk.

                                                                   
                                                                    

                                                                  Videos

                                                                  Watch comprehensive videos on features and other important topics that will help you master Zoho Desk.

                                                                   
                                                                    

                                                                  Webinar

                                                                  Sign up for our webinars and learn the Zoho Desk basics, from customization to automation and more

                                                                   
                                                                    
                                                                  • Desk Community Learning Series


                                                                  • Meetups


                                                                  • Ask the Experts


                                                                  • Kbase


                                                                  • Resources


                                                                  • Glossary


                                                                  • Desk Marketplace


                                                                  • MVP Corner



                                                                    Zoho Sheet Resources

                                                                     

                                                                        Zoho Forms Resources


                                                                          Secure your business
                                                                          communication with Zoho Mail


                                                                          Mail on the move with
                                                                          Zoho Mail mobile application

                                                                            Stay on top of your schedule
                                                                            at all times


                                                                            Carry your calendar with you
                                                                            Anytime, anywhere




                                                                                  Zoho Sign Resources

                                                                                    Sign, Paperless!

                                                                                    Sign and send business documents on the go!

                                                                                    Get Started Now




                                                                                            Zoho TeamInbox Resources





                                                                                                      Zoho DataPrep Demo

                                                                                                      Get a personalized demo or POC

                                                                                                      REGISTER NOW


                                                                                                        Design. Discuss. Deliver.

                                                                                                        Create visually engaging stories with Zoho Show.

                                                                                                        Get Started Now









                                                                                                                            • Related Articles

                                                                                                                            • Import incremental data from local files

                                                                                                                              Incremental data fetch is a method used to import new files or modified files from a source. Zoho DataPrep helps you import incremental data from your local files using Zoho Databridge. In DataPrep, you can incrementally import various file types, ...
                                                                                                                            • Import data from Microsoft Access

                                                                                                                              You can import data from the Microsoft Access database into Zoho DataPrep using Zoho Databridge. Click here to know about Zoho Databridge. Before looking into how to import data, let's take a look at the pre-requisites to connect to the Microsoft ...
                                                                                                                            • Import data from SAP Hana

                                                                                                                              You can import data from the SAP Hana database into Zoho DataPrep using Zoho Databridge. Click here to know about Zoho Databridge. Before looking into how to import data, let's take a look at the pre-requisites to connect to the SAP Hana database. ...
                                                                                                                            • Import data from local databases

                                                                                                                              You can import data from the following local databases into DataPrep using Zoho Databridge: MySQL MS SQL Server Oracle PostgreSQL Maria DB Pervasive SQL Sybase DB2 Exasol Sqlite Actian Vector Greenplum Denodo Progress OpenEdge YugabyteDB Microsoft ...
                                                                                                                            • Import data using Custom JDBC URL

                                                                                                                              You can import data using Custom JDBC URL into Zoho DataPrep using Zoho Databridge. Click here to know about Zoho Databridge. Before looking into how to import data, let's take a look at the pre-requisites to connect using the Custom JDBC URL. ...
                                                                                                                              Wherever you are is as good as
                                                                                                                              your workplace

                                                                                                                                Resources

                                                                                                                                Videos

                                                                                                                                Watch comprehensive videos on features and other important topics that will help you master Zoho CRM.



                                                                                                                                eBooks

                                                                                                                                Download free eBooks and access a range of topics to get deeper insight on successfully using Zoho CRM.



                                                                                                                                Webinars

                                                                                                                                Sign up for our webinars and learn the Zoho CRM basics, from customization to sales force automation and more.



                                                                                                                                CRM Tips

                                                                                                                                Make the most of Zoho CRM with these useful tips.



                                                                                                                                  Zoho Show Resources