Sync Data with Azure Data Factory

How to make your data available to Unified Contacts

The Azure Data Factory is a chargeable Microsoft Service. In normal use, these charges are nominal. Please visit https://azure.microsoft.com/en-us/pricing/details/data-factory/data-pipeline/ for more Details.

  • To set up a connection between a Data Source and the Azure Data Factory, please log in to Azure Portal, navigate to the Resource group of Unified Contacts and create a new resource by clicking "+ Create" and subsequently searching for Azure Data Factory.

  • This will take you through the steps to create a Data Factory

  • When the deployment has completed successfully, a confirmation screen will be displayed

  • Clicking on "Go to Resource" will bring you to the Azure Data Studio Launch Page

  • Click "Launch Studio" to begin the data source configuration

  • In the next step, we need to ingest (collect) the data from our external source that we want to use in the Data Factory

The Azure Data Factory supports more than 90 Data Sources natively (e.g. MySQL, Oracle DB, SAP, MongoDB, Microsoft Dynamics, Salesforce, Xero). For Details of the available connectors, please visit https://learn.microsoft.com/en-us/azure/data-factory/connector-overview.

  • Click on "Ingest" to begin the configuration

  • Select Built-In copy task and Schedule. Normally, a daily ingestion is sufficient but it can be configured to run more frequently if required.

  • In the Source Screen, select the data source that you want to use and then click "New connection"

  • In this example, we are ingesting data from a Dynamics Business Central Instance and are going to connect via HTTP to it.

  • Under the Source data store, select HTTP

  • In the next screen, enter a connection Name, the Base URL and chosen Connection method. Then click "Next".

  • In the following screen, all defaults can be accepted

  • Next, we need to provide the details of the database where the data collected from our Data Source will be stored. This database is created as part of the Unified Contacts installation (UC Database).

  • If you did not choose a custom naming when installing Unified Contacts Pro the DB-Server is called db-uc- followed by 13 digits and numbers and the DB itself sql-db by default.

  • By choosing the Account Selection Method "From Azure Subscription" the database details will be retrieved automatically.

  • Once you have entered all the data, click "Apply"

  • Then confirm the File format settings for your data source

  • Configure the Source -> Destination mappings

Please ensure that your data fits the specified database schema. Any rows not fitting the schema might cause the Copy activity to fail or the faulty rows to be skipped, depending on the configured fault tolerance. In the case of fields that are too long consider editing the data in your source (query) or adding a Data Flow activity to transform it into a valid form (i. e. shortening the text and adding "..." at the end.)

  • Finally, configure the settings for the copy data task

  • Under Fault tolerance, choose the required level

  • Once you have completed the final step, the Data Factory can be deployed

Last updated

Was this helpful?