Skip to main content
Adaptive Insights
Knowledge and Support - Adaptive Insights

Setting Up a JDBC Database Data Source

You can use Adaptive Integration to extract information from JDBC-compliant databases. Most databases used for accounting and finance applications are JDBC-compliant (Java database connectivity) databases. JDBC is a standard that uses the Java programming language for providing connectivity to a wide range of databases.

 

A JDBC-compliant database data source requires a driver.  The table below gives information on common JDBC drivers.

 

JDBC Source

Overview

Provider jar

Driver Class

JDBC/URL example

JDBC
URL info

Oracle

Introduction to the Oracle Database

ojdbc6-11.2.0.3.jar

oracle.jdbc.
OracleDriver

jdbc:oracle: thin:@jyates-win-db: 1521:ANACONDA

Class Ora­cleDriver

SQL Server

Microsoft SQL Server

jtds-1.3.1.jar

net.sourceforge
.jtds.jdbc.Driver

jdbc:jtds:sqlserver://127.0.0.1:1433/AGENT_TEST; domain=ADAPTIVE; instance=
SQLEXPRESS

jTDS for­mats

MySql

MySQL Refer­ence Manuals

mysql-connector-5.1.17.jar

com.mysql.jdbc
.Driver

jdbc:mysql://localhost: 3306/AGENT_TEST

URL Syn­tax

h2

H2 Database Engine

Look for h2*.jar in the Adaptive Data Agent Service Manager distribu­tion.

org.h2.Driver

jdbc:h2:/data/test

 

H2 Cheat Sheet

The jar files go into the C:/Program Files/Adaptive Data Agent/Plugins directory on the machine hosting the Adaptive Data Agent Service. After you add a jar file, you need to go into the local services and restart the Adaptive Data Agent Service.

Before you can set up a data source from a local database, you need to have an Adaptive Data Agent created that tells Adaptive Integration how to com­municate with the database. For information on how to create data agents, see  Using Adaptive Data Agents.

To set up a JDBC database data source, follow these steps:

  1. Access the Data Designer by going to Integration > Data Designer.

  2. In the Data Source folder in the Component Library on the right side of the screen, click Create New Data Source.
    The Create New dialog box appears.

  3. Select JDBC database as the data source type, enter the database’s name, and select the agent you want to use for this data source.

  4. Click Create.
    The center area of the screen displays your new data source’s settings and other information, as shown in below.

  5. Enter the data source’s information:

  • Driver class: Enter the driver class for the database. (You can obtain this infor­mation from the database vendor.)
  • URL: Enter the URL of the database.
  • Username: Enter the username used for accessing the database.
  • Password: Enter the password associated with the username.
  • Schema Filter: Enter the name of the schema(s) from which you would like to import data. If you are entering the names of more than one schema, enter the schema names separated by a semicolon. Use this field to restrict the number of tables that need to be refreshed in Adaptive Integration. This field is optional.
  • Log level: Select a log level from the drop-down list to specify the detail for the logging for this data source. Your options are:
    • Error: Only logs serious errors.
    • Info: Logs all basic information, such as when the data source was updated.
    • Verbose: Provides very detailed information about all phases and actions. (This level is used primarily for debugging or auditing, as it may produce more log information than is practical for typical use.)
  1. Click Save on the Actions menu.

You can test the connection information in the new data source by clicking Test connec­tion in the Actions menu. The system tries to connect to the database using the informa­tion in the data source fields. If the system can connect to the database, click OK. If there are connection problems, fix the errors and try again.

When you can connect to the database, you’re ready to import the data structure. This shows you the tables and columns in the database.

Only columns for supported types appear. “Supported Col­umn Types,” lists the supported column types for the various types of databases.

To import the JDBC database’s structure, do the following:

  1. Click Import structure in the Actions menu. The tables and columns appear in the Data Components menu.

The data import modes affect the way in which the data is synchronized. The default import mode (all records replaced) for a JDBC data source is as follows:

  1. All rows in staging are cleared.

  2. The JDBC data is imported.

  3. Data from JDBC source populates staging table.

  4. If there is a custom SQL filter. it is applied to the data to be imported from the data­base server to determine the subset of data to import.

You can also choose to append imported information, in which case imported data is appended to the existing staging table, or merge imported information, in which case existing rows in the JDBC data are merged with the rows in the staging area.

Adding Tables to a JDBC Data Source

After you have imported the database’s structure, you are ready to specify the information you want to import from the database. The Tables to Import section of the screen lets you configure the data imported from the database.

To specify the tables to import from the database, follow these steps:

  1. Expand the entry for the data source in the Data Components menu, as shown below.

  2. Each table in the database appears as a separate item.

 3. Drag and drop the tables you want to import to the Tables to Import section in the center of the screen, as shown below. As you drag each table to the Tables to Import section, a tab appears for the table and information from the database appears in the Tables to Import section letting you see the type of information you’ve added. (Several sample tables appear below.) Each table’s tab also has a drop-down menu, which you can access by clicking the arrow next to its name.

 

Managing Columns

If a table you are working with has a large number of columns, you can use the Manage Columns to do quick edits for which columns to import. The Manage Columns dialog box displays the columns in a table so you can then check or uncheck columns for importing.

To manage columns for importing, follow these steps:

  1. Click the arrow on the tab to the right of the table name.
    A floating menu with several options appears.

  2. Click Manage Columns on the floating menu.
    The Manage Columns dialog box appears.

 3. Check the box next to a column to import it. When you check a column, the column also appears within the preview window after the popup contents are saved. Unchecking a column removes the column from the preview window.

You can not change the import status on system-generated fields such as isDe­leted. You can also change the import status on individual columns by selecting col­umn properties and setting the property or by dragging and dropping the columns onto the designer.

Customizing the Tables

For each table, you can customize the way in which data is imported.

To customize the table settings, follow these steps:

  1. Click the arrow on the tab to the right of the table name.
    A floating menu with several options appears.

  2. Click Table Settings on the floating menu.
    The Table Settings dialog box appears, as shown below.

 3. Enter the table settings:

  • Data Import Mode: Select one of the options from the drop-down list.
    • All records replaced each time a data import is run
      This is the default. The data in the new version of the database table replaces the previous version in the staging area.
    • Remote cache used to identify and send changed records on import
      Only database records that have changed or are new are imported. Changed records overwrite existing records in the staging area. New records are added to the end of the information in the staging area.
    • All records that fall within a period range transmitted
      All of the records in the period range you specify are imported.
  • Data Import Filter: Use this to enter a SQL expression that filters or formats information being imported. The SQL filter is applied before data is extracted from the database to reduce the amount of data that needs to be imported, giving you an additional level of control for which data is brought into the staging area. Click the field to display the Edit SQL Filter dialog box, shown earlier. (The filter you enter here is applied to the actual data in the database rather than the preview data in the staging area.) Not all data sources support the same level of SQL fil­tering capability.
  1. Click Apply to apply the settings.
  2. Repeat steps 1-3 for each table.

If a database table has keys defined, this information is brought into the staging table as well. You can review this information and edit as needed.

  1. Click Save to save the settings in the data source.

Setting Column Options

While using the Tables to Import section, you can see a preview of the data as it is in the data source or as it is in the Staging area after it has been imported. You can change the options for each column by mousing over its heading and then clicking the drop-down menu next to its name. These options are:

  • Sort Ascending: Sort the entire table based on this column in ascending order.
  • Sort Descending: Sort the entire table based on this column in descending order.
  • Column Settings: This dialog is used to change the properties of the column within the staging table.
    • Name: The name of the column, displayed in the header.
    • Column Type: The type of data used in the column.
    • Convert Column Type: Select the new type from this drop-down menu.
  • Exclude column from import: Select this to remove this column from the data being imported.
  • Delete custom column: Select this to delete the column. (Available for custom col­umns only.)

Downloading Data from a Staging Table

You can download data from a single staging table.

To download data from a single staging table, follow these steps:

  1. Click the down arrow in a table heading in the Tables to Import section.

  2. Select Download Data.

  3. At the prompt “This operation will send an email with a url to download the data,” click Send. Adaptive Integration sends an email containing a URL to the email account associated with the data designer.

  4. Open the email and click the URL. The data is downloaded in .csv format.

Using the Advanced Filter

Below the Source drop-down menu, you can click Advanced Filter to reveal options for fil­tering the data in the Tables to Import preview area. The Advanced Filter provides tools to view and browse a subset of the data within a staging table or directly from the remote source (if you select JDBC Database from the Source drop-down menu). By changing fil­ter properties, you can change the view of the data shown within the preview window with­out changing the data in the staging area.

The preview area is specifically for previewing a small portion of the data in the staging area (or JDBC database). You can explore the data that is available from the JDBC Data source and verify that you are getting the data you expect by examining the staging area. You can also use this to refine the data import filter you want to use. (Filters set in the preview func­tion do not affect the actual data in the staging area.)

The options for the Advanced Filter are:

  • Distinct Rows: Check this box to hide duplicate rows.
  • Max Rows: Limit the number of rows displayed in the preview area by entering a number in the text box.
  • Columns: Check or uncheck columns in this drop-down menu to show or hide the columns displayed.

Removing a column from this list does not remove the column from the import list.

  • SQL Filter: Click the large text box to display the Edit SQL Filter dialog box (shown earlier), where you can enter a SQL filter to restrict the rows being previewed. Click Apply to automatically check the SQL syntax of the filter. (If there are any errors, the error is indicated in the expression.) The expression you entered appears in the Advanced Filter field. For detailed SQL syntax help, you can click online help in the Notes section.

Click Remove Filter (next to the Source drop-down menu) to clear all advanced filter set­tings.

As part of the table customization process, you can also create join tables and custom columns using SQL expressions.

Importing Data from a JDBC Database

When you have set up and saved your JDBC data source, you can import data.

To import data from a JDBC database, do the following:

  1. Click Import data in the Actions menu.
    The selected data is filtered by any SQL expressions and loaded into the staging area. You can preview the data in the staging area

Information remains in the staging area until you clear it manually, or until a subsequent data load clears it in accordance with the import mode.

To clear data from the staging area, do the following:

  1. Click Clear staging tables in the Actions menu.
    The staging area for all the tables in the data source is cleared.

To remove specified characters from Input Text Streams in JDBC data sources:

Refer to Remove Specified Characters from Input Text Streams in JDBC and Microsoft Dynamics GP  

JDBC Data Source Excluded Schemas

JDBC data sources that connect to Oralce or MS SQL Server explicitly exclude some schemas that do not relate to financial data.

Oracle JDBC Data Source Excluded Schemas

CTXSYS, MDSYS, SYSTEM, SYS, XDB

MS SQL Server JDBC Data Source Excluded Schemas

INFORMATION_SCHEMA, SYS

Troubleshooting Tips

There are a few data source problems you may run into that have solutions that are not immediately apparent.

When you import the structure for a database and the database has more than 1000 tables, you get an error advising you to filter the table list. You can get past this error by doing either of the following:

  • Editing the JDBC data source and entering a schema filter.
  • Changing the permissions associated with the login you’re using to access the server so fewer tables are accessible through that login.

When you import the structure of an Oracle database, the following schemas are not imported as part of the structure: CTXSYS, MDSYS, SYSTEM, SYS, and XDB. These are Oracle systems schemas and are not necessary for data integration purposes.

When you import the structure of an MS SQL database, the following schemas are not imported as part of the structure: INFORMATION_SCHEMA and sys. These are MS SQL systems schemas and are not necessary for data integration purposes.

  • Was this article helpful?