How to import my data ?

• Available with
• Free
• Tailored
• Team

Interactive CSV Import

iTop provides a powerful CSV Import feature to assist end-users (and administrators) in massively creating or updating objects in iTop.

The CSV import does not on purpose, apply any filter on values, as you would have when editing the object in the Console. It does not apply any prefill either. But checks made when you save the object, are identicals.

To launch the CSV Import Wizard, click on the CSV Import menu in the Data Administration section.

 

CSV Templates

iTop provides CSV templates for creating/importing any class of objects. You can either download or copy/paste the template from the third tab Templates in the CSV Import wizard.

Select the desired class from the drop-down list to generate the template. Then either click on the big Excel icon to download the template, or copy/paste the text from the text area below the icon.

Another way to obtain a pre-filled template is to export existing objects in CSV. Refer to Export data for detailed instructions about how to perform this task.

 

Expected format for values

In order to load some type of attributes, you need to provide their value in a particular format.

LinkedSet

Click here to check the format required to import relationships

Date

Date format of the imported data can be specified during the import, but all dates in your source file must be using the same format. This format can be either the default one defined on your iTop or any other one that you specify.

Text with HTML

You can provide the raw HTML with tags. Example: 

<p><strong>Bold</strong><br/>
Yes</p>

Image / file (New in 2.4)

Some classes contain an image or a file document as one of their field. For example, the "Person" class, contains a "Picture" field which holds the picture of the person. When editing a person interactively, the user can upload an image from its computer to provide the picture of the person. However, when performing a CSV import, it is not possible to “upload” such a file or to put its content inside the CSV file itself, since the CSV format does not support this.

In such a case the CSV file must provide - in the appropriate CSV column - an URL to upload the file from. This URL must be accessible from the iTop server and return the expected image (or file document). If the upload fails or if the format of the uploaded document does not match the expected format for the field (for example if the URL does not return an image for the picture field), the import/update will fail for this field (and the whole line of the CSV will be rejected).

Limitations:

• The URL must point directly to the image to upload (redirections are not supported)

• The URL must be accessible from the iTop web server (beware some public websites may deny access to a PHP script - they check the HTTP User Agent string)

• The URL must not require authentication, since the upload is done by the iTop server itself

• URLs pointing to iTop itself (like the URLs provided by CSV/Excel export) are supported. But only if they point to the same iTop instance (in such a case the import will parse the provided URL and will read the document from the iTop database).

• If you use an administrator account to perform the CSV import, you can specify a path to a file located on the iTop server itself (but this works only for administrators).

External object

For loading an attribute that is an external key to another object, you can provide any combination of attributes which are declared as reconciliation attributes for the remote class.

Example: if you were to load UserRequest and set the Caller (caller_id), you could provide any combination of those columns to identify the caller:

• Caller→Email

• Caller→Organization name

• Caller→First Name

• Caller→Last Name

• Caller→Full Name

• Caller→Employee number

TagSet

To load a TagSet attribute, the tags have to be imported with their label separated by a pipe "|". Better put a pipe at the beginning and at the end.

Uploading Data

The first step of the CSV import consists in uploading the data to the server. You can either upload a file from your computer or directly copy/paste the data in the second tab of the wizard.

• To upload the file from your computer, use the file upload form on the first tab of the wizard. Make sure you select the same character set that was used to save the file.If the character set that you use is not available in the drop-down list, contact your iTop administrator (this list is configurable via the csv_import_charsets configuration variable).
• If you choose to copy/paste the data, use the second tab "Copy and paste data" of the wizard:

Then click Next » to proceed with the next step of the wizard.

The following parameters must be adjusted during this step:

Parameter	Meaning
Separator Character	This is the character used for separating the different columns inside the CSV file. If the columns are properly displayed and separated in the preview at the bottom of the page, then the current choice is correct. (The typical choice for a CSV file is the comma)
Text Qualifier Character	Sometimes the values inside the cells contain the separator character. Such fields are then enclosed by the text qualifier character to prevent the separator character from being wrongly interpreted in the middle of a cell. (A typical value is either a simple or double quote)
Treat the first line as a header	Check this box if the first line of the file contains a header (column names) instead of actual data. Note that on the preview at the bottom of the page, the header line is displayed with a different color.
Skip x lines at the beginning	Some CSV files (typically CSV files generated by hand-made scripts) contain some comments at the beginning of the file. Use this setting to tell iTop to ignore as many lines as needed. The skipped lines are removed from the preview at the bottom of the page.
Date format	Specify in which format the date and date-time fields are written in your source data. You must use a single format for all dates in your source. Date do not need to specify time, if data-time do not contain any time information 00:00:00 will be used

If the accented characters do not displayed properly in the preview, go back to the first page of the wizard using the "Back" button and pick the appropriate character set before uploading again your CSV file.

Once the data looks correct in the preview, click on the "Next" button to move on to the next step:

The third step of the import wizard defines the mapping between the column of the CSV file and the fields of the object in iTop.
First: Select the class to import within the drop-down list. This determines the list of fields available for the mapping.
If the class you want is not in the list:

• You either don't have sufficient access rights, you need bulk-modify rights on that class to be allowed to CSV import objects.

• Or it's not possible to CSV import objects of that class, this is the case for some of the iTop internal classes (CMDBChange, CMDBChangeOp, Event,…), even for Administrators.

 

Fields Mapping

If the CSV file has been originally created using a template or an iTop CSV export, the header provided in the CSV should automatically give you the correct mapping.

If this is not the case, you have to create this mapping manually.

The wizard requires a mapping for each column of the imported data. However a column can be ignored (i.e. not imported) by selecting – ignore this field – as the mapping for this column.

 

Search Criteria

The CSV import wizard can be used to either create or update objects in iTop. The wizard uses the search criteria defined at this step to determine whether each line of the CSV file corresponds to an existing object in iTop or not. If the object exists, then it will be updated, otherwise it will be created. Typically in iTop search forms, all selected search criteria are combined using the AND operator. (i.e. an object must match each value of the search criterias to be considered as a match).

The exact same imported CSV file can produce different results in iTop depending on the selected search criteria. The number of objects created or updated depends on the search criteria defined at this stage.

Advanced usage for Administrators

Administrators have access to an advanced mode (a checkbox at the top-right of the page), which allows to import objects based on their internal identifiers (ID or Key number). This is useful for renaming objects exported from iTop. Do not use this option for objects not exported from iTop, or at least make sure that you know the data you are importing! The Advanced mode also allows administrators to import LinkedSet as described here.

Design tip for Administrators

Reconciliation attributes are defined for each class in the Datamodel. If such attributes are provided in the load, they are automatically set as Search criteria.
If the user wants to update some of those search criterias, with its load, they must uncheck them and use other criteria for retrieving the iTop objects.

Once the mapping and the search criteria looks correct, proceed to the simulation of the import by clicking on the Simulate Import: button.

Simulate import

After a few seconds, the following summary screen is displayed:

The lines in errors are highlighted with a red background. Errors will not prevent the import from running, but the corresponding lines will be rejected.

This screen provides feedback on what was not understood by the import wizard. In the example above:

• The “active” value for the Person status, which is an "enumeration" type of field, was not understood. "active" is the true code stored inside the database, but it is not what iTop expects here. It expects the label of that code in the user language, so you should provide “Active” with a capital A. → The CSV import is case-sensitive.

• There is no organization with the name “IT Department2”→ Cautious, it can be due to the fact that the current user is not allowed to see that particular Organization.

• iTop cannot determine which Paris location you are willing to use. In this example the database contains 2 Paris locations, one under the “IT Department” organization and another under “Demo”.

To solve the last error, restart the load and in excel add a new column to specify the “Location->Name of the owner organization”.

Note1: The mapping of the last row was done manually (first and second columns values are different) as we didn't know the exact expected naming for the “Location organization”.
Note2: When trying to map the provided location name, it doesn't check if this location maps the filter defined in the datamodel. This technical choice was made on purpose to get usable response time when loading large amount of data.

Note the number of objects to be created and the number of objects to be updated, as well as the number of errors. You can use the checkboxes at the top of the page to filter the display for a quicker analysis of the results.

If the results from the simulation look unconventionnal, maybe the Search criteria aren't correct. If so, then press "Back" button to change them and simulate again. If you need to modify the data, click on "Restart" (the "Copy and paste data" tab will contain a copy of the uploaded data). If the simulation looks correct, click on "Run the Import!" to perform the actual import.

 

Run import

If the number of objects imported and the percentage of objects to be created or updated exceeds a certain level, a confirmation dialog is displayed:

Click OK to close the dialog and proceed with the data import.

The last screen of the wizard displays the summary of the operations performed during the import.

In case of errors, the rejected lines are listed at the bottom of the report, they can be copied/pasted and corrected before being imported again.

An alternative is to proceed iteratively:

1. Complete the import wizard, and note which lines have errors. Lines containing errors are simply ignored.

2. Fix the lines in error in your reference CSV file.

3. Re-run the import with your reference CSV file (the whole file)

4. Repeat until no error is found.

iTop will ignore lines that have been unchanged, because the corresponding data is already up-to-date (unless modified between two imports by somebody else in the mean time ). Iterating on the whole CSV file may be the preferred option, if you need to maintain a reference CSV file of your data.