Upload Offline Customer Data into Adobe Audience Manager (Part 2)

This post is a continuation to Part 1 of the series where I’m covering how to upload offline customer data to AAM. This post goes even deeper into the weeds and covers how we can add, modify and remove (we cannot really delete data from the backend but can exclude users) customer data from AAM. There are different ways to configure the batch upload file for AAM, but this post only shows how to leverage key-value pairs to bring in data. The official Adobe documentation goes into some more detail on those other methods.

Let’s first take a look at a few steps we need to take as a prerequisite before uploading batch files:

1. File Extensions: There are two types of file extensions that are supported in the file name. The first one is “.sync” that is used to add records to Audience Manager and the other one is “.overwrite” that is used to overwrite or modify records. These two were covered in a lot of detail in the previous section.
2. File Name: The file name starts with 'ftp_dpm' (fixed) followed by a random number typically less than 10 digits along with the file extension. So, an example file name can be: ftp_dpm_Data-Source-ID_201831012.sync|.overwrite where the data source ID is the same as '#1' in the screenshot below, number used is in the “Year/Month/Date” format (can be anything) and .sync or .overwrite are the file extensions.
3. File Size: Each file should ideally not exceed more than 5GB and can be zipped using .gz so if you need to upload greater than 5GB then you can upload multiple files. The nomenclature of these files will be named like ftp_dpm_Data-Source-ID_123456789.sync.1.gz or ftp_dpm_Data-Source-ID_123456789.sync.2.gz.
4. Attribute Limit: According to Adobe’s documentation referred earlier, there is a limit of 200 lines Adobe can process for each user ID sent in the inbound data file.
5. Column Structure of the File: As covered in the previous section, the user ID is in the first column. The second column (separated by a tab) contains all key value pairs separated by a comma. Some points to consider:
• It’s easy to mix up the “ double quote with the " quotes being used in the second column. The double quote that will work is the " quote so it’s better to use a separate text editor to configure the file as excel might switch up the quotes being used.
• Numeric data can be sent as a string in the file but in the AAM UI, they will be evaluated as numbers so logical operators such as "<" OR ">" can be used while creating traits.
• Make sure that the tab between column one and column two is actually a tab and not a space as that will break the file format.
6. Data Source Setup: As discussed in the previous post, one of the first things we need to capture to initiate a batch upload process is to capture the authenticated user ID on the website. The data source ID (#1) is hard coded into the batch file name which covered in the third step. We tie this ID to Audience Manager via an integration code (#2 below) by initiating an ID sync. The user ID needs to be mapped to a data source by tying the user ID to the Customer ID inbound field (#3). Also, enable the option to sample errors so that you can pinpoint the exact cause in case of a failure. 

Source: Audience Manager UI

We're now going to cover how to add, modify and remove records from Audience Manager while leveraging key-value pairs:

• Add New Records to Audience Manager (".sync" format): This is the most common type of upload where we simply add new user IDs along to the first column and user attributes to the second column as shown below. The file format used here is “.sync”.

• Modify Already Uploaded Records (".overwrite" format): In this type of upload using the “.overwrite” format, we add all existing attributes of the user along with the modified records. In the example below, we’ve uploaded all existing records and only modified the ‘age’ and ‘purchases_last_month’ fields. If we don’t add certain attributes added previously, those attributes will not be part of user qualification moving forward.

• Add New Attributes to Existing Records: If you want to add a new attribute to an existing record (user ID), there are two ways to do this:
• (".overwrite" format): In this format, we include all existing records for a user along with the new attribute (company_size) as shown below:

• (".sync" format): In this format, we include just the new attributes (company_size) we need to add  for a user as shown below:

• Add Multiple Values for the same Dimension (".sync" format): We simply add the dimension that we wish to add data for multiple times in the same row as shown below:

• Remove Attributes from Existing Records (".overwrite" format): If you want to remove a certain attribute from exiting records, upload an ".overwrite" file with just the attributes you wish to keep and exclude the one you wish to remove. In the example below, we've removed the 'company_size' attribute that we added earlier. Also, note that we included the updated 'age' and 'purchases_last_month' fields:

• Delete Records from Audience Manager: There is no way to really "delete" a record from Audience Manager. The recommended approach is to include an attribute/flag (E.g. existing_customer=N) in the batch file and add it as a condition in AAM. Users with this flag can be part of an exclusion list that can be uploaded to a DSP so that they're not targeted moving forward.

The research went into these two posts took months but I'm hopeful that this post will act as a guide on how upload customer data into Audience Manager. I'll appreciate your feedback and comments.