Sunday, March 11, 2018

Upload Offline Customer Data into Adobe Audience Manager (Part 2)

This post is a continuation to Part 1 of the series where I’m covering how to upload offline customer data to AAM. This post goes even deeper into the weeds and covers how we can add, modify and remove (we cannot really delete data from the backend but can exclude users) customer data from AAM. There are different ways to configure the batch upload file for AAM, but this post only shows how to leverage key-value pairs to bring in data. The official Adobe documentation goes into some more detail on those other methods.

Let’s first take a look at a few steps we need to take as a prerequisite before uploading batch files:
  1. File Extensions: There are two types of file extensions that are supported in the file name. The first one is “.sync” that is used to add records to Audience Manager and the other one is “.overwrite” that is used to overwrite or modify records. These two were covered in a lot of detail in the previous section.
  2. File Name: The file name starts with 'ftp_dpm' (fixed) followed by a random number typically less than 10 digits along with the file extension. So, an example file name can be: ftp_dpm_Data-Source-ID_201831012.sync|.overwrite where the data source ID is the same as '#1' in the screenshot below, number used is in the “Year/Month/Date” format (can be anything) and .sync or .overwrite are the file extensions.
  3. File Size: Each file should ideally not exceed more than 5GB and can be zipped using .gz so if you need to upload greater than 5GB then you can upload multiple files. The nomenclature of these files will be named like ftp_dpm_Data-Source-ID_123456789.sync.1.gz or ftp_dpm_Data-Source-ID_123456789.sync.2.gz.
  4. Attribute Limit: According to Adobe’s documentation referred earlier, there is a limit of 200 lines Adobe can process for each user ID sent in the inbound data file.
  5. Column Structure of the File: As covered in the previous section, the user ID is in the first column. The second column (separated by a tab) contains all key value pairs separated by a comma. Some points to consider:
    • It’s easy to mix up the  double quote with the " quotes being used in the second column. The double quote that will work is the " quote so it’s better to use a separate text editor to configure the file as excel might switch up the quotes being used.
    • Numeric data can be sent as a string in the file but in the AAM UI, they will be evaluated as numbers so logical operators such as "<" OR ">" can be used while creating traits.
    • Make sure that the tab between column one and column two is actually a tab and not a space as that will break the file format.
  6. Data Source Setup: As discussed in the previous post, one of the first things we need to capture to initiate a batch upload process is to capture the authenticated user ID on the website. We tie this ID to Audience Manager via an integration code (#2 below) by initiating an ID sync. The user ID needs to be mapped to a data source by tying the user ID to the Customer ID inbound field (#3). The data source ID (#1) is hard coded into the batch file name which we’ll cover in the third step.
Source: Audience Manager UI

We're now going to cover how to add, modify and remove records from Audience Manager while leveraging key-value pairs:
  • Add New Records to Audience Manager (".sync" format): This is the most common type of upload where we simply add new user IDs along to the first column and user attributes to the second column as shown below. The file format used here is “.sync”.
  • Modify Already Uploaded Records (".overwrite" format): In this type of upload using the “.overwrite” format, we add all existing attributes of the user along with the modified records. In the example below, we’ve uploaded all existing records and only modified the ‘age’ and ‘purchases_last_month’ fields. If we don’t add certain attributes added previously, those attributes will not be part of user qualification moving forward.
  • Add New Attributes to Existing Records: If you want to add a new attribute to an existing record (user ID), there are two ways to do this:
    • (".overwrite" format): In this format, we include all existing records for a user along with the new attribute (company_size) as shown below:
    • (".sync" format): In this format, we include just the new attributes (company_size) we need to add  for a user as shown below:
  • Remove Attributes from Existing Records (".overwrite" format)If you want to remove a certain attribute from exiting records, upload an ".overwrite" file with just the attributes you wish to keep and exclude the one you wish to remove. In the example below, we've removed the 'company_size' attribute that we added earlier. Also, note that we included the updated 'age' and 'purchases_last_month' fields:
  • Delete Records from Audience Manager: There is no way to really "delete" a record from Audience Manager. The recommended approach is to include an attribute/flag (E.g. existing_customer=N) in the batch file and add it as a condition in AAM. Users with this flag can be part of an exclusion list that can be uploaded to a DSP so that they're not targeted moving forward.
The research went into these two posts took months but I'm hopeful that this post will act as a guide on how upload customer data into Audience Manager. I'll appreciate your feedback and comments.

Saturday, February 10, 2018

Upload Offline Customer Data into Adobe Audience Manager (Part 1)

Almost all customers I’ve worked with have a CRM (Customer Relationship Management) system where they store information about their current and prospective customers. A very common requirement for customers to make their CRM system actionable is to connect their offline CRM data with online data. 

Previously, I wrote about marketing use cases that are met using a Data Management Platform. In this article, I'm going to go a little deeper into Adobe Audience Manager (AAM) and cover how to import customer data from their backend system into the DMP. In the first of this two-part series, I'll go through the various steps needed to perform a successful CRM data upload and the second post will go into the granular details of how CRM attributes in these files are created, modified and deleted.

There are a lot of use cases (E.g. uploading Point of sale, customer lifetime value, last purchase date etc.) that can be accomplished using CRM data so it becomes necessary to understand WHY we’re even bringing this data into AAM. To get started, the first and foremost thing that we need to be aware of before setting up a CRM upload is that no PII (personally identifiable information) should be passed over to AAM. PII is data that can be tied directly to the user such as SSN email address, name, date of birth etc. and it’s imperative that no such data is brought in that can expose personal information about customers. 

We will now cover the various steps that are part of the offline file upload process. These may look like a lot of steps to just upload a flat file, but I believe it's important to follow these steps to make the most from this effort.

1.     Discuss Business ObjectivesBefore undertaking a CRM upload, it’s necessary to ask what business objectives are we trying to achieve. In my experience, I’ve seen clients request everything they can but I believe the right approach is to start simple to achieve small wins and then focus on additional components. Once clients have a few CRM segments they’ve acted on, they can start bringing in more data.

2.     Create and configure the file: The upload file format accepted by AAM is pretty simple as it only contains two columns. The first column is the hashed/anonymized user ID and the second column is a key value pair combination of CRM attributes separated by a comma (example below). In the second part, I’ll cover all aspects of how these files are created and modified.

3.     Create On boarded traits in AAM: Once we know the structure of the file, the next step is to create on boarded traits in Audience Manager. Traits are built off a combination of key value pairs and are the most granular data point in AAM. An example of a trait is shown below that is built off a condition where the gender is female. The trait logic can be expanded to include other conditions such as age, state etc. depending on what we’re looking to capture. It’s important to note that traits need to be created before we upload the offline file.

Source: Audience Manager Trait UI

4.     Perform an ID Sync: This is a critical piece of the CRM file upload process where we sync the CRM user ID with the Audience Manager cookie typically upon authentication. The CRM ID acts as the primary key that is the common link between the backend and the Web or App.
If we upload the CRM file without this sync, AAM will not be able to tie the backend system with its cookie thereby not populating anything in the UI. Aside from this, the other important point to keep in mind is that the user ID or email address captured upon authentication needs to be anonymized/hashed before it’s sent to AAM so that it cannot be traced back to the user. I will write more about the general ID sync process in a future post.

5.     Upload the file: Once the ID sync is setup, we now need to send the finalized CRM file to Audience Manager. AAM accepts offline files in either the S3 bucket or in a SFTP location. The most common approach is to upload a daily CRM file with new customer data every day via an automated file drop process. S3 is Adobe’s preferred data storage platform due to its scalability, security and speed just to name a few advantages. Some other advantages of S3 are documented here.

6.     Monitor the Onboarding Status report in AAM: The Onboarding Status report in AAM provides a complete summary of records/rows that were either successful or unsuccessful during the upload process. Adobe picks and processes these files once a day and expects the file contents and name in a certain format. The file contents and name need to match the expected format exactly as AAM requires otherwise it'll error out which is the most common reason for errors. Below is an example of a successfully uploaded file from this report as shown here.

                                                                                     Source: Audience Manager Onboarding Status report

7.     Tweak if necessary: It's not uncommon to see issues early on with the file format to get it right so it'll makes sense to set aside some time and be prepared to make changes. Once the changes and tweaks to the file is made, we can upload it again to the drop folder.

8.     Check On-Boarded traits for data: Once the file has been successfully uploaded and processed which typically takes between 24-48 hours, we need to monitor if data has started flowing into the various on boarded traits that we created (Step 3). If the trait specific metrics have data, we're good to proceed to the next step.

9.     Create Segments: Once these CRM data based traits have started populating with data, we need to filter the uploaded data tied to the business objectives that were defined upfront. This is done by building segments in AAM based off CRM data or other data sources. An example might be creating a segment comprising of users who haven't bought for the last six months (CRM) to provide them with a 15% off coupon when they visit the website (online).

10.  Activate: The final step of the process is to start sharing these segments with personalization and advertising platforms to deliver customized messages to our customers. A common example is to send these segments from the DMP to a personalization platform such as Adobe Target to deliver contextual and personalized content to customer. 

Now that we’ve covered all the steps required in this process, it's on us to determine if we can start working on these but it all has to start with documenting our business objectives. Once we've successfully setup the upload process, we need to build/learn from where we started to deliver a better experience to customers and monetize from our upfront effort. Do you use your CRM data to deliver tailored messages to customers coming to your site or App?

Sunday, January 21, 2018

Data Management Platform Use Cases

Marketers hope to get the best return for each campaign they run but is getting that return really that simple? Unfortunately, it’s not as there are a lot of factors that come into play for planning and executing these campaigns such as identifying/finding the target audience, communication channel, budget, technology platform and expected outcome to name a few. Luckily, there are tools out there to support marketers activate towards an outcome, measure performance and optimize continuously. One such tool that helps them is a Data Management Platform.
In my last post, I gave an overview of a Data Management Platform. In this article, I’ll go through some typical marketing scenarios that are realized using a DMP.
  1. Frequency Capping: Marketers use this feature to set a cap on how many impressions should users be exposed to for a particular campaign. The idea behind this is to not show the same campaign to users if they’ve seen it multiple times. As an example, the screenshot below taken from Adobe's DMP that shows a report where conversions start to drop once users see the same ad more than 2 times. The advantage of this feature is to save marketing dollars and not pay extra for potentially wasted impressions as users tend to convert in the first 2 interactions as per this report.
  2. Prospecting or Lookalike Audiences: Prospecting allows marketers to reach out to new audiences who wouldn’t have visited their site. This is where a DMP's connection to the 3rd party marketplace allows marketers to evaluate new audiences and overlap them with their 1st party data. This feature is called lookalike modeling where marketers pick a baseline 1st party attribute (such as users who bought something) and run that against 3 party audiences to see which new 3rd party audiences exhibit the same behavior. Once these audiences are identified, marketers can purchase these audiences at a premium and factor this cost as part of their marketing budget and look to further increase their overall CTR and conversion rate.
  3. Media Suppression: Media suppression or exclusion in the marketing world allow marketers to not display an ad to certain group of people who need to be excluded. Simple examples of exclusions can be at a geographic level or channel level that can be executed directly in the Demand Side Platform (DSP). A DMP however, is able to fulfill other use cases where a marketer would want to exclude users who already purchased something on their site or might have submitted a lead. In this case, marketers can build an exclusion audience in the DMP comprising of 1st party onsite behavioral data and share it out with a DSP for execution.
  4. Content Personalization: Personalization in simplest terms means serving up content tailor-made for customers based on their browsing patterns. A few examples of personalization, which are accomplished using a DMP are as follows:
    • Combine 1st party CRM data to gather demographic and past purchase information about a user and serve them discount offers to buy something again.
    • Personalize content based on product pages that users have visited in the past and deliver similar content to encourage them to go deeper into the purchase funnel. An integration between site analytics and an A/B Testing tool is required but a DMP can be used to find additional prospects that can be combined with 1st party site data.
    • Other simpler examples are serving personalized content based on city, weather, time of day so the opportunities are endless depending on your site.
  5. Visitor Retargeting: Retargeting (if done right) is a form of advertising in which you  follow visitors across the web who've visited your site but haven't converted. A retail site (shall remain unnamed) continued to retarget me even though I had bought from them. This tells me that the retailer doesn't share 1st party purchase data with their retargeting platform and should probably invest in a DMP. In terms of the technology, a client website has pixels that set a cookie on the user's browser -> that is shared with a retargeting partner who -> is able to show them ads across the web -> to bring them back to the original website. There are many retargeting use cases that marketers can execute on but I'm going to cover the most common ones which can be executed using a DMP:
    • Retarget users who added an item to their cart but didn't purchase. In this, first party data around cart add is shared with the retargeting platform via a DMP.
    • Retarget users who opened an email but did not convert. In this, data from the email captured in the DMP integrated with other data sources is sent over to the retargeting partner.
  6. Test multiple creatives and DSPs: DMPs also have the capability to test out multiple creatives or DSPs. The way it works (in Adobe Audience Manager as an example) is that a single segment is broken into mutually exclusive groups and then split into multiple subsets depending on the number of creatives or DSPs a marketer wants to test. The result of this is that the DMP will tell you which creative or DSP performed the best localized to its baseline segment picked as part of the test. As a next step, marketers can swap out underperforming ads and DSPs, pick the one that works and then repeat this process moving forward.
  7. Cross device targeting: It's common knowledge that consumers browse the web using multiple devices but how do we tie them together? DMPs can connect users browsing the web across multiple devices by using their authenticated profile. Once a user has visited a web property in an authenticated manner, the DMP is still able to identify that person even if the user visits the property as unauthenticated. A common scenario where this feature is leveraged is where a user may not have purchased something on desktop and sees an ad for the same product on their mobile app. The DMP in this case, collects 1st party data both from the desktop site and mobile app and shares that with the DSP for targeting. Here's a visual from Adobe's cross device tool called Profile Merge Rules.

We covered many aspects of digital marketing as individual use cases but in reality, they need to execute in tandem to get the most value from a DMP. Retargeting users and personalizing the website to show similar ad copy content is a great example of these done in tandem. Successful execution of these use cases will allow companies to reap the benefits of what a DMP has to offer as the possibilities are endless. Are you leveraging your DMP to act on these use cases?