Connect HubSpot and AWS S3 (BETA)

Learn how to connect HubSpot and AWS S3 to sync HubSpot data to your S3 bucket. 

Connect HubSpot and AWS S3

To connect HubSpot and AWS S3, you must follow the steps below in order: 

1. Configure access permissions for the AWS S3 bucket
2. Create an IAM policy
3. Create the IAM role in AWS
4. Connect HubSpot and AWS S3
5. Grant the IAM user permissions to access S3 bucket objects

1. Configure access permissions for the S3 bucket

HubSpot needs the following permissions on an AWS S3 bucket and folder to access files in the folder and sub-folders:

• s3:GetBucketLocation
• s3:GetObject
• s3:GetObjectVersion
• s3:ListBucket
• s3:PutObject
• s3:DeleteObject

2. Create an IAM policy

• Log into the AWS Management Console.
• On the home dashboard, click Services > Security, Identity, & Compliance > IAM.
• On the left sidebar, click Account settings.
• In the Security Token Service (STS) section, look for the AWS region that corresponds to the region your AWS account is located in. Click to toggle the switch on to activate the region.
• On the left side panel, click Policies. 
• Click Create policy. 
• On the JSON tab, click Add new statement. 
• Add the policy document below that will give HubSpot the permissions it needs to load or unload data using a single bucket and folder path. You can also purge data files using the PURGE copy option. 

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "s3:PutObject",
 "s3:GetObject",
 "s3:GetObjectVersion",
 "s3:DeleteObject",
 "s3:DeleteObjectVersion"
 ],
 "Resource": "arn:aws:s3:::<bucket>/<prefix>/*"
 },
 {
 "Effect": "Allow",
 "Action": [
 "s3:ListBucket",
 "s3:GetBucketLocation"
 ],
 "Resource": "arn:aws:s3:::<bucket>",
 "Condition": {
 "StringLike": {
 "s3:prefix": [
 "<prefix>/*"
 ]
 }
 }
 }
 ]
}

• Click Next.
• In the Review and create section, add the Policy name and an optional description. 
• Click Create policy. 

3. Create the IAM role in AWS

To configure access permissions for HubSpot in the AWS Management Console:

• Log into the AWS Management Console.
• On the home dashboard, click Services > Security, Identity, & Compliance > IAM.
• On the left sidebar, click Roles. 
• Click Create role. 
• In the Trusted entity type section, select AWS account.
• In the An AWS acount section, select Another AWS account and enter your own AWS account ID. You will later change the trust relationship to give access to HubSpot. 
• Select Require external ID and input a placeholder ID like 0000. You'll later change the trust relationship for your IAM role and specify the external ID for your storage integration. An external ID is used to grant access to your AWS resources to a third-party app like HubSpot. 

• Click Next. 
• Select the policy created in Step 1 and click Next.
• In the Role details section, enter a Role name and Description to identify the role. 
• Click Create role. 
• On the Role summary page, copy the ARN value. You will use this value in the next step. 

4. Connect HubSpot and AWS S3

• Make sure your HubSpot account is opted into this beta: Access HubSpot data in AWS S3. 
• Navigate to this URL and select your HubSpot account.
• In the upper-right, click Install app. 
• Enter the IAM Role Name configured in Step 3 and the S3 URL Path for the bucket you want the data to be dumped into. 
• Click Next.
• Review and copy the AWS IAM User and AWS External ID values as you will need them in the next step. Then, click Continue to Connected Apps. 

5. Grant the IAM user permissions to access bucket objects

• Log into the AWS Management Console.
• On the home dashboard, click Services > Security, Identity, & Compliance > IAM.
• In the left sidebar, click Roles.
• Select the role you created in Step 3. 
• Select the Trust relationships tab and click Edit trust policy. 
• Modify the policy documents with the values you recorded in Step 3. 

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "",
 "Effect": "Allow",
 "Principal": {
 "AWS": "<hubspot_user_arn>"
 },
 "Action": "sts:AssumeRole",
 "Condition": {
 "StringEquals": {
 "sts:ExternalId": "<hubspot_external_id>"
 }
 }
 }
 ]
}

In the above policy, change the following two fields to the values you recorded from Step 4:

• hubspot_user_arn to the STORAGE_AWS_IAM_USER_ARN value
• hubspot_external_id to the STORAGE_AWS_EXTERNAL_ID value 

After completing the above steps, HubSpot can sync data into AWS S3.

Data format for syncing between HubSpot and AWS S3

Available data for syncing

You can sync the following data from HubSpot to AWS S3: 

• Objects: standard and custom objects, such as contacts, companies, tickets, or products.
• Records: individual object records. 
• Associations: all association types and associated records.
• Events: event data for both standard HubSpot events and custom behavioral events. 
• Lists: HubSpot contact and company lists.
• Owners: record owners. 
• Pipelines: deal and ticket pipelines and pipeline stages. 
• Properties: properties and their values on the object records, including the property history. 

Object data of different types are organized and available in two ways:

• Individual: each object type is stored in its own table and has only records of the same object type. For example, objects_contacts contains only contact records.
• Combined: all records for all object types are combined into one database object. Thus, when querying, you need to specify a filter like “WHERE objectTypeId=’0-1’

Custom objects will be assigned an ID at the time of creation, and will be identified as 2-unique_ID. For example, 2-12345. 

objects_x tables

Each available object type, including custom objects, has its own specific table. These tables are named following the format OBJECTS_<object type name> where <object type name> is the plural form label of the objecttype.(Note that if the object type has no plural form, then <object type name> will be the singular form label of the object type name concatenated with “S”).

Below is a complete list of available HubSpot object types, a short description of the data they represent, and their corresponding tables.

object_with_object_properties

This table contains data from your HubSpot records. You can get one row for each record within an objecttype (e.g., all contact records in the contacts objecttype). 

object_properties_history

This table contains historical property values. For contacts, it contains the most recent 45 values. For all other objects, it contains the most recent 20 values. The table has a row for each historical property value. The current value of a property is determined by the most recent UPDATEDAT timestamp in the property's history.

Association queries

Like objects data, associations data is organized in two ways:

• Individual: each association type is stored in its own table and contains only records of that type. For example, associations_contacts_to_deals contains only association records between contact and deal objects. T
• Combined: all records for all association types are combined into one table.

associations_x_to_y 

Each association type has its own view, labeled as associations_<association type name> e.g. associations_contacts_to_deals. 

association_definitions

This table contains information about all of the available association types between HubSpot objects, with one row per association type.

property_definitions

This table contains information about object properties and event properties in your HubSpot account. Event properties include standard events, such as email opens and clicks, and custom behavioral event properties. This information can be helpful for defining values found in object_properties and events data.

Events

Events include standard HubSpot events and custom behavioral events. Each event has its own table, labeled as events_eventname.

object_and_event_type_definitions

This table contains definitions for all objects and events available in your HubSpot account.

Sync frequency and data organization

Data syncs daily between HubSpot and AWS S3. A folder will be created for each day that the data is synced. Each of the following tables will be available as a folder with gzipped CSV files:

ASSOCIATIONS
ASSOCIATION_DEFINITIONS
EVENTS_[Event name]
LISTS
LIST_MEMBERSHIPS
OBJECTS
OBJECT_AND_EVENT_TYPE_DEFINITIONS
OBJECT_PROPERTIES
OBJECT_PROPERTIES_HISTORY
OBJECT_WITH_OBJECT_PROPERTIES
OWNERS
OWNER_TEAM
PIPELINES
PIPELINE_STAGES
PROPERTY_DEFINITIONS
TEAMS

There will be several subfolders for each table. If the size of the table exceeds 15MB, there will be multiple gzipped CSV files under each table folder.