Connect HubSpot and AWS S3 (BETA)

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "s3:PutObject",
 "s3:GetObject",
 "s3:GetObjectVersion",
 "s3:DeleteObject",
 "s3:DeleteObjectVersion"
 ],
 "Resource": "arn:aws:s3:::<bucket>/<prefix>/*"
 },
 {
 "Effect": "Allow",
 "Action": [
 "s3:ListBucket",
 "s3:GetBucketLocation"
 ],
 "Resource": "arn:aws:s3:::<bucket>",
 "Condition": {
 "StringLike": {
 "s3:prefix": [
 "<prefix>/*"
 ]
 }
 }
 }
 ]
}

• Click Next.
• In the Review and create section, add the Policy name and an optional description. 
• Click Create policy. 

3. Create the IAM role in AWS

To configure access permissions for HubSpot in the AWS Management Console:

• Log into the AWS Management Console.
• On the home dashboard, click Services > Security, Identity, & Compliance > IAM.
• On the left sidebar, click Roles. 
• Click Create role. 
• In the Trusted entity type section, select AWS account.
• In the An AWS acount section, select Another AWS account and enter your own AWS account ID. You will later change the trust relationship to give access to HubSpot. 
• Select Require external ID and input a placeholder ID like 0000. You'll later change the trust relationship for your IAM role and specify the external ID for your storage integration. An external ID is used to grant access to your AWS resources to a third-party app like HubSpot. 

• Click Next. 
• Select the policy created in Step 1 and click Next.
• In the Role details section, enter a Role name and Description to identify the role. 
• Click Create role. 
• On the Role summary page, copy the ARN value. You will use this value in the next step. 

4. Connect HubSpot and AWS S3

• Make sure your HubSpot account is opted into this beta: Access HubSpot data in AWS S3. 
• Navigate to this URL and select your HubSpot account.
• In the upper-right, click Install app. 
• Enter the IAM Role Name configured in Step 3 and the S3 URL Path for the bucket you want the data to be dumped into. 
• Click Next.
• Review and copy the AWS IAM User and AWS External ID values as you will need them in the next step. Then, click Continue to Connected Apps. 

5. Grant the IAM user permissions to access bucket objects

• Log into the AWS Management Console.
• On the home dashboard, click Services > Security, Identity, & Compliance > IAM.
• In the left sidebar, click Roles.
• Select the role you created in Step 3. 
• Select the Trust relationships tab and click Edit trust policy. 
• Modify the policy documents with the values you recorded in Step 3. 

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "",
 "Effect": "Allow",
 "Principal": {
 "AWS": "<hubspot_user_arn>"
 },
 "Action": "sts:AssumeRole",
 "Condition": {
 "StringEquals": {
 "sts:ExternalId": "<hubspot_external_id>"
 }
 }
 }
 ]
}

In the above policy, change the following two fields to the values you recorded from Step 4:

• hubspot_user_arn to the STORAGE_AWS_IAM_USER_ARN value
• hubspot_external_id to the STORAGE_AWS_EXTERNAL_ID value 

After completing the above steps, HubSpot can sync data into AWS S3.

Data format for syncing between HubSpot and AWS S3

Available data for syncing

You can sync the following data from HubSpot to AWS S3: 

• Objects: standard and custom objects, such as contacts, companies, tickets, or products.
• Records: individual object records. 
• Associations: all association types and associated records.
• Events: event data for both standard HubSpot events and custom behavioral events. 
• Lists: HubSpot contact and company lists.
• Owners: record owners. 
• Pipelines: deal and ticket pipelines and pipeline stages. 
• Properties: properties and their values on the object records, including the property history. 

Object data of different types are organized and available in two ways:

• Individual: each object type is stored in its own table and has only records of the same object type. For example, objects_contacts contains only contact records.
• Combined: all records for all object types are combined into one database object. Thus, when querying, you need to specify a filter like “WHERE objectTypeId=’0-1’

Custom objects will be assigned an ID at the time of creation, and will be identified as 2-unique_ID. For example, 2-12345. 

objects_x tables

Each available object type, including custom objects, has its own specific table. These tables are named following the format OBJECTS_<object type name> where <object type name> is the plural form label of the objecttype.(Note that if the object type has no plural form, then <object type name> will be the singular form label of the object type name concatenated with “S”).

Below is a complete list of available HubSpot object types, a short description of the data they represent, and their corresponding tables.

HubSpot Object type	ObjectTypeId	Description
CONTACT	0-1	Information about individuals interacting with your business
COMPANY	0-2	Information about individual businesses or organizations
DEAL	0-3	Details about revenue opportunities with a contact or company.
ENGAGEMENT	0-4	Stores data from CRM actions, including notes, tasks, emails, meetings, and calls.
TICKET	0-5	Represent customer requests for help or support.
QUOTE	0-14	Used to share pricing information with potential buyers.
FORM_SUBMISSION	0-15	Details for individual submissions for a HubSpot form.
LINE_ITEM	0-8	Represent a subset of products sold in a deal. When a product is attached to a deal, it becomes a line item.
CONVERSATION	0-11	Details of incoming messages from multiple channels.
LANDING_PAGE	0-25	Details of your landing pages
TASK	0-27	Stores information about to-do lists
FORM	0-28	Used to collect lead information about your visitors and contacts
MARKETING_EMAIL	0-29	Details about emails from marketing hub
AD_ACCOUNT	0-30	Stores information about your ad accounts on Linkedin, Facebook and Google
AD_CAMPAIGN	0-31	Details about your ad campaigns. An ad campaign can contain one or more ads
AD_GROUP	0-32	Logical grouping of ads within an ad campaign
AD	0-33	Details about individual ads
CAMPAIGN	0-35	Information about related marketing assets and content, so you can easily measure the effectiveness of your collective marketing efforts.
SITE_PAGE	0-38	Data about individual pages on your websites
BLOG_POST	0-39	Data about blog posts
OBJECT LIST	0-45	Information about groupings of object records based on their properties or activities
CALL	0-48	Calls made by other CRM records, for example contacts
INVOICE	0-53	Manage and sync invoices with external accounting systems
MEDIA_BRIDGE	0-57	Information about media assets imported into HubSpot
SEQUENCE	0-58	A sequence is a series of targeted, timed email templates to nurture contacts over time
DEAL_SPLIT	0-72	Deal splits are used to share deal credit among multiple users
SALES_DOCUMENT	0-83	Sales documents build a library of content for your entire team to upload and share documents with your contacts.
FEEDBACK_SUBMISSION	0-19	Stores information submitted to a feedback survey.
SUBSCRIPTION	0-69	Subscriptions contain details of recurring payments
COMMERCE PAYMENT	0-101	Contains data about funds collected from customers

object_with_object_properties

This table contains data from your HubSpot records. You can get one row for each record within an objecttype (e.g., all contact records in the contacts objecttype). 

object_properties_history

This table contains historical property values. For contacts, it contains the most recent 45 values. For all other objects, it contains the most recent 20 values. The table has a row for each historical property value. The current value of a property is determined by the most recent UPDATEDAT timestamp in the property's history.

Association queries

Like objects data, associations data is organized in two ways:

• Individual: each association type is stored in its own table and contains only records of that type. For example, associations_contacts_to_deals contains only association records between contact and deal objects. T
• Combined: all records for all association types are combined into one table.

associations_x_to_y 

Each association type has its own view, labeled as associations_<association type name> e.g. associations_contacts_to_deals. 

association_definitions

This table contains information about all of the available association types between HubSpot objects, with one row per association type.

property_definitions

This table contains information about object properties and event properties in your HubSpot account. Event properties include standard events, such as email opens and clicks, and custom behavioral event properties. This information can be helpful for defining values found in object_properties and events data.

Events

Events include standard HubSpot events and custom behavioral events. Each event has its own table, labeled as events_eventname.

object_and_event_type_definitions

This table contains definitions for all objects and events available in your HubSpot account.

Sync frequency and data organization

Data syncs daily between HubSpot and AWS S3. A folder will be created for each day that the data is synced. Each of the following tables will be available as a folder with gzipped CSV files:

ASSOCIATIONS
ASSOCIATION_DEFINITIONS
EVENTS_[Event name]
LISTS
LIST_MEMBERSHIPS
OBJECTS
OBJECT_AND_EVENT_TYPE_DEFINITIONS
OBJECT_PROPERTIES
OBJECT_PROPERTIES_HISTORY
OBJECT_WITH_OBJECT_PROPERTIES
OWNERS
OWNER_TEAM
PIPELINES
PIPELINE_STAGES
PROPERTY_DEFINITIONS
TEAMS

There will be several subfolders for each table. If the size of the table exceeds 15MB, there will be multiple gzipped CSV files under each table folder.