Ingestion API
In this tutorial we will explain what is ingest API and How can we create Ingest API in Salesforce to push the data from External system to Salesforce Data Cloud.
- Ingestion API is a REST API and offers two interaction patterns: bulk and streaming.
- The streaming pattern accepts incremental updates to a dataset as those changes are captured.
- while the bulk pattern accepts CSV files in cases where data syncs occur periodically.
- You can push data from an external system into Data Cloud with Ingestion API. The Ingestion API Connector lets you connect with Ingestion API
Types of Ingestion API
Streaming Ingestion
- When updating small micro batches of records in near real time
- When creating change data capture event
Limitations:
- You can process maximum body size of 200 KB per request
- Total no of request 250 per second
Bulk Ingestion
- When moving large volume of data on a daily, weekly or monthly schedule
- When using legacy systems where you can use export data during peak hours
- When using new data cloud org that you want to back fill with 30,60,90 days of data
Limitations:
- You can upload one file at a time per bulk job. A job can have a maximum of 100 files.
- CSV files uploaded via Bulk API have a maximum size of 150 MB.
- 20 Number of request allowed per hour
Setting up the Ingestion API
Prerequisite:
- Must have access to Salesforce Data cloud Org
- Should have all correct permissions to create the Ingest API
Create Ingest API
- Login to Salesforce
- Once you login you can see Data Cloud and Data Cloud Setup under Setup Menu
Click on Data cloud setup and search Ingest API
Setup Ingestion API-> Click New
We need to upload the Schema, whatever the data you want to push, we need to create a schema in OAS Format and Upload
Sample YAML file:
| openapi: 3.0.3
components: schemas: Order: type: object properties: contact_name: type: string id: type: string total: type: number shipAddress: type: string my_email: type: string format: email my_phone: type: string format: phone |
Uploaded the schema
Once schema is created, you can see Ingest API is set up, currently its not in-use, we need to create the Data Stream which we need to create to fetch the data from Source system, Please see this link to learn how to create data stream and ingest data from MuleSoft to Data Cloud using Data Cloud Connector
