Upload Batch Data


Introduction

To simplify data integration, the Cohort Go platform supports accepting report data directly from external systems in either XLSX or CSV format. Import configuration is created through the Cohort Go platform UI, then on-going reports can be uploaded for automated processing.

Creating an Import Template

Within your Cohort Go platform interface, navigate to Settings -> Import Templates. Upload a sample file, and prepare an import template describing how each column in the data is mapped in. Take note of the ID (shown in the URL) of this template.

Uploading a file for processing

To process a file, the file first needs to be delivered to Cohort Go storage, then importing can be requested against this.

To upload a file, an upload endpoint is first requested like:

curl /files/request_upload_endpoint.json`

{
  "url": "https://cohortflow-assets.s3.amazonaws.com/uploads/2cb8e337-eb2a-48aa-a08a-3bced130ff05",
  "upload_params": {
    "AWSAccessKeyId": "123123",
    "Expires": "1234567890",
    "Signature": "098098098",
    "x-amz-acl": "public-read"
  }
}

The upload parameters should be concatenated to the returned URL, and a PUT request executed, like:

curl -XPUT [email protected] \
  'https://cohortflow-assets.s3.amazonaws.com/uploads/2cb8e337-eb2a-48aa-a08a-3bced130ff05?AWSAccessKeyId=123123&Expires=1234567890&Signature=098098098&x-amz-acl=public-read'

Once this has been completed, the url from this endpoint can be used to create an import request, such as:

curl -XPOST --data '{"import_request": {"document_url": "https://cohortflow-assets.s3.amazonaws.com/uploads/2cb8e337-eb2a-48aa-a08a-3bced130ff05", "import_template": {"id": 1234}}}' /import_requests.json`

{
  "id": 55,
  "status": "processing"
}

Monitoring the status of an upload

To monitor the status of this import, the ID returned in the initial creation call can be monitored:

curl /import_requests/55.json`

{
  "id": 55,
  "status": "processed"
}