Tutorial: File Load Task

Create a File Load Task

We will create a new file load task that will start loading data from a CSV file stored in an S3 bucket into our system. Here, we will follow the example to sync learning Skill Events into our API.

Step 1: Make a CSV file that is conforming to the API specification

Create a CSV file that contains the data you want to sync into the system. The CSV file should have all the columns that are required by the API specification flattened into a single row. For example, the learning Skill Event API requires the following columns (see Skill Event API):

• employee_external_id
• language
• external_id
• timestamp

Additionally it should contain the required columns of the content field: course_id or the two fields learning_title and learning_description

Optionally you can include the following columns:

• source

Side note: you do not need to specify the event_type column, as this will be inferred from the body of the File Load Task request.

Here is an example of a CSV file that contains the required columns for the learning Skill Event API:

employee_external_id,language,external_id,timestamp,course_id,source
employee-1,en,learning-1,2021-01-01T00:00:00Z,course-1,source-1
employee-2,en,learning-2,2021-01-01T00:00:00Z,course-2,source-2

Step 2: Upload the CSV file to an S3 bucket

Upload the CSV file to an S3 bucket. Make sure that you also have the necessary permissions to access the S3 bucket.

Step 3: Create a new File Load Task

Create a new File Load Task by sending a POST request to the /integrations/file-load-tasks endpoint. You need to copy the URI of the CSV file in the S3 bucket and include it in the body of the request. Additionally, you need to specify the entity_type and the method of the File Load Task. Finally, you need to include AWS credentials that have read access to the bucket where you uploaded the file to in the body of the request.

The body of the request should for example look like this:

{
  "entity_type": "learning",
  "method": "s3_iam_user",
  "uri": "s3://bucket-name/file-name.csv",
  "credentials": {
    "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
    "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "aws_session_token": "AQoDYXdzEJrtyd"
  }
}

The response will contain the task_id of the newly created File Load Task.

Step 4: Check the status of the File Load Task

You can check the status of the File Load Task by sending a GET request to the /integrations/file-load-tasks/{file_load_task_id} endpoint with the corresponding task_id of the File Load Task.