S3 File Organization
This document outlines the standard S3 bucket structure used by TechWolf for file organization. This structure is designed to facilitate automation and ensure compliance with data handling requirements.
This documentation is provided for reference purposes only. As a customer, you will be helped by our team to set up the correct structure for your files in case you are not working through SFTP.
Tenant Structure
The top-level folders in a customer’s S3 bucket correspond to different environment types. These are one of the following:
productionstagingtesting
Folder Organization
Each environment folder follows a standardized structure to maintain clear separation between input and output files.
The external folder contains all files that customers have access to. The rest
of the path will be of the following format:
File tree structure
File tree structure
-
input/: Path with write access, for input files and integrations-
integrations/: Integration-specific input files-
sap_skill_sync/: SAP skill synchronization data. Only used when the skill sync is enabled. -
file_based/: File-based integrations input files.The following items are examples, not rules. The main point is that files are separated per integration for easier navigation.
sap_datasource/LinkedInLearning_learnings/jobs/workday/workday_employee/WorkdayEmployee_WorkingHistory/- … (additional integration-specific folders)
-
-
-
output/: Files available for viewing and downloadexports/: Standard export files[export type]/: Exports grouped by type
Some examples:
testing/external/input/integrations/file_based/sap_datasource/production/external/output/exports/data-maturity-scan/dms_employees/quality_matrix_report.xlsx
We recommend following this structure as it provides several benefits:
- Organized grouping of integration files
- Standardized export locations
- Simplified automation and compliance management, improving the onboarding and security.