Skip to main content
This document outlines the standard file structure used by TechWolf for file organization. This structure is designed to facilitate automation and ensure compliance with data handling requirements.
This documentation is provided for reference purposes only. As a customer, you will be helped by our team to set up the correct structure for your files.We recommend following this structure as it provides several benefits:
  • Organized grouping of integration files
  • Standardized import and export locations
  • Simplified automation and compliance management, improving the onboarding and security.

​
S3 File Structure

​
Tenant Structure

The top-level folders in a customer’s S3 bucket correspond to different environment types. These are one of the following:
  • production
  • staging
  • testing

​
Folder Organization

Each environment folder follows a standardized structure to maintain clear separation between input and output files. The external folder contains all files that customers have access to. The rest of the path will be of the following format:
  • input/: Path with write access, for input files and integrations
    • integrations/: Integration-specific input files
      • sap_skill_sync/: SAP skill synchronization data. Only used when the SAP skill sync is enabled.
      • file_based/: File-based integrations input files.
        The following items are examples, not rules. The main point is that files are separated per integration for easier navigation.
        • sap_datasource/
        • LinkedInLearning_learnings/
        • jobs/
        • workday/
        • workday_employee/
        • WorkdayEmployee_WorkingHistory/
        • … (additional integration-specific folders)
  • output/: Files available for viewing and download
    • exports/: Standard export files
      • [export type]/: Exports grouped by type
Some examples:
  • testing/external/input/integrations/file_based/sap_datasource/
  • production/external/output/exports/data-maturity-scan/dms_employees/quality_matrix_report.xlsx

​
SFTP File Structure

​
Folder Organization

SFTP servers are not always owned by TechWolf, so it is possible that following an exact structure is not possible. To help set up a structure, here are our guidelines which improve onboarding, observability, and security. All of this improves clarity, observability, and security, and is a sustainable setup that will prevent people from “forgetting” what files are, and where they are.
Please communicate path decisions, and especially any deviations from this structure to TechWolf.
  • path/to/your/files: Any prefixing directories you need to organize your files on the server.
    • staging/: Environment-specific files
    • production/: Environment-specific files
      • input/: For input files and integrations
        • integrations/: Integration-specific input files
          The following items are examples, not rules. The main point is that files are separated per integration for easier navigation.
          • LinkedInLearning_learnings/
          • jobs/
          • workday/
          • workday_employees/
          • WorkdayEmployee_WorkingHistory/
          • … (additional integration-specific folders)
      • output/: Files available for viewing and download
        • exports/: Standard export files
          • [export type]/: Exports grouped by type