This document outlines the standard file structure used by TechWolf for file organization. This structure is designed to facilitate automation and ensure compliance with data handling requirements.
This documentation is provided for reference purposes only. As a customer, you will be helped by our team to set up the correct structure for your files.We recommend following this structure as it provides several benefits:
  • Organized grouping of integration files
  • Standardized import and export locations
  • Simplified automation and compliance management, improving the onboarding and security.

​
S3 File Structure

​
Tenant Structure

The top-level folders in a customer’s S3 bucket correspond to different environment types. These are one of the following:
  • production
  • staging
  • testing

​
Folder Organization

Each environment folder follows a standardized structure to maintain clear separation between input and output files. The external folder contains all files that customers have access to. The rest of the path will be of the following format:
  • input/: Path with write access, for input files and integrations
    • integrations/: Integration-specific input files
      • sap_skill_sync/: SAP skill synchronization data. Only used when the SAP skill sync is enabled.
      • file_based/: File-based integrations input files.
        The following items are examples, not rules. The main point is that files are separated per integration for easier navigation.
        • sap_datasource/
        • LinkedInLearning_learnings/
        • jobs/
        • workday/
        • workday_employee/
        • WorkdayEmployee_WorkingHistory/
        • … (additional integration-specific folders)
  • output/: Files available for viewing and download
    • exports/: Standard export files
      • [export type]/: Exports grouped by type
Some examples:
  • testing/external/input/integrations/file_based/sap_datasource/
  • production/external/output/exports/data-maturity-scan/dms_employees/quality_matrix_report.xlsx

​
SFTP File Structure

​
Folder Organization

SFTP servers are not always owned by TechWolf, so it is possible that following an exact structure is not possible. To help set up a structure, here are our guidelines which improve onboarding, observability, and security.
WhatWhy
1Any prefixing directories you need to organize your files on the server. Communicate this to TechWolf, this is where TechWolf will start looking for files.To allow flexible organization and clear communication of file locations.COULD
2Create a clear separation between environments if the same SFTP server is used for multiple environments. Call the environments production, staging, or testing. These should be readable/understandable by all parties involved.To avoid confusion and ensure all people involved can easily identify which environment files belong to.MUST
3Create a clear separation between input and output files (input/ and output/).To prevent accidental overwrites and to clarify the direction of data flow.MUST
4Add a subdirectory indicating what the files are about; their purpose (integrations/, integrations/file_based/, discovery/, manual_file_sharing_for_customer_request/, etc.)To make it clear what each set of files is for, improving navigation and management.COULD
5Create a subdirectory for each integration type.To improve organization, safety, and traceability of files related to different integrations.SHOULD
All of this improves clarity, observability, and security, and is a sustainable setup that will prevent people from “forgetting” what files are, and where they are.
Please communicate path decisions, and especially any deviations from this structure to TechWolf.
  • path/to/your/files: Any prefixing directories you need to organize your files on the server.
    • staging/: Environment-specific files
    • production/: Environment-specific files
      • input/: For input files and integrations
        • integrations/: Integration-specific input files
          The following items are examples, not rules. The main point is that files are separated per integration for easier navigation.
          • LinkedInLearning_learnings/
          • jobs/
          • workday/
          • workday_employees/
          • WorkdayEmployee_WorkingHistory/
          • … (additional integration-specific folders)
      • output/: Files available for viewing and download
        • exports/: Standard export files
          • [export type]/: Exports grouped by type