> ## Documentation Index
> Fetch the complete documentation index at: https://developers.techwolf.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# File Organization

This document outlines the standard file structure used by TechWolf for file
organization. This structure is designed to facilitate automation and ensure
compliance with data handling requirements.

<Note>
  This documentation is provided for reference purposes only. As a customer, you
  will be helped by our team to set up the correct structure for your files.

  We recommend following this structure as it provides several benefits:

  * Organized grouping of integration files
  * Standardized import and export locations
  * Simplified automation and compliance management, improving the onboarding
    and security.
</Note>

# S3 File Structure

## Tenant Structure

The top-level folders in a customer's S3 bucket correspond to different
environment types. These are one of the following:

* `production`
* `staging`
* `testing`

## Folder Organization

Each environment folder follows a standardized structure to maintain clear
separation between input and output files.

The `external` folder contains all files that customers have access to. The rest
of the path will be of the following format:

<Accordion title="S3 File tree structure">
  * `input/`: Path with write access, for input files and integrations

    * `integrations/`: Integration-specific input files

      * `sap_skill_sync/`: SAP skill synchronization data. Only used when
        the SAP skill sync is enabled.
      * `file_based/`: File-based integrations input files.

          <Note>
            The following items are examples, not rules. The main point is
            that files are separated per integration for easier navigation.
          </Note>

        * `sap_datasource/`
        * `LinkedInLearning_learnings/`
        * `jobs/`
        * `workday/`
        * `workday_employee/`
        * `WorkdayEmployee_WorkingHistory/`
        * ... (additional integration-specific folders)

  * `output/`: Files available for viewing and download
    * `exports/`: Standard export files
      * `[export type]/`: Exports grouped by type
</Accordion>

Some examples:

* `testing/external/input/integrations/file_based/sap_datasource/`
* `production/external/output/exports/data-maturity-scan/dms_employees/quality_matrix_report.xlsx`

# SFTP File Structure

## Folder Organization

SFTP servers are not always owned by TechWolf, so it is possible that following
an exact structure is not possible. To help set up a structure, here are our
guidelines which improve onboarding, observability, and security.

|   | What                                                                                                                                                                                                                                    | Why                                                                                                      |        |
| - | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | ------ |
| 1 | Any prefixing directories you need to organize your files on the server. Communicate this to TechWolf, this is where TechWolf will start looking for files.                                                                             | To allow flexible organization and clear communication of file locations.                                | COULD  |
| 2 | Create a clear separation between environments if the same SFTP server is used for multiple environments. Call the environments `production`, `staging`, or `testing`. These should be readable/understandable by all parties involved. | To avoid confusion and ensure all people involved can easily identify which environment files belong to. | MUST   |
| 3 | Create a clear separation between input and output files (`input/` and `output/`).                                                                                                                                                      | To prevent accidental overwrites and to clarify the direction of data flow.                              | MUST   |
| 4 | Add a subdirectory indicating what the files are about; their purpose (`integrations/`, `integrations/file_based/`, `discovery/`, `manual_file_sharing_for_customer_request/`, etc.)                                                    | To make it clear what each set of files is for, improving navigation and management.                     | COULD  |
| 5 | Create a subdirectory for each integration type.                                                                                                                                                                                        | To improve organization, safety, and traceability of files related to different integrations.            | SHOULD |

All of this improves clarity, observability, and security, and is a sustainable
setup that will prevent people from "forgetting" what files are, and where they
are.

<Note>
  Please communicate path decisions, and especially any deviations from this
  structure to TechWolf.
</Note>

<Accordion title="SFTP Example File tree structure">
  * `path/to/your/files`: Any prefixing directories you need to organize your files on the server.
    * `staging/`: Environment-specific files
    * `production/`: Environment-specific files
      * `input/`: For input files and integrations
        * `integrations/`: Integration-specific input files
            <Note>
              The following items are examples, not rules. The main point
              is that files are separated per integration for easier
              navigation.
            </Note>
          * `LinkedInLearning_learnings/`
          * `jobs/`
          * `workday/`
          * `workday_employees/`
          * `WorkdayEmployee_WorkingHistory/`
          * ... (additional integration-specific folders)
      * `output/`: Files available for viewing and download
        * `exports/`: Standard export files
          * `[export type]/`: Exports grouped by type
</Accordion>