Skip to main content

Onboard your data on EDITO

Discover the requirements and current process for onboarding datasets intended to be made available through the EDITO platform.

Context


In order to contribute to EDITO, a number of requirements need to be respected when onboarding datasets, applications, computational services, and model components.

These requirements help ensure consistency, interoperability, discoverability, and operational reliability within EDITO's cloud-native environment.

📌 Note: EDITO's data onboarding capabilities are evolving. In particular, the future automated ingestion workflow, including cloud-optimised data transformation and STAC metadata generation, is not yet available.

Technical requirements for Datasets


Self-Describing Data Principle

All datasets onboarded to EDITO must follow a self-describing data approach.

This means that datasets must either contain embedded metadata sufficient to interpret the data without external documentation, or they must be accompanied by comprehensive external metadata files.

​The objective of this requirement is to prevent ambiguities and to ensure that datasets remain interpretable over time, even when accessed independently from their original context.

Self-describing formats significantly reduce the need for custom parsing tools and promote interoperability within cloud-based processing environments.

Spatial and Temporal Referencing

​As marine and environmental datasets are inherently spatial and temporal, adequate geospatial and time descriptors are mandatory.

Datasets must include explicit coordinate information, preferably expressed in the WGS 84 coordinate reference system (EPSG:4326), unless a justified alternative projection is required. In all cases, the coordinate reference system must be clearly documented.

​Temporal information must follow the ISO 8601 standard for date and time formatting.

Where relevant, depth or elevation information must also be included and clearly specified.

Spatial bounding boxes and temporal extents must be provided within the dataset metadata to support indexing and discovery.

Controlled vocabularies and semantic interoperability

​To ensure semantic consistency across datasets, parameter names and variable descriptions should rely on recognized controlled vocabularies.

Recommended vocabularies include those maintained by the CF Standard Name Table, Darwin Core, and BODC Parameter Usage Vocabulary.

​The use of controlled vocabulary enhances machine readability, improves cross-dataset integration, and supports harmonization across marine data infrastructures.

Accepted Data Formats


Standard Formats

​EDITO supports widely adopted, open, and interoperable data formats.

For gridded environmental data, NetCDF files conforming to CF conventions (version 1.6 or later) are the recommended delivery format.

NetCDF is particularly suitable for multidimensional scientific datasets because it can embed information describing variables, dimensions, coordinates, units, and other metadata required to interpret the data.

Tabular datasets may be submitted in CSV format, provided that spatial and temporal fields are included and properly structured. Since CSV does not inherently support extensive metadata, a separate metadata document must accompany the dataset.

Vector geospatial data may be provided in formats such as Shapefile, GeoJSON, GeoParquet or GeoPackage, provided that coordinate reference system information is included and documented.

📌 Note: supported formats and onboarding procedures may evolve as EDITO's publishing and ingestion capabilities develop.

Analysis Ready Cloud Optimized Formats (ARCO)

For large-scale datasets or datasets intended for high-performance cloud processing, Analysis Ready Cloud Optimized (ARCO) formats enable scalable and efficient access in distributed computing environments.

Supported ARCO formats include Cloud Optimized GeoTIFF (COG), GeoParquet, and Zarr.

ARCO formats may be used as target formats for cloud-optimized access on EDITO. They are not necessarily the format in which data providers are required to deliver their datasets. For example, gridded model outputs may be delivered as CF-compliant NetCDF files and subsequently converted to ARCO/Zarr as part of the EDITO ingestion process.

📌 Note (COMING SOON): EDITO plans to provide an ingestion workflow to support the conversion of suitable NetCDF datasets into ARCO/Zarr representations and the generation of standardized metadata (STAC).

Metadata catalogue registration


All datasets intended for discovery or public access must be ultimately be referenced in the EDITO SpatioTemporal Asset Catalogue (STAC).

To enable catalogue registration, data providers must ensure that their datasets are self-describing and include sufficiently complete, standardised metadata, as described in the requirements above. These metadata must provide the information needed to interpret, process, discover and reference the dataset, including its spatial and temporal coverage.

📌 Note: in the future EDITO ingestion workflow, STAC metadata generation and catalogue registration are expected to be handled automatically, based on the metadata provided.

Until this automated workflow is fully available, data providers should follow the current EDITO documentation and onboarding procedure for catalogue registration. Depending on the applicable procedure, providers may currently be required to prepare or complete STAC metadata themselves.

💡 EDITO Pro Tip: ​see the STAC specifications​ ​to ensure that your metadata will fit into the EDITO STAC.

Storage and directory structure


​Upon approval, data providers are assigned a designated S3 object storage bucket within the EDITO Data Lake. Providers must follow the recommended directory structure to ensure logical organization, traceability, and version control.

​Any deviation from the recommended structure must be clearly described in the metadata. Published datasets should be managed carefully to avoid untracked changes. Where dataset versions are maintained by the data provider, version information should be clearly reflected in the associated metadata.

📌 Note: integrated dataset versioning is not currently supported by EDITO.

​The default allocation of 20 GB of S3 object storage per EDITO user may be increased upon request through the EDITO User Support.

Responsibilities of data providers


Data providers remain responsible for the quality, integrity, and correctness of their datasets. They are expected to maintain accurate documentation, respond to user queries, and notify EDITO of significant structural or licensing changes.

⚠️ ​Failure to maintain compliance with technical or legal requirements may result in temporary suspension or removal of datasets from the platform.

Questions & Answers


Is it possible to publish datasets on EDITO with a DOI?

Currently there is not an integrated way to publish datasets on EDITO with a DOI i.e. you can eventually add a dataset with an existing DOI and add this information within the attached metadata, but this is not a straightforward task.

What about DOI and versioning?

EDITO does not currently provide an integrated mechanism for assigning DOIs to datasets or for managing dataset versioning.

Datasets that already have a DOI may still be onboarded on EDITO, and the existing DOI can be included in the associated metadata. However, DOI handling is not yet integrated into the EDITO publishing workflow.

Similarly, dataset versioning is not currently supported as an integrated EDITO capability.

Both DOI support and dataset versioning are planned for future integration as part of the evolution of the EDITO platform.

Do I need to convert my NetCDF files to ARCO/Zarr before onboarding them on EDITO?

No, unless explicitly requested.

When NetCDF is specified as the required project output, data providers are expected to produce and deliver CF-compliant NetCDF files.

In the future EDITO ingestion process, EDITO is expected to handle not only the conversion of NetCDF files to an ARCO/Zarr format, but also the generation of standardized metadata (STAC) required to reference and expose the dataset in the EDITO Catalogue.

Until this ingestion process is fully available, data providers should follow the existing EDITO documentation for onboarding their datasets into the catalogue.

The documentation will be updated as the new ingestion workflow, including ARCO/Zarr conversion and STAC metadata generation, becomes available.

What's next?


In the coming months, a new process will be implemented for integrating data into EDITO’s public catalogue, ensuring traceability and quality assurance from submission to publication.

Contributions may be integrated for restricted use or public dissemination through the EDITO catalogue, with scientific validation processes in place to safeguard data quality and reliability.

If you have any questions, problems, or suggestions, please feel free to contact us via chat using the widget available at the bottom right of the page.

Did this answer your question?