CDF Project Archiver

This notebook demonstrates how to use ProjectArchiver to streamline data archival for project teardown.

import os
from cognite.client import CogniteClient
from cognite.utils.infrastructure import ProjectArchiver

# Establish client connection to CDF
client = CogniteClient(
    client_name=os.environ.get("CLIENT_NAME"),
    project=os.environ.get("PROJECT"),
    api_key=os.environ.get("API_KEY"),
)

# Create archiver instance
archiver = ProjectArchiver(client)

ProjectArchiver object has methods which usually take two optional arguments:

• dirpath: Directory path to save the data. Defaults to "." (i.e. current directory).

• compress: Whether to compress the saved data. Defaults to True.

# Archive serializable data (i.e. modeled data)
archiver.archive_assets()
archiver.archive_datasets()
archiver.archive_events()
archiver.archive_file_metadata()
archiver.archive_labels()
archiver.archive_relationships()
archiver.archive_sequences()
archiver.archive_timeseries()

Archiving <assets>...
9647 records of type <assets> have been archived

Archiving <data_sets>...
57 records of type <data_sets> have been archived

Archiving <events>...
20407 records of type <events> have been archived

Archiving <files>...
1673 records of type <files> have been archived

Archiving <labels>...
47 records of type <labels> have been archived

Archiving <relationships>...
1 records of type <relationships> have been archived

Archiving <sequences>...
No data exists for type <sequences>

Archiving <time_series>...
11120 records of type <time_series> have been archived

# Archive file data
archiver.archive_files()

Downloading 1673 files...
100 files downloaded
200 files downloaded
300 files downloaded
400 files downloaded
500 files downloaded
600 files downloaded
700 files downloaded
800 files downloaded
900 files downloaded
1000 files downloaded
1100 files downloaded
1200 files downloaded
1300 files downloaded
1400 files downloaded
1500 files downloaded
1600 files downloaded
1673 files downloaded

Compressing files...
Files compressed

# Check results
os.listdir(".")

['some-project_relationships.json.gzip',
 'some-project_events.json.gzip',
 'some-project_datasets.json.gzip',
 'some-project_file_metadata.json.gzip',
 'some-project_labels.json.gzip',
 'some-project_file_downloads.zip',
 'some-project_assets.json.gzip',
 'some-project_timeseries.json.gzip']