Class CogniteDestinationWithIDM

Class with utility methods supporting extraction of data into CDF. These methods complement the ones offered by the CogniteSdk.Client and use a CogniteConfig object to determine chunking of data and throttling of requests against the client

Inheritance

object

CogniteDestination

CogniteDestinationWithIDM

Implements

IRawDestination

Inherited Members

CogniteDestination.Throttling

CogniteDestination.Chunking

CogniteDestination.NanReplacement

CogniteDestination.CogniteClient

CogniteDestination.TestCogniteConfig(CancellationToken)

CogniteDestination.GetOrCreateTimeSeriesAsync(IEnumerable<string>, Func<IEnumerable<string>, IEnumerable<TimeSeriesCreate>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.GetOrCreateTimeSeriesAsync(IEnumerable<string>, Func<IEnumerable<string>, Task<IEnumerable<TimeSeriesCreate>>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.EnsureTimeSeriesExistsAsync(IEnumerable<TimeSeriesCreate>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.UpdateTimeSeriesAsync(IEnumerable<TimeSeriesUpdateItem>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.UpsertTimeSeriesAsync(IEnumerable<TimeSeriesCreate>, RetryMode, SanitationMode, UpsertParams, CancellationToken)

CogniteDestination.GetOrCreateAssetsAsync(IEnumerable<string>, Func<IEnumerable<string>, IEnumerable<AssetCreate>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.GetOrCreateAssetsAsync(IEnumerable<string>, Func<IEnumerable<string>, Task<IEnumerable<AssetCreate>>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.EnsureAssetsExistsAsync(IEnumerable<AssetCreate>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.UpdateAssetsAsync(IEnumerable<AssetUpdateItem>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.UpsertAssetsAsync(IEnumerable<AssetCreate>, RetryMode, SanitationMode, UpsertParams, CancellationToken)

CogniteDestination.InsertDataPointsAsync(IDictionary<Identity, IEnumerable<Datapoint>>, SanitationMode, RetryMode, CancellationToken)

CogniteDestination.InsertDataPointsCreateMissingAsync(IDictionary<Identity, IEnumerable<Datapoint>>, SanitationMode, RetryMode, long?, CancellationToken)

CogniteDestination.DeleteDataPointsIgnoreErrorsAsync(IDictionary<Identity, IEnumerable<TimeRange>>, CancellationToken)

CogniteDestination.InsertRawRowsAsync<T>(string, string, IDictionary<string, T>, JsonSerializerOptions, CancellationToken)

CogniteDestination.InsertRawRowsAsync<T>(string, string, IDictionary<string, T>, CancellationToken)

CogniteDestination.CreateRawUploadQueue<T>(string, string, TimeSpan, int, Func<QueueUploadResult<(string key, T columns)>, Task>)

CogniteDestination.CreateTimeSeriesUploadQueue(TimeSpan, int, Func<QueueUploadResult<(Identity id, Datapoint dp)>, Task>, string)

CogniteDestination.CreateEventUploadQueue(TimeSpan, int, Func<QueueUploadResult<EventCreate>, Task>, string)

CogniteDestination.GetRowsAsync(string, string, JsonSerializerOptions, CancellationToken)

CogniteDestination.GetRowsAsync<T>(string, string, JsonSerializerOptions, CancellationToken)

CogniteDestination.DeleteRowsAsync(string, string, IEnumerable<string>, CancellationToken)

CogniteDestination.GetExtractedRanges(IEnumerable<Identity>, CancellationToken, bool, bool, bool)

CogniteDestination.GetExtractedRanges(IEnumerable<(Identity id, TimeRange limit)>, CancellationToken, bool, bool, bool)

CogniteDestination.GetOrCreateEventsAsync(IEnumerable<string>, Func<IEnumerable<string>, IEnumerable<EventCreate>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.GetOrCreateEventsAsync(IEnumerable<string>, Func<IEnumerable<string>, Task<IEnumerable<EventCreate>>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.EnsureEventsExistsAsync(IEnumerable<EventCreate>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.GetOrCreateSequencesAsync(IEnumerable<string>, Func<IEnumerable<string>, IEnumerable<SequenceCreate>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.GetOrCreateSequencesAsync(IEnumerable<string>, Func<IEnumerable<string>, Task<IEnumerable<SequenceCreate>>>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.EnsureSequencesExistsAsync(IEnumerable<SequenceCreate>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.InsertSequenceRowsAsync(IEnumerable<SequenceDataCreate>, RetryMode, SanitationMode, CancellationToken)

CogniteDestination.GetOrCreateStreamAsync(StreamWrite, CancellationToken)

CogniteDestination.InsertRecordsAsync(string, ICollection<StreamRecordWrite>, CancellationToken)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Cognite.Extractor.Utils

Assembly: ExtractorUtils.dll

Syntax

public class CogniteDestinationWithIDM : CogniteDestination, IRawDestination

Constructors

CogniteDestinationWithIDM(Client, ILogger<CogniteDestination>, CogniteConfig, ViewIdentifier?)

Initializes the Cognite destination with the provided parameters

Declaration

public CogniteDestinationWithIDM(Client client, ILogger<CogniteDestination> logger, CogniteConfig config, ViewIdentifier? viewIdentifier = null)

Parameters

Type	Name	Description
Client	client	CogniteSdk.Client object
ILogger<CogniteDestination>	logger	Logger
CogniteConfig	config	Configuration object
ViewIdentifier	viewIdentifier	Optional view identifier

CogniteDestinationWithIDM(Client, ILogger<CogniteDestination>, BaseCogniteConfig, string, ViewIdentifier?)

Initializes the Cognite destination with the provided parameters

Declaration

public CogniteDestinationWithIDM(Client client, ILogger<CogniteDestination> logger, BaseCogniteConfig config, string project, ViewIdentifier? viewIdentifier = null)

Parameters

Type	Name	Description
Client	client	CogniteSdk.Client object
ILogger<CogniteDestination>	logger	Logger
BaseCogniteConfig	config	Configuration object
string	project	Configured project
ViewIdentifier	viewIdentifier	Optional view identifier

Properties

IDMViewIdentifier

View identifier for IDM TimeSeries

Declaration

public static ViewIdentifier IDMViewIdentifier { get; protected set; }

Property Value

Type	Description
ViewIdentifier

Methods

EnsureStateSetsExistAsync<T>(IEnumerable<SourcedNodeWrite<T>>, RetryMode, SanitationMode, CancellationToken)

Ensures that all state sets in stateSets exist in CDF. Tries to create the state sets and returns when all are created or have been removed due to issues with the request. State sets are only available through the beta CDM state set API.

Declaration

public Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>> EnsureStateSetsExistAsync<T>(IEnumerable<SourcedNodeWrite<T>> stateSets, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token) where T : CogniteStateSet

Parameters

Type	Name	Description
IEnumerable<SourcedNodeWrite<T>>	stateSets	List of CogniteSdk StateSet objects
RetryMode	retryMode	How to do retries. Keeping duplicates is not valid for this method.
SanitationMode	sanitationMode	The type of sanitation to apply to state sets before creating
CancellationToken	token	Cancellation token

Returns

Type	Description
Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the created state sets

Type Parameters

Name	Description
T

EnsureTimeSeriesExistsAsync<T>(IEnumerable<SourcedNodeWrite<T>>, RetryMode, SanitationMode, CancellationToken, bool)

Ensures that all time series in timeSeries exist in CDF. Tries to create the time series and returns when all are created or have been removed due to issues with the request. By default, if any items fail to be created due to missing asset, duplicated externalId, duplicated legacy name, or missing dataSetId, they can be removed before retrying by setting retryMode Timeseries will be returned in the same order as given in timeSeries

Declaration

public Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>> EnsureTimeSeriesExistsAsync<T>(IEnumerable<SourcedNodeWrite<T>> timeSeries, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token, bool isBeta = false) where T : CogniteTimeSeriesBase

Parameters

Type	Name	Description
IEnumerable<SourcedNodeWrite<T>>	timeSeries	List of CogniteSdk TimeSeries objects
RetryMode	retryMode	How to do retries. Keeping duplicates is not valid for this method.
SanitationMode	sanitationMode	The type of sanitation to apply to timeseries before creating
CancellationToken	token	Cancellation token
bool	isBeta	If true, use the beta CDM time series API instead of the IDM view. Required to create state time series.

Returns

Type	Description
Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the created timeseries

Type Parameters

Name	Description
T

GetOrCreateStateSetsAsync<T>(IEnumerable<InstanceIdentifier>, Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>>, RetryMode, SanitationMode, CancellationToken)

Ensures the the state sets with the provided instanceIds exist in CDF. If one or more do not exist, use the buildStateSets function to construct the missing state set objects and upload them to CDF. State sets are only available through the beta CDM state set API.

Declaration

public Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>> GetOrCreateStateSetsAsync<T>(IEnumerable<InstanceIdentifier> instanceIds, Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>> buildStateSets, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token) where T : CogniteStateSet

Parameters

Type	Name	Description
IEnumerable<InstanceIdentifier>	instanceIds	Instance Ids
Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>>	buildStateSets	Function that builds CogniteSdk StateSet objects
RetryMode	retryMode	How to handle failed requests
SanitationMode	sanitationMode	The type of sanitation to apply to state sets before creating
CancellationToken	token	Cancellation token

Returns

Type	Description
Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occurred and a list of the created and found state sets

Type Parameters

Name	Description
T

GetOrCreateStateSetsAsync<T>(IEnumerable<InstanceIdentifier>, Func<IEnumerable<InstanceIdentifier>, Task<IEnumerable<SourcedNodeWrite<T>>>>, RetryMode, SanitationMode, CancellationToken)

Ensures the the state sets with the provided instanceIds exist in CDF. If one or more do not exist, use the buildStateSets function to construct the missing state set objects and upload them to CDF. State sets are only available through the beta CDM state set API.

Declaration

public Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>> GetOrCreateStateSetsAsync<T>(IEnumerable<InstanceIdentifier> instanceIds, Func<IEnumerable<InstanceIdentifier>, Task<IEnumerable<SourcedNodeWrite<T>>>> buildStateSets, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token) where T : CogniteStateSet

Parameters

Type	Name	Description
IEnumerable<InstanceIdentifier>	instanceIds	Instance Ids
Func<IEnumerable<InstanceIdentifier>, Task<IEnumerable<SourcedNodeWrite<T>>>>	buildStateSets	Async function that builds CogniteSdk StateSet objects
RetryMode	retryMode	How to handle failed requests
SanitationMode	sanitationMode	The type of sanitation to apply to state sets before creating
CancellationToken	token	Cancellation token

Returns

Type	Description
Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the created and found state sets

Type Parameters

Name	Description
T

GetOrCreateTimeSeriesAsync<T>(IEnumerable<InstanceIdentifier>, Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>>, RetryMode, SanitationMode, CancellationToken, bool)

Ensures the the time series with the provided instanceIds exist in CDF. If one or more do not exist, use the buildTimeSeries function to construct the missing time series objects and upload them to CDF. This method uses the CogniteConfig object to determine chunking of items and throttling against CDF If any items fail to be created due to missing asset, duplicated externalId, duplicated legacy name, or missing dataSetId, they can be removed before retrying by setting retryMode

Declaration

public Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>> GetOrCreateTimeSeriesAsync<T>(IEnumerable<InstanceIdentifier> instanceIds, Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>> buildTimeSeries, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token, bool isBeta = false) where T : CogniteTimeSeriesBase

Parameters

Type	Name	Description
IEnumerable<InstanceIdentifier>	instanceIds	Instance Ids
Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>>	buildTimeSeries	Function that builds CogniteSdk TimeSeries objects
RetryMode	retryMode	How to handle failed requests
SanitationMode	sanitationMode	The type of sanitation to apply to TimeSeries before creating
CancellationToken	token	Cancellation token
bool	isBeta	If true, use the beta CDM time series API instead of the IDM view. Required to create state time series.

Returns

Type	Description
Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occurred and a list of the created and found TimeSeries

Type Parameters

Name	Description
T

GetOrCreateTimeSeriesAsync<T>(IEnumerable<InstanceIdentifier>, Func<IEnumerable<InstanceIdentifier>, Task<IEnumerable<SourcedNodeWrite<T>>>>, RetryMode, SanitationMode, CancellationToken, bool)

Ensures the the time series with the provided instanceIds exist in CDF. If one or more do not exist, use the buildTimeSeries function to construct the missing time series objects and upload them to CDF. This method uses the CogniteConfig object to determine chunking of items and throttling against CDF By default, if any items fail to be created due to missing asset, duplicated externalId, duplicated legacy name, or missing dataSetId, they can be removed before retrying by setting retryMode

Declaration

public Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>> GetOrCreateTimeSeriesAsync<T>(IEnumerable<InstanceIdentifier> instanceIds, Func<IEnumerable<InstanceIdentifier>, Task<IEnumerable<SourcedNodeWrite<T>>>> buildTimeSeries, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token, bool isBeta = false) where T : CogniteTimeSeriesBase

Parameters

Type	Name	Description
IEnumerable<InstanceIdentifier>	instanceIds	Instance Ids
Func<IEnumerable<InstanceIdentifier>, Task<IEnumerable<SourcedNodeWrite<T>>>>	buildTimeSeries	Async function that builds CogniteSdk TimeSeries objects
RetryMode	retryMode	How to handle failed requests
SanitationMode	sanitationMode	The type of sanitation to apply to TimeSeries before creating
CancellationToken	token	Cancellation token
bool	isBeta	If true, use the beta CDM time series API instead of the IDM view. Required to create state time series.

Returns

Type	Description
Task<CogniteResult<SourcedNode<T>, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the created and found TimeSeries

Type Parameters

Name	Description
T

GetStateSetsByIdsIgnoreErrors<T>(IEnumerable<Identity>, CancellationToken)

Gets state sets by ids in stateSets, ignoring errors. State sets are only available through the beta CDM state set API.

Declaration

public Task<IEnumerable<SourcedNode<T>>> GetStateSetsByIdsIgnoreErrors<T>(IEnumerable<Identity> stateSets, CancellationToken token) where T : CogniteStateSet

Parameters

Type	Name	Description
IEnumerable<Identity>	stateSets	List of StateSet instance ids to fetch
CancellationToken	token	Cancellation token

Returns

Type	Description
Task<IEnumerable<SourcedNode<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the created state sets

Type Parameters

Name	Description
T

GetTimeSeriesByIdsIgnoreErrors<T>(IEnumerable<Identity>, CancellationToken, bool)

Gets TimeSeries by ids in timeSeries, ignoring errors.

Declaration

public Task<IEnumerable<SourcedNode<T>>> GetTimeSeriesByIdsIgnoreErrors<T>(IEnumerable<Identity> timeSeries, CancellationToken token, bool isBeta = false) where T : CogniteTimeSeriesBase

Parameters

Type	Name	Description
IEnumerable<Identity>	timeSeries	List of TimeSeries instance ids to fetch
CancellationToken	token	Cancellation token
bool	isBeta	If true, use the beta CDM time series API instead of the IDM view. Required to fetch state time series.

Returns

Type	Description
Task<IEnumerable<SourcedNode<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the created timeseries

Type Parameters

Name	Description
T

InsertDataPointsCreateMissingAsync(IDictionary<Identity, IEnumerable<Datapoint>>?, SanitationMode, RetryMode, CancellationToken)

Insert datapoints to timeseries. Insertions are chunked and cleaned according to configuration, and can optionally handle errors. If any timeseries missing from the result and inserted by externalId, they are created before the points are inserted again.

Declaration

public Task<(CogniteResult<DataPointInsertError> DataPointResult, CogniteResult<SourcedNode<CogniteTimeSeriesBase>, SourcedNodeWrite<CogniteTimeSeriesBase>>? TimeSeriesResult)> InsertDataPointsCreateMissingAsync(IDictionary<Identity, IEnumerable<Datapoint>>? points, SanitationMode sanitationMode, RetryMode retryMode, CancellationToken token)

Parameters

Type	Name	Description
IDictionary<Identity, IEnumerable<Datapoint>>	points	Datapoints to insert
SanitationMode	sanitationMode	How to sanitize datapoints
RetryMode	retryMode	How to handle retries
CancellationToken	token	Cancellation token

Returns

Type	Description
Task<(CogniteResult<DataPointInsertError> DataPointResult, CogniteResult<SourcedNode<CogniteTimeSeriesBase>, SourcedNodeWrite<CogniteTimeSeriesBase>> TimeSeriesResult)>	Results with a list of errors. If TimeSeriesResult is null, no timeseries were attempted created.

InsertDataPointsIDMAsync(IDictionary<Identity, IEnumerable<Datapoint>>?, SanitationMode, RetryMode, CancellationToken)

Insert the provided data points into CDF. The data points are chunked according to CdfChunking and trimmed according to the CDF limits. The points dictionary keys are time series identities (Id or ExternalId) and the values are numeric or string data points

On error, the offending timeseries/datapoints can optionally be removed.

Declaration

public Task<CogniteResult<DataPointInsertError>> InsertDataPointsIDMAsync(IDictionary<Identity, IEnumerable<Datapoint>>? points, SanitationMode sanitationMode, RetryMode retryMode, CancellationToken token)

Parameters

Type	Name	Description
IDictionary<Identity, IEnumerable<Datapoint>>	points	Data points
SanitationMode	sanitationMode
RetryMode	retryMode
CancellationToken	token	Cancellation token

Returns

Type	Description
Task<CogniteResult<DataPointInsertError>>

UpsertStateSetsAsync<T>(IEnumerable<SourcedNodeWrite<T>>, RetryMode, SanitationMode, CancellationToken)

Upsert state sets in updates. If items fail due to duplicated instance ids, they can be removed before retrying by setting retryMode. State sets are only available through the beta CDM state set API.

Declaration

public Task<CogniteResult<SlimInstance, SourcedNodeWrite<T>>> UpsertStateSetsAsync<T>(IEnumerable<SourcedNodeWrite<T>> updates, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token) where T : CogniteStateSet

Parameters

Type	Name	Description
IEnumerable<SourcedNodeWrite<T>>	updates	List of StateSet objects
RetryMode	retryMode	How to do retries. Keeping duplicates is not valid for this method.
SanitationMode	sanitationMode	The type of sanitation to apply to state sets before updating
CancellationToken	token	Cancellation token

Returns

Type	Description
Task<CogniteResult<SlimInstance, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the updated state sets

Type Parameters

Name	Description
T

UpsertTimeSeriesAsync<T>(IEnumerable<SourcedNodeWrite<T>>, RetryMode, SanitationMode, CancellationToken, bool)

Upsert timeseries in updates. If items fail due to duplicated instance ids, they can be removed before retrying by setting retryMode. TimeSeries will be returned in the same order as given.

Declaration

public Task<CogniteResult<SlimInstance, SourcedNodeWrite<T>>> UpsertTimeSeriesAsync<T>(IEnumerable<SourcedNodeWrite<T>> updates, RetryMode retryMode, SanitationMode sanitationMode, CancellationToken token, bool isBeta = false) where T : CogniteTimeSeriesBase

Parameters

Type	Name	Description
IEnumerable<SourcedNodeWrite<T>>	updates	List of TimeSeries objects
RetryMode	retryMode	How to do retries. Keeping duplicates is not valid for this method.
SanitationMode	sanitationMode	The type of sanitation to apply to assets before updating
CancellationToken	token	Cancellation token
bool	isBeta	If true, use the beta CDM time series API instead of the IDM view. Required to update state time series.

Returns

Type	Description
Task<CogniteResult<SlimInstance, SourcedNodeWrite<T>>>	A CogniteResult<TResult, TError> containing errors that occured and a list of the updated TimeSeries

Type Parameters

Name	Description
T

Implements

IRawDestination

Extension Methods

DataModelUtils.GetOrCreateResourcesAsync<T, T2>(T2, IEnumerable<InstanceIdentifier>, Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>>, Func<IEnumerable<SourcedNodeWrite<T>>, SanitationMode, (IEnumerable<SourcedNodeWrite<T>>, IEnumerable<CogniteError<SourcedNodeWrite<T>>>)>, int, int, RetryMode, SanitationMode, CancellationToken)