Class LiteDBStateStore

Local database to store state between each run of the extractor. Used when the extractor needs to know the first/last points extracted from the source system and pushed to the destination.

Inheritance

object

LiteDBStateStore

Implements

IExtractionStateStore

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Cognite.Extractor.StateStorage

Assembly: Cognite.StateStorage.dll

Syntax

public class LiteDBStateStore : IExtractionStateStore, IDisposable

Constructors

LiteDBStateStore(StateStoreConfig, ILogger)

Create StateStore wrapper.

Declaration

public LiteDBStateStore(StateStoreConfig config, ILogger logger)

Parameters

Type	Name	Description
StateStoreConfig	config	Configuration of StateStore
ILogger	logger	Logger to use with StateStore

Properties

Database

Return a connection to the database. Must be disposed of after use. This uses the custom DateTime mapper.

Declaration

public LiteDatabase Database { get; }

Property Value

Type	Description
LiteDatabase

Mapper

BsonMapper used to convert objects into bson. Can be modified to add functionality.

Declaration

public BsonMapper Mapper { get; }

Property Value

Type	Description
BsonMapper

Methods

DeleteExtractionState(IEnumerable<IExtractionState>, string, CancellationToken)

Delete states from state store

Declaration

public Task DeleteExtractionState(IEnumerable<IExtractionState> extractionStates, string tableName, CancellationToken token)

Parameters

Type	Name	Description
IEnumerable<IExtractionState>	extractionStates	States to delete
string	tableName	Table to delete from
CancellationToken	token

Returns

Type	Description
Task

Dispose()

Dispose, for lite database.

Declaration

public void Dispose()

Dispose(bool)

Dispose lite database

Declaration

protected virtual void Dispose(bool disposing)

Parameters

Type	Name	Description
bool	disposing

GetAllExtractionStates<T>(string, CancellationToken)

Find all extraction states in the table.

Declaration

public Task<IEnumerable<T>> GetAllExtractionStates<T>(string tableName, CancellationToken token) where T : BaseStorableState

Parameters

Type	Name	Description
string	tableName	Name of table to restore from
CancellationToken	token

Returns

Type	Description
Task<IEnumerable<T>>

Type Parameters

Name	Description
T	Type of state to obtain

RestoreExtractionState<K>(IDictionary<string, K>, string, bool, CancellationToken)

Restore first and last timestamp from state store.

Declaration

public Task RestoreExtractionState<K>(IDictionary<string, K> extractionStates, string tableName, bool initializeMissing, CancellationToken token) where K : BaseExtractionState

Parameters

Type	Name	Description
IDictionary<string, K>	extractionStates	States to restore
string	tableName	Table to restore from
bool	initializeMissing	Initialize states missing from store to empty
CancellationToken	token

Returns

Type	Description
Task

Type Parameters

Name	Description
K	Subtype of BaseExtractionState used as state

RestoreExtractionState<T, K>(IDictionary<string, K>, string, Action<K, T>, CancellationToken)

Generic method to restore state with a custom type.

Declaration

public Task RestoreExtractionState<T, K>(IDictionary<string, K> extractionStates, string tableName, Action<K, T> restoreStorableState, CancellationToken token) where T : BaseStorableState where K : IExtractionState

Parameters

Type	Name	Description
IDictionary<string, K>	extractionStates	States to store
string	tableName	Collection to store into
Action<K, T>	restoreStorableState	Action for pair of stored object and state, to restore the state with information from the poco
CancellationToken	token

Returns

Type	Description
Task

Type Parameters

Name	Description
T	Subtype of BaseStorableState inserted into state store
K	Subtype of IExtractionState used as state

StoreExtractionState<K>(IEnumerable<K>, string, CancellationToken)

Store first and last timestamp to litedb state store collection given by tableName

Declaration

public Task StoreExtractionState<K>(IEnumerable<K> extractionStates, string tableName, CancellationToken token) where K : BaseExtractionState

Parameters

Type	Name	Description
IEnumerable<K>	extractionStates	States to store
string	tableName	Collection to store to
CancellationToken	token

Returns

Type	Description
Task

Type Parameters

Name	Description
K	Subtype of BaseExtractionState used as state

StoreExtractionState<T, K>(IEnumerable<K>, string, Func<K, T>, CancellationToken)

Store information from states into state store

Declaration

public Task StoreExtractionState<T, K>(IEnumerable<K> extractionStates, string tableName, Func<K, T> buildStorableState, CancellationToken token) where T : BaseStorableState where K : IExtractionState

Parameters

Type	Name	Description
IEnumerable<K>	extractionStates	States to store
string	tableName	Collection to store into
Func<K, T>	buildStorableState	Method to create a storable state from extraction state
CancellationToken	token

Returns

Type	Description
Task

Type Parameters

Name	Description
T	Subtype of BaseStorableState extracted from state store
K	Implementation of IExtractionState used as state

Implements

IExtractionStateStore

IDisposable

Extension Methods

StateStoreUtils.StoreStatesPeriodically<K>(IExtractionStateStore, TimeSpan, IEnumerable<K>, string, CancellationToken)

StateStoreUtils.StoreStatesPeriodically<T, K>(IExtractionStateStore, TimeSpan, IEnumerable<K>, string, Func<K, T>, CancellationToken)

DataModelUtils.GetOrCreateResourcesAsync<T, T2>(T2, IEnumerable<InstanceIdentifier>, Func<IEnumerable<InstanceIdentifier>, IEnumerable<SourcedNodeWrite<T>>>, Func<IEnumerable<SourcedNodeWrite<T>>, SanitationMode, (IEnumerable<SourcedNodeWrite<T>>, IEnumerable<CogniteError<SourcedNodeWrite<T>>>)>, int, int, RetryMode, SanitationMode, CancellationToken)