Data Stores lets you store, retrieve, and query your data with a universal, low-ceremony pattern that abstracts away the implementation details of any given storage technology.

Pluggable integrations are provided to many popular storage systems and techniques, including Local Storage, Azure Blob Storage, CosmosDb (formerly DocumentDb), PostgreSQL+Marten, Azure Table Storage, FTP, Amazon S3, Google Drive, JSON, YAML, XML, GZIP, LZMA, as well as super-efficient Bond and Protobuf serialization. More is planned in the Roadmap.

Data Stores lets you easily define a data context that describes any number of data stores. These data stores can exist on simple file storage, or on any other supported data store, such as a document database.

In the simplest form, Data Stores lets you use a file folder like a NoSQL (in fact NoDB) data store. This folder can be stored on local disk, or one of many cloud file storage services. These file-based data store implementations are called file stores.

In the most robust form, Data Stores lets you integrate to many different kinds of data storage services, such as document databases. You can use the advanced indexing and querying abilities of these stores, while making code that is indifferent to their implementation details.

First make some data classes to hold your data. These are typically simple POCOs with no behavior, only properties:

public class Vehicle
{
    public Vehicle(string vin, int year, string make, string model)
    {
        Vin = vin;
        Year = year;
        Make = make;
        Model = model;
    }

    public string Vin { get; }
    public int Year { get; }
    public string Make { get; }
    public string Model { get; }

    public class Key
    {
        public Key(string make, string vin)
        {
            Make = make;
            Vin = vin;
        }

        public string Make { get; }
        public string Vin { get; }
    }
}

In this example, we have defined a nested key type Vehicle.Key for uniquely identifying and storing things of this type. The key type does not need to be a nested class, and can also be a simple type, like int or Guid.

Now define a data context that describes all of your data stores:

public interface ITestDataContext : IContext
{
    [RootPath("data"), JsonSerialization, FileExtension(".json")]
    [KeyMap("vehicles/{Make}/{Vin}")]
    IDataStore<Vehicle.Key, Vehicle> Vehicles { get; }
}

This is an interface, with an IDataStore<,> property for each data store, decorated with facet attributes. These facets describe aspects of our data store. In this example:

• RootPath declares that we want the data store to write to a local folder called data.
• JsonSerialization declares that we want the data to be serialized using JSON.
• FileExtension declares that we want our data files to end with .json.
• KeyMap declares that we want our entity file to be in a subfolder like vehicles/{Make}, with the file named with Vin, e.g.:

  vehicles/Ford/1HGBH40JXMN109186.json
  

We can now use a ContextFactory to create an instance of our ITestDataContext:

var dataContext = new ContextFactory().Create<ITestDataContext>();

var vehicle = new Vehicle("1HGBH40JXMN109186", 2017, "Ford", "Fusion");

var vehicleKey = new Vehicle.Key(vehicle.Vin, vehicle.Make);

dataContext.Vehicles.Create(vehicleKey, vehicle).Wait();

var gottenVehicle = dataContext.Vehicles.Get(vehicleKey).Result;

var queriedVehicles = dataContext.Vehicles.ListValues(k => k.Make == "Ford").Result;

Most IDataStore<,> methods are Task-oriented to allow async behavior, so use await, .Result, .Wait(), etc. as appropriate.

Note that the consuming code here doesn't know or care too much about which data store technology is used. This code could operate against anything from a local folder to a database server. The IDataStore<,> interface abstracts these details away.

• Keying in Data Stores

• Data Contexts

• File Stores

Data Store integrations are provided for Local Storage, CosmosDb, Azure Blob Storage, Azure Table Storage, PostgreSQL/Marten Document Database, Amazon S3, FTP, HTTP, and Google Drive.

Serialization integrations are provided for JSON, YAML, XML, as well as super-efficient Bond and Protobuf protocols. Note that some serializers have special requirements for data classes to be serialized.

Optional compression via GZIP or LZMA (the compression used by 7-Zip) is provided as well. This can make a large difference in stored / transferred data payload size for formats like JSON.

Data Stores is available as a constellation of Nuget packages, broken up primarily by third party dependency requirements.

The Halforbit.DataStores package is required, and includes Local Storage, FTP, JSON, and GZIP capabilities.

Include any other packages with the functionality you desire:

Halforbit.DataStores

Halforbit.DataStores.FileStores.BlobStorage

Halforbit.DataStores.FileStores.AmazonS3

Halforbit.DataStores.FileStores.GoogleDrive

Halforbit.DataStores.FileStores.Compression.Lzma

Halforbit.DataStores.FileStores.Serialization.Yaml

Halforbit.DataStores.FileStores.Serialization.Protobuf

Halforbit.DataStores.FileStores.Serialization.Bond

Halforbit.DataStores.DocumentStores.CosmosDb

Halforbit.DataStores.DocumentStores.PostgresMarten

Halforbit.DataStores.TableStores.AzureTables