public static bool GetOrReserveEventType <TStreamId>(
this INameIndex <TStreamId> eventTypeIndex,
IRecordFactory <TStreamId> recordFactory,
string eventType,
long logPosition,
out TStreamId eventTypeId,
out IPrepareLogRecord <TStreamId> eventTypeRecord)
{
var preExisting = eventTypeIndex.GetOrReserve(eventType, out eventTypeId, out var addedNumber, out var addedName);
var appendNewEventType = recordFactory.ExplicitEventTypeCreation && !preExisting;
if (!appendNewEventType)
{
eventTypeRecord = null;
return(preExisting);
}
eventTypeRecord = recordFactory.CreateEventTypeRecord(
eventTypeId: Guid.NewGuid(),
parentEventTypeId: Guid.Empty,
eventType: addedName,
eventTypeNumber: addedNumber,
eventTypeVersion: 0,
logPosition: logPosition,
timeStamp: DateTime.UtcNow);
return(preExisting);
}
public override void TestFixtureSetUp()
{
base.TestFixtureSetUp();
_chunk = TFChunkHelper.CreateNewChunk(Filename);
var logFormat = LogFormatHelper <TLogFormat, TStreamId> .LogFormat;
logFormat.StreamNameIndex.GetOrAddId("test", out var streamId1, out _, out _);
logFormat.StreamNameIndex.GetOrAddId("test2", out var streamId2, out _, out _);
_prepare1 = LogRecord.Prepare(logFormat.RecordFactory, 0, _corrId, _eventId, 0, 0, streamId1, 1,
PrepareFlags.None, "Foo", new byte[12], new byte[15], new DateTime(2000, 1, 1, 12, 0, 0));
var r1 = _chunk.TryAppend(_prepare1);
_written1 = r1.Success;
_position1 = r1.OldPosition;
_prepare2 = LogRecord.Prepare(logFormat.RecordFactory, r1.NewPosition, _corrId, _eventId, 0, 0, streamId2, 2,
PrepareFlags.None, "Foo2", new byte[12], new byte[15], new DateTime(2000, 1, 1, 12, 0, 0));
var r2 = _chunk.TryAppend(_prepare2);
_written2 = r2.Success;
_position2 = r2.OldPosition;
_chunk.Flush();
}
public void SetUp()
{
_writerCheckpoint = new InMemoryCheckpoint();
_db = new TFChunkDb(TFChunkHelper.CreateDbConfig(PathName, _writerCheckpoint, new InMemoryCheckpoint(),
1024));
_db.Open();
_writer = new TFChunkWriter(_db);
_writer.Open();
var recordFactory = LogFormatHelper <TLogFormat, TStreamId> .RecordFactory;
var streamId = LogFormatHelper <TLogFormat, TStreamId> .StreamId;
var eventTypeId = LogFormatHelper <TLogFormat, TStreamId> .EventTypeId;
_record = LogRecord.Prepare(
factory: recordFactory,
logPosition: 0,
eventId: _eventId,
correlationId: _correlationId,
transactionPos: 0xDEAD,
transactionOffset: 0xBEEF,
eventStreamId: streamId,
expectedVersion: 1234,
timeStamp: new DateTime(2012, 12, 21),
flags: PrepareFlags.SingleWrite,
eventType: eventTypeId,
data: new byte[] { 1, 2, 3, 4, 5 },
metadata: new byte[] { 7, 17 });
long newPos;
_writer.Write(_record, out newPos);
_writer.Flush();
}
// todo: rename to GetOrReserveStream when we generalise to EventTypes too.
/// Generates a StreamRecord if necessary
public static bool GetOrReserve <TStreamId>(
this INameIndex <TStreamId> streamNameIndex,
IRecordFactory <TStreamId> recordFactory,
string streamName,
long logPosition,
out TStreamId streamId,
out IPrepareLogRecord <TStreamId> streamRecord)
{
var preExisting = streamNameIndex.GetOrReserve(streamName, out streamId, out var addedId, out var addedName);
var appendNewStream = recordFactory.ExplicitStreamCreation && !preExisting;
if (!appendNewStream)
{
streamRecord = null;
return(preExisting);
}
streamRecord = recordFactory.CreateStreamRecord(
streamId: Guid.NewGuid(),
logPosition: logPosition,
timeStamp: DateTime.UtcNow,
streamNumber: addedId,
streamName: addedName);
return(preExisting);
}
public override void TestFixtureSetUp()
{
base.TestFixtureSetUp();
_chunk = TFChunkHelper.CreateNewChunk(Filename);
var recordFactory = LogFormatHelper <TLogFormat, TStreamId> .RecordFactory;
var streamId1 = LogFormatHelper <TLogFormat, TStreamId> .StreamId;
var streamId2 = LogFormatHelper <TLogFormat, TStreamId> .StreamId2;
var eventTypeId1 = LogFormatHelper <TLogFormat, TStreamId> .EventTypeId;
var eventTypeId2 = LogFormatHelper <TLogFormat, TStreamId> .EventTypeId2;
_prepare1 = LogRecord.Prepare(recordFactory, 0, _corrId, _eventId, 0, 0, streamId1, 1,
PrepareFlags.None, eventTypeId1, new byte[12], new byte[15], new DateTime(2000, 1, 1, 12, 0, 0));
var r1 = _chunk.TryAppend(_prepare1);
_written1 = r1.Success;
_position1 = r1.OldPosition;
_prepare2 = LogRecord.Prepare(recordFactory, r1.NewPosition, _corrId, _eventId, 0, 0, streamId2, 2,
PrepareFlags.None, eventTypeId2, new byte[12], new byte[15], new DateTime(2000, 1, 1, 12, 0, 0));
var r2 = _chunk.TryAppend(_prepare2);
_written2 = r2.Success;
_position2 = r2.OldPosition;
_chunk.Flush();
}
protected override void WriteTestScenario()
{
_prepare0 = WritePrepare("ES", expectedVersion: -1);
_prepare1 = WritePrepare("ES", expectedVersion: -1);
_prepare2 = WritePrepare("ES", expectedVersion: -1);
WriteCommit(_prepare1.LogPosition, "ES", eventNumber: 0);
}
public override void TestFixtureSetUp()
{
base.TestFixtureSetUp();
var recordFactory = LogFormatHelper <TLogFormat, TStreamId> .RecordFactory;
var streamId = LogFormatHelper <TLogFormat, TStreamId> .StreamId;
_record = LogRecord.Prepare(recordFactory, 0, _corrId, _eventId, 0, 0, streamId, 1,
PrepareFlags.None, "Foo", new byte[12], new byte[15], new DateTime(2000, 1, 1, 12, 0, 0));
_chunk = TFChunkHelper.CreateNewChunk(Filename);
_result = _chunk.TryAppend(_record);
}
public override async Task TestFixtureSetUp()
{
await base.TestFixtureSetUp();
_db = new TFChunkDb(TFChunkHelper.CreateSizedDbConfig(PathName, 0, chunkSize: 16 * 1024));
_db.Open();
var chunk = _db.Manager.GetChunkFor(0);
var logFormat = LogFormatHelper <TLogFormat, TStreamId> .LogFormat;
var streamName = "es-to-scavenge";
logFormat.StreamNameIndex.GetOrAddId(streamName, out var streamId, out _, out _);
var expectedVersion = ExpectedVersion.NoStream;
_p1 = LogRecord.SingleWrite(logFormat.RecordFactory, 0, Guid.NewGuid(), Guid.NewGuid(), streamId, expectedVersion++, "et1",
new byte[2048], new byte[] { 5, 7 });
_res1 = chunk.TryAppend(_p1);
_c1 = LogRecord.Commit(_res1.NewPosition, Guid.NewGuid(), _p1.LogPosition, 0);
_cres1 = chunk.TryAppend(_c1);
_p2 = LogRecord.SingleWrite(logFormat.RecordFactory, _cres1.NewPosition,
Guid.NewGuid(), Guid.NewGuid(), streamId, expectedVersion++, "et1",
new byte[2048], new byte[] { 5, 7 });
_res2 = chunk.TryAppend(_p2);
_c2 = LogRecord.Commit(_res2.NewPosition, Guid.NewGuid(), _p2.LogPosition, 1);
_cres2 = chunk.TryAppend(_c2);
_p3 = LogRecord.SingleWrite(logFormat.RecordFactory, _cres2.NewPosition,
Guid.NewGuid(), Guid.NewGuid(), streamId, expectedVersion++, "et1",
new byte[2048], new byte[] { 5, 7 });
_res3 = chunk.TryAppend(_p3);
_c3 = LogRecord.Commit(_res3.NewPosition, Guid.NewGuid(), _p3.LogPosition, 2);
_cres3 = chunk.TryAppend(_c3);
chunk.Complete();
_originalFileSize = chunk.FileSize;
_db.Config.WriterCheckpoint.Write(chunk.ChunkHeader.ChunkEndPosition);
_db.Config.WriterCheckpoint.Flush();
_db.Config.ChaserCheckpoint.Write(chunk.ChunkHeader.ChunkEndPosition);
_db.Config.ChaserCheckpoint.Flush();
var scavenger = new TFChunkScavenger <TStreamId>(_db, new FakeTFScavengerLog(), new FakeTableIndex <TStreamId>(),
new FakeReadIndex <TLogFormat, TStreamId>(x => EqualityComparer <TStreamId> .Default.Equals(x, streamId)),
logFormat.SystemStreams);
await scavenger.Scavenge(alwaysKeepScavenged : true, mergeChunks : false);
_scavengedChunk = _db.Manager.GetChunk(0);
}
public override void TestFixtureSetUp()
{
base.TestFixtureSetUp();
var logFormat = LogFormatHelper <TLogFormat, TStreamId> .LogFormat;
logFormat.StreamNameIndex.GetOrAddId("test", out var streamId, out _, out _);
_record = LogRecord.Prepare(logFormat.RecordFactory, 0, _corrId, _eventId, 0, 0, streamId, 1,
PrepareFlags.None, "Foo", new byte[12], new byte[15], new DateTime(2000, 1, 1, 12, 0, 0));
_chunk = TFChunkHelper.CreateNewChunk(Filename);
_result = _chunk.TryAppend(_record);
}
protected CommitLogRecord WriteDeleteCommit(IPrepareLogRecord prepare)
{
LogFormatHelper <TLogFormat, TStreamId> .CheckIfExplicitTransactionsSupported();
long pos;
var commit = LogRecord.Commit(WriterCheckpoint.ReadNonFlushed(),
prepare.CorrelationId,
prepare.LogPosition,
EventNumber.DeletedStream);
Assert.IsTrue(Writer.Write(commit, out pos));
return(commit);
}
protected override void WriteTestScenario()
{
_id1 = Guid.NewGuid();
_id2 = Guid.NewGuid();
long pos0, pos1, pos2, pos3, pos4;
GetOrReserve("test1", out var streamId1, out pos0);
_prepare1 = LogRecord.SingleWrite(_recordFactory, pos0, _id1, _id1, streamId1, ExpectedVersion.NoStream, "type", new byte[0], new byte[0]);
Writer.Write(_prepare1, out pos1);
_prepare2 = LogRecord.SingleWrite(_recordFactory, pos1, _id2, _id2, streamId1, 0, "type", new byte[0], new byte[0]);
Writer.Write(_prepare2, out pos2);
Writer.Write(new CommitLogRecord(pos2, _id1, pos0, DateTime.UtcNow, 0), out pos3);
Writer.Write(new CommitLogRecord(pos3, _id2, pos1, DateTime.UtcNow, 1), out pos4);
}
public override void TestFixtureSetUp()
{
base.TestFixtureSetUp();
var recordFactory = LogFormatHelper <TLogFormat, TStreamId> .RecordFactory;
var streamId = LogFormatHelper <TLogFormat, TStreamId> .StreamId;
_record = LogRecord.Prepare(recordFactory, 0, _corrId, _eventId, 0, 0, streamId, 1,
PrepareFlags.None, "Foo", new byte[12], new byte[15], new DateTime(2000, 1, 1, 12, 0, 0));
_chunk = TFChunkHelper.CreateNewChunk(Filename);
_result = _chunk.TryAppend(_record);
_chunk.Flush();
_chunk.Complete();
_cachedChunk = TFChunk.FromCompletedFile(Filename, verifyHash: true, unbufferedRead: false,
initialReaderCount: Constants.TFChunkInitialReaderCountDefault, maxReaderCount: Constants.TFChunkMaxReaderCountDefault, reduceFileCachePressure: false);
_cachedChunk.CacheInMemory();
}
public override async Task TestFixtureSetUp()
{
await base.TestFixtureSetUp();
_db = new TFChunkDb(TFChunkHelper.CreateSizedDbConfig(PathName, 0, chunkSize: 16 * 1024));
_db.Open();
var chunk = _db.Manager.GetChunkFor(0);
_p1 = LogRecord.SingleWrite(new LogV2RecordFactory(), 0, Guid.NewGuid(), Guid.NewGuid(), "es-to-scavenge", ExpectedVersion.Any, "et1",
new byte[2048], new byte[] { 5, 7 });
_res1 = chunk.TryAppend(_p1);
_c1 = LogRecord.Commit(_res1.NewPosition, Guid.NewGuid(), _p1.LogPosition, 0);
_cres1 = chunk.TryAppend(_c1);
_p2 = LogRecord.SingleWrite(new LogV2RecordFactory(), _cres1.NewPosition,
Guid.NewGuid(), Guid.NewGuid(), "es-to-scavenge", ExpectedVersion.Any, "et1",
new byte[2048], new byte[] { 5, 7 });
_res2 = chunk.TryAppend(_p2);
_c2 = LogRecord.Commit(_res2.NewPosition, Guid.NewGuid(), _p2.LogPosition, 1);
_cres2 = chunk.TryAppend(_c2);
_p3 = LogRecord.SingleWrite(new LogV2RecordFactory(), _cres2.NewPosition,
Guid.NewGuid(), Guid.NewGuid(), "es-to-scavenge", ExpectedVersion.Any, "et1",
new byte[2048], new byte[] { 5, 7 });
_res3 = chunk.TryAppend(_p3);
_c3 = LogRecord.Commit(_res3.NewPosition, Guid.NewGuid(), _p3.LogPosition, 2);
_cres3 = chunk.TryAppend(_c3);
chunk.Complete();
_originalFileSize = chunk.FileSize;
_db.Config.WriterCheckpoint.Write(chunk.ChunkHeader.ChunkEndPosition);
_db.Config.WriterCheckpoint.Flush();
_db.Config.ChaserCheckpoint.Write(chunk.ChunkHeader.ChunkEndPosition);
_db.Config.ChaserCheckpoint.Flush();
var scavenger = new TFChunkScavenger <string>(_db, new FakeTFScavengerLog(), new FakeTableIndex <string>(),
new FakeReadIndex <string>(x => x == "es-to-scavenge"),
LogFormatAbstractor.V2.SystemStreams);
await scavenger.Scavenge(alwaysKeepScavenged : true, mergeChunks : false);
_scavengedChunk = _db.Manager.GetChunk(0);
}
protected override void WriteTestScenario()
{
WriteSingleEvent("ES", 0, new string('.', 3000)); // chunk 1
WriteSingleEvent("ES", 1, new string('.', 3000));
WriteSingleEvent("ES", 2, new string('.', 3000));
WriteSingleEvent("ES", 3, new string('.', 3000), retryOnFail: true); // chunk 2
WriteSingleEvent("ES", 4, new string('.', 3000));
_event7prepare = WriteDeletePrepare("ES");
_event7commit = WriteDeleteCommit(_event7prepare);
_event7 = new EventRecord(EventNumber.DeletedStream, _event7prepare);
_event9 = WriteSingleEvent("ES2", 0, new string('.', 5000), retryOnFail: true); //chunk 3
Scavenge(completeLast: false, mergeChunks: false);
}
public EventRecord(long eventNumber, IPrepareLogRecord prepare, string eventStreamName)
{
Ensure.Nonnegative(eventNumber, "eventNumber");
EventNumber = eventNumber;
LogPosition = prepare.LogPosition;
CorrelationId = prepare.CorrelationId;
EventId = prepare.EventId;
TransactionPosition = prepare.TransactionPosition;
TransactionOffset = prepare.TransactionOffset;
EventStreamId = eventStreamName;
ExpectedVersion = prepare.ExpectedVersion;
TimeStamp = prepare.TimeStamp;
Flags = prepare.Flags;
EventType = prepare.EventType ?? string.Empty;
Data = prepare.Data;
Metadata = prepare.Metadata;
}
public WriteResult(long writtenPos, long newPos, IPrepareLogRecord <TStreamId> prepare)
{
WrittenPos = writtenPos;
NewPos = newPos;
Prepare = prepare;
}
public override async Task TestFixtureSetUp()
{
await base.TestFixtureSetUp();
_logFormat = LogFormatHelper <TLogFormat, TStreamId> .LogFormatFactory.Create(new() {
IndexDirectory = GetFilePathFor("index"),
});
_db = new TFChunkDb(TFChunkHelper.CreateSizedDbConfig(PathName, 0, chunkSize: 16 * 1024));
_db.Open();
var chunk = _db.Manager.GetChunkFor(0);
var streamName = "es-to-scavenge";
var pos = 0L;
_logFormat.StreamNameIndex.GetOrReserve(_logFormat.RecordFactory, streamName, 0, out var streamId, out var streamRecord);
if (streamRecord is not null)
{
var res = chunk.TryAppend(streamRecord);
pos = res.NewPosition;
}
_logFormat.EventTypeIndex.GetOrReserveEventType(_logFormat.RecordFactory, "et1", pos, out var eventTypeId, out var eventTypeRecord);
if (eventTypeRecord is not null)
{
var res = chunk.TryAppend(eventTypeRecord);
pos = res.NewPosition;
}
var expectedVersion = ExpectedVersion.NoStream;
_p1 = LogRecord.SingleWrite(_logFormat.RecordFactory, pos, Guid.NewGuid(), Guid.NewGuid(), streamId, expectedVersion++, eventTypeId,
new byte[2048], new byte[] { 5, 7 });
_res1 = chunk.TryAppend(_p1);
_c1 = LogRecord.Commit(_res1.NewPosition, Guid.NewGuid(), _p1.LogPosition, 0);
_cres1 = chunk.TryAppend(_c1);
_p2 = LogRecord.SingleWrite(_logFormat.RecordFactory, _cres1.NewPosition,
Guid.NewGuid(), Guid.NewGuid(), streamId, expectedVersion++, eventTypeId,
new byte[2048], new byte[] { 5, 7 });
_res2 = chunk.TryAppend(_p2);
_c2 = LogRecord.Commit(_res2.NewPosition, Guid.NewGuid(), _p2.LogPosition, 1);
_cres2 = chunk.TryAppend(_c2);
_p3 = LogRecord.SingleWrite(_logFormat.RecordFactory, _cres2.NewPosition,
Guid.NewGuid(), Guid.NewGuid(), streamId, expectedVersion++, eventTypeId,
new byte[2048], new byte[] { 5, 7 });
_res3 = chunk.TryAppend(_p3);
_c3 = LogRecord.Commit(_res3.NewPosition, Guid.NewGuid(), _p3.LogPosition, 2);
_cres3 = chunk.TryAppend(_c3);
chunk.Complete();
_originalFileSize = chunk.FileSize;
_db.Config.WriterCheckpoint.Write(chunk.ChunkHeader.ChunkEndPosition);
_db.Config.WriterCheckpoint.Flush();
_db.Config.ChaserCheckpoint.Write(chunk.ChunkHeader.ChunkEndPosition);
_db.Config.ChaserCheckpoint.Flush();
var scavenger = new TFChunkScavenger <TStreamId>(_db, new FakeTFScavengerLog(), new FakeTableIndex <TStreamId>(),
new FakeReadIndex <TLogFormat, TStreamId>(x => EqualityComparer <TStreamId> .Default.Equals(x, streamId), _logFormat.Metastreams),
_logFormat.Metastreams);
await scavenger.Scavenge(alwaysKeepScavenged : true, mergeChunks : false);
_scavengedChunk = _db.Manager.GetChunk(0);
}
// called from tests only
public EventRecord(long eventNumber, IPrepareLogRecord <string> prepare) : this(eventNumber, prepare, prepare.EventStreamId)
{
}
protected override void WriteTestScenario()
{
_prepare = WritePrepare("ES", -1);
}
