Skip to content

EntityFrameworkCore Bulk Extensions for Insert Update and Delete (CUD) operations

License

Notifications You must be signed in to change notification settings

javafun/EFCore.BulkExtensions

 
 

Repository files navigation

EFCore.BulkExtensions

EntityFrameworkCore extensions for Bulk operations (Insert, Update, Delete).
Library is targeting NetStandard 1.4 so it can used on project targeting both NetCore(1.0+) or NetFramework(4.6.1+).
It is Lightweight and very Efficient, having all mostly used CUD operation.
Under the hood uses SqlBulkCopy for Insert, for Update/Delete combines BulkInsert with raw Sql 'MERGE' (MsSQL 2008+).

Available on NuGet. Latest Version: 1.0.4
Package manager console command for installation: Install-Package EFCore.BulkExtensions

Usage is simple and pretty straightforward. Extensions are made on DbContext class and can be used like this:

context.BulkInsert(entitiesList);
context.BulkUpdate(entitiesList);
context.BulkDelete(entitiesList);
context.BulkInsertOrUpdate(entitiesList);

Each of these operations are separate transactions.
So when using multiple operations in single procedure and if one would break because of some Db constraint, previous would stay executed.
In scenario where All or Nothing is required, there should be additional logic with try/catch block, catch having methods that would revert previously executed operations.

BulkConfig arguments

BulkInsertOrUpdate method can be used when there is need for both operations but in one connection to database.
It makes Update when PK is matched, otherwise does Insert.

Additionally BulkInsert and BulkInsertOrUpdate methods can have optional argument BulkConfig with bool properties:
{ PreserveInsertOrder, SetOutputIdentity }.
Default behaviour is { false, false } and if we want to change it, BulkConfig should be added explicitly with one or both properties set to true.
This argument has purpose only when PK has Identity (usually int type with AutoIncrement), while if PK is Guid(sequential) created in Application there is no need for it.

context.BulkInsert(entitiesList, new BulkConfig { PreserveInsertOrder = true, SetOutputIdentity = true});
context.BulkInsertOrUpdate(entitiesList, new BulkConfig { PreserveInsertOrder = true });

PreserveInsertOrder makes sure that entites are inserted to Db as they are ordered in entitiesList.
However for this to work Id column needs to be set for the proper order.
For example if table already has rows, let's say it has 1000 rows with Id-s (1:1000), and we now want to add 300 more.
Since Id-s are generated in Db we could not set them, they would all be 0 (int default) in list.
But if we want to keep the order as they are ordered in list then those Id-s should be set say 1 to 300.
Here single Id value itself doesn't matter, db will change it to (1001:1300), what matters is their mutual relationship for sorting.
Insertion order is implemented with TOP in conjuction with ORDER BY. stackoverflow:merge-into-insertion-order.

When using PreserveInsertOrder with BulkInsertOrUpdate method, Id value does matter for those that will be updated.
If we need to sort those for insert and not have conflict with existing Id-s, there are 2 ways:
One is set Id to really high values, order of magnitude 10^10, and another even better setting them to negative values.
So if we have list of 8000, say 3000 for update (they keep the real Id) and 5000 for insert then Id-s could be (-5000:-1).

SetOutputIdentity is useful when BulkInsert is done to multiple related tables, that have Identity column.
So after Insert is done to first table, we need Id-s that were generated in Db becasue they are FK in second table.
It is implemented with OUTPUT as part of MERGE Query, so in this case even the Insert is not done directly to TargetTable but to TempTable and then Merged with TargetTable.
When used if PreserveInsertOrder is also set to true Id-s will be updated in entitiesList, and if PreserveInsertOrder is false then entitiesList will be cleared and reloaded.

Performances

Following are performances (in seconds):

Operations\Rows 100,000 EF 100,000 EFBulk 1,000,000 EFBulk
Insert 38.98 s 2.10 s 17.99 s
Update 109.25 s 3.96 s 31.45 s
Delete 7.26 s 2.04 s 12.18 s
----------------- ------------ ---------------- ------------------
Together 70.70 s 5.88 s 56.84 s

TestTable has 6 column(Guid, string, string, int, decimal?, DateTime).
All were inserted and 2 of them (string, DateTime) were updated.
Test was done locally on following configuration: INTEL Core i5-3570K 3.40GHz, DDRIII 8GB x 2, SSD 840 EVO 128 GB.

If you find this project useful you can mark it by leaving a Github Star.

Contributing

Please read CONTRIBUTING.md for details on code of conduct, and the process for submitting pull requests.

NuGet

About

EntityFrameworkCore Bulk Extensions for Insert Update and Delete (CUD) operations

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C# 100.0%