Collection of low-level HTTP and FTP file transfer tools
https://www.nuget.org/packages/Shift-it/
.Net has some pretty good FTP and HTTP libraries built in. However, they aren't amazingly unit-test friendly. They also wrap a lot of status results in exceptions. The protocol strictness of the libraries is also problematic when dealing with the general internet.
An old and battle hardend FTP client library, works with many old and cranky FTP servers that the standard .Net libraries can't handle.
A HTTP client that can accept a reasonable level of invalid protocol from servers. This is a blocking, synchronous library that uses only .Net sockets, not WebClient or HttpWebRequest, and has a lot of replaceable components.
Example GET:
var rq = new HttpRequestBuilder().Get(new Uri("https://www.nuget.org/")).Build();
using (var result = _subject.Request(rq)) {
var body = result.BodyReader.ReadStringToLength();
// . . .
}Example POST string data, without reading result:
var client = new HttpClient();
var postRq = new HttpRequestBuilder().Put(new Uri("https://target.example.com/resource")).Build("Hello, world");
client.Request(postRq).Dispose();Example with upload and download with files and progress:
long uploadBytes = 0;
long downloadBytes = 0;
using (var fs = File.OpenRead(@"C:\my\file.dat"))
{
var rq = new HttpRequestBuilder()
.Post(new Uri("http://myserver.com/whatever"))
.Build(fs, fs.Length);
using (var result = new HttpClient().Request(rq, b => uploadBytes = b))
{
var responseString = result.BodyReader.ReadStringToLength(b => downloadBytes = b);
}
}Thanks to skolima for these:
=== 14MB file
Measured on Windows
Time taken Mem Method
->2849ms 29MB Web client
->2657ms 20MB Web Client - stream copy, no hash
->2647ms 16MB Web Client - stream copy, Shift-It hash
->1575ms 0MB ShiftIt [current]
->1592ms 0MB ShiftIt - no hashing [current]
->1616ms 0MB ShiftIt - no compression [current]
->1574ms 0MB ShiftIt - no hashing, no compression [current]
Measured on Mono + Linux + fast connection
Time taken Mem Method
->1763ms 30MB Web client
->403ms 1MB Web Client - stream copy, no hash
->651ms 16MB Web Client - stream copy, Shift-It hash
->755ms 0MB ShiftIt [current]
->603ms 0MB ShiftIt - no hashing [current]
->762ms 0MB ShiftIt - no compression [current]
->669ms 0MB ShiftIt - no hashing, no compression [current]
->833ms 0MB ShiftIt [previous]
->602ms 0MB ShiftIt - no hashing [previous]
->903ms 0MB ShiftIt - no compression [previous]
->601ms 0MB ShiftIt - no hashing, no compression [previous]
=== 191MB file
Measured on Windows
Time taken Mem Method
->35378ms 638MB Web client
->34211ms 334MB Web Client - stream copy, no hash
->35382ms 320MB Web Client - stream copy, Shift-It hash
->20292ms 0MB ShiftIt [current]
->20157ms 0MB ShiftIt - no hashing [current]
->20446ms 0MB ShiftIt - no compression [current]
->19689ms 0MB ShiftIt - no hashing, no compression [current]
Measured on Mono + Linux + fast connection
Time taken Mem Method
->17435ms 383MB Web client
->5843ms 69MB Web Client - stream copy, no hash
->9327ms 260MB Web Client - stream copy, Shift-It hash
->12948ms 0MB ShiftIt [current]
->8529ms 0MB ShiftIt - no hashing [current]
->13472ms 0MB ShiftIt - no compression [current]
->8108ms 0MB ShiftIt - no hashing, no compression [current]
->13584ms 0MB ShiftIt [previous]
->8237ms 0MB ShiftIt - no hashing [previous]
->10648ms 0MB ShiftIt - no compression [previous]
->7308ms 0MB ShiftIt - no hashing, no compression [previous]
- Digest authentication
- chunked upload