public Task <GetObjectResponse> GetObjectAsync(string bucketName, string objectKey, Action <GetObjectRequest>?config = null, CancellationToken token = default) { GetObjectRequest req = new GetObjectRequest(bucketName, objectKey); config?.Invoke(req); return(_objectOperations.GetObjectAsync(req, token)); }
/// <summary> /// Get object - With data /// </summary> /// <param name="objectId">Object Id</param> /// <param name="bucketName">Bucket name - Optional if passed throuhg constructor</param> /// <returns>Returns actual object data - bytes</returns> /// <exception cref="EndpointUnreachableException">Thrown when S3 endpoint is unreachable.</exception> /// <exception cref="ObjectNotFoundException">Thrown when object is not found.</exception> /// <exception cref="S3BaseException">Thrown when exception is not handled.</exception> public async Task <S3Object> GetObject(string objectId) { var s3Obj = BucketHelper.ExtractObjectInfo(objectId); var bucket = s3Obj.bucketName?.ToLower() ?? _bucketName; var objectName = s3Obj.objectName; var result = await GetObjectInfo(objectId); try { result.Data = await _requestRetryPolicy.ExecuteAsync(async() => { byte[] data = null; await _objectOperationsClient.GetObjectAsync(bucket, objectName, (s) => { using (var ms = new MemoryStream()) { byte[] buffer = new byte[result.Size]; int read; while ((read = s.Read(buffer, 0, buffer.Length)) > 0) { ms.Write(buffer, 0, read); } ms.Seek(0, SeekOrigin.Begin); data = ms.ToArray(); } }); return(data); }); } catch (MinioException ex) when(ex is ConnectionException || ex is InternalServerException || ex is InternalClientException || ex is InvalidEndpointException ) { throw new EndpointUnreachableException(_endpoint, ex.ToString()); } catch (S3BaseException) { throw; } catch (Exception ex) { throw new S3BaseException(ex.Message, ex.ToString()); } return(result); }
private static async Task <GetObjectResponse> DownloadPartAsync(IObjectOperations operations, string bucketName, string objectKey, Stream output, long partSize, int partNumber, int bufferSize, SemaphoreSlim semaphore, Mutex mutex, Action <GetObjectRequest>?config, CancellationToken token) { try { GetObjectRequest getReq = new GetObjectRequest(bucketName, objectKey); getReq.PartNumber = partNumber; config?.Invoke(getReq); GetObjectResponse getResp = await operations.GetObjectAsync(getReq, token).ConfigureAwait(false); using (Stream stream = getResp.Content) { long offset = (partNumber - 1) * partSize; byte[] buffer = new byte[bufferSize]; while (true) { int read = await stream.ReadUpToAsync(buffer, 0, bufferSize, token).ConfigureAwait(false); if (read > 0) { mutex.WaitOne(); output.Seek(offset, SeekOrigin.Begin); await output.WriteAsync(buffer, 0, read, token).ConfigureAwait(false); offset += read; mutex.ReleaseMutex(); } else { break; } } } return(getResp); } finally { semaphore.Release(); } }
public static async IAsyncEnumerable <GetObjectResponse> MultipartDownloadAsync(this IObjectOperations operations, string bucketName, string objectKey, Stream output, int bufferSize = 16777216, int numParallelParts = 4, Action <GetObjectRequest>?config = null, [EnumeratorCancellation] CancellationToken token = default) { Validator.RequireNotNull(output, nameof(output)); //Use a HEAD request on the object key to determine if the file was originally uploaded with multipart HeadObjectRequest headReq = new HeadObjectRequest(bucketName, objectKey); headReq.PartNumber = 1; HeadObjectResponse headResp = await operations.HeadObjectAsync(headReq, token).ConfigureAwait(false); Queue <Task <GetObjectResponse> > queue = new Queue <Task <GetObjectResponse> >(); if (headResp.NumberOfParts == null) { GetObjectRequest getReq = new GetObjectRequest(bucketName, objectKey); config?.Invoke(getReq); GetObjectResponse getResp = await operations.GetObjectAsync(getReq, token).ConfigureAwait(false); if (!getResp.IsSuccess) { throw new Exception(); } await getResp.Content.CopyToAsync(output, 81920, token).ConfigureAwait(false); yield return(getResp); } else { int parts = headResp.NumberOfParts.Value; using (SemaphoreSlim semaphore = new SemaphoreSlim(numParallelParts)) using (Mutex mutex = new Mutex()) { for (int i = 1; i <= parts; i++) { await semaphore.WaitAsync(token).ConfigureAwait(false); if (token.IsCancellationRequested) { yield break; } queue.Enqueue(DownloadPartAsync(operations, bucketName, objectKey, output, headResp.ContentLength, i, bufferSize, semaphore, mutex, config, token)); } while (queue.TryDequeue(out Task <GetObjectResponse>?task)) { if (token.IsCancellationRequested) { yield break; } GetObjectResponse response = await task !.ConfigureAwait(false); yield return(response); } } } }
public Task <GetObjectResponse> DownloadAsync(CancellationToken token = default) { return(_objectOperations.GetObjectAsync(_request, token)); }