private async Task DownloadAndSaveHttpFileAsync <T>(IRemoteFileSpec <T> fileSpec, CancellationToken cancellationToken) where T : class { var requestOptions = new HttpRequestOptions { Url = fileSpec.Url, CancellationToken = cancellationToken, EnableHttpCompression = false }; using (var stream = await this.httpClient.Get(requestOptions).ConfigureAwait(false)) { var unzippedStream = stream; if (fileSpec.IsGZipped) { unzippedStream = new GZipStream(stream, CompressionMode.Decompress); } using (var reader = new StreamReader(unzippedStream, Encoding.UTF8, true)) { var text = await reader.ReadToEndAsync().ConfigureAwait(false); await this.SaveFileContentAsync(text, fileSpec, cancellationToken); } } }
private async Task SaveFileContentAsync <T>(string text, IRemoteFileSpec <T> fileSpec, CancellationToken cancellationToken) where T : class { using (var file = File.Open(fileSpec.LocalPath, FileMode.Create, FileAccess.Write)) using (var writer = new StreamWriter(file)) { this.log.Debug($"Saving {text.Length} characters to {fileSpec.LocalPath}"); text = text.Replace("�", string.Empty); await writer.WriteAsync(text).ConfigureAwait(false); await writer.FlushAsync().ConfigureAwait(false); } }
public async Task DownloadFileAsync <T>(IRemoteFileSpec <T> fileSpec, CancellationToken cancellationToken) where T : class { await this.requestLimiter.TickAsync(); if (fileSpec is ICustomDownload <T> customDownloadFile) { var content = await customDownloadFile.DownloadFileAsync(fileSpec, cancellationToken); await this.SaveFileContentAsync(content, fileSpec, cancellationToken); } else { await this.DownloadAndSaveHttpFileAsync(fileSpec, cancellationToken); } }
public async Task <Option <T> > GetFileContentAsync <T>(IRemoteFileSpec <T> fileSpec, CancellationToken cancellationToken) where T : class { var cacheFile = new FileInfo(fileSpec.LocalPath); if (this.IsRefreshRequired(cacheFile)) { this.CreateDirectoryIfNotExists(cacheFile.DirectoryName); this.ClearCacheFilesFromDirectory(cacheFile.DirectoryName); await this.DownloadFileAsync(fileSpec, cancellationToken); } return(fileSpec.Serialiser.Deserialise <T>(File.ReadAllText(cacheFile.FullName))); }
public async Task DownloadFileAsync <T>(IRemoteFileSpec <T> fileSpec, CancellationToken cancellationToken) where T : class { await _requestLimiter.TickAsync(); var requestOptions = new HttpRequestOptions { Url = fileSpec.Url, CancellationToken = cancellationToken, EnableHttpCompression = false }; using (var stream = await _httpClient.Get(requestOptions).ConfigureAwait(false)) { var unzippedStream = stream; if (fileSpec.IsGZipped) { unzippedStream = new GZipStream(stream, CompressionMode.Decompress); } using (var reader = new StreamReader(unzippedStream, Encoding.UTF8, true)) using (var file = File.Open(fileSpec.LocalPath, FileMode.Create, FileAccess.Write)) using (var writer = new StreamWriter(file)) { var text = await reader.ReadToEndAsync().ConfigureAwait(false); _log.Debug($"Saving {text.Length} characters to {fileSpec.LocalPath}"); text = text.Replace("�", ""); await writer.WriteAsync(text).ConfigureAwait(false); await writer.FlushAsync().ConfigureAwait(false); } } }
private async Task DownloadFileAsync <T>(IRemoteFileSpec <T> fileSpec, CancellationToken cancellationToken) where T : class { await this.fileDownloader.DownloadFileAsync(fileSpec, cancellationToken); }
public async Task <string> DownloadFileAsync(IRemoteFileSpec <IDictionary <string, string[]> > fileSpec, CancellationToken cancellationToken) { var mappingsFromTvDb = await this.xemApiClient.GetAllNames(new AllNamesQuery(EntityType.TvDb)); return(JsonConvert.SerializeObject(mappingsFromTvDb.NameValues)); }