private async Task OpenRemoteFiles(RootPathObject rootPath, int limit = -1) { if (rootPath.FileReadMode != Constants.FileModeLocal) { try { await LoadRemoteSemaphoreSlim.WaitAsync(); var filesPulled = 0; switch (rootPath.FileReadMode) { case Constants.FileModeFtp: using (var client = Utility.GetFtpClient()) { var mask = new Regex(rootPath.Filter .Replace(".", "[.]") .Replace("*", ".*") .Replace("?", ".")); foreach (var item in await client.GetListingAsync(rootPath.RootPath)) { // if this is a file if (item.Type == FtpFileSystemObjectType.File && mask.IsMatch(item.Name)) { if (limit != -1 && filesPulled == limit) { break; } using (await client.OpenReadAsync(item.FullName)) { } filesPulled++; Logger.Debug($"Opened file {item.Name}"); } } await client.DisconnectAsync(); } break; case Constants.FileModeSftp: using (var client = Utility.GetSftpClient()) { var mask = new Regex(rootPath.Filter .Replace(".", "[.]") .Replace("*", ".*") .Replace("?", ".")); var allFiles = client.ListDirectory(rootPath.RootPath); var downloadFiles = allFiles.Where(f => f.IsDirectory == false && mask.IsMatch(f.Name)); foreach (var file in downloadFiles) { if (limit != -1 && filesPulled == limit) { break; } using (client.OpenRead(file.FullName)) { filesPulled++; Logger.Debug($"Opened file {file.Name}"); } } client.Disconnect(); } break; } } finally { LoadRemoteSemaphoreSlim.Release(); } } }
private async Task <List <string> > GetRemoteFiles(RootPathObject rootPath) { var files = new List <string>(); if (rootPath.FileReadMode != Constants.FileModeLocal) { try { await GetRemoteSemaphoreSlim.WaitAsync(); switch (rootPath.FileReadMode) { case Constants.FileModeFtp: using (var client = Utility.GetFtpClient()) { var mask = new Regex(rootPath.Filter .Replace(".", "[.]") .Replace("*", ".*") .Replace("?", ".")); foreach (var item in await client.GetListingAsync(rootPath.RootPath)) { // if this is a file if (item.Type == FtpFileSystemObjectType.File && mask.IsMatch(item.Name)) { files.Add(item.FullName); Logger.Debug($"Added file {item.Name}"); } } await client.DisconnectAsync(); } break; case Constants.FileModeSftp: using (var client = Utility.GetSftpClient()) { var mask = new Regex(rootPath.Filter .Replace(".", "[.]") .Replace("*", ".*") .Replace("?", ".")); var allFiles = client.ListDirectory(rootPath.RootPath); var downloadFiles = allFiles.Where(f => f.IsDirectory == false && mask.IsMatch(f.Name)); foreach (var file in downloadFiles) { files.Add(file.FullName); Logger.Debug($"Added file {file.Name}"); } client.Disconnect(); } break; } } finally { GetRemoteSemaphoreSlim.Release(); } } return(files); }