public async Task Collect(List <string> suppliers, LocalityCollectionModes mode = LocalityCollectionModes.Incremental, CancellationToken cancellationToken = default) { var previousCollectionDates = await GetPreviousCollectionDate(suppliers, mode, cancellationToken); foreach (var(countryCode, countryId, countryName) in await GetCountries(cancellationToken)) { _logger.LogStartCheckingLocalityCandidates(countryCode); var countryLocalities = await GetLocalityNamesForCountry(countryCode, cancellationToken); var countryRawLocalities = await GetLocalityCandidatesFromRawAccommodations(countryCode, previousCollectionDates.Select(s => (s.Key, s.Value)).ToList(), cancellationToken); foreach (var supplier in suppliers) { var supplierValue = supplier; if (countryRawLocalities.TryGetValue(supplierValue, out var rawLocalityCandidateNames)) { var previousCollectionDate = previousCollectionDates[supplierValue]; var previousCollectionDateText = previousCollectionDate.ToString("O"); _logger.LogStartCollectingLocalityCandidates(supplier, countryCode, previousCollectionDateText); await CollectDefaultLocalitiesForSupplier(supplierValue, previousCollectionDate, rawLocalityCandidateNames, countryCode, countryId, countryName, countryLocalities, cancellationToken); _logger.LogFinishCollectingLocalityCandidates(supplier, countryCode, previousCollectionDateText); } } _logger.LogFinishCheckingLocalityCandidates(countryCode); } await AddToHistory(suppliers, cancellationToken); }
private async Task <Dictionary <string, DateTimeOffset> > GetPreviousCollectionDate(List <string> suppliers, LocalityCollectionModes mode, CancellationToken cancellationToken) { if (mode == LocalityCollectionModes.Full) { return(suppliers.ToDictionary(s => s, _ => default(DateTimeOffset))); } var previousCollectionDateStore = new Dictionary <string, DateTimeOffset>(suppliers.Count); foreach (var supplier in suppliers) { var supplierUpdateEvent = await _dbContext.DataUpdateHistories.OrderByDescending(h => h.UpdateTime) .FirstOrDefaultAsync(h => h.Supplier == supplier && h.Type == DataUpdateTypes.LocalityCollection, cancellationToken); previousCollectionDateStore.Add(supplier, supplierUpdateEvent?.UpdateTime ?? default); } return(previousCollectionDateStore); }