/* UPDATE ZLASTSYNC set lastSync = "23/01/2020 4:49:41 AM" * where Mapping_ID not in (SELECT ID FROM ZMAPPING WHERE TYPE IN ("API") ); * DELETE FROM ZQUEUE; */ public static bool RunSQLLiteCmd(string token, Zmapping cmd) { using (var db = new ZudelloContext()) { try { //Exceute the custom command db.Database.ExecuteSqlCommand(cmd.Body); db.SaveChanges(); //Remove custom command from database FK var lastSyncQuery = db.Zlastsync.Where(z => z.MappingId == cmd.Id).FirstOrDefault(); var mappingTbl = db.Zmapping.Where(z => z.uuid == cmd.uuid); foreach (Zmapping sqlQuery in mappingTbl) { //Dont need to remove as its never added into sync table // db.Remove(lastSyncQuery); // db.SaveChanges(); db.Remove(sqlQuery); db.SaveChanges(); //Send delete request to Zudello to remove from mappings table Console.WriteLine(ZudelloLogin.DeleteMapping(token, cmd.uuid)); } return(true); } catch (Exception ex) { return(false); } } }
public static async Task SetSyncTable() { using (var db = new ZudelloContext()) { var toSet = (from zm in db.Zmapping join zs in db.Zlastsync on zm.Id equals zs.MappingId into r from zs in r.DefaultIfEmpty() where zs == null select zm).ToList(); foreach (var s in toSet) { Zlastsync lastSyncTable = new Zlastsync(); lastSyncTable.MappingId = s.Id; if (s.Type == "API") { lastSyncTable.LastSync = "2000-11-20T05:55:29.000Z"; } else { lastSyncTable.LastSync = DateTime.UtcNow.AddYears(-100).ToString(); } db.Add(lastSyncTable); db.SaveChanges(); Console.WriteLine(String.Format("Sync Table Updated:{0}, {1} ", s.DocType, s.Type)); } } }
public static void SaveToDB(string kfiName, string jsonBody) { using (var db = new ZudelloContext()) { //db.Files.Add() // db.Zfiles.Add(new Zfiles { JsonData = jsonBody }); db.SaveChanges(); } }
public static void SaveToDBQueue(string rawJson, int type, string status, GpMappingAndDatabase map) { using (var db = new ZudelloContext()) { db.Zqueue.Add(new Zqueue { Body = rawJson, MappingId = type, Status = status, ConnectionId = map.Id }); db.SaveChanges(); } }
public static async Task AttacheResponseSender() { string token = ZudelloLogin.Login(); ResponseZudello ResponseStatus = new ResponseZudello(); using (var db = new ZudelloContext()) { var queueList = db.Zqueue.Where(i => i.Status == "Success" && (i.ResponseSent == 0)).ToList(); try { foreach (var l in queueList) { ResponseObject successList = new ResponseObject(); dynamic Body = JObject.Parse(l.Body); successList.body = "Success"; successList.uuid = Body.uuid; successList.queueid = l.Id; successList.status = 200; ResponseStatus.data.Add(successList); } } catch (Exception ex) { Console.WriteLine(ex.Message); Console.WriteLine("Master Data"); } foreach (var data in ResponseStatus.data) { string responseBody = JsonConvert.SerializeObject(data); HttpStatusCode Response = ZudelloLogin.SendProcessResponse(token, responseBody); if (Response == HttpStatusCode.OK) { var updateSent = db.Zqueue.Where(i => i.Id == data.queueid).FirstOrDefault(); updateSent.ResponseSent = 1; db.SaveChanges(); } } db.Dispose(); } }
public static bool updateQueue(string queueId, string status) { try { int id = 0; if (Int32.TryParse(queueId, out id)) { using (var db = new ZudelloContext()) { var queueUpdate = db.Zqueue.Where(i => i.Id == id).FirstOrDefault(); queueUpdate.Status = status; db.SaveChanges(); return(true); } } return(false); } catch (Exception ex) { return(false); } }
public static int?SaveToDBQueue(string rawJson, int type, string status, ExoMappingAndDB map, int?linkedId = null) { try { using (var db = new ZudelloContext()) { var res = db.Zqueue.Add(new Zqueue { Body = rawJson, MappingId = type, Status = status, ConnectionId = map.Id, Queue_Id = linkedId }); db.SaveChanges(); db.DisposeAsync(); return(res.Entity.Id); } } catch (Exception ex) { return(null); } }
public static bool Hash(string Json, int mappingId) { string sSourceData; byte[] tmpSource; byte[] tmpHash; bool bEqual = false; sSourceData = Json; //Create a byte array from source data tmpSource = ASCIIEncoding.ASCII.GetBytes(sSourceData); //Compute hash based on source data tmpHash = new MD5CryptoServiceProvider().ComputeHash(tmpSource); List <byte[]> sent = new List <byte[]>(); using (var db = new ZudelloContext()) { var HashLog = (from a in db.Zhashlog where a.MappingId == mappingId select a); Zhashlog h = new Zhashlog(); if (HashLog.Count() < 1) { h.MappingId = mappingId; h.Hash = tmpHash; db.Add(h); db.SaveChanges(); } else { foreach (var hash in HashLog) { sent.Add(hash.Hash); } foreach (var hsh in sent) { if (hsh.Length == tmpHash.Length) { int i = 0; while ((i < hsh.Length) && (hsh[i] == tmpHash[i])) { i += 1; } if (i == hsh.Length) { bEqual = true; } } } if (bEqual == false) { Console.WriteLine("The two hash values are not the same"); Zhashlog ha = new Zhashlog(); ha.MappingId = mappingId; ha.Hash = tmpHash; db.Add(ha); db.SaveChanges(); Console.WriteLine("hash Added"); return(bEqual); } else { Console.WriteLine("Values are the same"); return(bEqual); } } } return(bEqual); }
public static List <ErrorData> KfiErrorGetter() { List <ErrorData> errList = new List <ErrorData>(); try { using (var db = new ZudelloContext()) { //Get a list of all failed items. var failureQueue = db.Zqueue.Where(i => i.Status == "Failure").Select(s => s.Id).ToList(); //Get a List of all .eer Files AttacheConfiguration FolderNames = ZudelloSetup.GetAttacheSettings(); var Fdir = FolderNames.AttacheInbox + "Failure"; DirectoryInfo d = new DirectoryInfo(Fdir); FileInfo[] Files = d.GetFiles("*.err"); List <string> err = new List <string>(); foreach (FileInfo file in Files) { ErrorData errData = new ErrorData(); //Check if failed item is part of queue try { int index1 = file.FullName.LastIndexOf('_'); string queueId = Regex.Match(file.FullName.Substring(index1), @"\d+").Value; // Convert to int to check int id = 0; Int32.TryParse(queueId, out id); if (failureQueue.Contains(id)) { //Send to Zudello this error File. errData.errorFile = File.ReadAllText(file.FullName); // Console.WriteLine(File.ReadAllText(file.FullName)); FileInfo[] originalKfi = d.GetFiles(String.Format("*_{0}.kfi", id.ToString())); //Get the Original KFI //Console.WriteLine(File.ReadAllText(originalKfi.FirstOrDefault().FullName)); try { errData.originalFile = File.ReadAllText(originalKfi.FirstOrDefault().FullName); // Console.ReadLine(); } catch (Exception ex) { //If we cannot locate original errData.originalFile = ex.Message; } //Bool check to confirm error file was sent to zudello. var updateQuery = db.Zqueue.Where(i => i.Id == id).FirstOrDefault(); errData.body = JsonConvert.DeserializeObject(updateQuery.Body); updateQuery.Status = "ZudelloReview"; db.SaveChanges(); errList.Add(errData); } else { //Next Item already been sent to Zudello continue; } } catch (Exception ex) { return(errList); } } return(errList); } } catch (Exception ex) { return(errList); } }
public static async Task ProcessRecords() { AttacheConfiguration Folder = ZudelloSetup.GetAttacheSettings(); using (var db = new ZudelloContext()) { List <int> Success = new List <int>(); var MasterDataAwaiting = (from a in db.Zqueue join c in db.Zmapping on a.MappingId equals c.Id where c.IsMasterData == 1 && a.Status == "Processing" select a).Count(); //The Cleaner Should be checking if these are in which folders if (MasterDataAwaiting > 0) { //run procedure to check folders return; } if (MasterDataAwaiting == 0) { //Get All Master Data in waiting status var MasterData = (from queue in db.Zqueue join map in db.Zmapping on queue.MappingId equals map.Id where map.IsMasterData == 1 && queue.Status.Trim() == "Waiting" //Will make more generic later select new { queue, map }).ToList(); if (MasterData.Count() > 0) { foreach (var queueData in MasterData) { bool success = ProcessMethod(queueData); if (success == true) { //Mark Record as processing queueData.queue.Status = "Processing"; db.SaveChanges(); } } return; //exit } } //Process transactional Data var TransData = (from queue in db.Zqueue join map in db.Zmapping on queue.MappingId equals map.Id where map.IsMasterData == 0 && queue.Status.Trim() == "Waiting" //Will make more generic later select new { queue, map }).ToList(); foreach (var queueData in TransData) { bool success = ProcessMethod(queueData); if (success == true) { queueData.queue.Status = "Processing"; db.SaveChanges(); } } } }
public static async Task AddtoQueue() { //Check Mappings ZudelloLogin.GetZudelloMappings().Wait(); string Token = ZudelloLogin.Login(); List <Zconnections> teams = new List <Zconnections>(); using (var db = new ZudelloContext()) { //Maybe hash the records teams = db.Zconnections.ToList(); await db.DisposeAsync(); } int numberOfLogicalCores = Environment.ProcessorCount; List <Thread> threads = new List <Thread>(numberOfLogicalCores); int sizeOfOneChunk = (teams.Count / numberOfLogicalCores) + 1; ConcurrentDictionary <int, dynamic> cb = new ConcurrentDictionary <int, dynamic>(); ConcurrentDictionary <dynamic, LastSync> lastSyncBag = new ConcurrentDictionary <dynamic, LastSync>(); for (int i = 0; i < numberOfLogicalCores; i++) { int ab = i; var thr = new Thread( () => { int count = 0; List <Zconnections> teamChunked = teams.Skip(ab * sizeOfOneChunk) .Take(sizeOfOneChunk).ToList(); foreach (var team in teamChunked) { try { dynamic ZudelloTeam = JsonConvert.DeserializeObject <ExpandoObject>(team.ZudelloCredentials); string InvoiceList = ZudelloLogin.CallZudelloDocs(Token, ZudelloTeam.team, team.Id); dynamic ToSync = JObject.Parse(InvoiceList); foreach (var data in ToSync.data) { try { string myDataString = data.ToString(); // Console.WriteLine(Id.uuid + "," + Id.items); Console.WriteLine(data); ExoMappingAndDB mapId = GetMappingAndDatabase(data.document.docType.ToString(), data.document.connectionUuid.ToString()); int?rowId = null; if (mapId != null) { rowId = SaveToDBQueue(myDataString, mapId.mappingID, "Waiting", mapId); } var validate = Validations.ExoSupplierInvoice(myDataString, mapId); if (validate.CreateSupplier == true) { //HAVE to hard code type for the supplier and inventory? mapId = GetMappingAndDatabase(data.document.supplier.docType.ToString(), data.document.connectionUuid.ToString()); // Check if already in the queue if (InQueue(mapId.mappingID, data.document.supplier.code.ToString()) == false) { SaveToDBQueue(data.document.supplier.ToString(), mapId.mappingID, "Waiting", mapId, rowId); } } if (validate.CreateInventory == true) { foreach (var itemData in data.document.lines) { mapId = GetMappingAndDatabase(itemData.docType.ToString(), data.document.connectionUuid.ToString()); if (validate.InventoryToCreate.Contains(itemData.item.sku.ToString())) { // Check if already in the queue if (InQueue(mapId.mappingID, itemData.item.sku.ToString()) == false) { SaveToDBQueue(itemData.ToString(), mapId.mappingID, "Waiting", mapId, rowId); } } } } LastSync lastSync = new LastSync(); lastSync.teamId = team.Id; lastSync.lastUpdated = data.created_at.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'"); lastSyncBag.TryAdd(data.uuid, lastSync); } catch (Exception ex) { Console.WriteLine(ex.Message); } } } catch (Exception ex) { //Write to loggings tables Console.WriteLine(ex.Message); } } } ); threads.Add(thr); } foreach (var thread in threads) { thread.Start(); } foreach (var thread in threads) { thread.Join(); } var dataList = lastSyncBag.Values.ToList(); foreach (var lastSync in dataList) { using (var db = new ZudelloContext()) { var updateLastSync = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.DocType == "CallZudelloDocs" && c.connection_id == lastSync.teamId select a).FirstOrDefault(); if (DateTime.Parse(updateLastSync.LastSync) > DateTime.Parse(lastSync.lastUpdated)) { } else { updateLastSync.LastSync = lastSync.lastUpdated; db.SaveChanges(); } db.Dispose(); } } }
public static async Task GetZudelloMappings(string team = null) { string Token = ZudelloLogin.Login(); //Output object should be this List <Zconnections> teams = new List <Zconnections>(); using (var db = new ZudelloContext()) { //Maybe hash the records teams = db.Zconnections.ToList(); await db.DisposeAsync(); } int numberOfLogicalCores = Environment.ProcessorCount; List <Thread> threads = new List <Thread>(numberOfLogicalCores); int sizeOfOneChunk = (teams.Count / numberOfLogicalCores) + 1; ConcurrentBag <Zmapping> cb = new ConcurrentBag <Zmapping>(); ConcurrentDictionary <int, dynamic> cbd = new ConcurrentDictionary <int, dynamic>(); for (int i = 0; i < numberOfLogicalCores; i++) { int ab = i; var thr = new Thread( () => { int count = 0; List <Zconnections> teamChunked = teams.Skip(ab * sizeOfOneChunk) .Take(sizeOfOneChunk).ToList(); foreach (var t in teamChunked) { dynamic ZudelloTeam = JsonConvert.DeserializeObject <ExpandoObject>(t.ZudelloCredentials); bool newMappings = false; string GetMapping = ZudelloLogin.CallZudelloMapping(Token, ZudelloTeam.team); ZmappingJson MappingFiles = JsonConvert.DeserializeObject <ZmappingJson>(GetMapping); dynamic zudelloObject = JObject.Parse(GetMapping); foreach (Zmapping map in MappingFiles.data) { Console.WriteLine(map.UpdatedAt); cb.Add(map); } } } ); threads.Add(thr); } foreach (var thread in threads) { thread.Start(); } foreach (var thread in threads) { thread.Join(); } foreach (var map in cb) { bool newMappings = false; using (var db = new ZudelloContext()) { try { Zmapping AddtoMap = map; Zmapping MapID = db.Zmapping.Where(i => i.DocType == map.DocType && i.Type == map.Type && i.connection_id == map.connection_id).FirstOrDefault(); if (MapID.Id > 0) { MapID.Body = map.Body; //= map.ShallowCopy(); MapID.IsOutgoing = map.IsOutgoing; MapID.ProcessOrder = map.ProcessOrder; MapID.IsMasterData = map.IsMasterData; MapID.connection_id = map.connection_id; MapID.Type = map.Type; MapID.IntergrationUuid = map.IntergrationUuid; // MapID.uuid = map.uuid; db.Update(MapID); db.SaveChanges(); var updateLastSync = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.DocType == "CallZudelloMapping" select a).FirstOrDefault(); string pr = String.Format(DateTime.UtcNow.ToString(), "yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'"); updateLastSync.LastSync = pr; // db.SaveChanges(); } else { //add to mapping if it does not exist. db.Zmapping.Add(AddtoMap); db.SaveChanges(); Console.WriteLine(String.Format("Doc Type {0} has been added: Type {1} has been added", map.DocType, map.Type)); newMappings = true; } } catch (Exception ex) { Console.WriteLine(ex.Message);//some error with mappings try { Zmapping AddtoMap = map; db.Zmapping.Add(AddtoMap); db.SaveChanges(); Console.WriteLine(String.Format("Doc Type {0} has been added: Type {1} has been added", map.DocType, map.Type)); newMappings = true; } catch (Exception exm) { Console.WriteLine(exm.Message); } finally { if (map.Type == "SQL_LITE_CMD") { Console.WriteLine(Tools.RunSQLLiteCmd(Token, map)); } } } finally { if (map.Type == "SQL_LITE_CMD") { Console.WriteLine(Tools.RunSQLLiteCmd(Token, map)); } } db.DisposeAsync(); } if (newMappings == true) { try //Add into sync table if there are any new records { SetSyncTable().Wait(); } catch (Exception ex) { Console.WriteLine(ex.Message); } } } }
public static string CallZudelloSettings(string token, string team = "") { string result = ""; string website = ""; string key = ""; string apiurl; string ApiResult = ""; string entity = @"https://api.zudello.com/api/thinclient/attache/settings"; // work out API call apiurl = website + entity; // do the api call ServicePointManager.ServerCertificateValidationCallback = delegate { return(true); }; var request = (HttpWebRequest)WebRequest.Create(apiurl); // var data = Encoding.ASCII.GetBytes(apibody); request.Method = "GET"; //request.ContentLength = data.Length; request.ContentType = "application/json"; request.Accept = "application/json"; string platform = "WEBSITE"; request.Headers.Add("x-team", team); request.Headers.Add("x-platform", platform); request.Headers.Add("Authorization", "Bearer " + token); var response = (HttpWebResponse)request.GetResponse(); var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd(); if (response.StatusCode == HttpStatusCode.OK) { result = responseString; try { using (var db = new ZudelloContext()) { var updateLastSync = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.DocType == "CallZudelloSettings" select a).FirstOrDefault(); updateLastSync.LastSync = DateTime.UtcNow.ToString(); db.SaveChanges(); } } catch (Exception ex) { } } else { result = responseString; } return(result); }
public static string SendToZudelloQueue(string token, string body, string myTeam = null) { string result = ""; string website = ""; string key = ""; string apiurl; string ApiResult = ""; string team = ""; string entity = ZudelloInitalSettings.GetZudelloSettings().ZudelloQueue; // work out API call //Multiple Database Teams if (myTeam == null) { team = ZudelloInitalSettings.GetZudelloSettings().ZudelloTeam; } else { team = myTeam; } apiurl = entity + team + "/queueforwarding"; // do the api call ServicePointManager.ServerCertificateValidationCallback = delegate { return(true); }; var request = (HttpWebRequest)WebRequest.Create(apiurl); var data = Encoding.ASCII.GetBytes(body); request.Method = "POST"; request.ContentLength = data.Length; request.ContentType = "application/json"; request.Accept = "application/json"; request.Headers.Add("x-team", team); request.Headers.Add("Authorization", "Bearer " + token); using (var stream = request.GetRequestStream()) { stream.Write(data, 0, data.Length); } var response = (HttpWebResponse)request.GetResponse(); var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd(); if (response.StatusCode == HttpStatusCode.OK) { result = responseString; using (var db = new ZudelloContext()) { var updateLastSync = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.DocType == "SendToZudelloQueue" select a).FirstOrDefault(); updateLastSync.LastSync = DateTime.UtcNow.ToString(); db.SaveChanges(); } } else { result = responseString; } return(result); }
public static async Task PushDataToZudello() { string Token = ZudelloLogin.Login(); foreach (var Mappings in getBody()) { string myReplacement = ""; string myObjType = Mappings.Value.DocType; dynamic myObj = JsonConvert.DeserializeObject <ExpandoObject>(Mappings.Value.Body); string myLinkValue = myObj.LINK.ToString(); string myQuery = myObj.HDR_QUERY.ToString(); if (myQuery.Contains("{LAST_ID}")) { try { using (var db = new ZudelloContext()) { var lastSyncValue = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.Id == Mappings.Value.Id select a).FirstOrDefault(); myQuery = myQuery.Replace("{LAST_ID}", lastSyncValue.lastID.ToString()); } } catch (Exception ex) { Console.WriteLine(ex.Message); } } if (myQuery.Contains("{LAST_SYNC}")) { try { using (var db = new ZudelloContext()) { var lastSyncValue = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.Id == Mappings.Value.Id select a).FirstOrDefault(); DateTime MyQueryDate = DateTime.Parse(lastSyncValue.LastSync); myQuery = myQuery.Replace("{LAST_SYNC}", MyQueryDate.ToString("yyyy-MM-dd HHmmss")); } } catch { } } var headerDs = AttacheFetchData.GetDataSet(myQuery); try { myReplacement = myObj.REPLACE_ME.ToString(); } catch { } bool LineQuery = false; if (myLinkValue != "") { LineQuery = true; } foreach (DataTable dt in headerDs.Tables) { foreach (DataRow dr in dt.Rows) { Data QData = new Data(); Output output = new Output(); QData.object_type = myObjType; //dynamic obj = new DynamicClass(fields); Dictionary <string, object> temp = new Dictionary <string, object>(); foreach (DataColumn dc in dt.Columns) { dynamic Colvalue; if (dr[dc.ColumnName] is String) { Colvalue = dr[dc.ColumnName].ToString(); Colvalue = Colvalue.Trim(); } else { Colvalue = dr[dc.ColumnName]; } temp.Add(dc.ColumnName, Colvalue); } QData.data = temp; output.data = QData; if (LineQuery == true) { var query = temp[myLinkValue]; // output.data.data = temp; string myQuery2 = myObj.LINE_QUERY.ToString().Replace("{0}", query.ToString()); var lineDs = GetDataSet(myQuery2); List <object> mArray = new List <object>(); foreach (DataTable lt in lineDs.Tables) { foreach (DataRow lr in lt.Rows) { Dictionary <string, object> lstemp = new Dictionary <string, object>(); foreach (DataColumn lc in lt.Columns) { dynamic lineValue; if (lr[lc.ColumnName] is String) { lineValue = lr[lc.ColumnName].ToString(); lineValue = lineValue.Trim(); } else { lineValue = lr[lc.ColumnName]; } lstemp.Add(lc.ColumnName, lineValue); } mArray.Add(lstemp); } } Dictionary <int, object> myDic = new Dictionary <int, object>(); object REPLACEME = new object(); string data = JsonConvert.SerializeObject(output); dynamic obj = JsonConvert.DeserializeObject <ExpandoObject>(data); obj.data.data.REPLACEME = mArray; string lineData = JsonConvert.SerializeObject(obj); lineData = lineData.Replace("REPLACEME", myReplacement); Console.WriteLine(lineData); //If this record already exisits in the hashing table then continue with the loop. if (HashingRecords.Hash(lineData, Mappings.Value.Id) == true) { continue; } Console.WriteLine(ZudelloLogin.SendToZudelloQueue(Token, lineData)); using (var db = new ZudelloContext()) { var updateLastSync = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.Id == Mappings.Value.Id select a).FirstOrDefault(); updateLastSync.LastSync = DateTime.UtcNow.ToString(); try { int myInternalId = Convert.ToInt32(temp[myLinkValue]); updateLastSync.lastID = myInternalId; } catch { } db.SaveChanges(); } } else { string data = JsonConvert.SerializeObject(output); Console.WriteLine(data); //If this record already exisits in the hashing table then continue with the loop. if (HashingRecords.Hash(data, Mappings.Value.Id) == true) { continue; } Console.WriteLine(ZudelloLogin.SendToZudelloQueue(Token, data)); using (var db = new ZudelloContext()) { var updateLastSync = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.Id == Mappings.Value.Id select a).FirstOrDefault(); updateLastSync.LastSync = DateTime.UtcNow.ToString(); db.SaveChanges(); } } } } } }
public static async Task PushExoDataToZudello() { string Token = ZudelloLogin.Login(); SQLCredentials ConnectionString = new SQLCredentials(); Dictionary <int, string> Connection = ConnectionString.ConnectionStringBuilder(); /**** * * Add threading into here for the foreach loop * will need to chunk based on amount of connections */ //get the query from mappings table Dictionary <int, Zmapping> MappingsBody = getBody(); int numberOfLogicalCores = Environment.ProcessorCount; List <Thread> threads = new List <Thread>(numberOfLogicalCores); int sizeOfOneChunk = (MappingsBody.Count / numberOfLogicalCores) + 1; ConcurrentBag <string> cb = new ConcurrentBag <string>(); ConcurrentBag <int> cbMaps = new ConcurrentBag <int>(); for (int i = 0; i < numberOfLogicalCores; i++) { int ab = i; var thr = new Thread( () => { try { int count = 0; Dictionary <int, Zmapping> MappingsChunked = MappingsBody.Skip(ab * sizeOfOneChunk) .Take(sizeOfOneChunk).ToDictionary(p => p.Key, p => p.Value); foreach (var Mappings in MappingsChunked) { bool isXMl = true; //Get the SQL statement string mySql = Mappings.Value.Body; //Get the Mappings dynamic zudelloObject = Mappings.Value; string SQLQuery = ExoTools.RenderToSql(mySql, zudelloObject); #warning make this in config also for process method string cmd = SQLQuery; //Open SQL connection and run the SQL query //Get connection details by Key ID SqlConnection con = new SqlConnection(Connection[Mappings.Value.connection_id]); SqlCommand SelectCommand = new SqlCommand(cmd, con); // SqlDataReader myreader; con.Open(); if (cmd.ToLower().Contains("for xml")) { isXMl = true; } if (String.IsNullOrEmpty(cmd)) { continue; } var jsonResult = new StringBuilder(); var xmlConvert = new StringBuilder(); //SelectCommand.CommandText = cmd; var myreader = SelectCommand.ExecuteReader(); if (!myreader.HasRows) { //if there is no data then close connection and next loop con.Close(); con.Dispose(); continue; } else { while (myreader.Read()) { jsonResult.Append(myreader.GetValue(0).ToString()); } //Console.WriteLine(jsonResult); dynamic obj = ""; string ConvertedXml = ""; if (isXMl == true) { ConvertedXml = ExoTools.ZudelloXMLConverter(jsonResult.ToString()); // string ConvertedXml = ExoTools.ZudelloXMLConverter(jsonResult.ToString()); obj = JsonConvert.DeserializeObject <ExpandoObject>(ConvertedXml); } else { obj = JsonConvert.DeserializeObject <ExpandoObject>(jsonResult.ToString()); } foreach (var ObjectType in obj) { //not sure why .Value is causing error but not affecting build foreach (var dataValues in ObjectType.Value) { string data = ""; if (isXMl == true) { data = JsonConvert.SerializeObject(dataValues); } else { MyQueueObject dataWrapper = new MyQueueObject(); dataWrapper.data = dataValues; data = JsonConvert.SerializeObject(dataWrapper); } // add to concurrent bag cb.Add(data); count++; // Console.WriteLine(data); } } cbMaps.Add(Mappings.Value.Id); } con.Close(); con.Dispose(); } } catch (Exception ex) { } } ); threads.Add(thr); } foreach (var thread in threads) { thread.Start(); } foreach (var thread in threads) { thread.Join(); } threads.Clear(); int sizeOfOneChunkQueue = (cb.Count / numberOfLogicalCores) + 1; for (int x = 0; x < numberOfLogicalCores; x++) { int abx = x; var SendToQueuethr = new Thread( () => { try { int count = 0; List <string> queuedBag = cb.Skip(abx * sizeOfOneChunkQueue) .Take(sizeOfOneChunkQueue).ToList(); foreach (var Mappings in queuedBag) { Console.WriteLine(ZudelloLogin.SendToZudelloQueue(Token, Mappings)); } } catch (Exception ex) { } }); threads.Add(SendToQueuethr); } foreach (var qthread in threads) { qthread.Start(); } foreach (var qthread in threads) { qthread.Join(); } /* * List<Task> bagConsumeTasks = new List<Task>(); * int itemsInBag = 0; * while (!cb.IsEmpty) * { * bagConsumeTasks.Add(Task.Run(() => * { * string item; * if (cb.TryTake(out item)) * { * * Console.WriteLine(ZudelloLogin.SendToZudelloQueue(Token, item)); * itemsInBag++; * } * })); * } */ // Task.WaitAll(bagConsumeTasks.ToArray()); foreach (int id in cbMaps) { using (var db = new ZudelloContext()) { var lastSync = db.Zlastsync.Where(s => s.MappingId == id).FirstOrDefault(); lastSync.LastSync = DateTime.Now.ToString(); db.SaveChanges(); db.DisposeAsync(); } } }
public static void PushGpDataToZudello() { string Token = ZudelloLogin.Login(); SQLCredentials ConnectionString = new SQLCredentials(); Dictionary <int, string> Connection = ConnectionString.ConnectionStringBuilder(); /**** * * Add threading into here for the foreach loop * will need to chunk based on amount of connections */ //get the query from mappings table foreach (var Mappings in getBody()) { //bool isXml = false; //Get the SQL statement string mySql = Mappings.Value.Body; //Get the Mappings dynamic zudelloObject = Mappings.Value; string SQLQuery = GpTools.RenderToSql(mySql, zudelloObject); //if (mySql.Contains("XML PATH")) isXml = true; #warning make this in config also for process method string cmd = SQLQuery; //Open SQL connection and run the SQL query //Get connection details by Key ID SqlConnection con = new SqlConnection(Connection[Mappings.Value.connection_id]); SqlCommand SelectCommand = new SqlCommand(cmd, con); // SqlDataReader myreader; con.Open(); var jsonResult = new StringBuilder(); var myreader = SelectCommand.ExecuteReader(); if (!myreader.HasRows) { //if there is no data then close connection and next loop con.Close(); continue; } else { while (myreader.Read()) { jsonResult.Append(myreader.GetValue(0).ToString()); } dynamic obj; //Console.WriteLine(jsonResult); obj = JsonConvert.DeserializeObject <ExpandoObject>(jsonResult.ToString()); foreach (var ObjectType in obj) { foreach (var dataValues in ObjectType.Value) { MyQueueObject dataWrapper = new MyQueueObject(); dataWrapper.data = dataValues; string data = JsonConvert.SerializeObject(dataWrapper); Console.WriteLine(data); Console.WriteLine(ZudelloLogin.SendToZudelloQueue(Token, data)); } } //Update last sync time for this mapping ID using (var db = new ZudelloContext()) { var lastSync = db.Zlastsync.Where(s => s.MappingId == Mappings.Value.Id).FirstOrDefault(); lastSync.LastSync = DateTime.Now.ToString(); db.SaveChanges(); } } con.Close(); } }
public static void AddtoQueue() { //Check Mappings ZudelloLogin.GetZudelloMappings(); string Token = ZudelloLogin.Login(); string InvoiceList = ZudelloLogin.CallZudelloDocs(Token); dynamic ToSync = JObject.Parse(InvoiceList); foreach (var data in ToSync.data) { try { string myDataString = data.ToString(); // Console.WriteLine(Id.uuid + "," + Id.items); Console.WriteLine(data); GpMappingAndDatabase mapId = GetMappingAndDatabase(data.document.docType.ToString(), data.document.connectionUuid.ToString()); if (mapId != null) { SaveToDBQueue(myDataString, mapId.mappingID, "Waiting", mapId); } var validate = GpValidations.SupplierInvoiceValidate(myDataString, mapId); if (validate.CreateSupplier == true) { //HAVE to hard code type for the supplier and inventory? mapId = GetMappingAndDatabase(data.document.supplier.docType.ToString(), data.document.connectionUuid.ToString()); // Check if already in the queue if (InQueue(mapId.mappingID, data.document.supplier.code.ToString()) == false) { SaveToDBQueue(data.document.supplier.ToString(), mapId.mappingID, "Waiting", mapId); } } if (validate.CreateInventory == true) { foreach (var itemData in data.document.lines) { mapId = GetMappingAndDatabase(itemData.docType.ToString(), data.document.connectionUuid.ToString()); if (validate.InventoryToCreate.Contains(itemData.item.sku.ToString())) { // Check if already in the queue if (InQueue(mapId.mappingID, itemData.item.sku.ToString()) == false) { SaveToDBQueue(itemData.ToString(), mapId.mappingID, "Waiting", mapId); } } } } using (var db = new ZudelloContext()) { var updateLastSync = (from a in db.Zlastsync join c in db.Zmapping on a.MappingId equals c.Id where c.DocType == "CallZudelloDocs" select a).FirstOrDefault(); //var dateParse = DateTime.ParseExact(data.created_at.ToString(), "yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'", CultureInfo.InvariantCulture); //string zudelloDate = data.created_at; // dateParse = dateParse string pr = data.created_at.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'"); updateLastSync.LastSync = pr; db.SaveChanges(); } } catch (Exception ex) { Console.WriteLine(ex.Message); } } }