private void threadStartScrape() { List<string> lst_structTweetFollowersIDs = new List<string>(); List<string> lst_structTweetFollowingsIds = new List<string>(); GlobusHttpHelper globusHttpHelper = new GlobusHttpHelper(); string user_id = string.Empty; foreach (string keyword in lstscrapeUsername) { if (!GlobusRegex.ValidateNumber(keyword))//(!IsItNumber(user_id_toFollow)) { user_id = TwitterDataScrapper.GetUserIDFromUsername(keyword); } else { user_id = keyword; } TwitterDataScrapper dataScrapeer = new TwitterDataScrapper(); if (chkboxScrapeFollowers.Checked) { try { if (!File.Exists(Globals.Path_ScrapedFollowersList)) { GlobusFileHelper.AppendStringToTextfileNewLine("User_ID , FollowersUserID", Globals.Path_ScrapedFollowersList); } lst_structTweetFollowersIDs = dataScrapeer.GetFollowers(user_id); //lst_structTweetFollowingsIds = dataScrapeer.GetFollowings(user_id); AddToScrapeLogs("Added " + lst_structTweetFollowersIDs.Count + " Followers to list"); //AddToScrapeLogs("Added " + lst_structTweetFollowingsIds.Count + " Followings to list"); foreach (string data in lst_structTweetFollowersIDs) { try { Globals.lstScrapedUserIDs.Add(data); GlobusFileHelper.AppendStringToTextfileNewLine(user_id + "," + data, Globals.Path_ScrapedFollowersList); AddToScrapeLogs(data); } catch (Exception ex) { Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine(DateTime.Now + " --> Error --> btnScrapeKeyword_Click() -- lst_structTweetFollowersIDs foreach --> " + ex.Message, Globals.Path_ScrapeUsersErroLog); Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine("Error --> btnScrapeKeyword_Click() -- lst_structTweetFollowersIDs foreach --> " + ex.Message, Globals.Path_TwtErrorLogs); } } //AddToScrapeLogs("Added " + lst_structTweetFollowersIDs.Count + " Followers from User: "******"Data Exported to " + Globals.Path_ScrapedFollowersList); if (Globals.IsDirectedFromFollower) { AddToLog_Follower("Added " + lst_structTweetFollowersIDs.Count + " Followers from User: "******" --> Error --> btnScrapeUser_Click() -- chkboxScrapeFollowers.Checked --> " + ex.Message, Globals.Path_ScrapeUsersErroLog); Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine("Error --> btnScrapeUser_Click() -- chkboxScrapeFollowers.Checked --> " + ex.Message, Globals.Path_TwtErrorLogs); } } if (chkboxScrapeFollowings.Checked) { try { if (!File.Exists(Globals.Path_ScrapedFollowingsList)) { GlobusFileHelper.AppendStringToTextfileNewLine("User_ID , FollowingsUserID", Globals.Path_ScrapedFollowingsList); } lst_structTweetFollowingsIds = dataScrapeer.GetFollowings(user_id); AddToScrapeLogs("Added " + lst_structTweetFollowingsIds.Count + " Followings to list"); foreach (string data in lst_structTweetFollowingsIds) { try { Globals.lstScrapedUserIDs.Add(data); GlobusFileHelper.AppendStringToTextfileNewLine(user_id + "," + data, Globals.Path_ScrapedFollowingsList); AddToScrapeLogs(data); } catch (Exception ex) { Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine(DateTime.Now + " --> Error --> btnScrapeKeyword_Click() -- lst_structTweetFollowingsIds foreach --> " + ex.Message, Globals.Path_ScrapeUsersErroLog); Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine("Error --> btnScrapeKeyword_Click() -- lst_structTweetFollowingsIds foreach --> " + ex.Message, Globals.Path_TwtErrorLogs); } } AddToScrapeLogs("Added " + lst_structTweetFollowingsIds.Count + " Followings from User: "******"Data Exported to " + Globals.Path_ScrapedFollowingsList); if (Globals.IsDirectedFromFollower) { AddToLog_Follower("Added " + lst_structTweetFollowingsIds.Count + " Followings from User: "******" --> Error --> btnScrapeKeyword_Click() -- lst_structTweetFollowingsIds foreach --> " + ex.Message, Globals.Path_ScrapeUsersErroLog); Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine("Error --> btnScrapeKeyword_Click() -- lst_structTweetFollowingsIds foreach --> " + ex.Message, Globals.Path_TwtErrorLogs); } } Globals.lstScrapedUserIDs = Globals.lstScrapedUserIDs.Distinct().ToList(); ////new Thread(() => ////{ //foreach (string data in lst_structTweetFollowersIDs) //{ // try // { // clsDBQueryManager DataBase = new clsDBQueryManager(); // DataBase.InsertOrUpdateScrapeSetting(data, ""); // } // catch (Exception ex) // { // } //} //// } ////).Start(); //AddToScrapeLogs("Added " + lst_structTweetFollowingsIds.Count + " Followings from User: "******"" , ""); } catch (Exception ex) { Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine(DateTime.Now + " --> Error --> lstScrapedUserIDs --> " + ex.Message, Globals.Path_ScrapeUsersErroLog); Globussoft.GlobusFileHelper.AppendStringToTextfileNewLine("Error --> lstScrapedUserIDs --> " + ex.Message, Globals.Path_TwtErrorLogs); } } } ).Start(); }
public List<string> GetFollowingsUsingUserID(string userID) { List<string> list_Followings = new List<string>(); TwitterDataScrapper followingScrapper = new TwitterDataScrapper(); list_Followings = followingScrapper.GetFollowings(userID); return list_Followings; }
public List<string> GetFollowingsUsingUserID(string userID) { List<string> list_Followings = new List<string>(); TwitterDataScrapper followingScrapper = new TwitterDataScrapper(); string returnStatus = string.Empty; list_Followings = followingScrapper.GetFollowings(userID, out returnStatus); return list_Followings; }