public static List <MSBuildOutputFile> GetMSBuildOutputFiles(PackageSpec project,
LockFile assetsFile,
IEnumerable <RestoreTargetGraph> targetGraphs,
IReadOnlyList <NuGetv3LocalRepository> repositories,
RestoreRequest request,
string assetsFilePath,
bool restoreSuccess,
ILogger log)
{
// Generate file names
var targetsPath = GetMSBuildFilePath(project, request, "targets");
var propsPath = GetMSBuildFilePath(project, request, "props");
// Targets files contain a macro for the repository root. If only the user package folder was used
// allow a replacement. If fallback folders were used the macro cannot be applied.
// Do not use macros for fallback folders. Use only the first repository which is the user folder.
var repositoryRoot = repositories.First().RepositoryRoot;
// Invalid msbuild projects should write out an msbuild error target
if (!targetGraphs.Any())
{
return(GenerateMultiTargetFailureFiles(
targetsPath,
propsPath,
request.ProjectStyle));
}
// Add additional conditionals for multi targeting
var multiTargetingFromMetadata = (request.Project.RestoreMetadata?.CrossTargeting == true);
var isMultiTargeting = multiTargetingFromMetadata ||
request.Project.TargetFrameworks.Count > 1;
// ItemGroups for each file.
var props = new List <MSBuildRestoreItemGroup>();
var targets = new List <MSBuildRestoreItemGroup>();
// MultiTargeting imports are shared between TFMs, to avoid
// duplicate import warnings only add each once.
var multiTargetingImportsAdded = new HashSet <string>(StringComparer.OrdinalIgnoreCase);
// Skip runtime graphs, msbuild targets may not come from RID specific packages.
var ridlessTargets = assetsFile.Targets
.Where(e => string.IsNullOrEmpty(e.RuntimeIdentifier));
foreach (var ridlessTarget in ridlessTargets)
{
// There could be multiple string matches from the MSBuild project.
var frameworkConditions = GetMatchingFrameworkStrings(project, ridlessTarget.TargetFramework)
.Select(match => string.Format(CultureInfo.InvariantCulture, TargetFrameworkCondition, match))
.ToArray();
// Find matching target in the original target graphs.
var targetGraph = targetGraphs.FirstOrDefault(e =>
string.IsNullOrEmpty(e.RuntimeIdentifier) &&
ridlessTarget.TargetFramework == e.Framework);
// Sort by dependency order, child package assets should appear higher in the
// msbuild targets and props files so that parents can depend on them.
var sortedGraph = TopologicalSortUtility.SortPackagesByDependencyOrder(ConvertToPackageDependencyInfo(targetGraph.Flattened));
// Filter out to packages only, exclude projects.
var packageType = new HashSet <string>(
targetGraph.Flattened.Where(e => e.Key.Type == LibraryType.Package)
.Select(e => e.Key.Name),
StringComparer.OrdinalIgnoreCase);
// Package -> PackageInfo
// PackageInfo is kept lazy to avoid hitting the disk for packages
// with no relevant assets.
var sortedPackages = sortedGraph.Where(e => packageType.Contains(e.Id))
.Select(sortedPkg =>
new KeyValuePair <LockFileTargetLibrary, Lazy <LocalPackageSourceInfo> >(
key: ridlessTarget.Libraries.FirstOrDefault(assetsPkg =>
sortedPkg.Version == assetsPkg.Version &&
sortedPkg.Id.Equals(assetsPkg.Name, StringComparison.OrdinalIgnoreCase)),
value: new Lazy <LocalPackageSourceInfo>(() =>
NuGetv3LocalRepositoryUtility.GetPackage(
repositories,
sortedPkg.Id,
sortedPkg.Version))))
.Where(e => e.Key != null)
.ToArray();
// build/ {packageId}.targets
var buildTargetsGroup = new MSBuildRestoreItemGroup();
buildTargetsGroup.RootName = MSBuildRestoreItemGroup.ImportGroup;
buildTargetsGroup.Position = 2;
buildTargetsGroup.Items.AddRange(sortedPackages.SelectMany(pkg =>
pkg.Key.Build.WithExtension(TargetsExtension)
.Where(e => pkg.Value.Exists())
.Select(e => pkg.Value.GetAbsolutePath(e)))
.Select(path => GetPathWithMacros(path, repositoryRoot))
.Select(GenerateImport));
targets.AddRange(GenerateGroupsWithConditions(buildTargetsGroup, isMultiTargeting, frameworkConditions));
// props/ {packageId}.props
var buildPropsGroup = new MSBuildRestoreItemGroup();
buildPropsGroup.RootName = MSBuildRestoreItemGroup.ImportGroup;
buildPropsGroup.Position = 2;
buildPropsGroup.Items.AddRange(sortedPackages.SelectMany(pkg =>
pkg.Key.Build.WithExtension(PropsExtension)
.Where(e => pkg.Value.Exists())
.Select(e => pkg.Value.GetAbsolutePath(e)))
.Select(path => GetPathWithMacros(path, repositoryRoot))
.Select(GenerateImport));
props.AddRange(GenerateGroupsWithConditions(buildPropsGroup, isMultiTargeting, frameworkConditions));
if (isMultiTargeting)
{
// buildMultiTargeting/ {packageId}.targets
var buildCrossTargetsGroup = new MSBuildRestoreItemGroup();
buildCrossTargetsGroup.RootName = MSBuildRestoreItemGroup.ImportGroup;
buildCrossTargetsGroup.Position = 0;
buildCrossTargetsGroup.Items.AddRange(sortedPackages.SelectMany(pkg =>
pkg.Key.BuildMultiTargeting.WithExtension(TargetsExtension)
.Where(e => pkg.Value.Exists())
.Select(e => pkg.Value.GetAbsolutePath(e)))
.Where(path => multiTargetingImportsAdded.Add(path))
.Select(path => GetPathWithMacros(path, repositoryRoot))
.Select(GenerateImport));
targets.AddRange(GenerateGroupsWithConditions(buildCrossTargetsGroup, isMultiTargeting, CrossTargetingCondition));
// buildMultiTargeting/ {packageId}.props
var buildCrossPropsGroup = new MSBuildRestoreItemGroup();
buildCrossPropsGroup.RootName = MSBuildRestoreItemGroup.ImportGroup;
buildCrossPropsGroup.Position = 0;
buildCrossPropsGroup.Items.AddRange(sortedPackages.SelectMany(pkg =>
pkg.Key.BuildMultiTargeting.WithExtension(PropsExtension)
.Where(e => pkg.Value.Exists())
.Select(e => pkg.Value.GetAbsolutePath(e)))
.Where(path => multiTargetingImportsAdded.Add(path))
.Select(path => GetPathWithMacros(path, repositoryRoot))
.Select(GenerateImport));
props.AddRange(GenerateGroupsWithConditions(buildCrossPropsGroup, isMultiTargeting, CrossTargetingCondition));
}
// Write out contentFiles only for XPlat PackageReference projects.
if (request.ProjectStyle != ProjectStyle.ProjectJson &&
request.Project.RestoreMetadata?.SkipContentFileWrite != true)
{
// Create a group for every package, with the nearest from each of allLanguages
props.AddRange(sortedPackages.Select(pkg =>
pkg.Key.ContentFiles
.Where(e => pkg.Value.Exists())
.OrderBy(e => e.Path, StringComparer.Ordinal)
.Select(e =>
Tuple.Create(
item1: pkg.Key,
item2: e,
item3: GetPathWithMacros(pkg.Value.GetAbsolutePath(e), repositoryRoot))))
.SelectMany(e => GetLanguageGroups(e))
.SelectMany(group => GenerateGroupsWithConditions(group, isMultiTargeting, frameworkConditions)));
}
}
// Add exclude all condition to all groups
foreach (var group in props.Concat(targets))
{
group.Conditions.Add(ExcludeAllCondition);
}
// Create XML, these may be null if the file should be deleted/not written out.
var propsXML = GenerateMSBuildFile(props, request.ProjectStyle);
var targetsXML = GenerateMSBuildFile(targets, request.ProjectStyle);
// Return all files to write out or delete.
var files = new List <MSBuildOutputFile>
{
new MSBuildOutputFile(propsPath, propsXML),
new MSBuildOutputFile(targetsPath, targetsXML)
};
var packageFolders = repositories.Select(e => e.RepositoryRoot);
AddNuGetPropertiesToFirstImport(files, packageFolders, repositoryRoot, request.ProjectStyle, assetsFilePath, restoreSuccess);
return(files);
}
public async Task CreatePathContextAsync_FromAssetsFile()
{
// Arrange
using (var testDirectory = TestDirectory.Create())
{
var userPackageFolder = Path.Combine(testDirectory.Path, "packagesA");
Directory.CreateDirectory(userPackageFolder);
var fallbackPackageFolder = Path.Combine(testDirectory.Path, "packagesB");
Directory.CreateDirectory(fallbackPackageFolder);
await SimpleTestPackageUtility.CreateFolderFeedV3Async(
userPackageFolder,
new PackageIdentity("Foo", NuGetVersion.Parse("1.0.1")));
await SimpleTestPackageUtility.CreateFolderFeedV3Async(
fallbackPackageFolder,
new PackageIdentity("Bar", NuGetVersion.Parse("1.0.2")));
var target = new VsPathContextProvider(
Mock.Of <ISettings>(),
Mock.Of <IVsSolutionManager>(),
Mock.Of <ILogger>(),
getLockFileOrNullAsync: _ =>
{
var lockFile = new LockFile
{
PackageFolders = new[]
{
new LockFileItem(userPackageFolder),
new LockFileItem(fallbackPackageFolder)
},
Libraries = new[]
{
new LockFileLibrary
{
Type = LibraryType.Package,
Name = "Foo",
Version = NuGetVersion.Parse("1.0.1")
},
new LockFileLibrary
{
Type = LibraryType.Package,
Name = "Bar",
Version = NuGetVersion.Parse("1.0.2")
}
}
};
return(Task.FromResult(lockFile));
});
var project = Mock.Of <BuildIntegratedNuGetProject>();
// Act
var actual = await target.CreatePathContextAsync(project, CancellationToken.None);
// Assert
Assert.NotNull(actual);
Assert.Equal(userPackageFolder, actual.UserPackageFolder);
Assert.Equal(
new[] { fallbackPackageFolder },
actual.FallbackPackageFolders.Cast <string>().ToArray());
string actualPackageDirectory = null;
var packageRootA = Path.Combine(userPackageFolder, "Foo", "1.0.1");
var assetFileA = Path.Combine(packageRootA, "lib", "net40", "a.dll");
Assert.True(actual.TryResolvePackageAsset(assetFileA, out actualPackageDirectory));
Assert.Equal(packageRootA, actualPackageDirectory, ignoreCase: true);
var packageRootB = Path.Combine(fallbackPackageFolder, "Bar", "1.0.2");
var assetFileB = Path.Combine(packageRootB, "lib", "net46", "b.dll");
Assert.True(actual.TryResolvePackageAsset(assetFileB, out actualPackageDirectory));
Assert.Equal(packageRootB, actualPackageDirectory, ignoreCase: true);
}
}
public ProjectContextBuilder WithLockFile(LockFile lockFile)
{
LockFile = lockFile;
return(this);
}
private static void RegisterServices(IKernel kernel)
{
var serverConfiguration = new ServerConfiguration();
// Make sure %HOME% is correctly set
EnsureHomeEnvironmentVariable();
EnsureSiteBitnessEnvironmentVariable();
IEnvironment environment = GetEnvironment();
EnsureDotNetCoreEnvironmentVariable(environment);
// Add various folders that never change to the process path. All child processes will inherit
PrependFoldersToPath(environment);
// Per request environment
kernel.Bind <IEnvironment>().ToMethod(context => GetEnvironment(context.Kernel.Get <IDeploymentSettingsManager>(), HttpContext.Current))
.InRequestScope();
// General
kernel.Bind <IServerConfiguration>().ToConstant(serverConfiguration);
kernel.Bind <IBuildPropertyProvider>().ToConstant(new BuildPropertyProvider());
System.Func <ITracer> createTracerThunk = () => GetTracer(environment, kernel);
System.Func <ILogger> createLoggerThunk = () => GetLogger(environment, kernel);
// First try to use the current request profiler if any, otherwise create a new one
var traceFactory = new TracerFactory(() => TraceServices.CurrentRequestTracer ?? createTracerThunk());
kernel.Bind <ITracer>().ToMethod(context => TraceServices.CurrentRequestTracer ?? NullTracer.Instance);
kernel.Bind <ITraceFactory>().ToConstant(traceFactory);
TraceServices.SetTraceFactory(createTracerThunk, createLoggerThunk);
// Setup the deployment lock
string lockPath = Path.Combine(environment.SiteRootPath, Constants.LockPath);
string deploymentLockPath = Path.Combine(lockPath, Constants.DeploymentLockFile);
string statusLockPath = Path.Combine(lockPath, Constants.StatusLockFile);
string sshKeyLockPath = Path.Combine(lockPath, Constants.SSHKeyLockFile);
string hooksLockPath = Path.Combine(lockPath, Constants.HooksLockFile);
_deploymentLock = new DeploymentLockFile(deploymentLockPath, kernel.Get <ITraceFactory>());
_deploymentLock.InitializeAsyncLocks();
var statusLock = new LockFile(statusLockPath, kernel.Get <ITraceFactory>());
var sshKeyLock = new LockFile(sshKeyLockPath, kernel.Get <ITraceFactory>());
var hooksLock = new LockFile(hooksLockPath, kernel.Get <ITraceFactory>());
kernel.Bind <IOperationLock>().ToConstant(sshKeyLock).WhenInjectedInto <SSHKeyController>();
kernel.Bind <IOperationLock>().ToConstant(statusLock).WhenInjectedInto <DeploymentStatusManager>();
kernel.Bind <IOperationLock>().ToConstant(hooksLock).WhenInjectedInto <WebHooksManager>();
kernel.Bind <IOperationLock>().ToConstant(_deploymentLock);
var shutdownDetector = new ShutdownDetector();
shutdownDetector.Initialize();
IDeploymentSettingsManager noContextDeploymentsSettingsManager =
new DeploymentSettingsManager(new XmlSettings.Settings(GetSettingsPath(environment)));
TraceServices.TraceLevel = noContextDeploymentsSettingsManager.GetTraceLevel();
var noContextTraceFactory = new TracerFactory(() => GetTracerWithoutContext(environment, noContextDeploymentsSettingsManager));
kernel.Bind <IAnalytics>().ToMethod(context => new Analytics(context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <IServerConfiguration>(),
noContextTraceFactory));
// Trace unhandled (crash) exceptions.
AppDomain.CurrentDomain.UnhandledException += (sender, args) =>
{
var ex = args.ExceptionObject as Exception;
if (ex != null)
{
kernel.Get <IAnalytics>().UnexpectedException(ex);
}
};
// Trace shutdown event
// Cannot use shutdownDetector.Token.Register because of race condition
// with NinjectServices.Stop via WebActivator.ApplicationShutdownMethodAttribute
Shutdown += () => TraceShutdown(environment, noContextDeploymentsSettingsManager);
// LogStream service
// The hooks and log stream start endpoint are low traffic end-points. Re-using it to avoid creating another lock
var logStreamManagerLock = hooksLock;
kernel.Bind <LogStreamManager>().ToMethod(context => new LogStreamManager(Path.Combine(environment.RootPath, Constants.LogFilesPath),
context.Kernel.Get <IEnvironment>(),
context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <ITracer>(),
shutdownDetector,
logStreamManagerLock));
kernel.Bind <InfoRefsController>().ToMethod(context => new InfoRefsController(t => context.Kernel.Get(t)))
.InRequestScope();
kernel.Bind <CustomGitRepositoryHandler>().ToMethod(context => new CustomGitRepositoryHandler(t => context.Kernel.Get(t)))
.InRequestScope();
// Deployment Service
kernel.Bind <ISettings>().ToMethod(context => new XmlSettings.Settings(GetSettingsPath(environment)))
.InRequestScope();
kernel.Bind <IDeploymentSettingsManager>().To <DeploymentSettingsManager>()
.InRequestScope();
kernel.Bind <IDeploymentStatusManager>().To <DeploymentStatusManager>()
.InRequestScope();
kernel.Bind <ISiteBuilderFactory>().To <SiteBuilderFactory>()
.InRequestScope();
kernel.Bind <IWebHooksManager>().To <WebHooksManager>()
.InRequestScope();
ITriggeredJobsManager triggeredJobsManager = new TriggeredJobsManager(
noContextTraceFactory,
kernel.Get <IEnvironment>(),
kernel.Get <IDeploymentSettingsManager>(),
kernel.Get <IAnalytics>(),
kernel.Get <IWebHooksManager>());
kernel.Bind <ITriggeredJobsManager>().ToConstant(triggeredJobsManager)
.InTransientScope();
TriggeredJobsScheduler triggeredJobsScheduler = new TriggeredJobsScheduler(
triggeredJobsManager,
noContextTraceFactory,
environment,
kernel.Get <IAnalytics>());
kernel.Bind <TriggeredJobsScheduler>().ToConstant(triggeredJobsScheduler)
.InTransientScope();
IContinuousJobsManager continuousJobManager = new ContinuousJobsManager(
noContextTraceFactory,
kernel.Get <IEnvironment>(),
kernel.Get <IDeploymentSettingsManager>(),
kernel.Get <IAnalytics>());
OperationManager.SafeExecute(triggeredJobsManager.CleanupDeletedJobs);
OperationManager.SafeExecute(continuousJobManager.CleanupDeletedJobs);
kernel.Bind <IContinuousJobsManager>().ToConstant(continuousJobManager)
.InTransientScope();
kernel.Bind <ILogger>().ToMethod(context => GetLogger(environment, context.Kernel))
.InRequestScope();
kernel.Bind <IDeploymentManager>().To <DeploymentManager>()
.InRequestScope();
kernel.Bind <ISSHKeyManager>().To <SSHKeyManager>()
.InRequestScope();
kernel.Bind <IRepositoryFactory>().ToMethod(context => _deploymentLock.RepositoryFactory = new RepositoryFactory(context.Kernel.Get <IEnvironment>(),
context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <ITraceFactory>()))
.InRequestScope();
kernel.Bind <IApplicationLogsReader>().To <ApplicationLogsReader>()
.InSingletonScope();
// Git server
kernel.Bind <IDeploymentEnvironment>().To <DeploymentEnvrionment>();
kernel.Bind <IGitServer>().ToMethod(context => new GitExeServer(context.Kernel.Get <IEnvironment>(),
_deploymentLock,
GetRequestTraceFile(context.Kernel),
context.Kernel.Get <IRepositoryFactory>(),
context.Kernel.Get <IDeploymentEnvironment>(),
context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <ITraceFactory>()))
.InRequestScope();
// Git Servicehook parsers
kernel.Bind <IServiceHookHandler>().To <GenericHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <GitHubHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <BitbucketHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <BitbucketHandlerV2>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <DropboxHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <CodePlexHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <CodebaseHqHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <GitlabHqHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <GitHubCompatHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <KilnHgHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <VSOHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <OneDriveHandler>().InRequestScope();
// SiteExtensions
kernel.Bind <ISiteExtensionManager>().To <SiteExtensionManager>().InRequestScope();
// Functions
kernel.Bind <IFunctionManager>().To <FunctionManager>().InRequestScope();
// Command executor
kernel.Bind <ICommandExecutor>().To <CommandExecutor>().InRequestScope();
MigrateSite(environment, noContextDeploymentsSettingsManager);
RemoveOldTracePath(environment);
RemoveTempFileFromUserDrive(environment);
// Temporary fix for https://github.com/npm/npm/issues/5905
EnsureNpmGlobalDirectory();
EnsureUserProfileDirectory();
// Skip SSL Certificate Validate
if (System.Environment.GetEnvironmentVariable(SettingsKeys.SkipSslValidation) == "1")
{
ServicePointManager.ServerCertificateValidationCallback = delegate { return(true); };
}
// Make sure webpages:Enabled is true. Even though we set it in web.config, it could be overwritten by
// an Azure AppSetting that's supposed to be for the site only but incidently affects Kudu as well.
ConfigurationManager.AppSettings["webpages:Enabled"] = "true";
// Kudu does not rely owin:appStartup. This is to avoid Azure AppSetting if set.
if (ConfigurationManager.AppSettings["owin:appStartup"] != null)
{
// Set the appSetting to null since we cannot use AppSettings.Remove(key) (ReadOnly exception!)
ConfigurationManager.AppSettings["owin:appStartup"] = null;
}
RegisterRoutes(kernel, RouteTable.Routes);
// Register the default hubs route: ~/signalr
GlobalHost.DependencyResolver = new SignalRNinjectDependencyResolver(kernel);
GlobalConfiguration.Configuration.Filters.Add(
new TraceDeprecatedActionAttribute(
kernel.Get <IAnalytics>(),
kernel.Get <ITraceFactory>()));
GlobalConfiguration.Configuration.Filters.Add(new EnsureRequestIdHandlerAttribute());
}
private void requireLocked(LockFile tmp)
{
if (liveFile == null)
throw new InvalidOperationException("DirCache is not locked");
if (tmp == null)
throw new InvalidOperationException("DirCache "
+ liveFile.FullName + " not locked.");
}
void Execute(DirectoryAssemblyResolver resolver)
{
foreach (var dir in ReferenceAssembliesDirectory.Split(new char[] { ';' }, StringSplitOptions.RemoveEmptyEntries))
{
resolver.SearchDirectories.Add(dir);
}
var assemblies = new Dictionary <string, ITaskItem> ();
var topAssemblyReferences = new List <AssemblyDefinition> ();
var logger = new NuGetLogger((s) => {
LogDebugMessage("{0}", s);
});
LockFile lockFile = null;
if (!string.IsNullOrEmpty(ProjectAssetFile) && File.Exists(ProjectAssetFile))
{
lockFile = LockFileUtilities.GetLockFile(ProjectAssetFile, logger);
}
try {
foreach (var assembly in Assemblies)
{
var assembly_path = Path.GetDirectoryName(assembly.ItemSpec);
if (!resolver.SearchDirectories.Contains(assembly_path))
{
resolver.SearchDirectories.Add(assembly_path);
}
// Add each user assembly and all referenced assemblies (recursive)
var assemblyDef = resolver.Load(assembly.ItemSpec);
if (assemblyDef == null)
{
throw new InvalidOperationException("Failed to load assembly " + assembly.ItemSpec);
}
if (MonoAndroidHelper.IsReferenceAssembly(assemblyDef))
{
// Resolve "runtime" library
var asmFullPath = Path.GetFullPath(assembly.ItemSpec);
if (lockFile != null)
{
assemblyDef = ResolveRuntimeAssemblyForReferenceAssembly(lockFile, resolver, asmFullPath);
}
if (lockFile == null || assemblyDef == null)
{
LogCodedWarning("XA0107", asmFullPath, 0, "Ignoring {0} as it is a Reference Assembly", asmFullPath);
continue;
}
}
topAssemblyReferences.Add(assemblyDef);
var taskItem = new TaskItem(assembly)
{
ItemSpec = Path.GetFullPath(assemblyDef.MainModule.FileName),
};
if (string.IsNullOrEmpty(taskItem.GetMetadata("ReferenceAssembly")))
{
taskItem.SetMetadata("ReferenceAssembly", taskItem.ItemSpec);
}
assemblies [assemblyDef.Name.Name] = taskItem;
}
} catch (Exception ex) {
LogError("Exception while loading assemblies: {0}", ex);
return;
}
try {
foreach (var assembly in topAssemblyReferences)
{
AddAssemblyReferences(resolver, assemblies, assembly, null);
}
} catch (Exception ex) {
LogError("Exception while loading assemblies: {0}", ex);
return;
}
// Add I18N assemblies if needed
AddI18nAssemblies(resolver, assemblies);
var mainapiLevel = MonoAndroidHelper.SupportedVersions.GetApiLevelFromFrameworkVersion(TargetFrameworkVersion);
foreach (var item in api_levels.Where(x => mainapiLevel < x.Value))
{
var itemOSVersion = MonoAndroidHelper.SupportedVersions.GetFrameworkVersionFromApiLevel(item.Value);
Log.LogCodedWarning("XA0105", ProjectFile, 0,
"The $(TargetFrameworkVersion) for {0} ({1}) is greater than the $(TargetFrameworkVersion) for your project ({2}). " +
"You need to increase the $(TargetFrameworkVersion) for your project.", Path.GetFileName(item.Key), itemOSVersion, TargetFrameworkVersion);
}
var resolvedAssemblies = new List <ITaskItem> (assemblies.Count);
var resolvedSymbols = new List <ITaskItem> (assemblies.Count);
var resolvedFrameworkAssemblies = new List <ITaskItem> (assemblies.Count);
var resolvedUserAssemblies = new List <ITaskItem> (assemblies.Count);
foreach (var assembly in assemblies.Values)
{
var mdb = assembly + ".mdb";
var pdb = Path.ChangeExtension(assembly.ItemSpec, "pdb");
if (File.Exists(mdb))
{
resolvedSymbols.Add(new TaskItem(mdb));
}
if (File.Exists(pdb) && Files.IsPortablePdb(pdb))
{
resolvedSymbols.Add(new TaskItem(pdb));
}
resolvedAssemblies.Add(assembly);
if (MonoAndroidHelper.IsFrameworkAssembly(assembly.ItemSpec, checkSdkPath: true))
{
resolvedFrameworkAssemblies.Add(assembly);
}
else
{
resolvedUserAssemblies.Add(assembly);
}
}
ResolvedAssemblies = resolvedAssemblies.ToArray();
ResolvedSymbols = resolvedSymbols.ToArray();
ResolvedFrameworkAssemblies = resolvedFrameworkAssemblies.ToArray();
ResolvedUserAssemblies = resolvedUserAssemblies.ToArray();
ResolvedDoNotPackageAttributes = do_not_package_atts.ToArray();
}
private void DowngradeLockFileToV2(LockFile lockFile)
{
// noop
}
public void LibrariesAreResolvedCorrectly(string packageName, string framework, bool resolved)
{
var net45Target = new LockFileTarget
{
TargetFramework = new FrameworkName(".NETFramework,Version=v4.5"),
Libraries = new List<LockFileTargetLibrary>
{
new LockFileTargetLibrary
{
Name = "MetaPackage",
Version = SemanticVersion.Parse("1.0.0"),
},
new LockFileTargetLibrary
{
Name = "Net451LibPackage",
Version = SemanticVersion.Parse("1.0.0"),
},
new LockFileTargetLibrary
{
Name = "Net451RefPackage",
Version = SemanticVersion.Parse("1.0.0"),
},
new LockFileTargetLibrary
{
Name = "AssemblyPlaceholderPackage",
Version = SemanticVersion.Parse("1.0.0"),
CompileTimeAssemblies = new List<LockFileItem> { Path.Combine("ref", "net45", "_._") },
RuntimeAssemblies = new List<LockFileItem> { Path.Combine("lib", "net45", "_._") }
}
}
};
var dnx451Target = new LockFileTarget
{
TargetFramework = new FrameworkName("DNX,Version=v4.5.1"),
Libraries = new List<LockFileTargetLibrary>
{
new LockFileTargetLibrary
{
Name = "MetaPackage",
Version = SemanticVersion.Parse("1.0.0")
},
new LockFileTargetLibrary
{
Name = "Net451LibPackage",
Version = SemanticVersion.Parse("1.0.0"),
CompileTimeAssemblies = new List<LockFileItem> { Path.Combine("lib", "net451", "Net451LibPackage.dll") },
RuntimeAssemblies = new List<LockFileItem> { Path.Combine("lib", "net451", "Net451LibPackage.dll") }
},
new LockFileTargetLibrary
{
Name = "Net451RefPackage",
Version = SemanticVersion.Parse("1.0.0"),
CompileTimeAssemblies = new List<LockFileItem> { Path.Combine("ref", "net451", "Net451LibPackage.dll") }
},
new LockFileTargetLibrary
{
Name = "AssemblyPlaceholderPackage",
Version = SemanticVersion.Parse("1.0.0")
}
}
};
var metaPackageLibrary = new LockFilePackageLibrary
{
Name = "MetaPackage",
Version = SemanticVersion.Parse("1.0.0")
};
var net451LibPackageLibrary = new LockFilePackageLibrary
{
Name = "Net451LibPackage",
Version = SemanticVersion.Parse("1.0.0"),
Files = new List<string>
{
Path.Combine("lib", "net451", "Net451LibPackage.dll"),
Path.Combine("lib", "net451", "Net451LibPackage.xml")
}
};
var net451RefPackageLibrary = new LockFilePackageLibrary
{
Name = "Net451RefPackage",
Version = SemanticVersion.Parse("1.0.0"),
Files = new List<string>
{
Path.Combine("ref", "net451", "Net451LibPackage.dll")
}
};
var assemblyPlaceholderPackageLibrary = new LockFilePackageLibrary
{
Name = "AssemblyPlaceholderPackage",
Version = SemanticVersion.Parse("1.0.0"),
Files = new List<string>
{
Path.Combine("lib", "net45", "_._"),
Path.Combine("ref", "net45", "_._"),
}
};
var lockFile = new LockFile()
{
Targets = new List<LockFileTarget> { net45Target, dnx451Target },
PackageLibraries = new List<LockFilePackageLibrary>
{
metaPackageLibrary,
net451LibPackageLibrary,
net451RefPackageLibrary,
assemblyPlaceholderPackageLibrary
}
};
var resolver = new PackageDependencyProvider("/path/to/packages");
var lockFileLookup = new LockFileLookup(lockFile);
var targetFramework = new FrameworkName(framework);
var libToLookup = lockFile.Targets.First(t => t.TargetFramework == targetFramework)
.Libraries
.First(l => l.Name == packageName);
var description = resolver.GetDescription(lockFileLookup.GetPackage(libToLookup.Name, libToLookup.Version), libToLookup);
Assert.Equal(resolved, description.Compatible);
Assert.NotNull(description);
}
/// <summary>
/// Prepares the dictionary that maps frameworks to packages top-level
/// and transitive.
/// </summary>
/// <param name="projectPath"> Path to the project to get versions for its packages </param>
/// <param name="userInputFrameworks">A list of frameworks</param>
/// <param name="assetsFile">Assets file for all targets and libraries</param>
/// <param name="transitive">Include transitive packages in the result</param>
/// <returns></returns>
internal IEnumerable <FrameworkPackages> GetResolvedVersions(
string projectPath, IEnumerable <string> userInputFrameworks, LockFile assetsFile, bool transitive)
{
if (userInputFrameworks == null)
{
throw new ArgumentNullException(nameof(userInputFrameworks));
}
if (projectPath == null)
{
throw new ArgumentNullException(nameof(projectPath));
}
if (assetsFile == null)
{
throw new ArgumentNullException(nameof(assetsFile));
}
var resultPackages = new List <FrameworkPackages>();
var requestedTargetFrameworks = assetsFile.PackageSpec.TargetFrameworks;
var requestedTargets = assetsFile.Targets;
// If the user has entered frameworks, we want to filter
// the targets and frameworks from the assets file
if (userInputFrameworks.Any())
{
//Target frameworks filtering
var parsedUserFrameworks = userInputFrameworks.Select(f =>
NuGetFramework.Parse(f.Split(new char[] { '/' }, StringSplitOptions.RemoveEmptyEntries).Select(s => s.Trim()).ToArray()[0]));
requestedTargetFrameworks = requestedTargetFrameworks.Where(tfm => parsedUserFrameworks.Contains(tfm.FrameworkName)).ToList();
//Assets file targets filtering by framework and RID
var filteredTargets = new List <LockFileTarget>();
foreach (var frameworkAndRID in userInputFrameworks)
{
var splitFrameworkAndRID = frameworkAndRID.Split(new char[] { '/' }, StringSplitOptions.RemoveEmptyEntries).Select(s => s.Trim()).ToArray();
// If a / is not present in the string, we get all of the targets that
// have matching framework regardless of RID.
if (splitFrameworkAndRID.Count() == 1)
{
filteredTargets.AddRange(requestedTargets.Where(target => target.TargetFramework.Equals(NuGetFramework.Parse(splitFrameworkAndRID[0]))));
}
else
{
//RID is present in the user input, so we filter using it as well
filteredTargets.AddRange(requestedTargets.Where(target => target.TargetFramework.Equals(NuGetFramework.Parse(splitFrameworkAndRID[0])) &&
target.RuntimeIdentifier != null && target.RuntimeIdentifier.Equals(splitFrameworkAndRID[1], StringComparison.OrdinalIgnoreCase)));
}
}
requestedTargets = filteredTargets;
}
// Filtering the Targets to ignore TargetFramework + RID combination, only keep TargetFramework in requestedTargets.
// So that only one section will be shown for each TFM.
requestedTargets = requestedTargets.Where(target => target.RuntimeIdentifier == null).ToList();
foreach (var target in requestedTargets)
{
// Find the tfminformation corresponding to the target to
// get the top-level dependencies
TargetFrameworkInformation tfmInformation;
try
{
tfmInformation = requestedTargetFrameworks.First(tfm => tfm.FrameworkName.Equals(target.TargetFramework));
}
catch (Exception)
{
Console.WriteLine(string.Format(Strings.ListPkg_ErrorReadingAssetsFile, assetsFile.Path));
return(null);
}
//The packages for the framework that were retrieved with GetRequestedVersions
var frameworkDependencies = tfmInformation.Dependencies;
var projPackages = GetPackageReferencesFromTargets(projectPath, "", tfmInformation.ToString());
var topLevelPackages = new List <InstalledPackageReference>();
var transitivePackages = new List <InstalledPackageReference>();
foreach (var library in target.Libraries)
{
var matchingPackages = frameworkDependencies.Where(d =>
d.Name.Equals(library.Name, StringComparison.OrdinalIgnoreCase)).ToList();
var resolvedVersion = library.Version.ToString();
//In case we found a matching package in requestedVersions, the package will be
//top level.
if (matchingPackages.Any())
{
var topLevelPackage = matchingPackages.Single();
InstalledPackageReference installedPackage;
//If the package is not auto-referenced, get the version from the project file. Otherwise fall back on the assets file
if (!topLevelPackage.AutoReferenced)
{
try
{ // In case proj and assets file are not in sync and some refs were deleted
installedPackage = projPackages.Where(p => p.Name.Equals(topLevelPackage.Name)).First();
}
catch (Exception)
{
Console.WriteLine(string.Format(Strings.ListPkg_ErrorReadingReferenceFromProject, projectPath));
return(null);
}
}
else
{
var projectFileVersion = topLevelPackage.LibraryRange.VersionRange.ToString();
installedPackage = new InstalledPackageReference(library.Name)
{
OriginalRequestedVersion = projectFileVersion
};
}
installedPackage.ResolvedVersion = library.Version;
installedPackage.AutoReference = topLevelPackage.AutoReferenced;
topLevelPackages.Add(installedPackage);
}
// If no matching packages were found, then the package is transitive,
// and include-transitive must be used to add the package
else if (transitive)
{
var installedPackage = new InstalledPackageReference(library.Name)
{
ResolvedVersion = library.Version
};
transitivePackages.Add(installedPackage);
}
}
var frameworkPackages = new FrameworkPackages(
target.TargetFramework.GetShortFolderName(),
topLevelPackages,
transitivePackages);
resultPackages.Add(frameworkPackages);
}
return(resultPackages);
}
public async Task BuildAssetsUtils_GeneratePathPropertyForTools(bool hasTools)
{
using (var pathContext = new SimpleTestPathContext())
using (var randomProjectDirectory = TestDirectory.Create())
{
// Arrange
var identity = new PackageIdentity("packagea", NuGetVersion.Parse("1.0.0"));
var packageDirectory = Directory.CreateDirectory(Path.Combine(pathContext.UserPackagesFolder, identity.Id, identity.Version.ToNormalizedString()));
File.WriteAllText(Path.Combine(packageDirectory.FullName, $"{identity.Id}.{identity.Version.ToNormalizedString()}.nupkg.sha512"), string.Empty);
var packagePath = await SimpleTestPackageUtility.CreateFullPackageAsync(
packageDirectory.FullName,
identity.Id,
identity.Version.ToString());
var logger = new TestLogger();
var spec = ToolRestoreUtility.GetSpec(
Path.Combine(pathContext.SolutionRoot, "tool", "fake.csproj"),
"a",
VersionRange.Parse("1.0.0"),
NuGetFramework.Parse("netcoreapp1.0"),
pathContext.UserPackagesFolder,
new List <string>()
{
pathContext.FallbackFolder
},
new List <PackageSource>()
{
new PackageSource(pathContext.PackageSource)
},
projectWideWarningProperties: null);
spec.RestoreMetadata.ProjectStyle = ProjectStyle.PackageReference;
spec.Dependencies.Add(new LibraryDependency
{
IncludeType = LibraryIncludeFlags.All,
LibraryRange = new LibraryRange(identity.Id, new VersionRange(identity.Version), LibraryDependencyTarget.Package)
});
var targetGraphs = new List <RestoreTargetGraph>
{
OriginalCaseGlobalPackageFolderTests.GetRestoreTargetGraph(pathContext.PackageSource, identity, packagePath, logger)
};
targetGraphs[0].Graphs.FirstOrDefault().Item.Data.Dependencies = spec.Dependencies;
var lockFile = new LockFile
{
Libraries =
{
new LockFileLibrary
{
Name = identity.Id,
Version = identity.Version,
Path = $"{identity.Id.ToLowerInvariant()}/{identity.Version.ToNormalizedString()}",
Type = LibraryType.Package,
HasTools = hasTools,
}
},
Targets =
{
new LockFileTarget
{
RuntimeIdentifier = targetGraphs[0].RuntimeIdentifier,
TargetFramework = targetGraphs[0].Framework,
Libraries =
{
new LockFileTargetLibrary
{
Name = identity.Id,
Version = identity.Version
}
}
}
}
};
var repositories = new List <NuGetv3LocalRepository>
{
new NuGetv3LocalRepository(pathContext.UserPackagesFolder)
};
var restoreRequest = new TestRestoreRequest(spec, new[] { new PackageSource(pathContext.PackageSource) }, pathContext.PackagesV2, logger)
{
ProjectStyle = spec.RestoreMetadata.ProjectStyle
};
var assetsFilePath = Path.Combine(randomProjectDirectory, "obj", "project.assets.json");
// Act
var outputFiles = BuildAssetsUtils.GetMSBuildOutputFiles(spec, lockFile, targetGraphs, repositories, restoreRequest, assetsFilePath, true, logger);
var expectedPropertyName = $"Pkg{identity.Id.Replace(".", "_")}";
var actualPropertyElement = outputFiles.FirstOrDefault().Content.Root.Descendants().Where(i => i.Name.LocalName.Equals(expectedPropertyName)).FirstOrDefault();
if (hasTools)
{
// Assert
Assert.NotNull(actualPropertyElement);
Assert.Equal($" '$({actualPropertyElement.Name.LocalName})' == '' ", actualPropertyElement.Attribute("Condition")?.Value);
Assert.Equal(packageDirectory.FullName, actualPropertyElement?.Value, ignoreCase: true);
Assert.Equal(" '$(ExcludeRestorePackageImports)' != 'true' ", actualPropertyElement.Parent.Attribute("Condition")?.Value);
}
else
{
Assert.Null(actualPropertyElement);
}
}
}
public PackagesLockFile CreateNuGetLockFile(LockFile assetsFile)
{
var lockFile = new PackagesLockFile(GetPackagesLockFileVersion(assetsFile));
var libraryLookup = assetsFile.Libraries.Where(e => e.Type == LibraryType.Package)
.ToDictionary(e => new PackageIdentity(e.Name, e.Version));
foreach (var target in assetsFile.Targets)
{
var nuGettarget = new PackagesLockFileTarget()
{
TargetFramework = target.TargetFramework,
RuntimeIdentifier = target.RuntimeIdentifier
};
var framework = assetsFile.PackageSpec.TargetFrameworks.FirstOrDefault(
f => EqualityUtility.EqualsWithNullCheck(f.FrameworkName, target.TargetFramework));
IEnumerable <LockFileTargetLibrary> libraries = target.Libraries;
// check if this is RID-based graph then only add those libraries which differ from original TFM.
if (!string.IsNullOrEmpty(target.RuntimeIdentifier))
{
var onlyTFM = assetsFile.Targets.First(t => EqualityUtility.EqualsWithNullCheck(t.TargetFramework, target.TargetFramework));
libraries = target.Libraries.Where(lib => !onlyTFM.Libraries.Any(tfmLib => tfmLib.Equals(lib)));
}
foreach (var library in libraries.Where(e => e.Type == LibraryType.Package))
{
var identity = new PackageIdentity(library.Name, library.Version);
var dependency = new LockFileDependency()
{
Id = library.Name,
ResolvedVersion = library.Version,
ContentHash = libraryLookup[identity].Sha512,
Dependencies = library.Dependencies
};
var framework_dep = framework?.Dependencies.FirstOrDefault(
dep => StringComparer.OrdinalIgnoreCase.Equals(dep.Name, library.Name));
CentralPackageVersion centralPackageVersion = null;
framework?.CentralPackageVersions.TryGetValue(library.Name, out centralPackageVersion);
if (framework_dep != null)
{
dependency.Type = PackageDependencyType.Direct;
dependency.RequestedVersion = framework_dep.LibraryRange.VersionRange;
}
// The dgspec has a list of the direct dependencies and changes in the direct dependencies will invalidate the lock file
// A dgspec does not have information about transitive dependencies
// At the restore time the transitive dependencies could be pinned from central package version management file
// By marking them will allow to evaluate when to invalidate the packages.lock.json
// in cases that a central transitive version is updated, removed or added the lock file will be invalidated
else if (centralPackageVersion != null)
{
// This is a transitive dependency that is in the list of central dependencies.
dependency.Type = PackageDependencyType.CentralTransitive;
dependency.RequestedVersion = centralPackageVersion.VersionRange;
}
else
{
dependency.Type = PackageDependencyType.Transitive;
}
nuGettarget.Dependencies.Add(dependency);
}
var projectFullPaths = assetsFile.Libraries
.Where(l => l.Type == LibraryType.Project || l.Type == LibraryType.ExternalProject)
.ToDictionary(l => new PackageIdentity(l.Name, l.Version), l => l.MSBuildProject);
foreach (var projectReference in libraries.Where(e => e.Type == LibraryType.Project || e.Type == LibraryType.ExternalProject))
{
var projectIdentity = new PackageIdentity(projectReference.Name, projectReference.Version);
var projectFullPath = projectFullPaths[projectIdentity];
var id = PathUtility.GetStringComparerBasedOnOS().Equals(Path.GetFileNameWithoutExtension(projectFullPath), projectReference.Name)
? projectReference.Name.ToLowerInvariant()
: projectReference.Name;
var dependency = new LockFileDependency()
{
Id = id,
Dependencies = projectReference.Dependencies,
Type = PackageDependencyType.Project
};
nuGettarget.Dependencies.Add(dependency);
}
nuGettarget.Dependencies = nuGettarget.Dependencies.OrderBy(d => d.Type).ToList();
lockFile.Targets.Add(nuGettarget);
}
return(lockFile);
}
private ResolvePackageDependencies GetExecutedTaskFromContents(string lockFileContents, out LockFile lockFile)
{
lockFile = TestLockFiles.CreateLockFile(lockFileContents);
return(GetExecutedTask(lockFile));
}
private ResolvePackageDependencies GetExecutedTaskFromPrefix(string lockFilePrefix, out LockFile lockFile)
{
lockFile = TestLockFiles.GetLockFile(lockFilePrefix);
return(GetExecutedTask(lockFile));
}
public void testUpdateRefLockFailureLocked()
{
ObjectId opid = db.Resolve("refs/heads/master");
ObjectId pid = db.Resolve("refs/heads/master^");
RefUpdate updateRef = db.UpdateRef("refs/heads/master");
updateRef.NewObjectId = pid;
var lockFile1 = new LockFile(new FileInfo(Path.Combine(db.Directory.FullName, "refs/heads/master")));
try
{
Assert.IsTrue(lockFile1.Lock()); // precondition to test
RefUpdate.RefUpdateResult update = updateRef.update();
Assert.AreEqual(RefUpdate.RefUpdateResult.LOCK_FAILURE, update);
Assert.AreEqual(opid, db.Resolve("refs/heads/master"));
var lockFile2 = new LockFile(new FileInfo(Path.Combine(db.Directory.FullName, "refs/heads/master")));
Assert.IsFalse(lockFile2.Lock()); // was locked, still is
}
finally
{
lockFile1.Unlock();
}
}
public async Task <RestoreResult> ExecuteAsync(CancellationToken token)
{
using (var telemetry = TelemetryActivity.CreateTelemetryActivityWithNewOperationIdAndEvent(parentId: ParentId, eventName: ProjectRestoreInformation))
{
_operationId = telemetry.OperationId;
var restoreTime = Stopwatch.StartNew();
// Local package folders (non-sources)
var localRepositories = new List <NuGetv3LocalRepository>
{
_request.DependencyProviders.GlobalPackages
};
localRepositories.AddRange(_request.DependencyProviders.FallbackPackageFolders);
var contextForProject = CreateRemoteWalkContext(_request, _logger);
CacheFile cacheFile = null;
using (var noOpTelemetry = TelemetryActivity.CreateTelemetryActivityWithNewOperationIdAndEvent(parentId: _operationId, eventName: RestoreNoOpInformation))
{
if (NoOpRestoreUtilities.IsNoOpSupported(_request))
{
noOpTelemetry.StartIntervalMeasure();
var cacheFileAndStatus = EvaluateCacheFile();
noOpTelemetry.EndIntervalMeasure(CacheFileEvaluateDuration);
cacheFile = cacheFileAndStatus.Key;
if (cacheFileAndStatus.Value)
{
noOpTelemetry.StartIntervalMeasure();
var noOpSuccess = NoOpRestoreUtilities.VerifyAssetsAndMSBuildFilesAndPackagesArePresent(_request);
noOpTelemetry.EndIntervalMeasure(MsbuildAssetsVerificationDuration);
noOpTelemetry.TelemetryEvent[MsbuildAssetsVerificationResult] = noOpSuccess;
if (noOpSuccess)
{
noOpTelemetry.StartIntervalMeasure();
// Replay Warnings and Errors from an existing lock file in case of a no-op.
await MSBuildRestoreUtility.ReplayWarningsAndErrorsAsync(_request.ExistingLockFile, _logger);
noOpTelemetry.EndIntervalMeasure(ReplayLogsDuration);
restoreTime.Stop();
return(new NoOpRestoreResult(
_success,
_request.ExistingLockFile,
_request.ExistingLockFile,
_request.ExistingLockFile.Path,
cacheFile,
_request.Project.RestoreMetadata.CacheFilePath,
_request.ProjectStyle,
restoreTime.Elapsed));
}
}
}
}
IEnumerable <RestoreTargetGraph> graphs = null;
using (var restoreGraphTelemetry = TelemetryActivity.CreateTelemetryActivityWithNewOperationIdAndEvent(parentId: _operationId, eventName: GenerateRestoreGraph))
{
// Restore
graphs = await ExecuteRestoreAsync(
_request.DependencyProviders.GlobalPackages,
_request.DependencyProviders.FallbackPackageFolders,
contextForProject,
token,
restoreGraphTelemetry);
}
LockFile assetsFile = null;
using (TelemetryActivity.CreateTelemetryActivityWithNewOperationIdAndEvent(parentId: _operationId, eventName: GenerateAssetsFile))
{
// Create assets file
assetsFile = BuildAssetsFile(
_request.ExistingLockFile,
_request.Project,
graphs,
localRepositories,
contextForProject);
}
IList <CompatibilityCheckResult> checkResults = null;
using (TelemetryActivity.CreateTelemetryActivityWithNewOperationIdAndEvent(parentId: _operationId, eventName: ValidateRestoreGraphs))
{
_success &= await ValidateRestoreGraphsAsync(graphs, _logger);
// Check package compatibility
checkResults = await VerifyCompatibilityAsync(
_request.Project,
_includeFlagGraphs,
localRepositories,
assetsFile,
graphs,
_request.ValidateRuntimeAssets,
_logger);
if (checkResults.Any(r => !r.Success))
{
_success = false;
}
}
// Generate Targets/Props files
var msbuildOutputFiles = Enumerable.Empty <MSBuildOutputFile>();
string assetsFilePath = null;
string cacheFilePath = null;
using (TelemetryActivity.CreateTelemetryActivityWithNewOperationIdAndEvent(parentId: _operationId, eventName: CreateRestoreResult))
{
// Determine the lock file output path
assetsFilePath = GetAssetsFilePath(assetsFile);
// Determine the cache file output path
cacheFilePath = NoOpRestoreUtilities.GetCacheFilePath(_request, assetsFile);
// Tool restores are unique since the output path is not known until after restore
if (_request.LockFilePath == null &&
_request.ProjectStyle == ProjectStyle.DotnetCliTool)
{
_request.LockFilePath = assetsFilePath;
}
if (contextForProject.IsMsBuildBased)
{
msbuildOutputFiles = BuildAssetsUtils.GetMSBuildOutputFiles(
_request.Project,
assetsFile,
graphs,
localRepositories,
_request,
assetsFilePath,
_success,
_logger);
}
// If the request is for a lower lock file version, downgrade it appropriately
DowngradeLockFileIfNeeded(assetsFile);
// Revert to the original case if needed
await FixCaseForLegacyReaders(graphs, assetsFile, token);
// Write the logs into the assets file
var logs = _logger.Errors
.Select(l => AssetsLogMessage.Create(l))
.ToList();
_success &= !logs.Any(l => l.Level == LogLevel.Error);
assetsFile.LogMessages = logs;
if (cacheFile != null)
{
cacheFile.Success = _success;
}
var errorCodes = ConcatAsString(new HashSet <NuGetLogCode>(logs.Where(l => l.Level == LogLevel.Error).Select(l => l.Code)));
var warningCodes = ConcatAsString(new HashSet <NuGetLogCode>(logs.Where(l => l.Level == LogLevel.Warning).Select(l => l.Code)));
if (!string.IsNullOrEmpty(errorCodes))
{
telemetry.TelemetryEvent[ErrorCodes] = errorCodes;
}
if (!string.IsNullOrEmpty(warningCodes))
{
telemetry.TelemetryEvent[WarningCodes] = warningCodes;
}
telemetry.TelemetryEvent[RestoreSuccess] = _success;
}
restoreTime.Stop();
// Create result
return(new RestoreResult(
_success,
graphs,
checkResults,
msbuildOutputFiles,
assetsFile,
_request.ExistingLockFile,
assetsFilePath,
cacheFile,
cacheFilePath,
_request.ProjectStyle,
restoreTime.Elapsed));
}
}
public void LibrariesAreResolvedCorrectly(string packageName, string framework, bool resolved)
{
var repo = new PackageRepository("path/to/packages");
var resolver = new NuGetDependencyResolver(repo);
var net45Target = new LockFileTarget
{
TargetFramework = new FrameworkName(".NETFramework,Version=v4.5"),
Libraries = new List<LockFileTargetLibrary>
{
new LockFileTargetLibrary
{
Name = "MetaPackage",
Version = SemanticVersion.Parse("1.0.0"),
},
new LockFileTargetLibrary
{
Name = "Net451LibPackage",
Version = SemanticVersion.Parse("1.0.0"),
},
new LockFileTargetLibrary
{
Name = "Net451RefPackage",
Version = SemanticVersion.Parse("1.0.0"),
},
new LockFileTargetLibrary
{
Name = "AssemblyPlaceholderPackage",
Version = SemanticVersion.Parse("1.0.0"),
CompileTimeAssemblies = new List<LockFileItem> { Path.Combine("ref", "net45", "_._") },
RuntimeAssemblies = new List<LockFileItem> { Path.Combine("lib", "net45", "_._") }
}
}
};
var dnx451Target = new LockFileTarget
{
TargetFramework = new FrameworkName("DNX,Version=v4.5.1"),
Libraries = new List<LockFileTargetLibrary>
{
new LockFileTargetLibrary
{
Name = "MetaPackage",
Version = SemanticVersion.Parse("1.0.0")
},
new LockFileTargetLibrary
{
Name = "Net451LibPackage",
Version = SemanticVersion.Parse("1.0.0"),
CompileTimeAssemblies = new List<LockFileItem> { Path.Combine("lib", "net451", "Net451LibPackage.dll") },
RuntimeAssemblies = new List<LockFileItem> { Path.Combine("lib", "net451", "Net451LibPackage.dll") }
},
new LockFileTargetLibrary
{
Name = "Net451RefPackage",
Version = SemanticVersion.Parse("1.0.0"),
CompileTimeAssemblies = new List<LockFileItem> { Path.Combine("ref", "net451", "Net451LibPackage.dll") }
},
new LockFileTargetLibrary
{
Name = "AssemblyPlaceholderPackage",
Version = SemanticVersion.Parse("1.0.0")
}
}
};
var metaPackageLibrary = new LockFilePackageLibrary
{
Name = "MetaPackage",
Version = SemanticVersion.Parse("1.0.0")
};
var net451LibPackageLibrary = new LockFilePackageLibrary
{
Name = "Net451LibPackage",
Version = SemanticVersion.Parse("1.0.0"),
Files = new List<string>
{
Path.Combine("lib", "net451", "Net451LibPackage.dll"),
Path.Combine("lib", "net451", "Net451LibPackage.xml")
}
};
var net451RefPackageLibrary = new LockFilePackageLibrary
{
Name = "Net451RefPackage",
Version = SemanticVersion.Parse("1.0.0"),
Files = new List<string>
{
Path.Combine("ref", "net451", "Net451LibPackage.dll")
}
};
var assemblyPlaceholderPackageLibrary = new LockFilePackageLibrary
{
Name = "AssemblyPlaceholderPackage",
Version = SemanticVersion.Parse("1.0.0"),
Files = new List<string>
{
Path.Combine("lib", "net45", "_._"),
Path.Combine("ref", "net45", "_._"),
}
};
var lockFile = new LockFile()
{
Targets = new List<LockFileTarget> { net45Target, dnx451Target },
PackageLibraries = new List<LockFilePackageLibrary>
{
metaPackageLibrary,
net451LibPackageLibrary,
net451RefPackageLibrary,
assemblyPlaceholderPackageLibrary
}
};
resolver.ApplyLockFile(lockFile);
var libToLookup = new LibraryRange(packageName, frameworkReference: false);
Assert.Equal(resolved, resolver.GetDescription(libToLookup, new FrameworkName(framework)).Resolved);
}
public NoOpRestoreResult(bool success, LockFile lockFile, LockFile previousLockFile, string lockFilePath, CacheFile cacheFile, string cacheFilePath, ProjectStyle projectStyle, TimeSpan elapsedTime) :
base(success: success, restoreGraphs: null, compatibilityCheckResults: new List <CompatibilityCheckResult>(),
msbuildFiles: null, lockFile: lockFile, previousLockFile: previousLockFile, lockFilePath: lockFilePath,
cacheFile: cacheFile, cacheFilePath: cacheFilePath, projectStyle: projectStyle, elapsedTime: elapsedTime)
{
}
/**
* Try to establish an update lock on the cache file.
*
* @return true if the lock is now held by the caller; false if it is held
* by someone else.
* @
* the output file could not be created. The caller does not
* hold the lock.
*/
public bool Lock()
{
if (liveFile == null)
throw new IOException("DirCache does not have a backing file");
LockFile tmp = new LockFile(liveFile);
if (tmp.Lock())
{
tmp.NeedStatInformation=true;
myLock = tmp;
return true;
}
return false;
}
public LockFile CreateLockFile(
LockFile previousLockFile,
PackageSpec project,
IEnumerable <RestoreTargetGraph> targetGraphs,
IReadOnlyList <NuGetv3LocalRepository> localRepositories,
RemoteWalkContext context,
IEnumerable <ToolRestoreResult> toolRestoreResults)
{
var lockFile = new LockFile();
lockFile.Version = _lockFileVersion;
var previousLibraries = previousLockFile?.Libraries.ToDictionary(l => Tuple.Create(l.Name, l.Version));
// Use empty string as the key of dependencies shared by all frameworks
lockFile.ProjectFileDependencyGroups.Add(new ProjectFileDependencyGroup(
string.Empty,
project.Dependencies
.Select(group => group.LibraryRange.ToLockFileDependencyGroupString())
.OrderBy(group => group, StringComparer.Ordinal)));
foreach (var frameworkInfo in project.TargetFrameworks
.OrderBy(framework => framework.FrameworkName.ToString(),
StringComparer.Ordinal))
{
lockFile.ProjectFileDependencyGroups.Add(new ProjectFileDependencyGroup(
frameworkInfo.FrameworkName.ToString(),
frameworkInfo.Dependencies
.Select(x => x.LibraryRange.ToLockFileDependencyGroupString())
.OrderBy(dependency => dependency, StringComparer.Ordinal)));
}
// Record all libraries used
foreach (var item in targetGraphs.SelectMany(g => g.Flattened).Distinct()
.OrderBy(x => x.Data.Match.Library))
{
var library = item.Data.Match.Library;
if (project.Name.Equals(library.Name, StringComparison.OrdinalIgnoreCase))
{
// Do not include the project itself as a library.
continue;
}
if (library.Type == LibraryType.Project || library.Type == LibraryType.ExternalProject)
{
// Project
LocalMatch localMatch = (LocalMatch)item.Data.Match;
var projectLib = new LockFileLibrary()
{
Name = library.Name,
Version = library.Version,
Type = LibraryType.Project,
};
// Set the relative path if a path exists
// For projects without project.json this will be empty
if (!string.IsNullOrEmpty(localMatch.LocalLibrary.Path))
{
projectLib.Path = PathUtility.GetRelativePath(
project.FilePath,
localMatch.LocalLibrary.Path,
'/');
}
// The msbuild project path if it exists
object msbuildPath;
if (localMatch.LocalLibrary.Items.TryGetValue(KnownLibraryProperties.MSBuildProjectPath, out msbuildPath))
{
var msbuildRelativePath = PathUtility.GetRelativePath(
project.FilePath,
(string)msbuildPath,
'/');
projectLib.MSBuildProject = msbuildRelativePath;
}
lockFile.Libraries.Add(projectLib);
}
else if (library.Type == LibraryType.Package)
{
// Packages
var packageInfo = NuGetv3LocalRepositoryUtility.GetPackage(localRepositories, library.Name, library.Version);
if (packageInfo == null)
{
continue;
}
var package = packageInfo.Package;
var resolver = packageInfo.Repository.PathResolver;
var sha512 = File.ReadAllText(resolver.GetHashPath(package.Id, package.Version));
LockFileLibrary previousLibrary = null;
previousLibraries?.TryGetValue(Tuple.Create(library.Name, library.Version), out previousLibrary);
var lockFileLib = previousLibrary;
// If we have the same library in the lock file already, use that.
if (previousLibrary == null || previousLibrary.Sha512 != sha512)
{
var path = resolver.GetPackageDirectory(package.Id, package.Version);
path = PathUtility.GetPathWithForwardSlashes(path);
lockFileLib = CreateLockFileLibrary(
package,
sha512,
path);
}
else if (Path.DirectorySeparatorChar != LockFile.DirectorySeparatorChar)
{
// Fix slashes for content model patterns
lockFileLib.Files = lockFileLib.Files
.Select(p => p.Replace(Path.DirectorySeparatorChar, LockFile.DirectorySeparatorChar))
.ToList();
}
lockFile.Libraries.Add(lockFileLib);
var packageIdentity = new PackageIdentity(lockFileLib.Name, lockFileLib.Version);
context.PackageFileCache.TryAdd(packageIdentity, lockFileLib.Files);
}
}
var libraries = lockFile.Libraries.ToDictionary(lib => Tuple.Create(lib.Name, lib.Version));
var warnForImports = project.TargetFrameworks.Any(framework => framework.Warn);
var librariesWithWarnings = new HashSet <LibraryIdentity>();
// Add the targets
foreach (var targetGraph in targetGraphs
.OrderBy(graph => graph.Framework.ToString(), StringComparer.Ordinal)
.ThenBy(graph => graph.RuntimeIdentifier, StringComparer.Ordinal))
{
var target = new LockFileTarget();
target.TargetFramework = targetGraph.Framework;
target.RuntimeIdentifier = targetGraph.RuntimeIdentifier;
var flattenedFlags = IncludeFlagUtils.FlattenDependencyTypes(_includeFlagGraphs, project, targetGraph);
var fallbackFramework = target.TargetFramework as FallbackFramework;
var warnForImportsOnGraph = warnForImports && fallbackFramework != null;
foreach (var graphItem in targetGraph.Flattened.OrderBy(x => x.Key))
{
var library = graphItem.Key;
if (library.Type == LibraryType.Project || library.Type == LibraryType.ExternalProject)
{
if (project.Name.Equals(library.Name, StringComparison.OrdinalIgnoreCase))
{
// Do not include the project itself as a library.
continue;
}
var localMatch = (LocalMatch)graphItem.Data.Match;
// Target framework information is optional and may not exist for csproj projects
// that do not have a project.json file.
string projectFramework = null;
object frameworkInfoObject;
if (localMatch.LocalLibrary.Items.TryGetValue(
KnownLibraryProperties.TargetFrameworkInformation,
out frameworkInfoObject))
{
// Retrieve the resolved framework name, if this is null it means that the
// project is incompatible. This is marked as Unsupported.
var targetFrameworkInformation = (TargetFrameworkInformation)frameworkInfoObject;
projectFramework = targetFrameworkInformation.FrameworkName?.DotNetFrameworkName
?? NuGetFramework.UnsupportedFramework.DotNetFrameworkName;
}
// Create the target entry
var lib = new LockFileTargetLibrary()
{
Name = library.Name,
Version = library.Version,
Type = LibraryType.Project,
Framework = projectFramework,
// Find all dependencies which would be in the nuspec
// Include dependencies with no constraints, or package/project/external
// Exclude suppressed dependencies, the top level project is not written
// as a target so the node depth does not matter.
Dependencies = graphItem.Data.Dependencies
.Where(
d => (d.LibraryRange.TypeConstraintAllowsAnyOf(
LibraryDependencyTarget.PackageProjectExternal)) &&
d.SuppressParent != LibraryIncludeFlags.All)
.Select(d => GetDependencyVersionRange(d))
.ToList()
};
object compileAssetObject;
if (localMatch.LocalLibrary.Items.TryGetValue(
KnownLibraryProperties.CompileAsset,
out compileAssetObject))
{
var item = new LockFileItem((string)compileAssetObject);
lib.CompileTimeAssemblies.Add(item);
lib.RuntimeAssemblies.Add(item);
}
// Add frameworkAssemblies for projects
object frameworkAssembliesObject;
if (localMatch.LocalLibrary.Items.TryGetValue(
KnownLibraryProperties.FrameworkAssemblies,
out frameworkAssembliesObject))
{
lib.FrameworkAssemblies.AddRange((List <string>)frameworkAssembliesObject);
}
target.Libraries.Add(lib);
continue;
}
else if (library.Type == LibraryType.Package)
{
var packageInfo = NuGetv3LocalRepositoryUtility.GetPackage(localRepositories, library.Name, library.Version);
if (packageInfo == null)
{
continue;
}
var package = packageInfo.Package;
// include flags
LibraryIncludeFlags includeFlags;
if (!flattenedFlags.TryGetValue(library.Name, out includeFlags))
{
includeFlags = ~LibraryIncludeFlags.ContentFiles;
}
var targetLibrary = LockFileUtils.CreateLockFileTargetLibrary(
libraries[Tuple.Create(library.Name, library.Version)],
package,
targetGraph,
dependencyType: includeFlags,
targetFrameworkOverride: null,
dependencies: graphItem.Data.Dependencies);
target.Libraries.Add(targetLibrary);
// Log warnings if the target library used the fallback framework
if (warnForImportsOnGraph && !librariesWithWarnings.Contains(library))
{
var nonFallbackFramework = new NuGetFramework(fallbackFramework);
var targetLibraryWithoutFallback = LockFileUtils.CreateLockFileTargetLibrary(
libraries[Tuple.Create(library.Name, library.Version)],
package,
targetGraph,
targetFrameworkOverride: nonFallbackFramework,
dependencyType: includeFlags,
dependencies: graphItem.Data.Dependencies);
if (!targetLibrary.Equals(targetLibraryWithoutFallback))
{
var libraryName = $"{library.Name} {library.Version}";
_logger.LogWarning(string.Format(CultureInfo.CurrentCulture, Strings.Log_ImportsFallbackWarning, libraryName, String.Join(", ", fallbackFramework.Fallback), nonFallbackFramework));
// only log the warning once per library
librariesWithWarnings.Add(library);
}
}
}
}
lockFile.Targets.Add(target);
}
PopulateProjectFileToolGroups(project, lockFile);
PopulateTools(toolRestoreResults, lockFile);
PopulatePackageFolders(localRepositories.Select(repo => repo.RepositoryRoot).Distinct(), lockFile);
return(lockFile);
}
public void ItFiltersAnalyzersByProjectLanguage()
{
string projectLanguage = "C#";
// expected included analyzers
string[] expectIncluded = new string[] {
"analyzers/dotnet/IncludedAlpha.dll",
"analyzers/dotnet/cs/IncludedBeta.dll",
"analyzers/dotnet/cs/vb/IncludedChi.dll",
};
// expected excluded files
string[] expectExcluded = new string[] {
"analyzers/dotnet/vb/ExcludedAlpha.dll",
"analyzers/dotnet/ExcludedBeta.txt",
"analyzers/dotnet/cs/ExcludedChi.txt",
"dotnet/ExcludedDelta.dll",
"dotnet/cs/ExcludedEpsilon.dll"
};
var libCFiles = new List <string>()
{
"lib/file/G.dll", "lib/file/H.dll", "lib/file/I.dll"
};
libCFiles.AddRange(expectIncluded);
libCFiles.AddRange(expectExcluded);
string libCDefn = CreateLibrary("LibC/1.2.3", "package", libCFiles.ToArray());
string lockFileContent = CreateLockFileSnippet(
targets: new string[] {
CreateTarget(".NETCoreApp,Version=v1.0", TargetLibA, TargetLibB, TargetLibC),
CreateTarget(".NETCoreApp,Version=v1.0/osx.10.11-x64", TargetLibB, TargetLibC),
},
libraries: new string[] {
LibADefn, LibBDefn, libCDefn
},
projectFileDependencyGroups: new string[] { ProjectGroup, NETCoreGroup, NETCoreOsxGroup }
);
LockFile lockFile = TestLockFiles.CreateLockFile(lockFileContent);
var task = new ResolvePackageDependencies(lockFile, new MockPackageResolver())
{
ProjectAssetsFile = lockFile.Path,
ProjectPath = null,
ProjectLanguage = projectLanguage // set language
};
task.Execute().Should().BeTrue();
IEnumerable <ITaskItem> fileDefns;
fileDefns = task.FileDefinitions
.Where(t => t.GetMetadata(MetadataKeys.Type) == "AnalyzerAssembly");
fileDefns.Count().Should().Be(3);
var expectedTargets = new string[] {
".NETCoreApp,Version=v1.0",
".NETCoreApp,Version=v1.0/osx.10.11-x64"
};
foreach (var analyzer in expectIncluded)
{
var fileKey = $"LibC/1.2.3/{analyzer}";
var item = task.FileDefinitions.Where(t => t.ItemSpec == fileKey).First();
item.GetMetadata(MetadataKeys.Type).Should().Be("AnalyzerAssembly");
item.GetMetadata(MetadataKeys.Path).Should().Be(analyzer);
// expect two file dependencies for each, one per target
var fileDeps = task.FileDependencies.Where(t => t.ItemSpec == fileKey);
fileDeps.Count().Should().Be(2);
fileDeps.Select(f => f.GetMetadata(MetadataKeys.ParentTarget))
.Should().Contain(expectedTargets);
fileDeps.All(f => f.GetMetadata(MetadataKeys.ParentPackage) == "LibC/1.2.3");
}
foreach (var otherFile in expectExcluded)
{
var fileKey = $"LibC/1.2.3/{otherFile}";
var item = task.FileDefinitions.Where(t => t.ItemSpec == fileKey).First();
item.GetMetadata(MetadataKeys.Type).Should().NotBe("AnalyzerAssembly");
// expect no file dependencies for each
task.FileDependencies.Where(t => t.ItemSpec == fileKey)
.Should().BeEmpty();
}
}
private static void PopulateTools(IEnumerable <ToolRestoreResult> toolRestoreResults, LockFile lockFile)
{
// Add the tool targets (this states what tools were actually restored)
var toolTargets = new Dictionary <string, LockFileTarget>();
foreach (var result in toolRestoreResults)
{
if (result.FileTargetLibrary == null)
{
continue;
}
var newTarget = new LockFileTarget
{
TargetFramework = result.LockFileTarget.TargetFramework,
RuntimeIdentifier = result.LockFileTarget.RuntimeIdentifier
};
LockFileTarget existingTarget;
if (!toolTargets.TryGetValue(newTarget.Name, out existingTarget))
{
toolTargets[newTarget.Name] = newTarget;
existingTarget = newTarget;
lockFile.Tools.Add(newTarget);
}
existingTarget.Libraries.Add(result.FileTargetLibrary);
}
}
public ProjectContext Build()
{
ProjectDirectory = Project?.ProjectDirectory ?? ProjectDirectory;
if (GlobalSettings == null)
{
RootDirectory = ProjectRootResolver.ResolveRootDirectory(ProjectDirectory);
GlobalSettings globalSettings;
if (GlobalSettings.TryGetGlobalSettings(RootDirectory, out globalSettings))
{
GlobalSettings = globalSettings;
}
}
RootDirectory = GlobalSettings?.DirectoryPath ?? RootDirectory;
PackagesDirectory = PackagesDirectory ?? PackageDependencyProvider.ResolvePackagesPath(RootDirectory, GlobalSettings);
ReferenceAssembliesPath = ReferenceAssembliesPath ?? GetDefaultReferenceAssembliesPath();
LockFileLookup lockFileLookup = null;
EnsureProjectLoaded();
var projectLockJsonPath = Path.Combine(ProjectDirectory, LockFile.FileName);
if (LockFile == null && File.Exists(projectLockJsonPath))
{
LockFile = LockFileReader.Read(projectLockJsonPath);
}
var validLockFile = true;
string lockFileValidationMessage = null;
if (LockFile != null)
{
validLockFile = LockFile.IsValidForProject(Project, out lockFileValidationMessage);
lockFileLookup = new LockFileLookup(LockFile);
}
var libraries = new Dictionary <LibraryKey, LibraryDescription>();
var projectResolver = new ProjectDependencyProvider();
var mainProject = projectResolver.GetDescription(TargetFramework, Project);
// Add the main project
libraries.Add(new LibraryKey(mainProject.Identity.Name), mainProject);
LockFileTarget target = null;
if (lockFileLookup != null)
{
target = SelectTarget(LockFile);
if (target != null)
{
var packageResolver = new PackageDependencyProvider(PackagesDirectory);
ScanLibraries(target, lockFileLookup, libraries, packageResolver, projectResolver);
}
}
var frameworkReferenceResolver = new FrameworkReferenceResolver(ReferenceAssembliesPath);
var referenceAssemblyDependencyResolver = new ReferenceAssemblyDependencyResolver(frameworkReferenceResolver);
bool requiresFrameworkAssemblies;
// Resolve the dependencies
ResolveDependencies(libraries, referenceAssemblyDependencyResolver, out requiresFrameworkAssemblies);
var diagnostics = new List <DiagnosticMessage>();
// REVIEW: Should this be in NuGet (possibly stored in the lock file?)
if (LockFile == null)
{
diagnostics.Add(new DiagnosticMessage(
ErrorCodes.NU1009,
$"The expected lock file doesn't exist. Please run \"dotnet restore\" to generate a new lock file.",
Path.Combine(Project.ProjectDirectory, LockFile.FileName),
DiagnosticMessageSeverity.Error));
}
if (!validLockFile)
{
diagnostics.Add(new DiagnosticMessage(
ErrorCodes.NU1006,
$"{lockFileValidationMessage}. Please run \"dotnet restore\" to generate a new lock file.",
Path.Combine(Project.ProjectDirectory, LockFile.FileName),
DiagnosticMessageSeverity.Warning));
}
if (requiresFrameworkAssemblies)
{
var frameworkInfo = Project.GetTargetFramework(TargetFramework);
if (string.IsNullOrEmpty(ReferenceAssembliesPath))
{
// If there was an attempt to use reference assemblies but they were not installed
// report an error
diagnostics.Add(new DiagnosticMessage(
ErrorCodes.DOTNET1012,
$"The reference assemblies directory was not specified. You can set the location using the DOTNET_REFERENCE_ASSEMBLIES_PATH environment variable.",
filePath: Project.ProjectFilePath,
severity: DiagnosticMessageSeverity.Error,
startLine: frameworkInfo.Line,
startColumn: frameworkInfo.Column
));
}
else if (!frameworkReferenceResolver.IsInstalled(TargetFramework))
{
// If there was an attempt to use reference assemblies but they were not installed
// report an error
diagnostics.Add(new DiagnosticMessage(
ErrorCodes.DOTNET1011,
$"Framework not installed: {TargetFramework.DotNetFrameworkName} in {ReferenceAssembliesPath}",
filePath: Project.ProjectFilePath,
severity: DiagnosticMessageSeverity.Error,
startLine: frameworkInfo.Line,
startColumn: frameworkInfo.Column
));
}
}
// Create a library manager
var libraryManager = new LibraryManager(libraries.Values.ToList(), diagnostics, Project.ProjectFilePath);
return(new ProjectContext(
GlobalSettings,
mainProject,
TargetFramework,
target?.RuntimeIdentifier,
PackagesDirectory,
libraryManager));
}
private static void AddProjectFileDependenciesForPackageReference(PackageSpec project, LockFile lockFile, IEnumerable <RestoreTargetGraph> targetGraphs)
{
// For NETCore put everything under a TFM section
// Projects are included for NETCore
foreach (var frameworkInfo in project.TargetFrameworks
.OrderBy(framework => framework.FrameworkName.ToString(),
StringComparer.Ordinal))
{
var dependencies = new List <LibraryRange>();
dependencies.AddRange(project.Dependencies.Select(e => e.LibraryRange));
dependencies.AddRange(frameworkInfo.Dependencies.Select(e => e.LibraryRange));
var targetGraph = targetGraphs.SingleOrDefault(graph =>
graph.Framework.Equals(frameworkInfo.FrameworkName) &&
string.IsNullOrEmpty(graph.RuntimeIdentifier));
var resolvedEntry = targetGraph?
.Flattened
.SingleOrDefault(library => library.Key.Name.Equals(project.Name, StringComparison.OrdinalIgnoreCase));
Debug.Assert(resolvedEntry != null, "Unable to find project entry in target graph, project references will not be added");
// In some failure cases where there is a conflict the root level project cannot be resolved, this should be handled gracefully
if (resolvedEntry != null)
{
dependencies.AddRange(resolvedEntry.Data.Dependencies.Where(lib =>
lib.LibraryRange.TypeConstraint == LibraryDependencyTarget.ExternalProject)
.Select(lib => lib.LibraryRange));
}
var seen = new HashSet <string>(StringComparer.OrdinalIgnoreCase);
var uniqueDependencies = new List <LibraryRange>();
foreach (var dependency in dependencies)
{
if (seen.Add(dependency.Name))
{
uniqueDependencies.Add(dependency);
}
}
// Add entry
var dependencyGroup = new ProjectFileDependencyGroup(
frameworkInfo.FrameworkName.ToString(),
uniqueDependencies.Select(x => x.ToLockFileDependencyGroupString())
.OrderBy(dependency => dependency, StringComparer.Ordinal));
lockFile.ProjectFileDependencyGroups.Add(dependencyGroup);
}
}
public ResolvePackageDependencies(LockFile lockFile, IPackageResolver packageResolver)
: this()
{
_lockFile = lockFile;
_packageResolver = packageResolver;
}
private static void PopulatePackageFolders(IEnumerable <string> packageFolders, LockFile lockFile)
{
lockFile.PackageFolders.AddRange(packageFolders.Select(path => new LockFileItem(path)));
}
// key goal is to create background tracer that is independent of request.
public static async Task <bool> PerformBackgroundDeployment(
DeploymentInfoBase deployInfo,
IEnvironment environment,
IDeploymentSettingsManager settings,
TraceLevel traceLevel,
Uri uri,
bool waitForTempDeploymentCreation)
{
var tracer = traceLevel <= TraceLevel.Off ? NullTracer.Instance : new CascadeTracer(new XmlTracer(environment.TracePath, traceLevel), new ETWTracer(environment.RequestId, "POST"));
var traceFactory = new TracerFactory(() => tracer);
var backgroundTrace = tracer.Step(XmlTracer.BackgroundTrace, new Dictionary <string, string>
{
{ "url", uri.AbsolutePath },
{ "method", "POST" }
});
// For waiting on creation of temp deployment
var tempDeploymentCreatedTcs = new TaskCompletionSource <object>();
// For determining whether or not we failed to create the deployment due to lock contention.
// Needed for deployments where deferred deployment is not allowed. Will be set to false if
// lock contention occurs and AllowDeferredDeployment is false, otherwise true.
var deploymentWillOccurTcs = new TaskCompletionSource <bool>();
// This task will be let out of scope intentionally
var deploymentTask = Task.Run(() =>
{
try
{
// lock related
string lockPath = Path.Combine(environment.SiteRootPath, Constants.LockPath);
string deploymentLockPath = Path.Combine(lockPath, Constants.DeploymentLockFile);
string statusLockPath = Path.Combine(lockPath, Constants.StatusLockFile);
string hooksLockPath = Path.Combine(lockPath, Constants.HooksLockFile);
var statusLock = new LockFile(statusLockPath, traceFactory);
var hooksLock = new LockFile(hooksLockPath, traceFactory);
var deploymentLock = DeploymentLockFile.GetInstance(deploymentLockPath, traceFactory);
var analytics = new Analytics(settings, new ServerConfiguration(SystemEnvironment.Instance), traceFactory);
var deploymentStatusManager = new DeploymentStatusManager(environment, analytics, statusLock);
var siteBuilderFactory = new SiteBuilderFactory(new BuildPropertyProvider(), environment);
var webHooksManager = new WebHooksManager(tracer, environment, hooksLock);
var deploymentManager = new DeploymentManager(siteBuilderFactory, environment, traceFactory, analytics, settings, deploymentStatusManager, deploymentLock, NullLogger.Instance, webHooksManager);
var fetchDeploymentManager = new FetchDeploymentManager(settings, environment, tracer, deploymentLock, deploymentManager, deploymentStatusManager);
IDisposable tempDeployment = null;
try
{
// Perform deployment
deploymentLock.LockOperation(() =>
{
deploymentWillOccurTcs.TrySetResult(true);
ChangeSet tempChangeSet = null;
if (waitForTempDeploymentCreation)
{
// create temporary deployment before the actual deployment item started
// this allows portal ui to readily display on-going deployment (not having to wait for fetch to complete).
// in addition, it captures any failure that may occur before the actual deployment item started
tempDeployment = deploymentManager.CreateTemporaryDeployment(
Resources.ReceivingChanges,
out tempChangeSet,
deployInfo.TargetChangeset,
deployInfo.Deployer);
tempDeploymentCreatedTcs.TrySetResult(null);
}
fetchDeploymentManager.PerformDeployment(deployInfo, tempDeployment, tempChangeSet).Wait();
}, "Performing continuous deployment", TimeSpan.Zero);
}
catch (LockOperationException)
{
if (tempDeployment != null)
{
tempDeployment.Dispose();
}
if (deployInfo.AllowDeferredDeployment)
{
deploymentWillOccurTcs.TrySetResult(true);
using (tracer.Step("Update pending deployment marker file"))
{
// REVIEW: This makes the assumption that the repository url is the same.
// If it isn't the result would be buggy either way.
FileSystemHelpers.SetLastWriteTimeUtc(fetchDeploymentManager._markerFilePath, DateTime.UtcNow);
}
}
}
}
catch (Exception ex)
{
tracer.TraceError(ex);
}
finally
{
// Will no-op if already set
deploymentWillOccurTcs.TrySetResult(false);
backgroundTrace.Dispose();
}
});
#pragma warning disable 4014
// Run on BG task (Task.Run) to avoid ASP.NET Request thread terminated with request completion and
// it doesn't get chance to clean up the pending marker.
Task.Run(() => PostDeploymentHelper.TrackPendingOperation(deploymentTask, TimeSpan.Zero));
#pragma warning restore 4014
// When the frontend/ARM calls /deploy with isAsync=true, it starts polling
// the deployment status immediately, so it's important that the temp deployment
// is created before we return.
if (waitForTempDeploymentCreation)
{
// deploymentTask may return withoout creating the temp deployment (lock contention,
// other exception), in which case just continue.
await Task.WhenAny(tempDeploymentCreatedTcs.Task, deploymentTask);
}
// If deferred deployment is not permitted, we need to know whether or not the deployment was
// successfully requested. Otherwise, to preserve existing behavior, we assume it was.
if (!deployInfo.AllowDeferredDeployment)
{
return(await deploymentWillOccurTcs.Task);
}
else
{
return(true);
}
}
public LockFile CreateLockFile(LockFile previousLockFile,
PackageSpec project,
IEnumerable <RestoreTargetGraph> targetGraphs,
IReadOnlyList <NuGetv3LocalRepository> localRepositories,
RemoteWalkContext context)
{
var lockFile = new LockFile()
{
Version = _lockFileVersion
};
var previousLibraries = previousLockFile?.Libraries.ToDictionary(l => Tuple.Create(l.Name, l.Version));
if (project.RestoreMetadata?.ProjectStyle == ProjectStyle.PackageReference ||
project.RestoreMetadata?.ProjectStyle == ProjectStyle.DotnetToolReference)
{
AddProjectFileDependenciesForPackageReference(project, lockFile, targetGraphs);
}
else
{
AddProjectFileDependenciesForSpec(project, lockFile);
}
// Record all libraries used
foreach (var item in targetGraphs.SelectMany(g => g.Flattened).Distinct()
.OrderBy(x => x.Data.Match.Library))
{
var library = item.Data.Match.Library;
if (project.Name.Equals(library.Name, StringComparison.OrdinalIgnoreCase))
{
// Do not include the project itself as a library.
continue;
}
if (library.Type == LibraryType.Project || library.Type == LibraryType.ExternalProject)
{
// Project
var localMatch = (LocalMatch)item.Data.Match;
var projectLib = new LockFileLibrary()
{
Name = library.Name,
Version = library.Version,
Type = LibraryType.Project,
};
// Set the relative path if a path exists
// For projects without project.json this will be empty
if (!string.IsNullOrEmpty(localMatch.LocalLibrary.Path))
{
projectLib.Path = PathUtility.GetRelativePath(
project.FilePath,
localMatch.LocalLibrary.Path,
'/');
}
// The msbuild project path if it exists
object msbuildPath;
if (localMatch.LocalLibrary.Items.TryGetValue(KnownLibraryProperties.MSBuildProjectPath, out msbuildPath))
{
var msbuildRelativePath = PathUtility.GetRelativePath(
project.FilePath,
(string)msbuildPath,
'/');
projectLib.MSBuildProject = msbuildRelativePath;
}
lockFile.Libraries.Add(projectLib);
}
else if (library.Type == LibraryType.Package)
{
// Packages
var packageInfo = NuGetv3LocalRepositoryUtility.GetPackage(localRepositories, library.Name, library.Version);
// Add the library if it was resolved, unresolved packages are not added to the assets file.
if (packageInfo != null)
{
var package = packageInfo.Package;
var resolver = packageInfo.Repository.PathResolver;
var sha512 = package.Sha512;
var path = PathUtility.GetPathWithForwardSlashes(resolver.GetPackageDirectory(package.Id, package.Version));
LockFileLibrary lockFileLib = null;
LockFileLibrary previousLibrary = null;
if (previousLibraries?.TryGetValue(Tuple.Create(package.Id, package.Version), out previousLibrary) == true)
{
// Check that the previous library is still valid
if (previousLibrary != null &&
StringComparer.Ordinal.Equals(path, previousLibrary.Path) &&
StringComparer.Ordinal.Equals(sha512, previousLibrary.Sha512))
{
// We mutate this previous library so we must take a clone of it. This is
// important because later, when deciding whether the lock file has changed,
// we compare the new lock file to the previous (in-memory) lock file.
lockFileLib = previousLibrary.Clone();
}
}
// Create a new lock file library if one doesn't exist already.
if (lockFileLib == null)
{
lockFileLib = CreateLockFileLibrary(package, sha512, path);
}
// Create a new lock file library
lockFile.Libraries.Add(lockFileLib);
}
}
}
var libraries = lockFile.Libraries.ToDictionary(lib => Tuple.Create(lib.Name, lib.Version));
var librariesWithWarnings = new HashSet <LibraryIdentity>();
var rootProjectStyle = project.RestoreMetadata?.ProjectStyle ?? ProjectStyle.Unknown;
// Cache package data and selection criteria across graphs.
var builderCache = new LockFileBuilderCache();
// Add the targets
foreach (var targetGraph in targetGraphs
.OrderBy(graph => graph.Framework.ToString(), StringComparer.Ordinal)
.ThenBy(graph => graph.RuntimeIdentifier, StringComparer.Ordinal))
{
var target = new LockFileTarget
{
TargetFramework = targetGraph.Framework,
RuntimeIdentifier = targetGraph.RuntimeIdentifier
};
var flattenedFlags = IncludeFlagUtils.FlattenDependencyTypes(_includeFlagGraphs, project, targetGraph);
// Check if warnings should be displayed for the current framework.
var tfi = project.GetTargetFramework(targetGraph.Framework);
var warnForImportsOnGraph = tfi.Warn &&
(target.TargetFramework is FallbackFramework ||
target.TargetFramework is AssetTargetFallbackFramework);
foreach (var graphItem in targetGraph.Flattened.OrderBy(x => x.Key))
{
var library = graphItem.Key;
// include flags
LibraryIncludeFlags includeFlags;
if (!flattenedFlags.TryGetValue(library.Name, out includeFlags))
{
includeFlags = ~LibraryIncludeFlags.ContentFiles;
}
if (library.Type == LibraryType.Project || library.Type == LibraryType.ExternalProject)
{
if (project.Name.Equals(library.Name, StringComparison.OrdinalIgnoreCase))
{
// Do not include the project itself as a library.
continue;
}
var projectLib = LockFileUtils.CreateLockFileTargetProject(
graphItem,
library,
includeFlags,
targetGraph,
rootProjectStyle);
target.Libraries.Add(projectLib);
continue;
}
else if (library.Type == LibraryType.Package)
{
var packageInfo = NuGetv3LocalRepositoryUtility.GetPackage(localRepositories, library.Name, library.Version);
if (packageInfo == null)
{
continue;
}
var package = packageInfo.Package;
var targetLibrary = LockFileUtils.CreateLockFileTargetLibrary(
libraries[Tuple.Create(library.Name, library.Version)],
package,
targetGraph,
dependencyType: includeFlags,
targetFrameworkOverride: null,
dependencies: graphItem.Data.Dependencies,
cache: builderCache);
target.Libraries.Add(targetLibrary);
// Log warnings if the target library used the fallback framework
if (warnForImportsOnGraph && !librariesWithWarnings.Contains(library))
{
var nonFallbackFramework = new NuGetFramework(target.TargetFramework);
var targetLibraryWithoutFallback = LockFileUtils.CreateLockFileTargetLibrary(
libraries[Tuple.Create(library.Name, library.Version)],
package,
targetGraph,
targetFrameworkOverride: nonFallbackFramework,
dependencyType: includeFlags,
dependencies: graphItem.Data.Dependencies,
cache: builderCache);
if (!targetLibrary.Equals(targetLibraryWithoutFallback))
{
var libraryName = DiagnosticUtility.FormatIdentity(library);
var message = string.Format(CultureInfo.CurrentCulture,
Strings.Log_ImportsFallbackWarning,
libraryName,
GetFallbackFrameworkString(target.TargetFramework),
nonFallbackFramework);
var logMessage = RestoreLogMessage.CreateWarning(
NuGetLogCode.NU1701,
message,
library.Name,
targetGraph.TargetGraphName);
_logger.Log(logMessage);
// only log the warning once per library
librariesWithWarnings.Add(library);
}
}
}
}
lockFile.Targets.Add(target);
}
PopulatePackageFolders(localRepositories.Select(repo => repo.RepositoryRoot).Distinct(), lockFile);
// Add the original package spec to the lock file.
lockFile.PackageSpec = project;
return(lockFile);
}
private void UpdateLockFile(PublishRoot root)
{
var lockFileFormat = new LockFileFormat();
string lockFilePath;
if (root.NoSource)
{
lockFilePath = Path.Combine(TargetPath, "root", LockFileFormat.LockFileName);
}
else
{
lockFilePath = Path.Combine(TargetPath, LockFileFormat.LockFileName);
}
LockFile lockFile;
if (File.Exists(lockFilePath))
{
lockFile = lockFileFormat.Read(lockFilePath);
}
else
{
lockFile = new LockFile
{
Islocked = false
};
var project = GetCurrentProject();
// Restore dependency groups for future lockfile validation
lockFile.ProjectFileDependencyGroups.Add(new ProjectFileDependencyGroup(
string.Empty,
project.Dependencies.Select(x => x.LibraryRange.ToString())));
foreach (var frameworkInfo in project.GetTargetFrameworks())
{
lockFile.ProjectFileDependencyGroups.Add(new ProjectFileDependencyGroup(
frameworkInfo.FrameworkName.ToString(),
frameworkInfo.Dependencies.Select(x => x.LibraryRange.ToString())));
}
}
if (root.NoSource)
{
// The dependency group shared by all frameworks should only contain the main nupkg (i.e. entrypoint)
lockFile.ProjectFileDependencyGroups.RemoveAll(g => string.IsNullOrEmpty(g.FrameworkName));
lockFile.ProjectFileDependencyGroups.Add(new ProjectFileDependencyGroup(
string.Empty,
new[] { _libraryDescription.LibraryRange.ToString() }));
}
var repository = new PackageRepository(root.TargetPackagesPath);
var resolver = new DefaultPackagePathResolver(root.TargetPackagesPath);
// For dependency projects that were published to nupkgs
// Add them to lockfile to ensure the contents of lockfile are still valid
using (var sha512 = SHA512.Create())
{
foreach (var publishProject in root.Projects.Where(p => p.IsPackage))
{
var packageInfo = repository.FindPackagesById(publishProject.Name)
.SingleOrDefault();
if (packageInfo == null)
{
root.Reports.Information.WriteLine("Unable to locate published package {0} in {1}",
publishProject.Name.Yellow(),
repository.RepositoryRoot);
continue;
}
var package = packageInfo.Package;
var nupkgPath = resolver.GetPackageFilePath(package.Id, package.Version);
var project = publishProject.GetCurrentProject();
lockFile.Libraries.Add(LockFileUtils.CreateLockFileLibraryForProject(
project,
package,
sha512,
project.GetTargetFrameworks().Select(f => f.FrameworkName),
resolver));
}
}
lockFileFormat.Write(lockFilePath, lockFile);
}
private static IEnumerable <string> GetNormalizedPackageFolders(LockFile lockFile)
{
return(lockFile.PackageFolders.Select(pf => pf.Path.TrimEnd(Path.DirectorySeparatorChar)));
}
private static void RegisterServices(IKernel kernel)
{
var serverConfiguration = new ServerConfiguration();
// Make sure %HOME% is correctly set
EnsureHomeEnvironmentVariable();
EnsureSiteBitnessEnvironmentVariable();
IEnvironment environment = GetEnvironment();
// Per request environment
kernel.Bind <IEnvironment>().ToMethod(context => GetEnvironment(context.Kernel.Get <IDeploymentSettingsManager>()))
.InRequestScope();
// General
kernel.Bind <HttpContextBase>().ToMethod(context => new HttpContextWrapper(HttpContext.Current))
.InRequestScope();
kernel.Bind <IServerConfiguration>().ToConstant(serverConfiguration);
kernel.Bind <IBuildPropertyProvider>().ToConstant(new BuildPropertyProvider());
System.Func <ITracer> createTracerThunk = () => GetTracer(environment, kernel);
System.Func <ILogger> createLoggerThunk = () => GetLogger(environment, kernel);
// First try to use the current request profiler if any, otherwise create a new one
var traceFactory = new TracerFactory(() => TraceServices.CurrentRequestTracer ?? createTracerThunk());
kernel.Bind <ITracer>().ToMethod(context => TraceServices.CurrentRequestTracer ?? NullTracer.Instance);
kernel.Bind <ITraceFactory>().ToConstant(traceFactory);
TraceServices.SetTraceFactory(createTracerThunk, createLoggerThunk);
// Setup the deployment lock
string lockPath = Path.Combine(environment.SiteRootPath, Constants.LockPath);
string deploymentLockPath = Path.Combine(lockPath, Constants.DeploymentLockFile);
string statusLockPath = Path.Combine(lockPath, Constants.StatusLockFile);
string sshKeyLockPath = Path.Combine(lockPath, Constants.SSHKeyLockFile);
string hooksLockPath = Path.Combine(lockPath, Constants.HooksLockFile);
_deploymentLock = new DeploymentLockFile(deploymentLockPath, kernel.Get <ITraceFactory>());
_deploymentLock.InitializeAsyncLocks();
var statusLock = new LockFile(statusLockPath, kernel.Get <ITraceFactory>());
var sshKeyLock = new LockFile(sshKeyLockPath, kernel.Get <ITraceFactory>());
var hooksLock = new LockFile(hooksLockPath, kernel.Get <ITraceFactory>());
kernel.Bind <IOperationLock>().ToConstant(sshKeyLock).WhenInjectedInto <SSHKeyController>();
kernel.Bind <IOperationLock>().ToConstant(statusLock).WhenInjectedInto <DeploymentStatusManager>();
kernel.Bind <IOperationLock>().ToConstant(hooksLock).WhenInjectedInto <WebHooksManager>();
kernel.Bind <IOperationLock>().ToConstant(_deploymentLock);
var shutdownDetector = new ShutdownDetector();
shutdownDetector.Initialize();
IDeploymentSettingsManager noContextDeploymentsSettingsManager =
new DeploymentSettingsManager(new XmlSettings.Settings(GetSettingsPath(environment)));
var noContextTraceFactory = new TracerFactory(() => GetTracerWithoutContext(environment, noContextDeploymentsSettingsManager));
kernel.Bind <IAnalytics>().ToMethod(context => new Analytics(context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <IServerConfiguration>(),
noContextTraceFactory));
// Trace unhandled (crash) exceptions.
AppDomain.CurrentDomain.UnhandledException += (sender, args) =>
{
var ex = args.ExceptionObject as Exception;
if (ex != null)
{
kernel.Get <IAnalytics>().UnexpectedException(ex);
}
};
// Trace shutdown event
// Cannot use shutdownDetector.Token.Register because of race condition
// with NinjectServices.Stop via WebActivator.ApplicationShutdownMethodAttribute
Shutdown += () => TraceShutdown(environment, noContextDeploymentsSettingsManager);
// LogStream service
// The hooks and log stream start endpoint are low traffic end-points. Re-using it to avoid creating another lock
var logStreamManagerLock = hooksLock;
kernel.Bind <LogStreamManager>().ToMethod(context => new LogStreamManager(Path.Combine(environment.RootPath, Constants.LogFilesPath),
context.Kernel.Get <IEnvironment>(),
context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <ITracer>(),
shutdownDetector,
logStreamManagerLock));
kernel.Bind <InfoRefsController>().ToMethod(context => new InfoRefsController(t => context.Kernel.Get(t)))
.InRequestScope();
kernel.Bind <CustomGitRepositoryHandler>().ToMethod(context => new CustomGitRepositoryHandler(t => context.Kernel.Get(t)))
.InRequestScope();
// Deployment Service
kernel.Bind <ISettings>().ToMethod(context => new XmlSettings.Settings(GetSettingsPath(environment)))
.InRequestScope();
kernel.Bind <IDeploymentSettingsManager>().To <DeploymentSettingsManager>()
.InRequestScope();
kernel.Bind <IDeploymentStatusManager>().To <DeploymentStatusManager>()
.InRequestScope();
kernel.Bind <ISiteBuilderFactory>().To <SiteBuilderFactory>()
.InRequestScope();
kernel.Bind <IWebHooksManager>().To <WebHooksManager>()
.InRequestScope();
ITriggeredJobsManager triggeredJobsManager = new TriggeredJobsManager(
noContextTraceFactory,
kernel.Get <IEnvironment>(),
kernel.Get <IDeploymentSettingsManager>(),
kernel.Get <IAnalytics>(),
kernel.Get <IWebHooksManager>());
kernel.Bind <ITriggeredJobsManager>().ToConstant(triggeredJobsManager)
.InTransientScope();
IContinuousJobsManager continuousJobManager = new ContinuousJobsManager(
noContextTraceFactory,
kernel.Get <IEnvironment>(),
kernel.Get <IDeploymentSettingsManager>(),
kernel.Get <IAnalytics>());
triggeredJobsManager.CleanupDeletedJobs();
continuousJobManager.CleanupDeletedJobs();
kernel.Bind <IContinuousJobsManager>().ToConstant(continuousJobManager)
.InTransientScope();
kernel.Bind <ILogger>().ToMethod(context => GetLogger(environment, context.Kernel))
.InRequestScope();
kernel.Bind <IDeploymentManager>().To <DeploymentManager>()
.InRequestScope();
kernel.Bind <IAutoSwapHandler>().To <AutoSwapHandler>()
.InRequestScope();
kernel.Bind <ISSHKeyManager>().To <SSHKeyManager>()
.InRequestScope();
kernel.Bind <IRepositoryFactory>().ToMethod(context => _deploymentLock.RepositoryFactory = new RepositoryFactory(context.Kernel.Get <IEnvironment>(),
context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <ITraceFactory>(),
context.Kernel.Get <HttpContextBase>()))
.InRequestScope();
kernel.Bind <IApplicationLogsReader>().To <ApplicationLogsReader>()
.InSingletonScope();
// Git server
kernel.Bind <IDeploymentEnvironment>().To <DeploymentEnvrionment>();
kernel.Bind <IGitServer>().ToMethod(context => new GitExeServer(context.Kernel.Get <IEnvironment>(),
_deploymentLock,
GetRequestTraceFile(context.Kernel),
context.Kernel.Get <IRepositoryFactory>(),
context.Kernel.Get <IDeploymentEnvironment>(),
context.Kernel.Get <IDeploymentSettingsManager>(),
context.Kernel.Get <ITraceFactory>()))
.InRequestScope();
// Git Servicehook parsers
kernel.Bind <IServiceHookHandler>().To <GenericHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <GitHubHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <BitbucketHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <DropboxHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <CodePlexHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <CodebaseHqHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <GitlabHqHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <GitHubCompatHandler>().InRequestScope();
kernel.Bind <IServiceHookHandler>().To <KilnHgHandler>().InRequestScope();
// SiteExtensions
kernel.Bind <ISiteExtensionManager>().To <SiteExtensionManager>().InRequestScope();
// Command executor
kernel.Bind <ICommandExecutor>().ToMethod(context => GetCommandExecutor(environment, context))
.InRequestScope();
MigrateSite(environment, noContextDeploymentsSettingsManager);
RemoveOldTracePath(environment);
// Temporary fix for https://github.com/npm/npm/issues/5905
EnsureNpmGlobalDirectory();
RegisterRoutes(kernel, RouteTable.Routes);
// Register the default hubs route: ~/signalr
GlobalHost.DependencyResolver = new SignalRNinjectDependencyResolver(kernel);
GlobalConfiguration.Configuration.Filters.Add(
new TraceDeprecatedActionAttribute(
kernel.Get <IAnalytics>(),
kernel.Get <ITraceFactory>()));
GlobalConfiguration.Configuration.Filters.Add(new EnsureRequestIdHandlerAttribute());
}
private void PopulateProjectAndPackageReferences(PackageBuilder packageBuilder, LockFile assetsFile,
IDictionary <string, string> projectRefToVersionMap,
ISet <NuGetFramework> frameworksWithSuppressedDependencies)
{
var dependenciesByFramework = new Dictionary <NuGetFramework, HashSet <LibraryDependency> >();
InitializeProjectDependencies(assetsFile, dependenciesByFramework, projectRefToVersionMap, frameworksWithSuppressedDependencies);
InitializePackageDependencies(assetsFile, dependenciesByFramework, frameworksWithSuppressedDependencies);
foreach (var pair in dependenciesByFramework)
{
PackCommandRunner.AddDependencyGroups(pair.Value, pair.Key, packageBuilder);
}
}
public void tryRenameWhenLocked(string toLock, string fromName,
string toName, string headPointsTo)
{
// setup
writeSymref(Constants.HEAD, headPointsTo);
ObjectId oldfromId = db.Resolve(fromName);
ObjectId oldHeadId = db.Resolve(Constants.HEAD);
writeReflog(db, oldfromId, oldfromId, "Just a message",
fromName);
IList<ReflogReader.Entry> oldFromLog = db
.ReflogReader(fromName).getReverseEntries();
IList<ReflogReader.Entry> oldHeadLog = oldHeadId != null ? db
.ReflogReader(Constants.HEAD).getReverseEntries() : null;
Assert.IsTrue(new FileInfo(Path.Combine(db.Directory.FullName, "logs/" + fromName)).Exists, "internal check, we have a log");
// "someone" has branch X locked
var lockFile = new LockFile(new FileInfo(Path.Combine(db.Directory.FullName, toLock)));
try
{
Assert.IsTrue(lockFile.Lock());
// Now this is our test
RefRename renameRef = db.RenameRef(fromName, toName);
RefUpdate.RefUpdateResult result = renameRef.rename();
Assert.AreEqual(RefUpdate.RefUpdateResult.LOCK_FAILURE, result);
// Check that the involved refs are the same despite the failure
assertExists(false, toName);
if (!toLock.Equals(toName))
assertExists(false, toName + ".lock");
assertExists(true, toLock + ".lock");
if (!toLock.Equals(fromName))
assertExists(false, "logs/" + fromName + ".lock");
assertExists(false, "logs/" + toName + ".lock");
Assert.AreEqual(oldHeadId, db.Resolve(Constants.HEAD));
Assert.AreEqual(oldfromId, db.Resolve(fromName));
Assert.IsNull(db.Resolve(toName));
Assert.AreEqual(oldFromLog.ToString(), db.ReflogReader(fromName)
.getReverseEntries().ToString());
if (oldHeadId != null)
Assert.AreEqual(oldHeadLog.ToString(), db.ReflogReader(Constants.HEAD)
.getReverseEntries().ToString());
}
finally
{
lockFile.Unlock();
}
}
private static void InitializeProjectDependencies(
LockFile assetsFile,
IDictionary <NuGetFramework, HashSet <LibraryDependency> > dependenciesByFramework,
IDictionary <string, string> projectRefToVersionMap,
ISet <NuGetFramework> frameworkWithSuppressedDependencies)
{
// From the package spec, all we know is each absolute path to the project reference the the target
// framework that project reference applies to.
if (assetsFile.PackageSpec.RestoreMetadata == null)
{
return;
}
// Using the libraries section of the assets file, the library name and version for the project path.
var projectPathToLibraryIdentities = assetsFile
.Libraries
.Where(library => library.MSBuildProject != null)
.ToLookup(
library => Path.GetFullPath(Path.Combine(
Path.GetDirectoryName(assetsFile.PackageSpec.RestoreMetadata.ProjectPath),
PathUtility.GetPathWithDirectorySeparator(library.MSBuildProject))),
library => new PackageIdentity(library.Name, library.Version),
PathUtility.GetStringComparerBasedOnOS());
// Consider all of the project references, grouped by target framework.
foreach (var framework in assetsFile.PackageSpec.RestoreMetadata.TargetFrameworks)
{
var target = assetsFile.GetTarget(framework.FrameworkName, runtimeIdentifier: null);
if (target == null || frameworkWithSuppressedDependencies.Contains(framework.FrameworkName))
{
continue;
}
HashSet <LibraryDependency> dependencies;
if (!dependenciesByFramework.TryGetValue(framework.FrameworkName, out dependencies))
{
dependencies = new HashSet <LibraryDependency>();
dependenciesByFramework[framework.FrameworkName] = dependencies;
}
// For the current target framework, create a map from library identity to library model. This allows
// us to be sure we have picked the correct library (name and version) for this target framework.
var libraryIdentityToTargetLibrary = target
.Libraries
.ToLookup(library => new PackageIdentity(library.Name, library.Version));
foreach (var projectReference in framework.ProjectReferences)
{
var libraryIdentities = projectPathToLibraryIdentities[projectReference.ProjectPath];
var targetLibrary = libraryIdentities
.Select(identity => libraryIdentityToTargetLibrary[identity].FirstOrDefault())
.FirstOrDefault(library => library != null);
if (targetLibrary == null)
{
continue;
}
var versionToUse = targetLibrary.Version;
// Use the project reference version obtained at build time if it exists, otherwise fallback to the one in assets file.
if (projectRefToVersionMap.TryGetValue(projectReference.ProjectPath, out var projectRefVersion))
{
versionToUse = NuGetVersion.Parse(projectRefVersion);
}
// TODO: Implement <TreatAsPackageReference>false</TreatAsPackageReference>
// https://github.com/NuGet/Home/issues/3891
//
// For now, assume the project reference is a package dependency.
var projectDependency = new LibraryDependency
{
LibraryRange = new LibraryRange(
targetLibrary.Name,
new VersionRange(versionToUse),
LibraryDependencyTarget.All),
IncludeType = projectReference.IncludeAssets & ~projectReference.ExcludeAssets,
SuppressParent = projectReference.PrivateAssets
};
PackCommandRunner.AddLibraryDependency(projectDependency, dependencies);
}
}
}
private void updateFETCH_HEAD(FetchResult result)
{
LockFile @lock = new LockFile(new FileInfo(Path.Combine(_transport.Local.Directory.FullName, "FETCH_HEAD")));
try
{
if (@lock.Lock())
{
StreamWriter w = new StreamWriter(@lock.GetOutputStream());
try
{
foreach (FetchHeadRecord h in _fetchHeadUpdates)
{
h.Write(w);
result.Add(h);
}
}
finally
{
w.Close();
}
@lock.Commit();
}
}
finally
{
@lock.Unlock();
}
}
private void updateFETCH_HEAD(FetchResult result)
{
using (LockFile @lock = new LockFile(PathUtil.CombineFilePath(_transport.Local.Directory, "FETCH_HEAD")))
{
if (@lock.Lock())
{
using (StreamWriter w = new StreamWriter(@lock.GetOutputStream()))
{
foreach (FetchHeadRecord h in _fetchHeadUpdates)
{
h.Write(w);
result.Add(h);
}
}
@lock.Commit();
}
}
}
/// <summary>
/// Try to establish an update lock on the cache file.
/// </summary>
/// <returns>
/// True if the lock is now held by the caller; false if it is held
/// by someone else.
/// </returns>
/// <exception cref="IOException">
/// The output file could not be created. The caller does not
/// hold the lock.
/// </exception>
public bool Lock()
{
if (_liveFile == null)
{
throw new IOException("DirCache does not have a backing file");
}
_myLock = new LockFile(_liveFile);
if (_myLock.Lock())
{
_myLock.NeedStatInformation = true;
return true;
}
return false;
}
private void It_targets_the_right_framework(
string testIdentifier,
string targetFramework,
string runtimeFrameworkVersion,
bool selfContained,
bool isExe,
string expectedPackageVersion,
string expectedRuntimeVersion,
string extraMSBuildArguments = null)
{
string runtimeIdentifier = null;
if (selfContained)
{
runtimeIdentifier = EnvironmentInfo.GetCompatibleRid(targetFramework);
}
var testProject = new TestProject()
{
Name = "FrameworkTargetTest",
TargetFrameworks = targetFramework,
RuntimeFrameworkVersion = runtimeFrameworkVersion,
IsExe = isExe,
RuntimeIdentifier = runtimeIdentifier
};
var extraArgs = extraMSBuildArguments?.Split(' ') ?? Array.Empty <string>();
var testAsset = _testAssetsManager.CreateTestProject(testProject, testIdentifier);
NuGetConfigWriter.Write(testAsset.TestRoot, NuGetConfigWriter.DotnetCoreBlobFeed);
var buildCommand = new BuildCommand(testAsset);
buildCommand
.Execute(extraArgs)
.Should()
.Pass();
var outputDirectory = buildCommand.GetOutputDirectory(targetFramework, runtimeIdentifier: runtimeIdentifier);
if (isExe)
{
// Self-contained apps don't write a framework version to the runtimeconfig, so only check this for framework-dependent apps
if (!selfContained)
{
string runtimeConfigFile = Path.Combine(outputDirectory.FullName, testProject.Name + ".runtimeconfig.json");
string runtimeConfigContents = File.ReadAllText(runtimeConfigFile);
JObject runtimeConfig = JObject.Parse(runtimeConfigContents);
string actualRuntimeFrameworkVersion = ((JValue)runtimeConfig["runtimeOptions"]["framework"]["version"]).Value <string>();
actualRuntimeFrameworkVersion.Should().Be(expectedRuntimeVersion);
}
var runtimeconfigDevFileName = testProject.Name + ".runtimeconfig.dev.json";
outputDirectory.Should()
.HaveFile(runtimeconfigDevFileName);
string devruntimeConfigContents = File.ReadAllText(Path.Combine(outputDirectory.FullName, runtimeconfigDevFileName));
JObject devruntimeConfig = JObject.Parse(devruntimeConfigContents);
var additionalProbingPaths = ((JArray)devruntimeConfig["runtimeOptions"]["additionalProbingPaths"]).Values <string>();
// can't use Path.Combine on segments with an illegal `|` character
var expectedPath = $"{Path.Combine(FileConstants.UserProfileFolder, ".dotnet", "store")}{Path.DirectorySeparatorChar}|arch|{Path.DirectorySeparatorChar}|tfm|";
additionalProbingPaths.Should().Contain(expectedPath);
}
LockFile lockFile = LockFileUtilities.GetLockFile(Path.Combine(buildCommand.ProjectRootPath, "obj", "project.assets.json"), NullLogger.Instance);
var target = lockFile.GetTarget(NuGetFramework.Parse(targetFramework), null);
var netCoreAppLibrary = target.Libraries.Single(l => l.Name == "Microsoft.NETCore.App");
netCoreAppLibrary.Version.ToString().Should().Be(expectedPackageVersion);
}
/**
* Commit this change and release the lock.
* <p>
* If this method fails (returns false) the lock is still released.
*
* @return true if the commit was successful and the file contains the new
* data; false if the commit failed and the file remains with the
* old data.
* @throws InvalidOperationException
* the lock is not held.
*/
public bool commit()
{
LockFile tmp = myLock;
requireLocked(tmp);
myLock = null;
if (!tmp.Commit())
return false;
lastModified = tmp.CommitLastModified;
return true;
}
public async Task CreatePathContextAsync_FromAssetsFile()
{
// Arrange
using (var testDirectory = TestDirectory.Create())
{
var userPackageFolder = Path.Combine(testDirectory.Path, "packagesA");
Directory.CreateDirectory(userPackageFolder);
var fallbackPackageFolder = Path.Combine(testDirectory.Path, "packagesB");
Directory.CreateDirectory(fallbackPackageFolder);
await SimpleTestPackageUtility.CreateFolderFeedV3Async(
userPackageFolder,
new PackageIdentity("Foo", NuGetVersion.Parse("1.0.1")));
await SimpleTestPackageUtility.CreateFolderFeedV3Async(
fallbackPackageFolder,
new PackageIdentity("Bar", NuGetVersion.Parse("1.0.2")));
var projectUniqueName = Guid.NewGuid().ToString();
var project = new Mock <EnvDTE.Project>();
var vsProjectAdapter = new Mock <IVsProjectAdapter>();
vsProjectAdapter
.Setup(x => x.BuildProperties.GetPropertyValueAsync("ProjectAssetsFile"))
.Returns(Task.FromResult("project.aseets.json"));
var vsProjectAdapterProvider = new Mock <IVsProjectAdapterProvider>();
vsProjectAdapterProvider
.Setup(x => x.CreateAdapterForFullyLoadedProjectAsync(project.Object))
.Returns(Task.FromResult(vsProjectAdapter.Object));
var target = new VsPathContextProvider(
Mock.Of <ISettings>(),
Mock.Of <IVsSolutionManager>(),
Mock.Of <ILogger>(),
vsProjectAdapterProvider.Object,
getLockFileOrNull: _ =>
{
var lockFile = new LockFile
{
PackageFolders = new[]
{
new LockFileItem(userPackageFolder),
new LockFileItem(fallbackPackageFolder)
},
Libraries = new[]
{
new LockFileLibrary
{
Type = LibraryType.Package,
Name = "Foo",
Version = NuGetVersion.Parse("1.0.1")
},
new LockFileLibrary
{
Type = LibraryType.Package,
Name = "Bar",
Version = NuGetVersion.Parse("1.0.2")
}
}
};
return(lockFile);
});
// Act
var actual = await target.CreatePathContextAsync(vsProjectAdapter.Object, "project.aseets.json", projectUniqueName, CancellationToken.None);
// Assert
Assert.NotNull(actual);
Assert.Equal(userPackageFolder, actual.UserPackageFolder);
Assert.Equal(
new[] { fallbackPackageFolder },
actual.FallbackPackageFolders.Cast <string>().ToArray());
string actualPackageDirectory = null;
var packageRootA = Path.Combine(userPackageFolder, "Foo", "1.0.1");
var assetFileA = Path.Combine(packageRootA, "lib", "net40", "a.dll");
Assert.True(actual.TryResolvePackageAsset(assetFileA, out actualPackageDirectory));
Assert.Equal(packageRootA, actualPackageDirectory, ignoreCase: true);
var packageRootB = Path.Combine(fallbackPackageFolder, "Bar", "1.0.2");
var assetFileB = Path.Combine(packageRootB, "lib", "net46", "b.dll");
Assert.True(actual.TryResolvePackageAsset(assetFileB, out actualPackageDirectory));
Assert.Equal(packageRootB, actualPackageDirectory, ignoreCase: true);
}
}
/**
* Unlock this file and abort this change.
* <p>
* The temporary file (if created) is deleted before returning.
*/
public void unlock()
{
LockFile tmp = myLock;
if (tmp != null)
{
myLock = null;
tmp.Unlock();
}
}
public void ItAddsAnalyzerMetadataAndFileDependencies()
{
string projectLanguage = "VB";
string libCDefn = CreateLibrary("LibC/1.2.3", "package",
"lib/file/G.dll", "lib/file/H.dll", "lib/file/I.dll",
"analyzers/dotnet/cs/Microsoft.CodeAnalysis.Analyzers.dll",
"analyzers/dotnet/cs/Microsoft.CodeAnalysis.CSharp.Analyzers.dll",
"analyzers/dotnet/vb/Microsoft.CodeAnalysis.Analyzers.dll",
"analyzers/dotnet/vb/Microsoft.CodeAnalysis.VisualBasic.Analyzers.dll",
"analyzers/dotnet/cs/Microsoft.CodeAnalysis.CSharp.Analyzers.txt", // not analyzer
"lib/file/Microsoft.CodeAnalysis.VisualBasic.Analyzers.dll" // not analyzer
);
string lockFileContent = CreateLockFileSnippet(
targets: new string[] {
CreateTarget(".NETCoreApp,Version=v1.0", TargetLibA, TargetLibB, TargetLibC),
CreateTarget(".NETCoreApp,Version=v1.0/osx.10.11-x64", TargetLibB, TargetLibC),
},
libraries: new string[] {
LibADefn, LibBDefn, libCDefn
},
projectFileDependencyGroups: new string[] { ProjectGroup, NETCoreGroup, NETCoreOsxGroup }
);
LockFile lockFile = TestLockFiles.CreateLockFile(lockFileContent);
var task = new ResolvePackageDependencies(lockFile, new MockPackageResolver())
{
ProjectAssetsFile = lockFile.Path,
ProjectPath = null,
ProjectLanguage = projectLanguage // set language
};
task.Execute().Should().BeTrue();
IEnumerable <ITaskItem> fileDefns;
fileDefns = task.FileDefinitions
.Where(t => t.GetMetadata(MetadataKeys.Type) == "AnalyzerAssembly");
fileDefns.Count().Should().Be(2);
var analyzers = new string[] {
"analyzers/dotnet/vb/Microsoft.CodeAnalysis.Analyzers.dll",
"analyzers/dotnet/vb/Microsoft.CodeAnalysis.VisualBasic.Analyzers.dll",
};
var expectedTargets = new string[] {
".NETCoreApp,Version=v1.0",
".NETCoreApp,Version=v1.0/osx.10.11-x64"
};
foreach (var analyzer in analyzers)
{
var fileKey = $"LibC/1.2.3/{analyzer}";
var item = task.FileDefinitions.Where(t => t.ItemSpec == fileKey).First();
item.GetMetadata(MetadataKeys.Type).Should().Be("AnalyzerAssembly");
item.GetMetadata(MetadataKeys.Path).Should().Be(analyzer);
// expect two file dependencies for each, one per target
var fileDeps = task.FileDependencies.Where(t => t.ItemSpec == fileKey);
fileDeps.Count().Should().Be(2);
fileDeps.Select(f => f.GetMetadata(MetadataKeys.ParentTarget))
.Should().Contain(expectedTargets);
fileDeps.All(f => f.GetMetadata(MetadataKeys.ParentPackage) == "LibC/1.2.3");
}
}
