title: Publish website, aspnet_compiler null reference exception
body: <p>I'm having trouble publishing a solution through the command line.  It intermittently fails on various projects, but every so often succeeds.  I get both the null reference error and "The application domain in which the thread was running has been unloaded" which may just be the null reference with a race condition.  It seems to only fail on projects that have web services (original .asmx and WCF with .svc or file-less activation).  The fact that it sometimes succeeds makes me think it's not an issue with our code, though there may be some changes we can do to mitigate the underlying cause (whatever that may be).</p>
  
  <p>What I tried/checked:</p>
  
  <ul>
  <li>I ran <code>tfpt scorch</code> to clean up any local changes including build output without success, so I know it's not an issue with the source directories.</li>
  <li>I cleared the temp files at <code>C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files</code> (we're on 32-bit) and seemed to get further but still had issues.  After another publish the directory was still empty so probably unrelated.</li>
  <li>With Process Monitor I found <code>aspnet_compiler</code> was writing to <code>%LocalAppData%\Temp\Temporary ASP.NET Files</code>.  Clearing these files seems to sometimes work or maybe I got lucky, but then it starts failing again.  I didn't spot anything else that may be relevant.</li>
  <li>Make sure I have enough RAM, didn't seem to make a difference.</li>
  <li>Run msbuild single-threaded, no difference</li>
  <li>Run the aspnet_compiler from the command line (get command from the MSBuild log output), always succeeds</li>
  <li>There are no compressed folders</li>
  <li>I'm on the latest stable Visual Studio (VS 2013 update 2), but that shouldn't matter since I'm running from the command line and the executables used are in the framework directory.</li>
  <li>There is no App_Code folder in some of the failing projects</li>
  </ul>
  
  <p>The build output doesn't give any good information so I set up DebugDiag to output on crash.  Here are the results:</p>
  
  <pre><code>DetailID = 1
      Count:    1
      Type:     System.NullReferenceException
      Message:  
      Stack:    
          System.Web.Compilation.DiskBuildResultCache.CacheBuildResult(System.String, System.Web.Compilation.BuildResult, Int64, System.DateTime)
          System.Web.Compilation.BuildManager.CacheBuildResultInternal(System.String, System.Web.Compilation.BuildResult, Int64, System.DateTime)
          System.Web.Compilation.WebDirectoryBatchCompiler.CacheAssemblyResults(System.Web.Compilation.AssemblyBuilder, System.CodeDom.Compiler.CompilerResults)
          System.Web.Compilation.WebDirectoryBatchCompiler.CompileAssemblyBuilder(System.Web.Compilation.AssemblyBuilder)
          System.Web.Compilation.WebDirectoryBatchCompiler.CompileAssemblyBuilderParallel(System.Collections.ICollection)
          System.Web.Compilation.WebDirectoryBatchCompiler.CompileNonDependentBuildProviders(System.Collections.ICollection)
          System.Web.Compilation.WebDirectoryBatchCompiler.Process()
          System.Web.Compilation.BuildManager.BatchCompileWebDirectoryInternal(System.Web.Hosting.VirtualDirectory, Boolean)
          System.Web.Compilation.BuildManager.BatchCompileWebDirectory(System.Web.Hosting.VirtualDirectory, System.Web.VirtualPath, Boolean)
          System.Web.Compilation.BuildManager.PrecompileWebDirectoriesRecursive(System.Web.Hosting.VirtualDirectory, Boolean)
          System.Web.Compilation.BuildManager.PrecompileAppInternal(System.Web.VirtualPath, System.Collections.Generic.IEnumerable`1&lt;System.String&gt;)
          System.Web.Compilation.BuildManager.PrecompileApp(System.Web.VirtualPath, System.Collections.Generic.IEnumerable`1&lt;System.String&gt;)
          System.Web.Compilation.BuildManager.PrecompileApp(System.Web.Compilation.ClientBuildManagerCallback, System.Collections.Generic.IEnumerable`1&lt;System.String&gt;)
          System.Web.Compilation.BuildManagerHost.PrecompileApp(System.Web.Compilation.ClientBuildManagerCallback, System.Collections.Generic.List`1&lt;System.String&gt;)
          [GCFrame]
          [GCFrame]
          [ContextTransitionFrame]
          [GCFrame]
          [GCFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [TPMethodFrame]
          *** WARNING: Unable to verify checksum for C:\Windows\assembly\NativeImages_v4.0.30319_32\System.Web\3d247ccfb800c38a29cf91c27a6339da\System.Web.ni.dll
          Module load completed but symbols could not be loaded for C:\Windows\assembly\NativeImages_v4.0.30319_32\System.Web\3d247ccfb800c38a29cf91c27a6339da\System.Web.ni.dll
          System.Web.Compilation.ClientBuildManager.PrecompileApplication(System.Web.Compilation.ClientBuildManagerCallback)
          *** ERROR: Module load completed but symbols could not be loaded for C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe
          [GCFrame]
  
  
  DetailID = 2
      Count:    1
      Type:     System.NullReferenceException
      Message:  Object reference not set to an instance of an object.
      Stack:    
          [GCFrame]
          [GCFrame]
          [GCFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [GCSafeCollectionFrame]
          [TPMethodFrame]
          System.Web.Compilation.ClientBuildManager.PrecompileApplication(System.Web.Compilation.ClientBuildManagerCallback, Boolean)
          System.Web.Compilation.ClientBuildManager.PrecompileApplication(System.Web.Compilation.ClientBuildManagerCallback)
          System.Web.Compilation.Precompiler.Main(System.String[])
          [GCFrame]
  </code></pre>
  
  <p>I have found others with similar issues but they were either unanswered or the solutions didn't work for me.  Any more ideas?</p>
  
  <p><strong>Edit:</strong>
  This is the command line I'm using:</p>
  
  <pre><code>"C:\Program Files (x86)\MSBuild\12.0\bin\MSBuild.exe" Solution.sln /t:Build /m:1 /p:Configuration=Prod /p:Platform="Mixed Platforms" /p:DeployOnBuild=true /p:PublishProfile=Prod /p:AutoParameterizationWebConfigConnectionStrings=false /v:diag &gt; out.txt
  </code></pre>
  
  <p>Here is some of the console output (redirected to the file):</p>
  
  <pre><code>Target "AspNetPreCompile: (TargetId:3773)" in file "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\Web\Transform\Microsoft.Web.Publishing.AspNetCompileMerge.targets" from project "C:\Source\Project\Project.csproj" (target "PipelineAspNetCompileMergePhase" depends on it):
  Task "AspNetCompiler" (TaskId:2474)
    Task Parameter:PhysicalPath=C:\Source\Project\obj\x86\Release\AspnetCompileMerge\Source (TaskId:2474)
    Task Parameter:TargetPath=C:\Source\Project\obj\x86\Release\AspnetCompileMerge\TempBuildDir (TaskId:2474)
    Task Parameter:VirtualPath=/ (TaskId:2474)
    Task Parameter:Debug=False (TaskId:2474)
    Task Parameter:Updateable=False (TaskId:2474)
    Task Parameter:ToolPath=C:\Windows\Microsoft.NET\Framework\v4.0.30319 (TaskId:2474)
  
    C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe -v / -p C:\Source\Project\obj\x86\Release\AspnetCompileMerge\Source C:\Source\Project\obj\x86\Release\AspnetCompileMerge\TempBuildDir  (TaskId:2474)
    Microsoft (R) ASP.NET Compilation Tool version 4.0.30319.34209 (TaskId:2474)
    Utility to precompile an ASP.NET application (TaskId:2474)
    Copyright (C) Microsoft Corporation. All rights reserved. (TaskId:2474)
     (TaskId:2474)
  ASPNETCOMPILER : error ASPRUNTIME: Object reference not set to an instance of an object. [C:\Source\Project\Project.csproj]
    The command exited with code 1. (TaskId:2474)
  
  Done executing task "AspNetCompiler" -- FAILED. (TaskId:2474)
  Done building target "AspNetPreCompile" in project "Project.csproj" -- FAILED.: (TargetId:3773)
  </code></pre>
  
  <p>So as you can see, the msbuild diagnostic output isn't very helpful.</p>
  
  <p><strong>Edit 2:</strong>
  Trying to see if I can debug <code>aspnet_compiler</code> to shed more light on this.  I kicked off <code>gflags.exe</code> to enable auto-attaching to the debugger.  I was able to break on the <code>NullReferenceException</code> but I'm having trouble loading the symbols for <code>System.Web</code>.  It's coming from <code>C:\Windows\Microsoft.Net\assembly\GAC_32\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll</code>.</p>
  
  <p>From the stack trace, it's failing in this method:</p>
  
  <pre><code>internal override void CacheBuildResult(string cacheKey, BuildResult result, long hashCode, DateTime utcStart)
  {
    if (!result.CacheToDisk)
      return;
    if (HostingEnvironment.ShutdownInitiated)
    {
      BuildResultCompiledAssemblyBase compiledAssemblyBase = result as BuildResultCompiledAssemblyBase;
      if (compiledAssemblyBase == null)
        return;
      this.MarkAssemblyAndRelatedFilesForDeletion(compiledAssemblyBase.ResultAssembly.GetName().Name);
    }
    else
      new PreservationFileWriter(this.PrecompilationMode).SaveBuildResultToFile(this.GetPreservedDataFileName(cacheKey), result, hashCode);
  }
  </code></pre>
  
  <p>So any of these could be null and causing the exception: <code>result</code>, <code>HostingEnvironment</code>, <code>compiledAssemblyBase</code>.</p>