.NET's loading of binaries is great in most standard .NET applications. Assemblies are loaded out of the application folder or a special private bin folder (like ASP.NET applications) and it all works as you would expect.
But, once you need to dynamically load assemblies and load them out of different folders things start getting pretty ugly fast. I've been to this rodeo quite a few times, and I've used different approaches and pretty much all of them are ugly. Recently when working on Markdown Monster I ran into this again and had some really odd issues and - another solution that I hadn't used before that I want to share.
Loading assemblies out of separate folders is not something you do very frequently in typical business applications. Most applications add references to their projects and the compiler and tooling handles spitting out the final required DLLs in the right folders. You take those folders and you're done.
The most common case for dynamic assembly loading involves some sort of addin mechanism where users can extend the application by creating custom components and plugging them into the current application, thereby extending the functionality.
I came to addins via my Markdown Monster Markdown Editor, which has support for addins where you can create custom extensions that extend the editor and UI. Basically it's quite easy to create a custom form that performs some task with the full Markdown document or manipulates specific text in the document - or publishes the markdown to some custom location. It seems like a natural fit for an editor - you write you want to push your document out to somewhere.
Adding addins was a very early decision when I decided to create this tool, because first of all I wanted to be able to publish my Markdown directly to my blog. And I wanted to capture images from screen shots and embed them. These two features happen to be implemented as separate addins that plug into the core Markdown Monster editor. Since then I've created a couple more (an Image to Azure Blob Storage uploader, a Gist Code embedding Addin). A few other people have also created addins.
Although I was very clear on wanting to create addins I didn't deal with all the issues of how to store the addins and get them downloaded and installed initially. My first take (Take 0.1) was to just dump all addins into an .\Addins folder and call it a day.
The application now has some startup code that checks the .DLLs in the Addins folder, checks for the addin interface and if so loads the addin.
Behind the scenes this sets the AppDomain's PrivateBin path. This also happens to be the only way to properly add the PrivateBin path for the main executable. While there is AppDomain.AppendPrivatePath() this method is obsolete and can cause some potential load order problems. It also has to be called very early on in the application likely as the first line of code before additional assemblies beyond mscorlib are loaded.
In my application I actually didn't need this. Rather than dealing with Private Bin paths I've always used Appdomain.AssemblyResolve which is fired whenever an assembly cannot be resolved. If you have a single folder it's very easy to find assemblies because you know where to look for any missing references that the application can't resolve on its own.
To put this in perspective here is how addins are loaded. The original addin loader runs through all assemblies in the Addins folder and checks for the Addin interface and if found loads the assembly with Assembly.LoadFrom().
The assembly typically loads without a problem, but the problem usually comes from any dependent assemblies that get loaded when scanning the assembly for types that return or pass dependent assembly types.
The asm = Assembly.LoadFrom(assemblyFile); never fails by itself - loading an assembly typically works. When an assembly is loaded only that assembly is loaded and not any of its dependencies.
But, when running asm.GetTypes(); additional types and dependencies are accessed and that triggers an assembly load attempt from .NET natively. If there's no additional probing or assembly resolve the code bombs.
If you are very vigilant about not bleeding external dependencies in your public interfaces you may not see dependency exceptions here, but you will then hit them later at runtime when you actually invoke code that uses them.
.NET's assembly loading is smart and delay loads assemblies only when a method is called that uses a dependency (except ASP.NET applications which explicitly pre-load all BIN folder assemblies).
However, .NET only looks for dependencies in the startup folder or any additionally declared Private Bin paths. I don't have those, so the asm.GetTypes() in many cases causes assembly load failures.
This is pretty straight forward and it works easily because I can look for assemblies in a known folder.
Notice that before going to check for assemblies out on disk, there's a check against already loaded assemblies. Huh? This sounds counter intuitive. Why would this code actually trigger and find an already loaded assembly? I dunno, but I've had this happen consistently with CookComputing.XmlRpcV2.dll which is already loaded, yet somehow ends up in the AssemblyResolve handler. Simply returning the already loaded assembly instance oddly works, which is just strange.
If the assembly is not already loaded, I can then try and load it from the .\Addins folder. With the single folder this all worked just fine.
Fast forward a couple of months and now I'm looking at creating an addin manager with downloadable addins. Of course, I quickly realized that a single folder isn't going to work as each addin needs to provide some metadata and folders make it easy to see what's installed and easy to uninstall without having to track all the files.
In theory moving the code to use folders should work the same, but there's a catch - I no longer know where the assemblies are loading from specifically because there are no many addin folders.
Note that the AssemblyResolve handler has a args.RequestingAssembly property which maddeningly is always blank. If this value actually gave me the requesting or calling assembly things would be easy since I could just try loading from the same folder. But alas the value is always empty, so no go.
I tried a number of different approaches in order to figure out how to get at the assembly. Different ways of loading the assembly, moving the files, AppendPrivateBin() (which as mentioned earlier has no effect after assemblies have started to load).
In the end I ended up using an extension of what worked initially which is simply to load the assembly from disk. This time around I don't know the exact folder, but I know what the base folder is so I can simply scan the directory hierarchy for the DLLs. Yes, this definitely has some overhead, but after all the false starts this just seems to be the most reliable way to ensure assemblies are found and matched.
This works with everything I've thrown at it thus far so this seems like a good solution. There's definitely some overhead in this - both searching for the assemblies and then also from all the assembly preloading that occurs because of the type scanning in order to find the addin interface - which effectively preloads all used dependencies.
In order to minimize the overhead of this addin loading, I also load addins asynchronously, so they happen in the background while the rest of the application loads.
and this seems to help to mitigate the startup lag quite a bit.
No discussion of Add ins and assembly loading would be complete without mentioning AppDomains and loading Addins separately. A lot of the issues I've described here could be mitigated by using a custom AppDomain and explicitly setting up the private bin path before load time by pre-scanning the folders.
But after having struggled with AppDomain based addins in a couple of other applications and realizing that add-ins need to have access to the WPF UI, there's no easy way to deal with the cross domain serialization in such a tighly integrated addin. Although possible, the complexities that this raises are not worth the effort.
Along the same lines app Isolation is not a concern since addins have to have tight integration with the main application anyway in order to do what they need to. So addins run in-process.
It all depends on the solution used of course. More business service centric addins can be a good fit for AppDomain.
This may sound like a fairly esoteric problem, but while searching for solutions around not getting Assembly resolve errors and loading of assemblies from multiple folders, there are a lot of people running into these same problems. There are a lot of hacky workarounds and this one is just one more in a long line of hacks. But for me at least this one has been reliably working - in fact so much so I've retrofitted it to two other applications that were previously guessing at paths.
If you want to see all the pieces together in a working application you can check out the links below in the Markdown Monster source code on Github.
Or just throw MEF into your application and load from all the different folders you want to, and use DI to get whatever instances you want from them.
If I follow you correctly (and I may not), you're loading add-ins by folder now, not by DLL-at-a-time in the "Addins" folder. If that's correct, could you change the AssembyResolve handler before you begin loading each addin, perhaps as a closure over the addin's folder name so that you know which addin you are adding when the handler gets called?
I'm sure I'm missing something, but that's what went through my mind as I read your article.
@Paw - yep MEF is an option, but seems like overkill for a relatively simple task. While it wasn't exactly straight forward to arrive at this solution, it works and is easy to implement.
@Donnie - while that might work I don't think that's a good idea. Assembly resolve errors can occur at any time, and you may end up with oddball dependencies getting pulled in that might have to be loaded differently (or just passed back) - too risky. I suppose one could track the current addin folder with a static variable and based on that know the active addin folder, but again it's not generic. The solution here - brute force as it is - works and it simply runs through files to find what it needs. The folder strucuture isn't going to be large for addin assemblies so while not fast, it's not exactly a ball breaker either especially if the add-in loading is offloaded to a separate async/thread operation.
I have used MEF for that particular problem in a little side project called Nubot; a chat bot in C# connecting to HipChat. You can find the source here https://github.com/laurentkempe/Nubot It worked really nicely and I found it a clean solution and definitely not an overkill!
A slight performance improvement could be gained by scanning the addins directories recursively and building a static dictionary of all the assemblies. That would at least put the disk cost to very close to the single directory solution.
Since you cannot unload dlls once loaded you can either shadow copy them (app load option) or ignore the uninstall after start issue.
Then if you want a little bit of extra awesome you can add a file system watcher to rebuild your dictionary and load newly installed plugins.
I'm going to bookmark this for next time I need to update our LOB app that uses plugins. Always nice to pre-emptively prevent bugs.
To answer your question of why the AssemblyResolve gets called even for assemblies already loaded, you need to dive into .NET "Load Contexts".
Assemblies loaded through Assembly.LoadFrom(...) are not always treated the same as though found through the runtime probing.
Another vote for MEF 2.0 as it's designed with addin's in mind. MEF 1.0 is attribute heavy and has crazy defaults. MEF 2.0 is better.
My trajectory has been MEF, load assemblies from the same domain, from different appdomains and back to the same domain. For me this is the most easy, not overkill and maintainable solution. Currently I prefer to use a mixed approach with a host that act as the addin assembly referencing the specific module as a nuget package.
How we can unload an assembly?
@Rashidul - you cannot unload an assembly loaded into an appdomain. Once loaded always loaded until you unload the appdomain. Unless you load assemblies explicitly into a separate AppDomain, they can never be unloaded.
I had the same problem that .NET didn't find assemblies which were already loaded.. I suppose it's because I load from byte and paths are different thus dependencies are not found correctly. But it's still strange since I have all dlls loaded manually using brute force.
Anyway, the resolve callback fixed it. I'm very glad I saw your article in google search, it's difficult to find something a bit more advanced these days.