Issue: Memory leak in ConstantExpressionHelper.cs

[KentRollins]

I believe I've discovered a memory leak in the parsing of literal values (numerics at least) in expressions. I have a server that runs 24/7 and receives workunits. The workunits contain expressions that are compiled when received. I noticed a memory leak and used JetBrains dotMemory to investigate. It indicated that there was a ConcurrentDictionary that was slowly accumulating objects. My app does not use ConcurrentDictionary. I subsequently narrowed this down to ConstantExpressionHelper. After 30 minutes of execution the ConcurrentDictionary contained 89K nodes.

Below is an app that will reproduce this problem. Because it is stripped down to the bare minimum required to reproduce the problem and is multithreaded you can watch the Visual Studio memory monitor for 30 seconds and you will see it slowly creeping up. This app starts out for me at about 32MB of memory and after 2 minutes it is at 84MB.

This app is currently using an expression consisting of a single number (simplest possible repro), but I found this with a much more complex numeric expression involving about 6 calls to Math.X operations and some floating point literals.

using System;
using System.Linq.Dynamic.Core;
using System.Linq.Expressions;
using System.Threading;

namespace Utility.SoakTest
{
	class Program
	{
		static void Main(string[] asArgs)
		{
			RunThreads(8, RawMemoryLeak);
		}

		private static void RunThreads(int nThreads, Action actThreadStart)
		{
			for (int i = 0; i < nThreads; i++)
			{
				Thread thread = new Thread(new ThreadStart(actThreadStart));
				thread.Start();
			}

			while (true)
			{
				Thread.Sleep(5000);
			}
		}

		private static void RawMemoryLeak()
		{
			while (true)
			{
				string sExpr = "1234567890";
				LambdaExpression expr = DynamicExpressionParser.ParseLambda(new ParameterExpression[0], typeof(double), sExpr);
			}
		}
	}
}

[StefH]

Thank you for analyzing this issue. I was able to reproduce and find the leaky code.

I've created a new PR which includes this fix:
#168

If you like you can review the code and use this source code branch for testing. Else I can also create a NuGet for you to test your scenario.

[KentRollins]

Stef, the code update looks good. I am currently on Visual Studio 2015 and am not in a position to upgrade so I can't build the library. This fix looks very simple so I have high confidence in it and I am happy to wait for an official build. However, if you would like me to try an early build in my environment to confirm, I would be happy to accept a nuget.

[StefH]

I'll create a new NuGet.

[KentRollins]

Stef, I want to let you know that I downloaded and rebuilt with 1.0.8.8 which contains the ConstantExpressionHelper fix. After almost 24 hours, I could not detect a memory leak by looking at Task Manager. However, I ran the process in dotMemory and the data there seem to indicate that there is still a leak in this area. Things are improved by about an order of magnitude. The test case I submitted previously works without leaks and I have not been able to tweak it to induce this remaining problem. I will continue to try and create a reproducible case and report back as soon as I have something useful.

[KentRollins]

False alarm. The remaining leak was user error on my part. I failed to update a folder in one of the QA systems during a test. After I updated it with 1.0.8.8 and ran the system under dotMemory it showed stable memory use over 1 hour. This fix is good.

[StefH]

Ok, thanks a lot for finding this issue.