How to use Dynamic LINQ with custom types (i.e NodaTime) ?

[myshon]

Hi,
Let's use an entity with external types (in my case NodaTime). I use Postgres with Nodatime converters from Npgsql.NodaTime package (which provides not only mapping between Postgres types and Noda types but also allows query by these)

Here is an entity:

class Entity
{
   public LocalDate Date {get; set;}
   public LocalTime? Time {get;set:}
}

I would like to query Date and Time using dynamic Linq expression.

var list = entities
      .Where("Date >= @0", date)
      .ToList();

By default, it's not possible, the code above throws NotSupportedException. It can be easily solved using type converters.

TypeDescriptor.AddAttributes(typeof(LocalDate), new TypeConverterAttribute(typeof(LocalDateConverter)));
TypeDescriptor.AddAttributes(typeof(LocalTime), new TypeConverterAttribute(typeof(LocalTimeConverter)));

Then it works as expected... Almost.

The problem is that TypeDescriptor.AddAttributes adds converters globally which affects many places (like Newtonsoft JSON serializer) in a stable, working legacy system. It's very risky, I would like to avoid this way.

Is there any different approach how to register converter for custom type?

Here. list of package details I use:

<PackageReference Include="Microsoft.EntityFrameworkCore.DynamicLinq" Version="3.2.6" />
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="3.1.4" />
<PackageReference Include="Npgsql.NodaTime" Version="4.1.6" />
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="3.1.4" />

[KrzysztofBranicki]

Hi @StefH, since we are using NodaTime ubiquitously it means we can't use Dynamic LINQ without addressing this issue. If there is no built in mechanism that would help to solve this problem maybe you would be willing to accept contribution for one and maybe you have some suggestions on approach that would be prefered.

[StefH]

Maybe a possibility could be to add support for a new DynamicLinq attribute for each property?
Like:

class Entity
{
   [DynamicLinqTypeConverter(typeof(LocalDateConverter)]
   public LocalDate Date {get; set;}

   [DynamicLinqTypeConverter(typeof(LocalTimeConverter)]
   public LocalTime? Time {get;set:}
}

However once this is added, I don't know yet how this can be used/accessed within the logic...

[StefH]

@myshon Can you provide a full working example console app?

[KrzysztofBranicki]

@StefH annotating every property in a class which uses type from NodaTime with DynamicLinqTypeConverter is rather not an option for us. We are developing library that is later used to implement bunch of microservices in the system. We don't control classes on which this will be used nor we can't require them annotate them in a special way. Expectation is that it will be used on POCOs without any special adjustments. Because of that I was thinking rather about something that would register type converter for a specified type inside Dynamic LINQ engine eg. DynamicExpressionParser.RegisterTypeConverterFor<LocalDate>(typeof(LocalDateConverter);

[StefH]

@KrzysztofBranicki I see your point.

Can you please provide an example project which uses postgress database and uses Npgsql.NodaTime ?
I need this in order to start investigating the error.

[myshon]

@StefH
Please take look at this project in my repository: https://github.com/myshon/sandbox/blob/master/DynamicLinqSample/DynamicLinqSample/DynamicLinqTests.cs

It didn't use Postgres nor Npgsql.NodaTime, but I believe it's not needed to point the case. Key line is here:

var expr = DynamicExpressionParser.ParseLambda<Entity, bool>(config, false, 
   "BirthDate == @0", "1987-10-12");

without LocalDateConverter registered it fails throwingNotSupportedException because "1987-10-12" cannot be converted to LocalDate type. I know that apart from the converter, another solution that works could be

var expr = DynamicExpressionParser.ParseLambda<Entity, bool>(config, false, 
   "BirthDate == @0", new LocalDate(1987,10,12));

but it is not satisfying because I don't know the type of parameter during parsing. The solution we expect is a possibility to register converter for parsing only (not globally) like @KrzysztofBranicki mentioned

DynamicExpressionParser.RegisterTypeConverterFor<LocalDate>(typeof(LocalDateConverter);

or alternatively, provide a list/dictionary of converters in ParsingConfig would be also fine:

var config = new ParsingConfig()
{
   TypeConverters = new ()
   {
       {typeof(LocalDate), new LocalDateConverter()}, 
       {typeof(LocalTime), new LocalTimeConverter()}
   };
}

What do you think about it?
BTW. We, @KrzysztofBranicki and I are addressing the same issue.

[StefH]

BTW : I noticed that when you add this attribute to a property, it also seems to work fine:

private class Entity
        {
            public Guid Id { get; set; }
            public string FirstName { get; set; }

            [TypeConverter(typeof(LocalDateConverter))]  // <------
            public LocalDate BirthDate { get; set; }
        }

(However in your case this would not be an option? Because you don't want to annotate all classes....)

[StefH]

Hello @myshon,

I did add your test to the test project:
https://github.com/zzzprojects/System.Linq.Dynamic.Core/blob/TypeConverter/test/System.Linq.Dynamic.Core.Tests/TypeConvertors/NodaTimeConverterTests.cs

However the test does not fail?

[myshon]

Hi @StefH,
I suppose you've installed NodaTime 3.0.* where they added default converters for noda types 😄
See details: https://nodatime.org/versions
If you install the previous version 2.4.7 I installed in my repo you will have failed tests.

Will NodaTime 3 solves this issue? Partially...

First of all, we would like to have a solution which works not only for node types.
Secondly, some converters in Noda3 have a different pattern than converters provided for serialization (relates to Duration and ZonedDateTime). Why? I don't know 🤔 There is an issue with this: nodatime/nodatime.serialization#68
We would like to register our own type converters with different patterns that default one.

I already prepared PR for this feature in Dynamic Linq but I don't have permission to push it into repo. You can review my changes.

[StefH]

@myshon and @KrzysztofBranicki
Can you try version *** 9-preview-01 from MyGet : https://www.myget.org/F/system-linq-dynamic-core/api/v3/index.json ?

You can add extra converters via the parserconfig: TypeConverters

Example:

var parsingConfig = new ParsingConfig
            {
                TypeConverters = new Dictionary<Type, TypeConverter>
                {
                    { typeof(LocalDate), new LocalDateConverter() }
                }
            };

[myshon]

Thanks @StefH
I've checked preview-01 and it works correctly 👍

However, while testing, I found another intersting issue. I suppose it's not related with converters but null comparison.

Having entity

private class Entity
{
    public DateTime? EmployedAt { get; set; }
}

we can create the following expression successfully ✔️

DynamicExpressionParser.ParseLambda<Entity, bool>(config, false, "EmployedAt != null");

However, when I change DateTime? to LocalDate? then it fails with the exception

System.Linq.Dynamic.Core.Exceptions.ParseException
Operator '!=' incompatible with operand types 'LocalDate?' and 'Object'

Both types DateTime and LocalDate are struct so why DateTime? works? 🤔
Same result after updating to Noda 3.0.5 with their default converters.

[StefH]

@myshon and @KrzysztofBranicki
The != null should be fixed in *** 9-preview-02, can you please try?

[myshon]

preview-02 tests passed ✔️ 👍

[StefH]

Hello @myshon, did you also try to run this with a real Postgres database?

[myshon]

Hi @StefH
✔️ Tested both in-memory collections and with real Postgres database
✔️ Tested both config.TypeConverters and with default NodaTime 3 converters
Thanks for help!

[StefH]

Official version will be released within some time, until then, you can use the preview version.

[olaisen81]

Hi @StefH , i'm using .NET 6 with Dynamic Linq and Microsoft.EntityFrameworkCore.DynamicLinq 6.2.19 but i can't configure and override custom TypeConverters.
My config is:

var typeConverters = new Dictionary<Type, TypeConverter>();
typeConverters.Add(typeof(int), new EnumTypeConverter());

var config = new ParsingConfig
{
  TypeConverters = typeConverters
};

My custom converter class is:

using System;
using System.Collections;
using System.ComponentModel;
using System.Globalization;

namespace WikiPass.DynamicLinq.TypeConverters
{

    public class EnumTypeConverter : TypeConverter
    {

        public override StandardValuesCollection GetStandardValues(ITypeDescriptorContext context)
        {
            return base.GetStandardValues(context);
        }

        public override PropertyDescriptorCollection GetProperties(ITypeDescriptorContext context, object value, Attribute[] attributes)
        {
            return base.GetProperties(context, value, attributes);
        }

        public override bool GetPropertiesSupported(ITypeDescriptorContext context)
        {
            return base.GetPropertiesSupported(context);
        }

        public override bool GetStandardValuesExclusive(ITypeDescriptorContext context)
        {
            return base.GetStandardValuesExclusive(context);
        }

        public override bool GetStandardValuesSupported(ITypeDescriptorContext context)
        {
            return base.GetStandardValuesSupported(context);
        }

        public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
        {
            return base.ConvertFrom(context, culture, value);
        }

        public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
        {
            if (sourceType == typeof(int))
                return true;
            return base.CanConvertFrom(context, sourceType);
        }

        public override bool CanConvertTo(ITypeDescriptorContext context, Type destinationType)
        {
            return base.CanConvertTo(context, destinationType);
        }

        public override object ConvertTo(ITypeDescriptorContext context, CultureInfo culture, object value, Type destinationType)
        {
            return base.ConvertTo(context, culture, value, destinationType);
        }

        public override object CreateInstance(ITypeDescriptorContext context, IDictionary propertyValues)
        {
            return base.CreateInstance(context, propertyValues);
        }

        public override bool GetCreateInstanceSupported(ITypeDescriptorContext context)
        {
            return base.GetCreateInstanceSupported(context);
        }

        public override bool IsValid(ITypeDescriptorContext context, object value)
        {
            return base.IsValid(context, value);
        }

        public override bool Equals(object obj)
        {
            return base.Equals(obj);
        }

        public override int GetHashCode()
        {
            return base.GetHashCode();
        }

        public override string ToString()
        {
            return base.ToString();
        }
    }

    
}

I execute my query passing the config but none of the above methods is calling and i can't override converters:
var query = dbSet.Select(config, selectDinLinq); await query.ToDynamicListAsync();
Is there something wrong with this scenario?
Thanks in advance for any help.