title: Handling nullable vs. non-nullable types with generics
body: <p>I created a <code>LookupConverter : JsonConverter</code> class to perform JSON serialization and deserialization of <code>ILookup</code> objects. As you might imagine, it has some complexity, having to deal with generics and due to the lack of a public concrete <code>Lookup</code> class. For performance, it caches its type-specific reflection work inside a static generic class. It works perfectly!</p>
  
  <p>Well, almost perfectly. I just realized today that it can't cope with serializing an <code>ILookup</code> which contains a null <code>Key</code>. After some thought and realizing that in JSON, there's no easy way to represent a null key in an object (as every key is converted to a string), I thought I'd just make the output object a little bigger.</p>
  
  <p>If the former output was, e.g., <code>{"key1":[1,2,3]}</code>, then I figured that the new output can look like <code>{Groupings:{"key1":[1,2,3]},NullKeyValue:[4,5,6]}</code>. It's awkward, but so far so good. Or it can be <code>[{"key":"key1","values":[1,2,3]},{"key":null,"values":[4,5,6]}]</code>. No big deal either way.</p>
  
  <p>Adding serialization for this was a cinch.</p>
  
  <p>However, when the time comes to deserialize, I'm having a problem. My former deserializer was really easy (there's some complex caching going on here, try too look past that and just see that my function takes a <code>jObject</code> and a <code>serializer</code> and returns an object of the right type, which is used like <code>lookupmaker(JObject.Load(reader), serializer);</code></p>
  
  <pre><code>public static Func&lt;JObject, JsonSerializer, object&gt; GetLookupMaker() =&gt;
     (jObject, serializer) =&gt; ((IEnumerable&lt;KeyValuePair&lt;string, JToken&gt;&gt;) jObject)
        .SelectMany(
           kvp =&gt; kvp.Value.ToObject&lt;List&lt;TValue&gt;&gt;(),
           (kvp, value) =&gt; new KeyValuePair&lt;TKey, TValue&gt;(Convert&lt;TKey&gt;(kvp.Key), value)
        )
        .ToLookup(kvp =&gt; kvp.Key, kvp =&gt; kvp.Value);
  </code></pre>
  
  <p>Okay, so now I'm thinking, I'll just make a <code>List</code> of <code>KeyValuePair</code>s, add an extra value if there's a null key, then throw a <code>ToLookup</code> on it just like above:</p>
  
  <pre><code>var list = new List&lt;KeyValuePair&lt;TKey, List&lt;TValue&gt;&gt;&gt;();
  var nullKeyValue = jObject["NullKeyValue"];
  if (nullKeyValue != null) {
     list.Add(new KeyValuePair&lt;TKey, List&lt;TValue&gt;&gt;(null, nullKeyValue.ToObject&lt;List&lt;TValue&gt;&gt;()));
  } //                                             ^^^^ this null
  // Then here append the items from jObject["Groupings"], and finally ToLookup.
  </code></pre>
  
  <p>But now I get an error in the above <code>Add</code>:</p>
  
  <blockquote>
    <p>Argument type 'null' is not assignable to parameter type 'TKey'.</p>
  </blockquote>
  
  <p>Well, of course it's not. It has no assurance that <code>TKey</code> isn't a non-nullable value type. Great. I'll just throw a constraint <code>where TKey : class</code> on my static class <code>GenericMethodCache&lt;TKey, TValue&gt;</code>... only, the moment I want to have a <code>: struct</code> version, I'm in trouble because the whole point of the <code>GenericMethodCache</code> was to prevent the serializer code using <code>object</code> from having to deal with the generics part. I can't get automatic resolution because resolution can't use type constraints to distinguish method groups. Suddenly the complexity of this exploded and I'm not sure that continuing to hack my way deep into the jungle trying to get it working makes sense, so I'm seeking guidance!</p>
  
  <p>Since this is quite a complicated scenario, here's the full code that doesn't handle null Keys (more on FunctionResultCache next):</p>
  
  <pre><code>public sealed class LookupConverter : JsonConverter {
     // ReSharper disable once CollectionNeverUpdated.Local
     private static readonly FunctionResultCache&lt;Type, bool&gt; s_typeCanConvertDictionary =
        new FunctionResultCache&lt;Type, bool&gt;(type =&gt;
           new [] { type }
              .Concat(type.GetInterfaces())
              .Any(iface =&gt; iface.IsGenericType &amp;&amp; iface.GetGenericTypeDefinition() == typeof(ILookup&lt;,&gt;))
        );
     public override bool CanConvert(Type objectType) =&gt; s_typeCanConvertDictionary[objectType];
  
     public override bool CanWrite =&gt; true;
     public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
        writer.WriteStartObject();
        var groupings = (IEnumerable) value;
        var getKey = _keyFetcherForType[value.GetType()];
        foreach (dynamic grouping in groupings) {
           writer.WritePropertyName(getKey(grouping).ToString());
           serializer.Serialize(writer, (IEnumerable) grouping);
        }
        writer.WriteEndObject();
     }
  
     public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) =&gt;
        // ReSharper disable once AccessToStaticMemberViaDerivedType
        _deserializerForType[objectType](JObject.Load(reader), serializer);
  
     private static class GenericMethodCache&lt;TKey, TValue&gt; {
        public static Func&lt;JObject, JsonSerializer, object&gt; GetLookupMaker() =&gt;
           (jObject, serializer) =&gt; ((IEnumerable&lt;KeyValuePair&lt;string, JToken&gt;&gt;) jObject)
              .SelectMany(
                 kvp =&gt; kvp.Value.ToObject&lt;List&lt;TValue&gt;&gt;(),
                 (kvp, value) =&gt; new KeyValuePair&lt;TKey, TValue&gt;(Convert&lt;TKey&gt;(kvp.Key), value)
              )
              .ToLookup(kvp =&gt; kvp.Key, kvp =&gt; kvp.Value);
  
        public static Func&lt;object, object&gt; GetKeyFetcher() =&gt;
           grouping =&gt; ((IGrouping&lt;TKey, TValue&gt;) grouping)
              .Key;
  
        private static T Convert&lt;T&gt;(string input) {
           try {
              return (T) TypeDescriptor.GetConverter(typeof(T)).ConvertFromString(input);
           }
           catch (NotSupportedException) {
              return default(T);
           }
        }
     }
  
     // ReSharper disable once CollectionNeverUpdated.Local
     private readonly FunctionResultCache&lt;Type, Func&lt;JObject, JsonSerializer, object&gt;&gt; _deserializerForType =
        new FunctionResultCache&lt;Type, Func&lt;JObject, JsonSerializer, object&gt;&gt;(type =&gt; {
           var genericMethodCache = typeof(GenericMethodCache&lt;,&gt;).MakeGenericType(type.GetGenericArguments());
           return (Func&lt;JObject, JsonSerializer, object&gt;) genericMethodCache.GetMethod(nameof(GenericMethodCache&lt;int, int&gt;.GetLookupMaker)).Invoke(null, new object[0]);
        }
     );
  
     // ReSharper disable once CollectionNeverUpdated.Local
     private readonly FunctionResultCache&lt;Type, Func&lt;object, object&gt;&gt; _keyFetcherForType =
        new FunctionResultCache&lt;Type, Func&lt;object, object&gt;&gt;(type =&gt; {
           var genericMethodCache = typeof(GenericMethodCache&lt;,&gt;).MakeGenericType(type.GetGenericArguments());
           return (Func&lt;object, object&gt;) genericMethodCache.GetMethod(nameof(GenericMethodCache&lt;int, int&gt;.GetKeyFetcher)).Invoke(null, new object[0]);
        }
     );
  }
  </code></pre>
  
  <p><code>FunctionResultCache</code> is basically just a <code>Dictionary</code> with the special property that when you index into a key that isn't present, it runs a function (passed in the constructor) to get the value, then stores &amp; caches the value plus returns it to you, so the next time you index into the same key it returns the cached value.</p>
  
  <p>I'm sorry about the length of this question and the code. This is a bit complicated scenario and to get useful feedback, I have to show some details about what's going on.</p>
  
  <p>P.S. One little note about this: <code>genericMethodCache.GetMethod(nameof(GenericMethodCache&lt;int, int&gt;.GetKeyFetcher))</code>. <code>nameof</code> does NOT like generic type definitions such as <code>GenericMethodCache&lt;,&gt;</code>. It only likes generic types <code>GenericMethodCache&lt;int, int&gt;</code>. But, in the long run, the <code>int, int</code> is ignored and the name <code>GenericMethodCache</code> is returned.</p>