{"action":"opened","issue":{"url":"https://api.github.com/repos/dotnet/coreclr/issues/4607","repository_url":"https://api.github.com/repos/dotnet/coreclr","labels_url":"https://api.github.com/repos/dotnet/coreclr/issues/4607/labels{/name}","comments_url":"https://api.github.com/repos/dotnet/coreclr/issues/4607/comments","events_url":"https://api.github.com/repos/dotnet/coreclr/issues/4607/events","html_url":"https://github.com/dotnet/coreclr/issues/4607","id":151258816,"number":4607,"title":"Reverse P/Invoke Vector3 argument passing failure on Linux","user":{"login":"sejongoh","id":16199127,"avatar_url":"https://avatars.githubusercontent.com/u/16199127?v=3","gravatar_id":"","url":"https://api.github.com/users/sejongoh","html_url":"https://github.com/sejongoh","followers_url":"https://api.github.com/users/sejongoh/followers","following_url":"https://api.github.com/users/sejongoh/following{/other_user}","gists_url":"https://api.github.com/users/sejongoh/gists{/gist_id}","starred_url":"https://api.github.com/users/sejongoh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sejongoh/subscriptions","organizations_url":"https://api.github.com/users/sejongoh/orgs","repos_url":"https://api.github.com/users/sejongoh/repos","events_url":"https://api.github.com/users/sejongoh/events{/privacy}","received_events_url":"https://api.github.com/users/sejongoh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-04-26T22:57:33Z","updated_at":"2016-04-26T22:57:33Z","closed_at":null,"body":"On Linux, the following RPInvoke case fails. The second Vector3 object, v3f32_mem0 is assigned onto stack, and its z value is wrong. The testcase passes on Windows.\r\n\r\nNative side\r\n```\r\n//\r\n// RPInvoke native call for Vector3 argument\r\n//\r\ntypedef void (__stdcall *CallBack_RPInvoke_Vector3Arg_Unix)(\r\n    Vector3 v3f32_xmm0, \r\n    float   f32_xmm2,\r\n    float   f32_xmm3,\r\n    float   f32_xmm4,\r\n    float   f32_xmm5,\r\n    float   f32_xmm6,\r\n    float   f32_xmm7,\r\n    float   f32_mem0,\r\n    Vector3 v3f32_mem1,\r\n    float   f32_mem2,\r\n    float   f32_mem3);\r\n\r\nEXPORT(void) __stdcall nativeCall_RPInvoke_Vector3Arg_Unix(\r\n  CallBack_RPInvoke_Vector3Arg_Unix notify)\r\n{\r\n    Vector3 v1, v2;\r\n    v1.x = 1; v1.y = 2; v1.z = 3;\r\n    v2.x = 10; v2.y = 20; v2.z = 30;\r\n    float f0 = 0, f1 = 1, f2 = 2, f3 = 3, f4 = 4, f5 = 5, f6 = 6, f7 = 7, f8 = 8;\r\n    notify(\r\n        v1,  \r\n        f0, f1, f2, f3, f4, f5,\r\n        f6, // mapped onto stack\r\n        v2, \r\n        f7, f8);\r\n} \r\n```\r\n\r\nManaged side\r\n```\r\n   static void callBack_RPInvoke_Vector3Arg_Unix(\r\n        Vector3 v3f32_xmm0, \r\n        float   f32_xmm2,\r\n        float   f32_xmm3,\r\n        float   f32_xmm4,\r\n        float   f32_xmm5,\r\n        float   f32_xmm6,\r\n        float   f32_xmm7,\r\n        float   f32_mem0,\r\n        Vector3 v3f32_mem0,\r\n        float   f32_mem1,\r\n        float   f32_mem2)\r\n    {\r\n        result = v3f32_mem0.Z == 30;\r\n        /*\r\n        // sum = (1, 2, 3) dot (1, 2, 3) = 14\r\n        float sum0 = Vector3.Dot(v3f32_xmm0, v3f32_xmm0);   \r\n        // sum = (10, 20, 30) dot (10, 20, 30) = 1400\r\n        float sum1 = Vector3.Dot(v3f32_mem0, v3f32_mem0);       \r\n        // sum = (1, 2, 3) dot (10, 20, 30) = 140\r\n        float sum2 = Vector3.Dot(v3f32_xmm0, v3f32_mem0);      \r\n        // sum = 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 = 36\r\n        float sum3 = f32_xmm2 + f32_xmm3 + f32_xmm4 + f32_xmm5 + f32_xmm6 + f32_xmm7\r\n            + f32_mem0 + f32_mem1 + f32_mem2;\r\n        \r\n        Console.WriteLine(\"callBack_RPInvoke_Vector3Arg_Unix:\");\r\n        Console.WriteLine(\"    {0}, {1}, {2}\", v3f32_xmm0.X, v3f32_xmm0.Y, v3f32_xmm0.Z);\r\n        Console.WriteLine(\"    {0}, {1}, {2}\", v3f32_mem0.X, v3f32_mem0.Y, v3f32_mem0.Z);\r\n        Console.WriteLine(\"    Sum0,1,2,3 = {0}, {1}, {2}, {3}\", sum0, sum1, sum2, sum3);\r\n        \r\n        result = (sum0 == 14) && (sum1 == 1400) && (sum2 == 140) && (sum3==36);\r\n        */\r\n    }   \r\n```\r\n\r\nv3f32_mem0 is assigned (10,20,30) on the native side. \r\n\r\n1) Native stack right before calling the callback function\r\n$rsp = 0x7fffffffd0b0\r\n```\r\n(gdb) x /32f 0x7fffffffd0b0  \r\n; 6 is f32_mem0 and 10, 20, 30 are located at [rsp + 8, 12, 16].\r\n0x7fffffffd0b0: 6       4.59163468e-41  10      20    \r\n0x7fffffffd0c0: 30      0       7       4.59163468e-41\r\n```\r\n\r\n2) Before entering IL_STUB_ReversePInvoke, it looks like the stack arguments are copied to different stack locations. $rsp is adjusted from 0x7fffffffd0b0 to 0x7fffffffcfc8, and stack arguments are found at the new stack addresses below.\r\n\r\n```\r\n(gdb) print $rbp\r\n$6 = (void *) 0x7fffffffd0a0\r\n\r\n(gdb) print $rsp\r\n$7 = (void *) 0x7fffffffcfc8\r\n\r\n(gdb)  x /32f 0x7fffffffcfd0\r\n; 6. 10, 20 are correct but v3f32_mem0.z now has 1 instead of 30.\r\n0x7fffffffcfd0: 6       4.59163468e-41  10      20  \r\n0x7fffffffcfe0: 1       2       0       0\r\n\r\n```"}}