{"push_id":1137606671,"size":13,"distinct_size":13,"ref":"refs/heads/external-dependency-separation","head":"73b665c67f89872cb8d9d4e0498e505d47759a0e","before":"51044d9dfb79a8abe2a8154f4f7a547fd7559654","commits":[{"sha":"c239fe8f69bbbfe05bdc8a6b830db267453c9d4e","author":{"name":"Stephen Toub","email":"b60ab4ec0ae5032e51dd285ac67105c722f075d3@microsoft.com"},"message":"Dispose some X509 resources more aggressively\n\nIn a simple test on Unix that used an HttpClient to download https://bing.com with a server certificate custom validation callback (that just returned true), I saw a ton of memory in OpenSSL not being cleaned up.  Turns out it was being cleaned up, but only once the GC and finalizers ran, due to over 1500 SafeHandles from the X509Certificates library not being disposed of and instead left for finalization.\n\nThis commit addresses ~2/3rds of those, ones where it's clear that Dispose'ing is safe (the remaining 1/3, however, represent most of the memory being temporarily leaked).  The others should be looked at by @bartonjs, who can better determine where/when it's safe to clean up the remainder.  Most of the remainder of them are coming from:\n```\n   at Microsoft.Win32.SafeHandles.SafeX509Handle..ctor()\n   at Interop.Crypto.X509Duplicate(IntPtr handle)\n   at Internal.Cryptography.Pal.CertificatePal.FromHandle(IntPtr handle)\n   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(IntPtr handle)\n   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(IntPtr handle)\n   at Internal.Cryptography.Pal.StorePal.CloneStore(X509Certificate2Collection seed)\n   at Internal.Cryptography.Pal.StorePal.FromSystemStore(String storeName, StoreLocation storeLocation, OpenFlags openFlags)\n   at System.Security.Cryptography.X509Certificates.X509Store.Open(OpenFlags flags)\n   at Internal.Cryptography.Pal.OpenSslX509ChainProcessor.FindCandidates(X509Certificate2 leaf, X509Certificate2Collection extraStore, HashSet`1 downloaded, HashSet`1 systemTrusted, TimeSpan& remainingDownloadTime)\n   at Internal.Cryptography.Pal.ChainPal.BuildChain(Boolean useMachineContext, ICertificatePal cert, X509Certificate2Collection extraStore, OidCollection applicationPolicy, OidCollection certificatePolicy, X509RevocationMode revocationMode, X509RevocationFlag revocationFlag, DateTime verificationTime, TimeSpan timeout)\n   at System.Security.Cryptography.X509Certificates.X509Chain.Build(X509Certificate2 certificate)\n   at System.Net.Security.CertificateValidation.BuildChainAndVerifyProperties(X509Chain chain, X509Certificate2 remoteCertificate, Boolean checkCertName, String hostName)\n   at System.Net.Http.CurlHandler.SslProvider.VerifyCertChain(IntPtr storeCtxPtr, IntPtr curlPtr)\n   at Interop.Http.MultiPerform(SafeCurlMultiHandle multiHandle)\n   at Interop.Http.MultiPerform(SafeCurlMultiHandle multiHandle)\n   at System.Net.Http.CurlHandler.MultiAgent.WorkerBodyLoop()\n   at System.Net.Http.CurlHandler.MultiAgent.WorkerBody()\n   at System.Threading.Tasks.Task.Execute()\n   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)\n   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)\n   at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)\n   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)\n```\nand are being left for finalization due to https://github.com/dotnet/corefx/blob/d725f9f6a7232118f9179dc74f56af8ae54dc269/src/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/OpenSslX509ChainProcessor.cs#L19-L21 not disposing of the ChainElement's certificates (I wasn't sure if ownership of these was transferred anywhere).\n\nI also looked at using client certificates.  Similarly, in a simple test, several hundred SafeHandles are being left for finalization, coming from:\n```\n   at Microsoft.Win32.SafeHandles.SafeX509Handle..ctor()\n   at Interop.Crypto.X509Duplicate(IntPtr handle)\n   at Internal.Cryptography.Pal.CertificatePal.FromHandle(IntPtr handle)\n   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(IntPtr handle)\n   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(IntPtr handle)\n   at Internal.Cryptography.Pal.StorePal.CloneStore(X509Certificate2Collection seed)\n   at Internal.Cryptography.Pal.StorePal.FromSystemStore(String storeName, StoreLocation storeLocation, OpenFlags openFlags)\n   at System.Security.Cryptography.X509Certificates.X509Store.Open(OpenFlags flags)\n   at Internal.Cryptography.Pal.OpenSslX509ChainProcessor.FindCandidates(X509Certificate2 leaf, X509Certificate2Collection extraStore, HashSet`1 downloaded, HashSet`1 systemTrusted, TimeSpan& remainingDownloadTime)\n   at Internal.Cryptography.Pal.ChainPal.BuildChain(Boolean useMachineContext, ICertificatePal cert, X509Certificate2Collection extraStore, OidCollection applicationPolicy, OidCollection certificatePolicy, X509RevocationMode revocationMode, X509RevocationFlag revocationFlag, DateTime verificationTime, TimeSpan timeout)\n   at System.Security.Cryptography.X509Certificates.X509Chain.Build(X509Certificate2 certificate)\n   at System.Net.Http.TLSCertificateExtensions.BuildNewChain(X509Certificate2 certificate, Boolean includeClientApplicationPolicy)\n   at System.Net.Http.CurlHandler.ClientCertificateProvider.TlsClientCertCallback(IntPtr ssl, IntPtr& certHandle, IntPtr& privateKeyHandle)\n   at Interop.Http.MultiPerform(SafeCurlMultiHandle multiHandle)\n   at Interop.Http.MultiPerform(SafeCurlMultiHandle multiHandle)\n   at System.Net.Http.CurlHandler.MultiAgent.PerformCurlWork()\n   at System.Net.Http.CurlHandler.MultiAgent.WorkerBodyLoop()\n   at System.Net.Http.CurlHandler.MultiAgent.WorkerBody()\n   at System.Net.Http.CurlHandler.MultiAgent.<>c.<EnsureWorkerIsRunning>b__19_0(Object s)\n   at System.Threading.Tasks.Task.Execute()\n   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)\n   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)\n   at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)\n   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)\n```\nand are being left for finalization due to all of the certificates in these collections https://github.com/dotnet/corefx/blob/d725f9f6a7232118f9179dc74f56af8ae54dc269/src/System.Security.Cryptography.X509Certificates/src/Internal/Cryptography/Pal.Unix/OpenSslX509ChainProcessor.cs#L495-L498 not being disposed of (again, some of these appear to be handed out, so I've not touched them because I'm unclear as to the lifetime and ownership).\n\nSeparately, we should also look at whether there's anything we can do about so many SafeHandles being created.","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/c239fe8f69bbbfe05bdc8a6b830db267453c9d4e"},{"sha":"c17833dfc0bdd74e47b135fe23e0ef37adfa0f01","author":{"name":"Stephen Toub","email":"b60ab4ec0ae5032e51dd285ac67105c722f075d3@microsoft.com"},"message":"Address PR feedback","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/c17833dfc0bdd74e47b135fe23e0ef37adfa0f01"},{"sha":"5fd5a998637e4d143310edee8070eefe0b812d35","author":{"name":"Joel Hendrix","email":"3c474cc3d316ec37b4154de18f896ad19636031a@microsoft.com"},"message":"Update build tools to pick up fixes for the official build.","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/5fd5a998637e4d143310edee8070eefe0b812d35"},{"sha":"85a47ca5ce93d43b1a72a1ae1712b6ea70007e12","author":{"name":"Joel Hendrix","email":"3c474cc3d316ec37b4154de18f896ad19636031a@microsoft.com"},"message":"Merge pull request #8994 from jhendrixMSFT/master\n\nUpdate build tools to pick up fixes for the official build.","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/85a47ca5ce93d43b1a72a1ae1712b6ea70007e12"},{"sha":"9ff5dc53f2e6b807f50ed7f962bbda8aebac1099","author":{"name":"Stephen Toub","email":"b60ab4ec0ae5032e51dd285ac67105c722f075d3@microsoft.com"},"message":"Merge pull request #8978 from stephentoub/cert_disposal\n\nDispose some X509 resources more aggressively","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/9ff5dc53f2e6b807f50ed7f962bbda8aebac1099"},{"sha":"18a32a429fa441c2e6232a4a375c6aadda9e33ba","author":{"name":"Eric St. John","email":"5cba59b6016e12cb8b8404aa75c5312236fbcd78@microsoft.com"},"message":"Add rel-notes and project URL to nupkgs\n\nThis is a 1.1.0-beta specific link for relnotes.","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/18a32a429fa441c2e6232a4a375c6aadda9e33ba"},{"sha":"26e4753b5963ac8d533a26ef1114b9ef47a8e71a","author":{"name":"Eric StJohn","email":"5cba59b6016e12cb8b8404aa75c5312236fbcd78@microsoft.com"},"message":"Merge pull request #8997 from ericstj/packageUrls\n\nAdd rel-notes and project URL to nupkgs","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/26e4753b5963ac8d533a26ef1114b9ef47a8e71a"},{"sha":"a94db66d894212b9bffd3c7fd53d2522683a6689","author":{"name":"Maira Wenzel","email":"8faf3085af2f6a6a551b07f897a1cfa8a91c25da@users.noreply.github.com"},"message":"Update README.md to fix a few links (#8993)","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/a94db66d894212b9bffd3c7fd53d2522683a6689"},{"sha":"7307337eee2884a631d0c5495f360a0aa77b2d40","author":{"name":"Joel Hendrix","email":"3c474cc3d316ec37b4154de18f896ad19636031a@microsoft.com"},"message":"Fix build-packages.sh script so it works for Fedora and OpenSUSE.\n\nThe package building script had some poor hard-coded logic for detecting\ndifferent Linux distros.  When support for Fedora and OpenSUSE was added\nthis logic didn't work for them.\nI've tweaked the logic based on what's in build.sh so it should be more\nrobust in the future.","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/7307337eee2884a631d0c5495f360a0aa77b2d40"},{"sha":"a8b4814247d4eed8687dd3fc7ad9b2a2bf7d231b","author":{"name":"Deepak Shankargouda","email":"aa022906a0d2f301c61ea3d41a5218ab755fbd01@microsoft.com"},"message":"Fixing perf linux tests and revving xunit perf versions (#8862)","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/a8b4814247d4eed8687dd3fc7ad9b2a2bf7d231b"},{"sha":"d7fb66a9c0b3fda1f95d52656df0fb02e96950e7","author":{"name":"Joel Hendrix","email":"3c474cc3d316ec37b4154de18f896ad19636031a@microsoft.com"},"message":"Merge pull request #9001 from jhendrixMSFT/master\n\nFix build-packages.sh script so it works for Fedora and OpenSUSE.","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/d7fb66a9c0b3fda1f95d52656df0fb02e96950e7"},{"sha":"0ee435012400693e42866a42c7f2616b72c8a10d","author":{"name":"Eric St. John","email":"5cba59b6016e12cb8b8404aa75c5312236fbcd78@microsoft.com"},"message":"Switch test-runtime to NETStandard.Library\n\nMicrosoft.NETCore.Console soon won't be built with the same versioning\nas the rest of CoreFx: https://github.com/dotnet/core-setup/pull/14\n\nWe don't want to have to \"know\" which version of this package is\nconsistent with the CoreFx version we're picking up.\n\nIn addition to this Microsoft.NETCore.Console was too large: we really\nonly need the test-runtime to bring in a closure of dependencies for the\ntest runner and libraries.\n\nIn addition to this I've split up the validation rules for packages that\nwill soon start coming from other repos.\n\nAdd missing impl dependencies to tests\n\nThese dependencies are exclusive to the product assembly brought in via\nproject-reference.  Because our current restore process doesn't catch\ntransitive package dependencies these need to be added manually.\n\nFix up some of more project dependencies","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/0ee435012400693e42866a42c7f2616b72c8a10d"},{"sha":"73b665c67f89872cb8d9d4e0498e505d47759a0e","author":{"name":"Davis Goodin","email":"f91e8880f23763640898f5276faf1a7784d51e38@microsoft.com"},"message":"Split external dependency versions to a second expected prerelease.\n\nAdd UpdateDependencies.ps1 parameter to update either expected prerelease.","distinct":true,"url":"https://api.github.com/repos/dagood/corefx/commits/73b665c67f89872cb8d9d4e0498e505d47759a0e"}]}
{"actor":{"display_login":"dagood"}}