Add managed Mac signing implementation #107378
Draft
+50,758
−29,910
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jtschuster
force-pushed
the
AdHocSigningMac2
branch
2 times, most recently
from
46fda6b
to
0f8c240
Compare
This was referenced Sep 11, 2024
…o the code context (dotnet#106846)
Co-authored-by: pavelsavara <[email protected]>
Co-authored-by: campersau <[email protected]> Co-authored-by: Pavel Savara <[email protected]>
Shake's GetHashAndReset would perform a double-reset on Windows, this fixes it so that exactly one reset happens. This occurred because the high-level type like Shake128 implemented GetHashAndReset as a call to Finalize, followed by a call to Reset. On Linux, this is reasonable. The finalize gets the output, and reset puts it back in to a working state. On Windows, Finalize (BCryptFinishHash) always does a reset. So we do the finalize with the implicit reset, then do an explicit reset. This change fixes that by making finalize+reset an operation that is PAL specific. On Linux, it continues to call Finalize and Reset. On Windows, it calls Finalize, and skips the reset call.
…otnet#107121) The Tier1 IL may diverge from the Tier0+Instr IL, if the method is modified by a profiler at just the "right" time. Tolerate this by not asserting if there is a mismatch in the Tier0+Instr and Tier1 schemas (instead the JIT will ignore the profile data). The Tier1 JIT may initially read profile data and then switch over to minopt because of debugger override. If so, completely remove the profile data, so later phases are not confused and hit asserts. Fixes dotnet#106033.
…dotnet#107038) * Correct lengths on lunar months 8 and 9 in 2057. Fixes dotnet#107032 * Add test. * Correct comment. * Addresses PR feedback. Local testing undertaken. --------- Co-authored-by: Tarek Mahmoud Sayed <[email protected]>
…tnet#107130) * [LoongArch64] Add 'loongarch64' display in Crossgen2 and ILC cli. * Add missing 'Architecture.LoongArch64' in libraries tests.
The recently added AVX support in hardware exception handling path on macOS x64 has introduced a problem when running under Rosetta. When we extract the floating point part of the context of the failing thread, the thread can have AVX or AVX512 active, or none of these. The code accidentally leaves CONTEXT_XSTATE set on the context even when no AVX was enabled on the thread. Rosetta doesn't support AVX, so having CONTEXT_XSTATE set in the context flags can lead to later call to RtlRestoreContext attempting to set ymm registers using instructions that Rosetta cannot emulate and the app crashes due to that. This doesn't happen in .NET 9, since we always clear the CONTEXT_XSTATE before exception handling stack unwinding. But in .NET 8, this causes stack overflow under Rosetta, since the attemt to execute the ymm instruction triggers the hardware exception handling again and again.
This exists as the more used function bbIsHandlerBeg
This breaks the dependency on having dotnet installed when running tests.
…se (dotnet#107170) On macOS with system integrity protection enabled, if a code-signed binary is loaded in a process, modified, and loaded again in another process, the second run will crash with Code Signature Invalid). Enabling test-only behaviour modifies the binary (re-writes a placeholder value). The nethost tests were enabling/disabling test-only behaviour for the same nethost binary used by all the tests. This change updates the tests to just always enable it before any use of the binary.
- Add a proper constructor for `Compiler` and call it via placement new from `jitNativeCode`. - Remove `compInit`, `fgInit`, `impInit`, `lvaInit` and `optInit`. Move all their initialization into the constructor or, when possible, initialize the field directly with its declaration.
* add initial AssemblyNameInfo Fuzzer * fix the first bug that it has discovered
…et#107806) - `CheckRunClassInitThrowing` didn't check to see if the class had been initialized before taking a lock - `EnsureTlsIndexAllocated` didn't check if the Tls index had been allocated before setting the flag via an expensive Interlocked call to indicate that it had been allocated - And finally `JIT_GetNonGCThreadStaticBaseOptimized` and `JIT_GetGCThreadStaticBaseOptimized` were missing the fast paths which avoided even calling those apis at all. Perf with a small benchmark which does complex multithreaded work... | Runtime | Time | | ---- | ---- | | .NET 8 | 00.9414682 s | | .NET 9 before this fix | 22.8079382 | | .NET 9 with this fix | 00.2004539 | Fixes dotnet#107728
…sociation with busy lower-half register (dotnet#107714) * make sure the upperHalf is moved to ready pool when lowerHalf holding DOUBLE interval is done * remove the logging * added test cases * fix a typo * Add missing Xunit reference * Revert "remove the logging" This reverts commit d8f2f17. * better fix to free up upperHalf if lowerHalf is getting freed up * jit format * Revert "remove the logging" This reverts commit d8f2f17. * Handle upperHalf fix similar to lowerHalf code * formatting * jit format
* fixing System.Array constructor * adding back in co-varience check and tests
* copy comments and asserts from Levis internal code review * apply Levis suggestion: don't store Array.MaxLength as a const, as it may change in the future * add missing and fix some of the existing comments * first bug fix: SerializationRecord.TypeNameMatches should throw ArgumentNullException for null Type argument * second bug fix: SerializationRecord.TypeNameMatches should know the difference between SZArray and single-dimension, non-zero offset arrays (example: int[] and int[*]) * third bug fix: don't cast bytes to booleans * fourth bug fix: don't cast bytes to DateTimes * add one test case that I've forgot in previous PR
…ng it with JIT (dotnet#107401) * [mono][mini] Prefer llvmaot compiled method instead of inlining it with JIT The motivation for this is to fix crashes due to different HW intrinsics support between llvm and JIT. For example, a llvmaot compiled method might check if we have a set of HW intrinsics available and proceed with a code path that will end up calling a method that assumes intrinsics support that is not present in the aot image (because the gsharedvt version is not emitted for example). In this scenario, the called method will end up being compiled with JIT where we would crash due to missing HW intrinsics support. * [mono][mini] Prefer llvm compiled method only if it uses simd intrinsics We include a new flag when compiling aot method. When loading aot method, we will include mapping from code to flags so we can look it up later.
Normally we would have caught this oversight immediately, as tests would have failed to compile, but unlike the vast majority of our test suites, System.Collections.Immutable's test suite references the implementation assembly rather than the ref assembly, in order to access internals via InternalsVisibleTo. Grrr.
* introduce ArrayRecord.FlattenedLength * do not include invalid Type or Assembly names in the exception messages, as it's most likely corrupted/tampered/malicious data and could be used as a vector of attack. * It is possible to have binary array records have an element type of array without being marked as jagged
Those platforms are using hybrid globalization now and don't need the package anymore.
The RemoteInvokeOptions instance is mutated by RemoveExecutor.Invoke and cannot be shared.
* Remove redundant unsafe from APIs Fixes: dotnet#104845 * Restore incorrectly commented API
* Mark Encoding as nullable in StreamWriter's constructor * Update reference file * Align string constructors with Stream * add explicit test for null as encoding
* Started the Quality Week work of improving the workflow docs. * Made more progress on the general workflow README doc, as well as the CoreCLR one. * Rephrased some things to make them roll off the tongue better, and fixed a couple spelling and grammatical mistakes. * Basically finished the main workflow README, and the CoreCLR building docs. * Almost done with the CoreCLR building docs, finished the MacOS requirements one, and got far enough on the Windows requirements one. * Finished the first pass on the Windows requirements doc. * Finished the first pass of the Linux Environment Requirements. * Finished the first pass of the Linux requirements doc. * Made good progress on the 'Using Docker' new doc. * Finished the first pass of the Using Docker doc. * Initial merging of the new docs with the existing documentation. * Improved the redaction of some sentences, improved some formatting, and fixed a few broken links. * Forgot to commit a NativeAOT note. * Removed the cross signs in the tables and added general info about cross-compiling to the general workflow introduction segment. * Added some redaction improvements, and corrected a few misleading and/or outdated details. * Linked the Workflow's README to the CoreCLR Building README. * Added a few additional details, and improved some wording and phrasing to make the docs more friendly and clear as per received feedback. * Fixed typo in the -DCLR_CMAKE_APPLE_DYSM=TRUE flags.
* empty slice fix * PR comments and more testing * fixing check * fixing array rent init * fixes from PR comments
* fixing 0 length issue with constructor * Update src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs Co-authored-by: Stephen Toub <[email protected]> * comments from PR --------- Co-authored-by: Stephen Toub <[email protected]>
* Update ABI doc * Managed/native varargs are supported on Windows only Co-authored-by: Jan Kotas <[email protected]> * Add github issue links * Update varargs link ref * Managed varargs are supported on Windows only --------- Co-authored-by: Jan Kotas <[email protected]>
) * Initial debugging work for python pipeline * Use custom performance branch * Remove template reference * Move resources to perf.yml * Ensure runtime repo is checked out to SourcesDirectory * Try fix python run * Set python in perf-job.yml * More fixes * More fixes * Set most params in perf-job.yml * Fix variable declaration * Fix some bugs in setup parameters * Ensure Helix Access Token is passed in * Fix osGroup arg parsing * Fix ios args and use local commit time * Fix physical promotion yaml * Fix project file path * Ensure internal param is passed * Ensure project file is passed in * Try dynamically load performance repo * Try use condition inside repositories definition * Try put condition on repositories object * Try use variables to conditionally create repository * Fix conditional variable definition * Just use ADO for performance repo * Use updated repository name for scenarios jobs * Use HELIX_WORKITEM_ROOT for performance dir * Fix paths for android APKs * Remove accidental submodule * Use new project file location for arm64 jit job * Move archive of work item dir to python * Use github for performance repo checkout on public jobs * Another attempt at conditional repository definition * Another attempt at conditional repository * Use multi repo checkout only for internal runs * Ensure ios scenarios and wasm jobs are working * Only copy files from dotnet-runtime directory on internal jobs * Use branch instead of ref for performance repository clone * Clone performance repo to correct folder in public jobs
add instru to help with DATAS debugging + record heap count change history on blocking GC threads and BGC threads (I'm including a change to get the m_OSThreadId field of the Thread object since we recently discovered an issue where GC indicates a BGC thread was successfully created yet that field is 0. I'm only recording this on the OS we discovered this on. for future releases we should get it properly via a GCToEEInterface method) + record BGC thread creation history record LLC size to help with gen0 min size debugging
The old label was replaced with macos-latest-internal.
…106964) * Introduce new event for failed http requests * remove response code, because it is not applicable * fix log level from error to information and message * fix formating * fix message template and add tests * minor fixes
* Update dependencies from https://github.com/dotnet/xharness build 20240913.2 Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 10.0.0-prerelease.24462.1 -> To Version 10.0.0-prerelease.24463.2 * Update dependencies from https://github.com/dotnet/xharness build 20240916.1 Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 10.0.0-prerelease.24463.2 -> To Version 10.0.0-prerelease.24466.1 --------- Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
We don't use libtool files anymore. Contributes to dotnet#107883
dotnet#107811) Revise the fix done in dotnet#39105 to make the cast from byref to native int via an explicit cast, rather than retyping the node. Fixes dotnet#107045.
…ported (dotnet#107874) If a regex is created with RegexOptions.Compiled and RegexOptions.IgnoreCase, and it begins with a pattern that's a reasonably small number of alternating strings, it'll now end up using `SearchValues<string>` to find the next possible match location. However, the `SearchValues<string>` instance doesn't end up getting created if the interpreter is being used. If the implementation falls back to the interpreter because compilation isn't supported because dynamic code isn't supported, then it won't use any optimizations to find the next starting location. That's a regression from when it would previously at least use a vectorized search to find one character class from the set of starting strings. This fixes it to just always create the `SearchValues<string>`. This adds some overhead when using RegexOptions.Compiled, but it's typically just a few percentage points, and only applies in the cases where this `SearchValues<string>` optimization kicks in. At the moment, changing it to have perfect knowledge about whether it can avoid that creation is too invasive. This overhead also doesn't apply to the source generator.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a Mac signing library in managed code that can sign an apphost built on Windows for a Mac.
The signing is done with https://github.com/filipnavara/CodeSign, with much of the code unrelated to ad-hoc signing removed. We will only use the managed implementation to ad-hoc sign.