/// This is the new (optimized) signal handler.
///
/// Differences to the old signal handler:
/// 1. It can set up mapping for multiple pages in one go (prefetching).
/// 2. It uses the checkpoint file in `PageMap` for copy-on-write mapping.
/// 3. It uses the new access kind bit to optimize the case when the first
///    access to a page is a write access. This saves two `mprotect` calls
///    and expensive TLB flushing.
/// 4. It uses the new dirty-page-tracking bit to optimize for messages that
///    do not care about dirty pages. Currently these are replicated queries.
///    Update calls and non-replicated queries need dirty page tracking.
///
/// To describe the invariants, let's take some page and denote
/// - as `accessed` the corresponding bit in `tracker.accessed_bitmap`.
/// - as `prot_flags` the corresponding protection flags of the mapping.
/// - as `ignore_dirty` the dirty page tracking mode of `tracker`.
/// - as `dirty` the indication that the page is in `tracker.dirty_bitmap`.
///
/// Then the following must hold before and after the signal handler runs:
/// A. `prot_flags=NONE <=> !accessed`,
/// B. `ignore_dirty and accessed => prot_flags=READ_WRITE`,
/// C. `!ignore_dirty and prot_flags=READ_WRITE <=> dirty`,
/// D. `!ignore_dirty and prot_flags=READ => !dirty`,
/// E. `!ignore_dirty and !dirty => prot_flags=READ`,
/// F. `dirty` => `accessed`,
/// G. `dirty => the page is in either `tracker.dirty_pages` or in
///     `tracker.speculatively_dirty_pages`.
///
/// Invariant A ensures that we don't attempt to map the same page twice.
/// Invariant B boosts performance of messages that don't track dirty pages.
/// Invariants C - G ensure correct tracking of dirty pages. A speculatively
/// dirty page needs to be validated after the execution by comparing its
/// contents to the original page.
///
/// To understand how a faulting page `P` is handled we need to consider where
/// its backing memory is coming from in `PageMap`. There are three cases:
/// 1. The corresponding memory is not in `PageMap` meaning that the canister
///    has never written to it.
/// 2. The corresponding memory is in the checkpoint file meaning that the
///    canister has not written to it since the last checkpoint.
/// 3. The corresponding memory is in `PageDelta`.
///
/// In the first two cases the underlying memory has already been mmap'ed at
/// in the constructor of the memory tracker. So handler tries to mprotect `P`
/// and a few of its subsequent pages that have the same backing memory.
/// This optimization is referred to as prefetching in the code. Prefetching
/// is disabled if we are tracking dirty pages and the faulting access was a
/// write access.
///
/// The third case is handled similar to the old implementation with one
/// important optimization: if the faulting access is a write access and the
/// page has not been accessed yet, then it is mapped as `READ_WRITE` right away
/// without going through `READ_WRITE` => copy content => `READ` => `READ_WRITE`
/// like the old signal handler does.
This is the new (optimized) signal handler.
Differences to the old signal handler:
1. It can set up mapping for multiple pages in one go (prefetching).
2. It uses the checkpoint file in `PageMap` for copy-on-write mapping.
3. It uses the new access kind bit to optimize the case when the first
access to a page is a write access. This saves two `mprotect` calls
and expensive TLB flushing.
4. It uses the new dirty-page-tracking bit to optimize for messages that
do not care about dirty pages. Currently these are replicated queries.
Update calls and non-replicated queries need dirty page tracking.

To describe the invariants, let's take some page and denote
as `accessed` the corresponding bit in `tracker.accessed_bitmap`.
as `prot_flags` the corresponding protection flags of the mapping.
as `ignore_dirty` the dirty page tracking mode of `tracker`.
as `dirty` the indication that the page is in `tracker.dirty_bitmap`.

Then the following must hold before and after the signal handler runs:
A. `prot_flags=NONE <=> !accessed`,
B.

Invariant A ensures that we don't attempt to map the same page twice.
Invariant B boosts performance of messages that don't track dirty pages.
Invariants C - G ensure correct tracking of dirty pages. A speculatively
dirty page needs to be validated after the execution by comparing its
contents to the original page.

To understand how a faulting page `P` is handled we need to consider where
its backing memory is coming from in `PageMap`. There are three cases:
1. The corresponding memory is not in `PageMap` meaning that the canister
has never written to it.
2. The corresponding memory is in the checkpoint file meaning that the
canister has not written to it since the last checkpoint.
3. The corresponding memory is in `PageDelta`.

In the first two cases the underlying memory has already been mmap'ed at
in the constructor of the memory tracker. So handler tries to mprotect `P`
and a few of its subsequent pages that have the same backing memory.
This optimization is referred to as prefetching in the code. Prefetching
is disabled if we are tracking dirty pages and the faulting access was a
write access.


This is the new ( optimized ) signal handler . Differences to the old signal handler : 1 . It can set up mapping for multiple pages in one go ( prefetching ) . 2 . It uses the checkpoint file in ` PageMap ` for copy - on - write mapping . 3 . It uses the new access kind bit to optimize the case when the first access to a page is a write access . This saves two ` mprotect ` calls and expensive TLB flushing . 4 . It uses the new dirty - page - tracking bit to optimize for messages that do not care about dirty pages . Currently these are replicated queries . Update calls and non - replicated queries need dirty page tracking . To describe the invariants let ' s take some page and denote as ` accessed ` the corresponding bit in ` tracker . accessed_bitmap ` . as ` prot_flags ` the corresponding protection flags of the mapping . as ` ignore_dirty ` the dirty page tracking mode of ` tracker ` . as ` dirty ` the indication that the page is in ` tracker . dirty_bitmap ` . Then the following must hold before and after the signal handler runs : A . ` prot_flags = NONE < = > !accessed ` B . Invariant A ensures that we don ' t attempt to map the same page twice . Invariant B boosts performance of messages that don ' t track dirty pages . Invariants C - G ensure correct tracking of dirty pages . A speculatively dirty page needs to be validated after the execution by comparing its contents to the original page . To understand how a faulting page ` P ` is handled we need to consider where its backing memory is coming from in ` PageMap ` . There are three cases : 1 . The corresponding memory is not in ` PageMap ` meaning that the canister has never written to it . 2 . The corresponding memory is in the checkpoint file meaning that the canister has not written to it since the last checkpoint . 3 . The corresponding memory is in ` PageDelta ` . In the first two cases the underlying memory has already been mmap ' ed at in the constructor of the memory tracker . So handler tries to mprotect ` P ` and a few of its subsequent pages that have the same backing memory . This optimization is referred to as prefetching in the code . Prefetching is disabled if we are tracking dirty pages and the faulting access was a write access .
pub fn sigsegv_fault_handler_new(
    tracker: &SigsegvMemoryTracker,
    page_map: &PageMap,
    access_kind: AccessKind,
    fault_address: *mut libc::c_void,
) -> bool {
    if !tracker.memory_area.is_within(fault_address) {
        // This memory tracker is not responsible for handling this address.
        return false;
    };
    let faulting_page = tracker.page_index_from(fault_address);
    let mut accessed_bitmap = tracker.accessed_bitmap.borrow_mut();
    match (access_kind, tracker.dirty_page_tracking) {
        (_, DirtyPageTracking::Ignore) => {
            // We don't care about dirty pages here, so we can set up the page mapping for
            // for multiple pages as read/write right away.
            let prefetch_range = range_from_count(faulting_page, MAX_PAGES_TO_MAP);
            let max_prefetch_range = accessed_bitmap.restrict_range_to_unmarked(prefetch_range);
            let min_prefetch_range =
                accessed_bitmap.restrict_range_to_predicted(max_prefetch_range.clone());
            let prefetch_range = map_unaccessed_pages(
                tracker,
                page_map,
                ProtFlags::PROT_READ | ProtFlags::PROT_WRITE,
                faulting_page,
                min_prefetch_range,
                max_prefetch_range,
            );
            accessed_bitmap.mark_range(&prefetch_range);
        }
        (AccessKind::Read, DirtyPageTracking::Track) => {
            // Set up the page mapping as read-only in order to get a signal on subsequent
            // write accesses to track dirty pages. We can do this for multiple pages.
            let prefetch_range = range_from_count(faulting_page, MAX_PAGES_TO_MAP);
            let max_prefetch_range = accessed_bitmap.restrict_range_to_unmarked(prefetch_range);
            let min_prefetch_range =
                accessed_bitmap.restrict_range_to_predicted(max_prefetch_range.clone());
            let prefetch_range = map_unaccessed_pages(
                tracker,
                page_map,
                ProtFlags::PROT_READ,
                faulting_page,
                min_prefetch_range,
                max_prefetch_range,
            );
            accessed_bitmap.mark_range(&prefetch_range);
        }
        (AccessKind::Write, DirtyPageTracking::Track) => {
            let mut dirty_bitmap = tracker.dirty_bitmap.borrow_mut();
            assert!(!dirty_bitmap.is_marked(faulting_page));
            let prefetch_range = range_from_count(faulting_page, MAX_PAGES_TO_MAP);
            // Ensure that we don't overwrite an already dirty page.
            let prefetch_range = dirty_bitmap.restrict_range_to_unmarked(prefetch_range);
            if accessed_bitmap.is_marked(faulting_page) {
                // Ensure that all pages in the range have already been accessed because we are
                // going to simply `mprotect` the range.
                let prefetch_range = accessed_bitmap.restrict_range_to_marked(prefetch_range);
                // Amortize the prefetch work based on the previously written pages.
                let prefetch_range = dirty_bitmap.restrict_range_to_predicted(prefetch_range);
                let page_start_addr = tracker.page_start_addr_from(faulting_page);
                unsafe {
                    mprotect(
                        page_start_addr,
                        range_size_in_bytes(&prefetch_range),
                        ProtFlags::PROT_READ | ProtFlags::PROT_WRITE,
                    )
                    .unwrap()
                };
                dirty_bitmap.mark_range(&prefetch_range);
                tracker.add_dirty_pages(faulting_page, prefetch_range);
            } else {
                // The first access to the page is a write access. This is a good case because
                // it allows us to set up read/write mapping right away.
                // Ensure that all pages in the range have not been accessed yet because we are
                // going to set up a new mapping. Note that this implies that all pages in the
                // range have not been written to.
                let prefetch_range = accessed_bitmap.restrict_range_to_unmarked(prefetch_range);
                // Amortize the prefetch work based on the previously written pages.
                let prefetch_range = dirty_bitmap.restrict_range_to_predicted(prefetch_range);
                let prefetch_range = map_unaccessed_pages(
                    tracker,
                    page_map,
                    ProtFlags::PROT_READ | ProtFlags::PROT_WRITE,
                    faulting_page,
                    prefetch_range.clone(),
                    prefetch_range,
                );
                accessed_bitmap.mark_range(&prefetch_range);
                dirty_bitmap.mark_range(&prefetch_range);
                tracker.add_dirty_pages(faulting_page, prefetch_range);
            }
        }
    }
    true
}
pub fn sigsegv_fault_handler_new ( tracker : & SigsegvMemoryTracker , page_map : & PageMap , access_kind : AccessKind , fault_address : * mut libc :: c_void , ) -> bool { if ! tracker . memory_area . is_within ( fault_address ) { // This memory tracker is not responsible for handling this address. return false ; } ; let faulting_page = tracker . page_index_from ( fault_address ) ; let mut accessed_bitmap = tracker . accessed_bitmap . borrow_mut ( ) ; match ( access_kind , tracker . dirty_page_tracking ) { ( _ , DirtyPageTracking :: Ignore ) => { // We don't care about dirty pages here, so we can set up the page mapping for // for multiple pages as read/write right away. let prefetch_range = range_from_count ( faulting_page , MAX_PAGES_TO_MAP ) ; let max_prefetch_range = accessed_bitmap . restrict_range_to_unmarked ( prefetch_range ) ; let min_prefetch_range = accessed_bitmap . restrict_range_to_predicted ( max_prefetch_range . clone ( ) ) ; let prefetch_range = map_unaccessed_pages ( tracker , page_map , ProtFlags :: PROT_READ | ProtFlags :: PROT_WRITE , faulting_page , min_prefetch_range , max_prefetch_range , ) ; accessed_bitmap . mark_range ( & prefetch_range ) ; } ( AccessKind :: Read , DirtyPageTracking :: Track ) => { // Set up the page mapping as read-only in order to get a signal on subsequent // write accesses to track dirty pages. We can do this for multiple pages. let prefetch_range = range_from_count ( faulting_page , MAX_PAGES_TO_MAP ) ; let max_prefetch_range = accessed_bitmap . restrict_range_to_unmarked ( prefetch_range ) ; let min_prefetch_range = accessed_bitmap . restrict_range_to_predicted ( max_prefetch_range . clone ( ) ) ; let prefetch_range = map_unaccessed_pages ( tracker , page_map , ProtFlags :: PROT_READ , faulting_page , min_prefetch_range , max_prefetch_range , ) ; accessed_bitmap . mark_range ( & prefetch_range ) ; } ( AccessKind :: Write , DirtyPageTracking :: Track ) => { let mut dirty_bitmap = tracker . dirty_bitmap . borrow_mut ( ) ; assert ! ( dirty_bitmap is_marked ( faulting_page ) ) ; let prefetch_range = range_from_count ( faulting_page , MAX_PAGES_TO_MAP ) ; // Ensure that we don't overwrite an already dirty page. let prefetch_range = dirty_bitmap . restrict_range_to_unmarked ( prefetch_range ) ; if accessed_bitmap . is_marked ( faulting_page ) { // Ensure that all pages in the range have already been accessed because we are // going to simply `mprotect` the range. let prefetch_range = accessed_bitmap . restrict_range_to_marked ( prefetch_range ) ; // Amortize the prefetch work based on the previously written pages. let prefetch_range = dirty_bitmap . restrict_range_to_predicted ( prefetch_range ) ; let page_start_addr = tracker . page_start_addr_from ( faulting_page ) ; unsafe { mprotect ( page_start_addr , range_size_in_bytes ( & prefetch_range ) , ProtFlags :: PROT_READ | ProtFlags :: PROT_WRITE , ) . unwrap ( ) } ; dirty_bitmap . mark_range ( & prefetch_range ) ; tracker . add_dirty_pages ( faulting_page , prefetch_range ) ; } else { // The first access to the page is a write access. This is a good case because // it allows us to set up read/write mapping right away. // Ensure that all pages in the range have not been accessed yet because we are // going to set up a new mapping. Note that this implies that all pages in the // range have not been written to. let prefetch_range = accessed_bitmap . restrict_range_to_unmarked ( prefetch_range ) ; // Amortize the prefetch work based on the previously written pages. let prefetch_range = dirty_bitmap . restrict_range_to_predicted ( prefetch_range ) ; let prefetch_range = map_unaccessed_pages ( tracker , page_map , ProtFlags :: PROT_READ | ProtFlags :: PROT_WRITE , faulting_page , prefetch_range . clone ( ) , prefetch_range , ) ; accessed_bitmap . mark_range ( & prefetch_range ) ; dirty_bitmap . mark_range ( & prefetch_range ) ; tracker . add_dirty_pages ( faulting_page , prefetch_range ) ; } } } true }
This is the new (optimized) signal handler.
This is the new ( optimized ) signal handler .
tracker
&SigsegvMemoryTracker
page_map
&PageMap
access_kind
AccessKind
fault_address
*mut libc::c_void