Realtime control RTC is pivotal for any Adaptive Optics AO system including highcontrast imaging of exoplanets and circumstellar environments. It is the brain of the AO system and what wavefront sensing and control WFSampC techniques need to work with to achieve unprecedented image quality and contrast ultimately advancing our understanding of exoplanetary systems in the context of high contrast imaging HCI. Developing WFSampC algorithms first happens in simulation or a lab before deployment onsky. The transition to onsky testing is often challenging due to the different RTCs used. Sharing common RTC standards across labs and telescope instruments would considerably simplify this process. A data architecture based on the interprocess communication method known as shared memory is ideally suited for this purpose. The CACAO package an example of RTC based on shared memory was initially developed for the SubaruSCExAO instrument and now deployed on several benches and instruments. This proceeding discusses the challenges requirements implementation strategies and performance evaluations associated with integrating a shared memorybased RTC. The Santa Cruz Extreme AO Laboratory SEAL bench is a platform for WFSampC development for large groundbased segmented telescopes. Currently SEAL offers the user a nonrealtime version of CACAO a sharedmemory based RTC package initially developed for the SubaruSCExAO instrument and now deployed on several benches and instruments. We show here the example of the SEAL RTC upgrade as a precursor to both RTC upgrade at the 3m Shane telescopes at Lick Observatory ShaneAO and a future development platform for the Keck II AO. This paper is aimed at specialists in AO astronomers and WFSampC scientists seeking a deeper introduction to the world of RTCs.
['Astrophysics - Instrumentation and Methods for Astrophysics']
Realtime control and data standardization on various telescopes and benches
{'Real-time control and data standardization on various telescopes and benches': "Nour Skaf a , Rebecca Jensen-Clem a , Aaron Hunter a , Olivier Guyon b,d,e,f , Vincent Deo b , Phil Hinz a , Sylvain Cetre a , Vincent Chambouleyron a , J. Fowler a , Aditya Sengupta a , Maissa Salama a , Jared Males e , Eden McEwen e , Ewan S. Douglas e , Kyle Van Gorkom e , Emiel Por g , Miles Lucas b , Florian Ferreira h , Arnaud Sevin h , Rachel Bowens-Rubin a , Jesse Cranney i , and Ben Calvin j  \na Department of Astronomy & Astrophysics, University of California, Santa Cruz, CA 95064, USA b National Astronomical Observatory of Japan, Subaru Telescope, 650 North A'oh¯ok¯u Place, Hilo, HI 96720, U.S.A.  \nd Astrobiology Center of NINS, 2-21-1 Osawa, Mitaka, Tokyo 181-8588, Japan e Steward Observatory, University of Arizona, Tucson, AZ 85721, USA f Wyant College of Optical Sciences, University of Arizona, Tucson, AZ 85721, USA g Space Telescope Science Institute (STScI), 3700 San Martin Dr, Baltimore MD, 21218, USA h LESIA, Observatoire de Paris, Universit'e PSL, Sorbonne Universit'e, Universit'e de Paris, CNRS, 5 place Jules Janssen, 92195 Meudon, France i Research School of Astronomy and Astrophysics, Mt Stromlo Observatory, Weston Creek, ACT 2611, Australia  \nj University of California - Los Angeles, 430 Portola Plaza, Los Angeles, CA 90095, USA", 'ABSTRACT': 'Real-time control (RTC) is pivotal for any Adaptive Optics (AO) system, including high-contrast imaging of exoplanets and circumstellar environments. It is the brain of the AO system, and what wavefront sensing and control (WFS&C) techniques need to work with to achieve unprecedented image quality and contrast, ultimately advancing our understanding of exoplanetary systems in the context of high contrast imaging (HCI). Developing WFS&C algorithms first happens in simulation or a lab before deployment on-sky. The transition to on-sky testing is often challenging due to the di ff erent RTCs used. Sharing common RTC standards across labs and telescope instruments would considerably simplify this process. A data architecture based on the interprocess communication method known as shared memory is ideally suited for this purpose. The CACAO package, an example of RTC based on shared memory, was initially developed for the SubaruSCExAO instrument and now deployed on several benches and instruments. This proceeding discusses the challenges, requirements, implementation strategies, and performance evaluations associated with integrating a shared memory-based RTC. The Santa Cruz Extreme AO Laboratory (SEAL) bench is a platform for WFS&C development for large groundbased segmented telescopes. Currently, SEAL o ff ers the user a non-real-time version of CACAO, a shared-memory based RTC package initially developed for the Subaru-SCExAO instrument, and now deployed on several benches and instruments. We show here the example of the SEAL RTC upgrade as a precursor to both RTC upgrade at the 3-m Shane telescopes at Lick Observatory (Shane-AO) and a future development platform for the Keck II AO. This paper is aimed at specialists in AO, astronomers, and WFS&C scientists seeking a deeper introduction to the world of RTCs.  \nKeywords: Adaptive Optics, Real-Time Control', '1. INTRODUCTION': "With nearly 6,000 discovered exoplanets and numerous dedicated space missions and ground-based instruments, the exoplanet era is advancing towards more robust detection and characterization methods. Di ff erent detection techniques, such  \nFurther author information: (Send correspondence to N.S)  \nas transit, radial velocity, direct imaging, and astrometry, are detecting new planets, enabling extensive statistical searches and surveys. However, direct imaging remains a significant technical challenge due to the overwhelming brightness of exoplanet host stars and the Earth's atmospheric distortions impacting the incoming wavefront.  \nIn this context, AO has emerged as a pivotal technology for exoplanet science, o ff ering more and more sophisticated wavefront sensing and control (WFS&C) techniques to mitigate the deleterious e ff ects of Earth's turbulent atmosphere, as well as the non-common path aberrations (NCPA), which are aberrations induced by the optical path between the AO loop and the science camera. Every high contrast imaging (HCI) instrument is equipped with an AO system, and the next generation of space-based telescopes will also be equipped with AO elements. Every AO system includes at least three core components: a wavefront sensor (WFS), an RTC, and a deformable mirror (DM). The RTC is comparable to the brain of AO: it is the software and hardware that performs the computation of the correction needed for the DM to correct atmospheric aberrations in real-time. In many cases, the RTC has been seen as a 'set and forget' system, computing the corrections in the background of the science observations. However, interacting with an RTC for WFS&C purposes is essential yet frequently perceived as complex by astronomers, requiring significant time and e ff ort to adapt WFS&C techniques to di ff erent RTC systems across various benches and telescopes. Furthermore, because the RTC is intrinsically part of the AO ecosystem, it is part of the experimental framework that must be flexible rather than static.  \nIn response to these challenges, some members of the RTC community have moved towards establishing common standards across instruments, notably through the use of shared-memory architectures for data management and interaction with the hardware. Mainly, the use of shared memory has become a standard in dealing with the data flows, which represents several benefits, especially in terms of modularity and flexibility. The use of shared-memories for RTC is originally from the CACAO (Compute and Control for Adaptive Optics) software, 1 initially designed for the Subaru Coronagraph Extreme Adaptive Optics (SCExAO) 2 needs. It is now being used by several additional instruments and optical benches. The target audience of this paper is AO scientists, astronomers, and WFS&C scientists seeking a deeper introduction to the RTC world. We will focus especially on the RTC installation and integration using a few examples, focusing on the integration of CACAO on the Santa Cruz Extreme AO Lab (SEAL) testbed. 3 We will describe the power of the use of shared-memories as a standard for data management architecture, as part of a long-term global strategic move towards a more collaborative and e ffi cient scientific environment, in Sections 3 and 4. Nowadays, many RTCs are based on CACAO's shared-memory architecture, each being designed for di ff erent uses, yet all benefiting from the modularity and flexibility of the shared-memory architecture which we will develop in Sections 3 and 4. Although this paper focuses more on CACAO, we also note that other shared-memory-based RTCs are currently in use, some of which are described in Section 4.3. The focus on CACAO in this paper stems from the on-going e ff ort to develop CACAO as an open-source RTC industry standard. It is being upgraded with both community-oriented and performance-oriented goals, in order to facilitate adoption at new facilities. In this paper we use the SEAL testbed to demonstrate its implementation. In the process, we present some best-practices recommendations for successful integration and collaboration in the use of RTC systems. Finally, we lay out the vision of the opportunities for an integrated RTC system that includes SEAL, the Shane 3-m telescope (ShaneAO) at Lick Observatory, and Keck II AO, for both technical development and science observations.", '2.1 The place of the RTC in an AO system': 'The AO system is composed of three main components: the wavefront sensor (WFS), that measures the incoming wavefront; the deformable mirror (DM), that corrects the incoming wavefront; the real-time control (RTC), that links the WFS to the DM.  \nLight averrated by earth atmosphere travels through the telescope system to the WFS. Once measured by the WFS, the wavefront aberrations must be reconstructed, allowing us to estimate the wavefront phase. The information from the WFS is sent to the RTC that reconstructs the wavefront phase. The RTC proceeds in an open, pseudo-open, or more generally closed loop operating in the kHz regime to keep up with the rapidly changing turbulence of the atmosphere. The RTC calculates the commands that need to be sent to the DM, which is the hardware element that compensates for the distorted wavefront. In a nutshell, the WFS first samples the wavefront, the RTC calculates its phase, and the DM acts accordingly to correct the aberrations.  \nThe RTC is the interface between the wavefront received and its correction. It plays two significant roles: estimating the wavefront based on the WFS data and then applying a relevant command to send to the DM. There is usually a linear  \nrelationship between these two components, allowing us to rely on linear algebra, with matrix-vector multiplication (MVM) to run the control loop. This must be done in real-time as the atmosphere is continuously evolving.', '2.2 Real-time operating system': "Over the past half-century, advancements in computer science and processor development have led to increasingly sophisticated operating systems (OS) with complex task management capabilities designed to maximize the e ffi cient utilization of processor resources. This complexity, however, has rendered modern computers non-deterministic in their task management processes, causing them to perform tasks with variable latency and no guarantee of the order of execution. Real-time control systems, however, require deterministic (that is to say, fixed) latency to properly control the closed loop behavior of a dyanamic system such as an AO system. Modern computers are often fast enough that the variation in latency is small compared to the loop time, but AO systems increasingly require shorter loop times and more degrees of freedom to correct for atmospheric disturbances. These requirements stress the performance of the processor to the point where the latency variation a ff ects the controller performance.  \nTo illustrate this point, we explore the concept of determinism. On a single CPU, the computer can execute only one process at a time. There is a scheduler that is responsible for allocating the required time for each process and determining which task to perform next. The scheduler is designed to manage task order autonomously, balancing all processes with equal priority to manage resources e ffi ciently. Because the scheduler manages many processes simultaneously, its behavior is challenging to predict. The scheduler selects which task to execute based on various parameters, such as the task's readiness, but importantly not according to a predetermined task priority. There is minimal visibility into the scheduler's decision-making process and task execution order can change instantaneously, contributing to the computer's non-deterministic nature.  \nOne of the primary focuses of RTC development is to change the behavior of the core of the operating system to achieve deterministic behavior. Simplistically speaking, the RTC imposes specific priorities on tasks, thereby directing the scheduler to perform these tasks in order of importance. Less important tasks are preempted by more important tasks. By imposing these priorities, RTC systems achieve more deterministic behavior, ensuring predictable and reliable task execution essential for real-time control. For additional information on real-time kernel and performance, we refer to the SPIE Proceeding of Deo et al. 2024.", '2.3 Real-time control essentials': 'In this section, we lay out the basics of what an RTC has to do, including key definitions and linear algebra concepts.', '2.3.1 Matrices definitions': 'Linear algebra is at the base of AO control * . There are three main matrices taking place in the control loop:  \nResponse matrix: The response matrix (RM) is the linear mapping from the WFS measurements s to the DM commands c : s = Dc ; with D being the response matrix. It is usually measured by simply poking the DM with a combination of modes and recording the response on the WFS.  \nControl matrix: Once the relationship between the WFS and the DM is acquired with the RM, we aim to estimate the DM commands to compensate for the measured WFS signal. To this end, we must compute the control matrix, which is usually the pseudo-inverse of the response matrix: C = D † . However, some modes applied on the DM during the RM acquisition have a weak signal on the WFS. These modes must be removed to avoid any excess noise. This is usually performed with a singular value decomposition (SVD) of the RM, and the rejection of the weak modes, before calculating its pseudo-inverse.  \nReconstruction matrix: A modal reconstruction matrix translates the WFS measurements to modes. Di ff erent approaches include going from WFS to DM with a control matrix and going from WFS to modes with the reconstruction matrix, and then from modes to DM. This second approach is the modal control framework.', '2.3.2 Control loop': "The control loop allows us to link WFS measurements to DM commands, and its temporal behavior is inherently part of the AO system's error budget. As shown in Figure 1, control laws ensure that the DM applies the relevant correction, taking into account the noise propagation and latency e ff ects, while applying a gain to the correction. 4 Typically, an integrator filter is introduced, so only a fraction of the correction, defined by the gain g , is applied to avoid instabilities linked to latency between WFS measurements and DM correction and to allow the reduction of photon noise by averaging the noisy WFS measurements.  \nWe describe In the case of a closed loop, for a frame k , a perfectly corrected wavefront corresponds to: Φ DM [ k ] = Φ Atm [ k ]; and the WFS sees a perfectly flat wavefront. With the temporal evolution of the atmospheric turbulence, the WFS sees the frame k + 1 with residual errors:  \nΦ Res [ k + 1] = Φ Atm [ k + 1] -Φ DM [ k ] = Φ Atm [ k + 1] -Φ Atm [ k ] (1)  \nThere is hence a need to decrement the command Φ DM of Φ Res [ k + 1]. We then need to use a command law with an integrator filter: for each frame calculated with the RTC, the command c applied to the DM follows:  \nc [ k ] = c [ k -1] -g × C × s [ k ] (2)  \nwith s [ k ] being the WFS measurement at the frame k , C being the control matrix, and g being the gain of the integrator. This is a simplistic description of the control loop with its gain in the case of an integrator type of controller. The gain is usually lower than 0.5, to avoid any loop instability. In this case, that means that 50% of the correction as measured by the WFS is applied. There are of course several other types of control laws besides the integrator, such as the Linear Quadratic Gaussian control, 5 the predictive control 67 , 8 and reinforcement learning 9 . 10  \nFigure 1. Simplistic block diagram of the control loop, in closed-loop, with an integrator filter.  \n<!-- image -->  \nThe AO control loop includes several steps that need to be processed by the RTC. Representative steps of the control loop include:  \n- · Wait for WFS image s ;\n- · Calibrate: normalise WFS frame to the total intensity;\n- · Subtract the WFS reference;\n- · Compute and extract the WFS measurements;\n- · Multiply the resulting by the control matrix C ;\n- · Update DM state, including loop gain g ;\n- · Apply correction to DM;\n- · Record DM state for next loop iteration;\n- · Repeat until the loop opens.  \nSeveral remaining sources of error have a considerable impact on the final AO system performance at the science camera. They are characterized by the error variance they leave on the final phase, and the total variance of the residual phase corresponds to the sum of the individual variances, defining the error budget. There are errors due to the dimensioning of the system, mainly fitting and aliasing errors. Then there are errors due to the system itself, and the main error stemming from the RTC is the temporal error , linked with the lag between the measurement of the phase and its correction on the DM. Several studies expand on the AO error budget, such as the pure lag and the bandwidth error, and how these relate with the expose time and readouttime of the WFS detector, computation time on the RTC, and the reduction time of the DM. 11", '2.4 Next-generation adaptive optics control': "This section discusses several possibilities for future directions in RTC for AO. In present-day conventional AO systems, including the current generation of extreme-AO systems, the RTC follows a well-defined linear control scheme: input measurements from a single WFS are multiplied by a control matrix to produce an output DM control command. Thanks to recent high-performance computing capabilities and new algorithms allowing greater computation speed and power, new extensions to this framework are now emerging to help address various sources of error. Below are a few examples:  \nSensor fusion techniques can ingest measurements from multiple WFSs and possibly other sensors in the AO system, aiming to improve the correction's sensitivity, completeness, and robustness. Sensitivity is improved by combining WFSs that, together, lead to a greater resolution of the wavefront. Completeness is improved by gathering information from WFSs that are complementary to the measurements they perform. Robustness gains result from eliminating measurement ambiguities. Sensor fusion techniques, especially those that include focal plane wavefront sensing, contribute to addressing the non-common path aberrations (NCPAs).  \nPredictive control improves the AO system's temporal error without sacrificing measurement noise. It is the solution to the AO system's most fundamental limit: the trade-o ff between flux and correction speed. In a conventional AO system, the WFS measurement must be su ffi ciently fast to keep up with rapidly changing turbulence, but su ffi ciently slow to average photon noise, and overcome readout noise. The conventional RTC operates at a speed that is a painful compromise between time lag and measurement noise. With predictive control, the time lag is mitigated and replaced by a much smaller temporal prediction error. Recent measurements can be extrapolated in a way that o ff ers the noise-reduction benefits of time averaging without incurring the temporal lag cost. 6 Several studies on predictive control have recently been published on that topic. 7, 8, 12, 13  \nNon-linear control extends the range of WFS solutions to include, for example, focal plane images (but not necessarily), in which information about optical aberrations is encoded with high sensitivity,s but for which the relationship to measurements is highly non-linear and induces inaccurate corrections over the wavefront. Artificial Intelligence (AI) plays a significant role in improving non-linear controls, particularly through techniques such as machine learning, neural networks, reinforcement learning, 14 and has been playing a growing role in the last couple of years. 15 For example, reinforcement learning allows a system to learn optimal control strategies through trial and error, adjusting to non-linear dynamics by maximizing a reward signal.  \nSelf-calibration is essential for the robust operation of the above techniques. In conventional AO, calibration is performed o ffl ine prior to running the AO system. Typically, a response matrix will be measured by actuating individual DM actuators or modes and recording the corresponding linear response in the WFS. This approach cannot measure the actual on-sky system response which evolves due to variations in conditions, and it cannot extend to the sensor fusion, predictive control, and non-linear control schemes described above. Instead, the calibration needs to be continuously derived from recent (approximately less than 10 mn for predictive control) on-sky measurements. 16,17", '2.5.1 Introduction': 'High-performance computing (HPC) refers to the software technique designed to get the most out of the use of computers, often supercomputers. The hardware provides the computing power (whether it is GPU, CPU or FPGA), the HPC uses parallel processing techniques to solve complex computational problems quickly and e ffi ciently. HPC systems typically consist of multiple processors or compute nodes that work together to perform computations in parallel, allowing them to tackle large-scale problems that would be impractical or impossible to solve with traditional computing methods. HPC systems often utilize specialized hardware components optimized for computationally demanding tasks, as for example graphics processing units (GPUs). HPC is crucial for RTC systems for several reasons, and has been an active field of development. 18-22  \n- · Speed: RTCs require rapid processing of large datasets to make timely corrections. HPC provides the necessary computational power to handle these tasks quickly, ensuring that control actions are executed within the required time frames, often in milliseconds or microseconds.\n- · Complexity: RTCs involve complex algorithms and computations, which HPC enables by providing the computational resources needed to perform complex calculations in real-time.\n- · Accuracy: RTCs often require high precision and accuracy in decision-making to ensure the safety and reliability of the controlled processes. HPC facilitates the use of advanced algorithms and numerical techniques that can produce accurate results even under tight time constraints.\n- · Reliability: RTCs must be resilient to failures to ensure uninterrupted operations in the right order, and minimize the impact of hardware or software failures on system performance and reliability.', '2.5.2 How to determine the HPC needs for a RTC system: the example of the Matrix-Vector Multiply (MVM) and GPU / CPU Specs': 'The most compute-heavy operation in closing the AO loop is often the matrix-vector-multiply (MVM) converting the input WFS pixel values to output wavefront modes. This MVM must be completed in a fraction of the AO loop period, typically well under 1 ms. The MVM computation time is most often limited by the number of memory accesses needed to perform the computation: it is a memory-bound problem. Then, the bigger the memory bandwidth of the hardware is, the faster is the computation time. Taking, for example, a large system with 87k input pixels, 33k output modes. Performing this MVM will require to access 11.4 GB of data (considering single precision, i.e. 32 bit per element). Considering the requirement of 1 ms for the RTC time budget, it leads to a requirement of 11.4 TB / s in terms of memory bandwidth.  \nAs of today (year 2024), current GPU such as NVIDIA H100 † have memory bandwidth of approximately 4 TB / s (note this is terabytes, not terabits). Comparing these specs with the requirement derived above reveals that the system will need at least 3 H100 GPU to do the computation on time.', 'SCExAO computer as an on-sky RTC example:': 'SCExAO relies on five di ff erent computers. The computer on which the CACAO RTC runs is called scexaortc , and is the most powerful of all the SCExAO computers, with the most demanding high-performance computing requirements. It has two AMD Epyc 7763 processors with 128 cores total, 1TB RAM, PCI Gen 4, 5 GPUs (A6000, 3080Ti, 2080Ti), and 600TB + of storage, including 20GB / s + temp storage SSDs. scexao6 is operating a fast 100Gbps downlink from the instrument (located in the IR Nasmyth platform of the Subaru Telescope) to the server room without data collisions and without introducing too much timing variability. It deals with managing everything linked with the RTC, and controls the following aspects:  \n- · the PyWFS data processing to drive the ExAO loop;\n- · the DM display and control;\n- · the Tip-Tilt (TT) loop;  \n- · the astrogrid status;\n- · the low-order WFS loop.  \nSCExAO performs the second-stage AO thanks to a visible pyramid WFS, operating in the 600-950 nm wavelength range. 23 It controls a 2000-actuator MEMS DM from Boston Micromachines. There are 45 actuators across the pupil, giving a control region of ± 22.5 λ/ D squared. The Compute And Control for Adaptive Optic (CACAO) software manages the real-time control (RTC) of the AO system. 24 We use the CACAO software to interact with the system and implement the algorithm to communicate between the PyWFS and the DM. CACAO enables the control of the DM through 12 di ff erent channels, and each channel is a potential input map, o ff ering a layer of correction, given that several corrections are implemented from di ff erent wavefront sensors or wavefront control techniques. CACAO then sums up all the maps provided by the user and applies the summed map to the DM. This allows for summing di ff erent corrections in parallel, based on multiple WFSs and control algorithms.', 'The SEAL computer as a bench RTC example :': "SEAL 3 has an Intel Xeon Gold 6126 CPU operating at 2.60GHz, and an NVIDIA GPU GeForce GTX 1080. SEAL has Python and Julia interfaces to the krtc / DAO real-time control system, 25 which is an o ff -shoot of CACAO. Since SEAL's focus is technology demonstrations, implementations of experiments often prioritize flexibility over speed and therefore do not yet focus on high-speed operations. However, experiments can achieve high AO loop rates when required. An implementation of linear-quadratic-Gaussian control 26 operated at 100 Hz, with the main limiting factor being the di ffi culty of synchronization between the WFS, DM, and simulated disturbances across threads. This was a limitation imposed by the Python interface rather than the hardware or RTC, and it was addressed within these constraints by spinlocking to ensure precise timing across threads. Similarly, tests of multi-WFS single-conjugate AO 27 successfully operated with two AO loops in di ff erent Python sessions at 100 Hz and 200 Hz. Most of the latency in these cases came from the reconstruction algorithm for the FAST focal-plane wavefront sensor. The algorithm required two FFTs, requiring 5ms with significant jitter on scales that a ff ected the AO loop rate. Using quicker implementations, for example via the Julia interface and / or GPU, or computationally easier AO algorithms, could enable even faster operation up to kHz scales.  \nSEAL is also often a ff ected by latency due to concurrent hardware access. Operating several cameras at once can limit the loop rate of each one, and in some cases can completely stall operations. Updates to the RTC or the interface may help to address this.", '3.1 Shared memory data management description': 'Shared memory refers to a type of inter-process communication (IPC) mechanism in computing where multiple processes can access common memory locations. In shared memory systems, processes can communicate and synchronize by reading from and writing to the same memory addresses. This shared memory region is typically managed by the operating system and can be accessed by multiple processes concurrently. Shared memory is a fundamental concept in parallel and distributed computing, enabling e ffi cient communication and coordination among processes or threads sharing a common address space. It is widely used in various computing environments, and ideally suited for RTC. It is especially an e ffi cient data management method, because it avoids the overhead associated with copying data between processes. Since processes directly access shared memory locations, data can be transferred quickly and with low latency. Shared memory requires mechanisms for synchronizing access to shared data to prevent race conditions and ensure consistency. Techniques such as semaphores are commonly used to coordinate access to shared resources and enforce mutual exclusion among processes.', '3.2 Problem statement and intention': "The realm of RTC systems is software-centric, posing significant challenges for astronomers and WFS&C scientists who often lack the specialized training needed to directly modify RTC code. Developing a thorough understanding of the RTC's behavior, parameters, and the impact of various inputs on outputs can be the work of months or years, depending on the background in software engineering. Moreover, the interfaces for RTC systems often present complex information, leading to information overload on the user end. To address some of these challenges, some RTC developers across various AO systems have adopted a standard for data management and interaction: the shared-memory architecture. This section  \ndescribes the advantages of shared-memory architecture and its importance for facilitating ease of use and collaboration in RTC development. By using a shared-memory approach, and allowing access to the shared-memories, the interaction with the RTC becomes more transparent, allowing users to implement their algorithms without needing to understand the details of the system's inner workings.  \nThe user still needs to understand how the data in the shared memory is treated by the RTC, hence there must be an understanding of how the wavefront control happens within the RTC. Ideally, and the RTC community has been working on this aspect, the user does not need to dive into the deeper parts of the software that can remain a form of 'black box'. Below is a concrete example of a code that takes both the PSF and WFS data from the shared memory, and synchronizes them. Figure 2 shows an example of a script that collects PSF and WFS frames, and synchronizes them.  \nFigure 2. Example of a python function that gets PSF and WFS frames and synchronizes them. Lines 11 and 12 correspond to the name associated to the shared memory of the PSF and WFS respectively (for example, 'pueo' and 'vcam1' for SCExAO's pyramid WFS and Vampires' first camera). These two lines are su ffi cient alone, the rest is simply an example of use. Lines 18 and 19 then assign a name for these specific shared memories that will be used later in the code. In this case, it is to allow for changing which shared memory one wants to work with. Line 21 and 23 shows an example of a function that gives some information on the shared memory, in this case the keyword for each frame.  \n<!-- image -->  \nThis architecture not only simplifies the interface for end-users but also fosters collaborative work by providing a common ground for data interaction. It enables closer-to-seamless integration and communication between di ff erent components of the system, promoting a cooperative environment where complex tasks are made more manageable. The sharedmemory system thus o ff ers an elegant solution to the inherent challenges of RTC systems, making the technology more  \nFigure 3. Schematic view of the data management with shared memories between the WFS and the DM. The light blue boxes are processes, linked together with streams, represented in dark blue boxes. The signaling between streams and processes is achieved through semaphores: a process 'waits' to a semaphore, and will start as soon as the semaphore is 'posted' to 1, at which point it will decrement the semaphore value back to zero. Figure adapted from. 16  \n<!-- image -->  \naccessible and inspiring collaborative e ff orts in the pursuit of scientific progress. In some cases, however, the sharedmemories are nested with the real-time processes, which is necessary to understand to keep an optimal performance. In such a case, providing free access to all the shared memories to the user can bring conflicts within the processes, and some communication with the RTC developers is necessary to avoid any issues.", '3.3 The CACAO RTC': 'CACAO ‡ is presented in several proceedings 24 , 1 and in the proceeding of Deo et al. 2024. Because many RTCs worldwide are based on some aspects of CACAO (mainly the shared-memory structure), or got inspired by it, we present here a description of the CACAO RTC, its architecture, and its data management system. This is targeted to WFS&C users and astronomers rather than RTC developers.', '3.3.1 CACAO data management': 'The data flow from the di ff erent streams is managed with shared memory structures, accessible directly from computer RAM. To avoid any collision, a system of semaphores, locking shared memory bu ff ers, arises when waiting for the appropriate timing for the right data flow. There are ten stream semaphores. This architecture allows users to plug and play their own codes without diving into the hardware camera interfaces. Figure 3 presents a schematic view of this shared memory architecture, and the steps taken within the RTC in an AO system.', 'ImageStreamIO': "ImageStreamIO § is the core library of all of CACAO: it contains all the data format and synchronization. It allows the  \n§ Open source, available here: https://github.com/milk-org/ImageStreamIO .  \nFigure 4. Canonical modal control in CACAO. The pink rectangular boxes represent shared memories. CACAO first reads the sharedmemory that is given by the WFS, canonically called 'aol wfsim'. It then processes the signal and writes into the shared-memory named 'dmvolt', which is then read by the DM.  \n<!-- image -->  \nuse of the data streams in shared memory image format. This deals directly with the camera FITS files, allowing to read and write data streams, at low latency and a high throughput input / output (I / O) of data. There are two main scripts for data conversion, either converting from shared memory to FITS files ( shmim2fits ) or converting from FITS files to shared memory ( Fits2shm ).", 'Milk': "The Milk (Modular Image processing Library toolKit) software framework ¶ mainly manages the data architecture, providing tools for image processing and analysis in real-time. Milk also provides a command line interface for easy access to functions and arguments, manages runtime loading of modules to extend capabilities, and provides inter-process communication (IPC), for sharing process status and function parameters, and images. Milk is coded in C, which isn't the most used language in the high-contrast imaging community. To make it more user-friendly, a python version was developed, pyMilk ∥ , which is a python wrapper to ImageStreamIO. It provides an easy way to link python scripts and the shared memory structures used in Milk .", '3.4 Hardware interface': "The hardware interface is often the main challenge when interfacing with a software. Connecting with the hardware drivers requires working with the libraries of the company of the hardware. Each hardware company has their own software that needs to be communicated with to translate the data into shared memory format.  \nUnderstanding the various hardware layers involved in communication is essential for e ff ective integration and operation of systems like cameras. The communication stack of a given camera typically includes multiple layers, each with specific functions, including:  \n- · Physical layer: this is the most fundamental layer, consisting of the camera's physical components and the connections to the frame grabber. The physical layer involves the conversion of light into electrical signals by the camera's sensor.\n- · Frame Grabber: this is a crucial hardware component that captures the electrical signals from the camera's sensor. It contains electronic chips and associated firmware. The firmware provides instructions to the chips, enabling them to convert the electrical signals (voltages) into digital pixel data.\n- · Driver: the driver is a software component that interfaces with the operating system (OS). It translates the pixel data from the frame grabber into a format that the OS can understand and process. The driver ensures that the OS can read the pixel data and make it accessible to other software applications on the computer.  \nFigure 5. Example of the creation of a driver to use a camera library to turn the data into a shared-memory stream. The code is available at: https://github.com/milk-org/milk/blob/dev/src/milk module example/examplefunc3 updatestreamloop.c  \n<!-- image -->  \n- · Application Programming Interface (API): the API is a software layer that provides a set of routines and protocols for interacting with the hardware (e.g., the electronic card within the frame grabber). The API abstracts the hardware details and o ff ers a simplified interface for users to use in their applications.\n- · Software Development Kit (SDK): the SDK is an extension of the API, providing additional tools, libraries, and documentation to facilitate software development. The SDK includes a user interface and sample code, making it easier for users to create applications that interact with the camera and process the images. It is through the SDK that the user communicates with everything else.  \nE ff ective hardware communication and integration requires a clear understanding of these layers. Each layer plays a critical role in converting raw sensor data into usable digital information, ultimately enabling the development of sophisticated applications. Usually for the user, the word 'driver' is often used to refer to all of these elements.  \nCACAO has adopted a straightforward approach to interfacing custom software with dedicated hardware, where one stream is defined as a hardware endpoint and is the only interface exposed to the rest of the CACAO ecosystem. SCExAO has developed a collection of interfaces that cover a variety of scientific camera manufacturers (Andor, First Light Imaging, Teledyne / FLIR, Hamamatsu, Nuvu, ...) as well as deformable mirrors (Boston Micromachines, ALPAO). The easiest hardware integration is with hardware that was already integrated into the shared-memory format with the manufacturers mentioned above. If this is not the case, the steps for successful hardware integration are to first work with the hardware manufacturer and have the library run in C or C ++ . Then, this can be used in a CACAO template that will transfer the data into a shared-memory. The template can be found in the Milk module, Figure 5 presents the four main steps to create a new driver from scratch.", '4. RATIONAL FOR SHARED-MEMORY-BASED RTC': "This section highlights the benefits of using the shared-memory solution for data management. Although the sharedmemory isn't the solution to all the problems, the main point here is to present this tool as a form of standard used by some of the RTC community for increased collaboration.", '4.1 Highlight of flexibility and cost': "The use of shared-memory based RTC systems introduces significant flexibility in hardware selection, allowing for the use of 'commercial o ff -the-shelf' (COTS) components. This approach highlights the primary advantage of shared-memory systems: hardware agnosticism. The software operates independently of the underlying hardware, enabling nearly seamless hardware upgrades with minimal software modifications and simplifying the integration of spare parts.  \nThis flexibility o ff ers several benefits. Firstly, new developments, including the integration of new hardware from familiar manufacturers, can be expedited. Secondly, redeploying hardware from one computer to another becomes more easily achievable, as reconfiguration is relatively rather straightforward. Lastly, the relative simplicity of the hardwaresoftware interface, compared to more complex software stacks, allows users to focus on performance optimization rather than extensive feature coverage.", '4.2 Collaborative development and open source': "Whether a specific RTC is open source or close source, the use of shared-memories allows for greater collaborative opportunities through e ffi cient data sharing, and hence seamless integration of hardware and WFS&C algorithms. Shared memories furthermore provides a standardized method of communication with the di ff erent hardware components in different AO systems. This standardization is crucial for collaboration, as it enables communication with hardware components from di ff erent manufacturers or research groups to interoperate smoothly, facilitating projects that involve several institutions with common endeavors. Indeed, several instruments and benches worldwide use shared-memory-based RTC, from CACAO's ImageStreamIO format. Below is a non-exhaustive list of these cases, we will expand in the next session on some of these RTCs.  \nCACAO was adopted for the control of the MagAO-X 28 platform at the Magellan telescope in Chile. CACAO is the RTC solution used by the MAPS project, 29 currently mid-commissioning at the 6.5-m MMT telescope in Arizona. CACAO is used by the experimental platform KalAO, 30 led by the observatory of Geneva and operational at the 1.2 m Euler Telescope at La Silla Observatory. Promoted by these early adopters, CACAO is already the chosen solution for future projects at Subaru Telescope - the AO3K [59] upgrade and the ULTIMATE laser tomography [60,61] system; and for the GMagAO-X high-contrast platform on the upcoming 24 m Giant Magellan telescope [19]. CACAO's sharedmemory format is also at the base of the COSMIC RTC, which is the RTC for MICADO on the E-ELT, 31 Keck I and II, 32 as well as MAVIS on the VLT. 33 Likewise, the DAO RTC, 25 which is the RTC for HARMONI on the E-ELT. Along with these instruments, some optical benches, such as HiCAT at STSCI, got inspired by CACAO's shared-memory data management style. 34", '4.3.1 MagAO-X': 'The Magellan Adaptive Optics Extreme - MagAO-X - instrument, is installed on the 6.5 m Magellan Clay telescope at Las Campanas Observatory in Chile. It is the prototype platform for the GMagAO-X instrument, which will be on the Giant Magellan Telescope (GMT). 35 MagAO-X is designed for HCI of exoplanetary systems. Similar to SCExAO, MagAO-X has a dual purpose: WSF&C development, along with astronomy-science observations.  \nMagAO-X uses CACAO for its core RTC functionality. The MagAO-X instrument control software system is a c ++ application framework designed to work with CACAO. In particular, it uses the ImageStreamIO library for all image acquisition, processing, and low-latency IPC. A key rationale for this architecture is that any camera, including science detectors, can be used as a wavefront sensor feeding corrections to any deformable mirror (or other corrector, such as tip / tilt stages).  \nA unique feature of MagAO-X is that it uses a woofer-tweeter configuration, using o ffl oading, It uses a woofer-tweeter configuration to provide su ffi cient low-order stroke while enabling high order correction with a MEMS DM. The shape on the tweeter is averaged for a configurable number of loop steps. That average shape is then multiplied by the tweeter-towoofer response matrix, and the result is applied to the woofer with a gain and a leak. With long averaging ( ∼ 10 loop steps) and a small gain it causes a small perturbation to the high-order loop. The tweeter-to-woofer response matrix is calibrated from the woofer and tweeter zonal response matrices taken on the WFS in the same alignment. The o ffl oading modes are normally filtered, with typically only 5 to 10 low-order modes being o ffl oaded (which depends on the observing conditions).', '4.3.2 MAPS': 'The MMT AO exoPlanet characterization System (MAPS) is an extensive upgrade to the adaptive secondary mirror (ASM) AO system at the 6.5 m MMT telescope on Mt. Hopkins, Arizona 29 (see Morzinski et al, in these proceedings). The new MAPS system includes both visible and IR pyramid sensors, as well as several IR science instruments. Like MagAO-X, MAPS uses CACAO for its core RTC functionality and MILK and ImageStreamIO are used extensively for instrument control. MAPS has two interesting features compared to typical CACAO-based systems: it uses pyramid slopes, and the 336 actuators of the ASM are not laid out in a regular grid. Slopes are calculated by the instrument software, and the output is treated as the raw WFS signal by CACAO. The use of slopes necessitates several di ff erences in the pre-processing done by CACAO, e.g. no dark or reference subtraction. The DM command is handled as a 1x336 vector. Tools were implemented to enable 2D display of these commands.', '4.3.3 The DAO RTC': 'The Durham Adaptive Optics (DAO) Real-Time Controller (RTC) is designed for the HARMONI and MOSAIC instruments on the E-ELT. 36 It utilizes a shared-memory format originally forked from ImageStreamIO in 2016, following work on the IR pyramid RTC at W.M. Keck Observatory. Until recently, DAO was still based on this fork, but the team has since o ff ered an option to resynchronize the ImageStreamIO component, making DAO fully compatible with other CACAO RTC systems. DAO primarily operates on a CPU but also provides GPU solutions. It o ff ers a suite of tools dedicated to AO and supports a wide range of hardware. Although developed mainly for HARMONI, DAO is now used in other systems at Santa Cruz on SEAL for certain cameras. A collaboration has begun with the INAF Padova Observatory to build their new MCAO bench. A new DAO RTC has been deployed on PAPYRUS at the Observatoire de Haute Provence (OHP) in France. DAO will also be used for the European Solar Telescope and a turbulence profiler for DKIST. Additionally, DAO is used at Durham University for multiple AO projects (WIVERN, NESSIE, MKIDS) and is a potential solution for the VLT RISTRETTO instrument.', '4.3.4 The COSMIC RTC': 'COSMIC is a robust software stack built around the shared-memory paradigm, providing a stable high-level API to its users. COSMIC aims to abstract away all low-level interface and function calls, allowing the user / AOscientist to configure a set of processes in high-level configuration files. This turn-key solution has been proven on the Keck telescopes, and is proposed for a number of other instruments with even higher demand (e.g., HARMONI, MAVIS 33,37 ).  \nCOSMIC is fast and scalable (see, e.g., 31), but historically has been closed-source, introducing a challenge to users wishing to deviate from classical RTC functionality. That said, an open-source release of COSMIC has recently (June, 2024) been announced. Such a release would improve accessibility for a host of research institutions who wish to develop novel RTC techniques for themselves and the community.', '4.3.5 ImageStreamIO in space': "WFS&C for space-based telescopes is fundamentally di ff erent from that for ground-based systems. Unlike ground-based telescopes, space telescopes do not need to contend with real-time atmospheric turbulence corrections. Instead, the aberrations in space-based systems are typically induced by the instrument's optics and mechanical and thermal disturbances from the telescope and spacecraft. Moreover, space-based wavefront control often focuses on optimizing the wavefront to create a dark hole, thereby increasing the contrast necessary for direct imaging of exoplanets. Using shared-memory architectures for WFS&C in space telescopes o ff ers several significant advantages. Firstly, having a control system onboard the spacecraft provides a more cost-e ff ective and relevant means of managing the wavefront. This onboard control system reduces the latency and complexity associated with real-time adjustments from ground-based operations. Additionally, it facilitates seamless collaboration with ground-based telescopes, allowing for the integration of advanced WFS&C techniques developed for terrestrial applications.  \nA prime example of this approach is the Space Coronagraph Optical Bench (SCoOB) 38 which is demonstrating flight ready technologies wavefront sensing and control for potential future missions such as the Coronagraphic Debris and Exoplanet Exploring Pioneer concept 39 and other low-cost exoplanet imaging missions under study. 40 SCoOB is equipped with a high-contrast ( < 1e-8, Van Gorkom et al, these proceedings) coronagraph and utilizes a wavefront control system based on a shared-memory architecture derived from CACAO and MagAO-X ** . This shared-memory framework enhances  \nthe e ffi ciency and flexibility of the control system, allowing for smoother software updates compared to how other spacecrafts are dealing with their control computers. It also supports collaborative e ff orts by enabling the reuse of established algorithms and techniques from ground-based AO systems, including leveraging space-rated GPUs. 41,42  \nThe implementation of shared-memory architectures in space-based WFS&C systems, such as SCoOB, underscores the potential for combining and using WFS&C systems from both space-based and ground-based telescopes, leveraging shared-memory architectures to enhance overall performance and collaboration. This integration can lead to more e ffi cient and advanced technological solutions across di ff erent types of astronomical instruments and environments.", '4.3.6 The HiCAT RTC': "The High-Contrast Imager for Complex Aperture Telescopes (HiCAT) testbed, 43 located at STScI, is a testbed dedicated to system-level technology demonstration of high-contrast imaging on segmented telescopes. With future space telescopes requiring multiple concurrent control loops to stabilize the wavefront, HiCAT transitioned to a shared-memory-based RTC software, called catkit2, 34 running on o ff -the-shelf computing hardware. For hardware-compatibility reasons, catkit2 uses their own shared-memory format inspired by ImageStreamIO. In addition, catkit2 uses a service-oriented architecture to manage all RTC processes and data streams. This process management, and accompanying service discovery, enables increased hardware safety for performing long-running experiments without human intervention. Being written in C ++ and Python, it enables fast prototyping of new wavefront control algorithms 44,45 while maintaining easy integration of new hardware.  \nAlthough developed for the HiCAT testbed, catkit2 is starting to be used at other testbeds. The Tr'es Haute Dynamique 2 (THD2) testbed located at the Observatoire de Paris, is currently transitioning to catkit2 as their new RTC.", '5.1 Introduction to the SEAL testbed': "The Santa Cruz Extreme AO Lab (SEAL) 3 is a testbed in the visible and infrared, designed to advance the state of the art in wavefront control for high contrast imaging on large, segmented, ground-based telescopes.  \nFigure 6 presents the schematic optical and hardware layout of the SEAL testbed. The AO system comprises a pair of deformable mirrors, with one acting as a woofer with 97 actuators from ALPAO DM and the other as a tweeter with 1024 actuators from Boston Micromachines MEMs DM. Additionally, it includes four wavefront sensor arms:  \n- · a high-speed Shack-Hartmann WFS,\n- · a reflective pyramid WFS,\n- · a vector-Zernike WFS 46  \n,  \n47  \n- · a self-coherent camera,  \nThe testbed also features science arms initially equipped with a vector-vortex coronagraph and the self-coherent camera, which serves both as a wavefront sensor and a science camera. SEAL's initial RTC system, named kRTC, is an o ff -shoot of the CACAO RTC, and featured several di ff erences. The intention was to facilitate the seamless exchange of software between SEAL and the Keck II AO system, given kRTC was the implemented RTC on the infrared pyramid WFS.", '5.2 RTC challenges and requirements': "AObenches and instruments on telescopes face di ff erent challenges and requirements. Benches have a more controlled and stable environment, with easy accessibility for adjustments, maintenance and upgrade, while not having the responsibility to perform observations on sky as instruments on telescopes do. That main di ff erence leads to a high flexibility and modularity that makes benches ideal for performing WFS&C development, with modifying, adding and removing hardware components and software upgrades, allowing tests of new technologies and refine algorithms before deploying them on a telescope. Furthermore, AO benches do not have the same real-time needs as instruments do, hence the latency can be very slow. In the context of SEAL, the long-term intention is to implement new WFS&C technologies on the Keck telescope. In 2024, Keck's pyramid wavefront sensor and associated kRTC machine were retired, and the Keck II AO system RTC was upgraded to the COSMIC RTC framework. In the future, a development-only CACAO-based RTC at Keck II would allow for technology transfer from other CACAO-bases systems including SEAL in a way that does not a ff ect the operational COSMIC-based system.  \nFigure 6. Schematic layout of the SEAL testbed.  \n<!-- image -->", '5.3.1 Current status': "In any RTC implementation, the communication with the hardware remains the main challenge, as described in Section 3.4. With kRTC being previously implemented with the SEAL hardware with a di ff erent shared-memory format than the one used by CACAO, one of the major benefits in implementing CACAO on SEAL thus far has been to make the sharedmemory format compatible between kRTC / DAO and CACAO, as briefly described in 4.3.3. Once the shared-memory data structure is implemented on the hardware (WFS and DM) as described in Section 3.4, there are three main steps for calibration and closing the loop:  \n- · accurately measure the hardware latency (HL);\n- · take a response matrix;\n- · compute a control matrix and close the loop.  \nThe HL is the time delay between issuing a DM command and having the corresponding wavefront change visible in the arriving WFS data. HL includes data transfers between the RTC and hardware devices, digital-to-analog conversion of DM commands, mechanical motion of the DM surface, and WFS camera readout. Measuring HL requires real-time performance, as DM commands must be issued at known time o ff sets relative to WFS frames arrival. It is implemented in CACAO, and is a powerful diagnostic of all critical hardware as well as RTC configuration.  \nAccurate knowledge of HL is essential to modern AO systems. Operating an AO loop critically relies on a precise linear mapping between DM commands and WFS signals. Historically, early AO systems measured this response by sequentially poking the DM and waiting su ffi cient time for the WFS to include the corresponding signal, an approach that does not scale to modern high-order AO systems with thousands of actuators. Instead, a rapid fire of DM pokes should be issued,  \nprecisely timed such that the DM moves between ∼ kHz WFS frames, allowing a high precision measurement of the system linear response to many actuators.  \nThe step-by-step guide can be found on the CACAO GitHub page:  \nhttps://github.com/cacao-org/cacao/blob/dev/AOloopControl/examples/scexao-vispyr-bin2-conf/README.md  \nThere have been several challenges along the way making the progress tedious. First, the di ff erence of shared memory format was not initially not well understood, it took several iterations to communicate properly with the hardware, both from hardware to software, and from software to hardware. Then, the speed of the pyramid wavefront sensor camera, running at ∼ 15Hz: CACAO's HL measurement is not well-suited for such slow speed, making it challenging to map the WFS and DM. The Shack-Hartmann WFS has a greater speed and is more suitable in this specific context.", '5.3.2 Control on SEAL': "SEAL o ff ers a woofer-tweeter configuration. To deal with this configuration, one control method used on the testbed uses the framework developed in Calvin et al. (in prep) for simultaneously controlling a segmented primary mirror along side the AO system, which shares the same mathematics as the modal decoupling algorithm first presented by Conan et al. (2007). ? The interaction matrices for both deformable mirrors are calculated independently, and then the command matrix (found as the pseudo inverse of the interaction matrix) of the primary deformable mirror (in our case, the woofer, for its larger amplitude range) is used to subtract the primary DM's controllable space from the auxiliary's interaction matrix. Mathematically, this is seen as IM 2 i = IM 2 -IM 1 × IM 1 -1 × IM 2, where the i represents that the resulting interaction matrix is the 'independent' interaction matrix. With this, the pseudo-inverse of the independent IM can identify the signal from the wavefront sensor that cannot be controlled by the woofer and commands the tweeter to correct that independent signal, while the same WFS output simultaneously uses the woofer to correct the wavefront errors within the woofer's controllable space.", '5.4 Status of RTC upgrade on UC Lick Shane and Automated Planet Finder Telescope': 'The University of California Lick Observatory, located on Mount Hamilton, has two mid-sized telescopes that are suitable for adopting the CACAO RTC package for their current and future AO systems.  \nThe Shane telescope has a 3-meter primary mirror and is equipped with an AO system, ShaneAO, that has gone through several generations of upgrades. 48-50 With the intention of using Shane as a development platform for the Keck Telescopes, there is an on-going e ff ort in upgrading the RTC of the AO system. Given the telescope is being used for science observations, the current work is focused on installing CACAO on an upgraded computer, that works in parallel with the existing instrument.  \nThe Automated Planet Finder (APF) is a 2.4-meter telescope that operates fully robotically to collect precision spectra to discover exoplanets using the radial velocity method. A concept for an adaptive secondary mirror based AO system has been proposed for APF 51 which could more than double the spectrographic throughput. The upgrade would incorporate a TNO 61-actuator large-format convex deformable mirror in the position of the secondary mirror. While the APF adaptive secondary mirror awaits funding, members of the UCSC Lab for Adaptive Optics are building a large-format deformable mirror on-sky testbench that will be located post-focal plane at the open Nasmyth port of the APF. CACAO is currently being developed as the RTC software for this large-format deformable testbench to work in conjunction with the TNO DM3 52 57-actuator flat deformable mirror lab prototype. The software developed as part of these e ff orts at the APF provides a future pathway to use the CACAO RTC package to control the TNO large-format deformable mirrors when being used as adaptive secondary mirrors.', '6.1 Collaborative software practices': 'To ensure the stability and reproductibility of the AO system configuration, it is imperative to maintain a workflow that documents the optimal setup. It is also important to always secure a working and robust configuration, as updates (e.g., via a git pull ) may introduce bugs that could disrupt the system functionality. In the context of collaborative development across various instruments, e ff ective version control is crucial. It is essential to track the specific Git hash (version number) of functional configurations. This practice enables the identification and replication of stable system states. To facilitate  \ncollaboration and maintain system integrity, each collaborator should maintain their own fork of the repository, periodically synchronizing with the development branch. This approach accounts for the diversity in hardware, operating systems, and drivers (e.g., di ff erent NVIDIA driver versions), which necessitates rigorous version control. In summary, diligent version control and documentation of functional configurations are fundamental to successful collaborative development in AO systems, ensuring compatibility and stability across diverse environments.', '6.2 Successful bench and instrument integration': "Integrating the RTC within an instrument and telescope requires three main considerations.  \nFirst, the cyber infrastructure of the telescope has to communicate with the RTC, which requires a multifaceted approach to ensure seamless operation and user e ffi ciency. Firstly, the RTC must be harmoniously integrated with the existing observatory systems, ensuring compatibility and communication between various software and hardware components. This necessitates a comprehensive understanding of the observatory's cyber infrastructure to facilitate smooth data flow and coordination, as each telescope has their own software infrastructure.  \nThen, the user experience is another critical factor. A well-designed graphical user interface (GUI) is essential to provide intuitive and e ffi cient interaction with the RTC system. The GUI should present complex data and control options in a user-friendly manner, reducing the cognitive load on astronomers and operators. This is particularly important given the intricate nature of AO systems and the real-time adjustments required to correct atmospheric distortions.  \nFinally, comparable to what is described in Section 6.1, the successful feature integration of the RTC within the telescopes includes implementing robust version control to track changes and maintain a stable system configuration. Each new configuration involves rigorous testing and validation to ensure that new functionalities work seamlessly with existing components.", '7. RTC CARBON FOOTPRINT': "The carbon footprint of RTC systems and HPC in general is a growing concern as these systems require significant computational power and energy consumption. 53 Here's a closer look at the factors that contribute to the carbon footprint of RTC:  \n- · Energy consumption: whether it is from the computational power or the servers cooling systems, RTCs lead to high energy consumption.\n- · Data management: an average night on the SCExAO instrument at the Subaru telescope corresponds to ∼ 10 Tbs of data. Storing this data is not only energy-intensive, but also not sustainable on the long run.\n- · Hardware: the production of the hardware used in RTC systems, including servers, GPUs, and specialized control devices, has a significant carbon footprint. The disposal and recycling of hardware at the end of its life cycle also have environmental impacts. Sustainable practices in hardware lifecycle management can reduce the overall carbon footprint.  \nMitigating this impact and reaching carbon neutrality has been a field of research going further than the field of AO for astronomy. First, the e ffi ciency of the algorithms used in RTC systems can significantly impact energy consumption. More e ffi cient algorithms require less computational power and can reduce the overall energy demand. Optimizing the code for energy e ffi ciency, such as reducing unnecessary computations and improving data handling, can also contribute to lowering the carbon footprint. 18 Furthermore, integrating renewable energy and improving cooling technologies within observatories would play a key role. Along these lines, the use of shared-memories can play a role with respect to the data management: solving the PSF reconstruction challenge would allow for considerably reducing the amount of data being stored.  \nFurther work is necessary to accurately assess the carbon footprint of RTC, and how to mitigate that to reach carbon neutrality.", '8. CONCLUSION': "This proceeding covers an introduction to the realm of RTC for users and WFS&C developers, and present a form of step-by-step guide on how to install and interact with a RTC. Along with this intention, the purpose is equally to encourage the RTC community to adopt common software standard and practices for increased collaboration. Collaborative development across various instruments and telescopes benefits significantly from standardized protocols and shared-memory architectures, which allow for more straightforward adaptation and integration of new features. Indeed, gathering several contributing teams and a diversity of experiences will promote and fast-track the development of RTCs, enabling passing the tipping point for a self-sustaining community of astronomers and instrument scientists.  \nFinally, successful RTC integration within an AO system's cyber infrastructure involves harmonizing with observatory systems, enhancing user experience through an e ff ective GUI, and ensuring reliable feature integration through meticulous calibration, testing and version control. These practices collectively contribute to the e ffi cient and e ff ective operation of high-contrast imaging systems for exoplanet detection and characterization.", 'ACKNOWLEDGMENTS': 'The development of SCExAO is supported by the Japan Society for the Promotion of Science (Grant-in-Aid for Research numbers 23340051, 26220704, 23103002, 19H00703, 19H00695 and 21H04998), the Subaru Telescope, the National Astronomical Observatory of Japan, the Astrobiology Center of the National Institutes of Natural Sciences, Japan, the Mt Cuba Foundation and the Heising-Simons Foundation. The authors wish to recognize and acknowledge the very significant cultural role and reverence that the summit of Maunakea has always had within the indigenous Hawaiian community, and are most fortunate to have the opportunity to conduct observations from this mountain.', 'REFERENCES': "- [1] Guyon, O., Sevin, A., Ferreira, F., et al. , 'Adaptive optics real-time control with the compute and control for adaptive optics (Cacao) software framework,' in [ Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series ], Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 11448 , 114482N (2020).\n- [2] Lozi, J., Guyon, O., Jovanovic, N., et al. , 'SCExAO, an instrument with a dual purpose: perform cutting-edge science and develop new technologies,' in [ Proc. SPIE. ], Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 10703 , 1070359 (2018).\n- [3] Jensen-Clem, R., Dillon, D., Gerard, B., et al. , 'The Santa Cruz Extreme AO Lab (SEAL): design and first light,' in [ Techniques and Instrumentation for Detection of Exoplanets X ], Shaklan, S. B. & Ruane, G. J., eds., Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 11823 , 118231D (2021).\n- [4] Demerle, M., Madec, P. Y., & Rousset, G., 'Servo-Loop Analysis for Adaptive Optics,' in [ Adaptive Optics for Astronomy ], Alloin, D. M. & Mariotti, J. M., eds., NATO Advanced Study Institute (ASI) Series C 423 , 73 (1994).\n- [5] Kulcs'ar, C., Raynaud, H.-F., Petit, C., Conan, J.-M., & Viaris de Lesegno, P., 'Optimal control, observers and integrators in adaptive optics,' Optics Express 14 , 7464 (2006).\n- [6] Guyon, O. & Males, J., 'Adaptive Optics Predictive Control with Empirical Orthogonal Functions (EOFs),' arXiv e-prints , arXiv:1707.00570 (2017).\n- [7] van Kooten, M. A. M., Jensen-Clem, R., Cetre, S., et al. , 'Predictive wavefront control on Keck II adaptive optics bench: on-sky coronagraphic results,' Journal of Astronomical Telescopes, Instruments, and Systems 8 , 029006 (2022).\n- [8] Fowler, J., Van Kooten, M. A. M., & Jensen-Clem, R., 'Battle of the predictive wavefront controls: comparing data and model-driven predictive control for high contrast imaging,' in [ Adaptive Optics Systems VIII ], Schreiber, L., Schmidt, D., & Vernet, E., eds., Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 12185 , 1218582 (2022).\n- [9] Nousiainen, J., Rajani, C., Kasper, M., et al. , 'Toward on-sky adaptive optics control using reinforcement learning. Model-based policy optimization for adaptive optics,' Astronomy & Astrophysics 664 , A71 (2022).\n- [10] Recht, B., 'A Tour of Reinforcement Learning: The View from Continuous Control,' arXiv e-prints , arXiv:1806.09460 (2018).\n- [11] Ferreira, F., Gendron, E., Rousset, G., & Gratadour, D., 'Numerical estimation of wavefront error breakdown in adaptive optics,' Astronomy & Astrophysics 616 , A102 (2018).  \n- [49] Kupke, R., Gavel, D., Roskosi, C., et al. , 'ShaneAO: an enhanced adaptive optics and IR imaging system for the Lick Observatory 3-meter telescope,' in [ Adaptive Optics Systems III ], Ellerbroek, B. L., Marchetti, E., & V'eran, J.-P., eds., Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 8447 , 84473G (2012).\n- [50] Gavel, D. T., Kupke, R., Rudy, A. R., et al. , 'Lick Observatory's Shane telescope adaptive optics system (ShaneAO): research directions and progress,' in [ Adaptive Optics Systems V ], Marchetti, E., Close, L. M., & V'eran, J.-P., eds., Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 9909 , 99092W (2016).\n- [51] Bowens-Rubin, R., Bos, A., Hinz, P., Holden, B., & Radovan, M., 'An adaptive optics upgrade for the Automated Planet Finder telescope using an adaptive secondary mirror,' in [ Adaptive Optics Systems VIII ], Schreiber, L., Schmidt, D., & Vernet, E., eds., Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 12185 , 121851X (2022).\n- [52] Bowens-Rubin, R., Hinz, P., Jonker, W., et al. , 'Performance of large-format deformable mirrors constructed with TNO variable reluctance actuators,' in [ Adaptive Optics Systems VII ], Schreiber, L., Schmidt, D., & Vernet, E., eds., Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 11448 , 114485V (2020).\n- [53] Portegies Zwart, S., 'The ecological impact of high-performance computing in astrophysics,' Nature Astronomy 4 , 819-822 (2020)."}