Let’s Architect! Leveraging in-memory databases
AWS Architecture BlogLet’s Architect! Leveraging in-memory databasesby Luca Mezzalira, Federica Ciuffo, Vittorio Denti, and Zamira Jaupaj | on13 SEP 2023| inAmazon ElastiCache,Amazon MemoryDB for Redis,Architecture,Database,Thought Leadership|Permalink|Comments|ShareIn-memory databases play a critical role in modern computing, particularly in reducing the strain on existing resources, scaling workloads efficiently, and minimizing the cost of infrastructure. The advanced performance capabilities of in-memory databases make them vital for demanding applications characterized by voluminous data, real-time analytics, and rapid response requirements.In this edition ofLet’s Architect!, we are introducing caching strategies and, further, examining case studies that useAmazon Web Services(AWS), likeAmazon ElastiCacheorAmazon MemoryDB for Redis, in real workloads where customers share the reasoning behind their approaches. It is very important understanding the context for leveraging a specific solution or pattern, and many common questions can be answered with these resources.Caching challenges and strategiesMany services built at Amazon rely on caching systems in the background to speed up performance, deal with low latency requirements, and avoid overloading on source databases and other microservices. Operating caches and adding caches into our systems may present complex challenges in terms of monitoring, data consistency, and load on the other components of the system. Indeed, a cache can give big benefits, but it’s also a new component to run and keep healthy. Furthermore, engineers may need to use empirical methods to choose the cache size, expiration policy, and eviction policy: we always have to perform tests and use the metrics to tune the setup.With thisAmazon Builder’s Libraryresource, you can learn strategies for using caching in your architecture and best practices directly from Amazon’s engineers.Take me to this Amazon Builder’s Library article!Strategies applied in Amazon applications at scale, explained and contextualized by Amazon engineersHow Yahoo cost optimizes their in-memory workloads with AWSDiscover how Yahoo effectively leverages the power of Amazon ElastiCache and data tiering to process an astounding 1.3 million advertising data events per second, all while generating savings of up to 50% on their overall bill.Data tiering is an ingenious method to scale up to hundreds of terabytes of capacity by intelligently managing data. It achieves this by automatically shifting the least-recently accessed data between RAM and high-performance SSDs.In this video, you will gain insights into how data tiering operates and how you can unlock ultra-fast speeds and seamless scalability for your workloads in a cost-efficient manner. Furthermore, you can also learn how it’s implemented under the hood.Take me to this re:Invent 2022 video!A snapshot of how Yahoo architecture leverages Amazon ElastiCacheUse MemoryDB to build real-time applications for performance and durabilityMemoryDB is a robust, durable database marked by microsecond reads, low single-digit millisecond writes, scalability, and fortified enterprise security. It guarantees an impressive 99.99% availability, coupled with instantaneous recovery without any data loss.In this session, we explore multiple use cases across sectors, such as Financial Services, Retail, and Media & Entertainment, like payment processing, message brokering, and durable session store applications. Moreover, through a practical demonstration, you can learn how to utilize MemoryDB to establish a microservices message broker for a Media & Entertainment application.Take me to this AWS Online Tech Talks video!A sample use case for retail applicationSamsung SmartThings powers home automation with Amazon MemoryDBMemoryDB offers the kind of ultra-fast performance that only an in-memory database can deliver, curtailing latency to microseconds and processing 160+ million requests per second —without data loss. In this re:Invent 2022 session, you will understand why Samsung SmartThings selected MemoryDB as the engine to power the next generation of their IoT device connectivity platform, one that processes millions of events every day.You can also discover the intricate design of MemoryDB and how it ensures data durability without compromising the performance of in-memory operations, thanks to the utilization of a multi-AZ transactional log. This session is an enlightening deep-dive into durable, in-memory data operations.Take me to this re:Invent 2022 video!The architecture leveraged by Samsung SmartThings using Amazon MemoryDB for RedisAmazon ElastiCache: In-memory datastore fundamentals, use cases and examplesIn this edition of AWS Online Tech Talks, explore Amazon ElastiCache, a managed service that facilitates the seamless setup, operation, and scaling of widely used, open-source–compatible, in-memory datastores in the cloud environment. This service positions you to develop data-intensive applications or enhance the performance of your existing databases through high-throughput, low-latency, in-memory datastores. Learn how it is leveraged for caching, session stores, gaming, geospatial services, real-time analytics, and queuing functionalities.This course can help cultivate a deeper understanding of Amazon ElastiCache, and how it can be used to accelerate your data processing while maintaining robustness and reliability.Take me to this AWS Online Tech Talks course!A free training course to increase your skills and leverage better in-memory databasesSee you next time!Thanks for joining us to discuss in-memory databases! In 2 weeks, we’ll talk about SQL databases.To find all the blogs from this series, visit theLet’s Architect!list of content on theAWS Architecture Blog.TAGS:cost optimization,database,Let's Architect,re:Invent,RedisLuca MezzaliraLuca is Principal Solutions Architect based in London. He has authored several books and is an international speaker. He lent his expertise predominantly in the solution architecture field. Luca has gained accolades for revolutionizing the scalability of front-end architectures with micro-frontends, from increasing the efficiency of workflows, to delivering quality in products.Federica CiuffoFederica is a Solutions Architect at Amazon Web Services. She is specialized in container services and is passionate about building infrastructure with code. Outside of the office, she enjoys reading, drawing, and spending time with her friends, preferably in restaurants trying out new dishes from different cuisines.Vittorio DentiVittorio Denti is a Machine Learning Engineer at Amazon based in London. After completing his M.Sc. in Computer Science and Engineering at Politecnico di Milano (Milan) and the KTH Royal Institute of Technology (Stockholm), he joined AWS. Vittorio has a background in distributed systems and machine learning. He's especially passionate about software engineering and the latest innovations in machine learning science.Zamira JaupajZamira is an Enterprise Solutions Architect based in the Netherlands. She is highly passionate IT professional with over 10 years of multi-national experience in designing and implementing critical and complex solutions with containers, serverless, and data analytics for small and enterprise companies.CommentsView Comments
In-memory databases play a critical role in modern computing, particularly in reducing the strain on existing resources, scaling workloads efficiently, and minimizing the cost of infrastructure. The advanced performance capabilities of in-memory databases make them vital for demanding applications characterized by voluminous data, real-time analytics, and rapid response requirements. In this edition of Let’s Architect!, […]
Luca Mezzalira
Amazon ElastiCache Amazon MemoryDB for Redis Architecture Database Thought Leadership