Get more than a cache back!
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Balliauw, Maarten
The Microsoft Azure Redis Cache must be the fourth or fifth generation of a hosted cache service on Azure. This time, an open-source solution has been embraced: Redis. In this session, we’ll see that it’s more than just an in-memory cache system we can use in our applications. Let’s explore what Redis is, what the different data types are and why we should care. And once we grasp how Redis stores its stuff, we’ll delve into how we can use it to its fullest extent: searching the key-value store, transactions, pub/sub support and scripting.
 All right, let's wait for the screen to clear and there we are. Welcome this morning in this session on get more than a cash back on the Redis cash on Azure. Now, before we start, a quick word of warning, this talk is going to be mostly about Redis, a little bit about Azure. And the reason for that is that the Redis cash on Azure is actually just the Redis cash. So if you want to use Redis and you want to start with Redis, you can simply download it and start working with it. So the goal of this talk is actually to make you more familiar with Redis. To see all the things that are in there, to see that it's more than just storing a value, retrieving a value like you would use a cash normally. And then if you're interested, you can always start using it on Azure. But it's basically the same. So whatever reason you're here, if it's for Azure, perfect. If it's for Redis, even better. It's just going to be about Redis mostly. Let's quickly introduce myself. I'm Martin Balliao. I flew in from Antwerp in Belgium. I'm a software engineer at Microsoft and also work on my gets in my evening hours. I'm the co-founder of the Azure user group in Belgium. So I've been doing a lot of Azure in the past and I continue on doing that. And if you want to follow me on Twitter or read my blog on all these topics, you can find them there. Oh, and if you want to buy me a beer, especially here in Norway, buy my book by twice or at least with the beer prices here by three times, I would really appreciate that. Thank you. All right. So the agenda for today, like I said, is going to be mostly about Redis, but we'll start a little bit with the Azure cash story, the things that happened there, which caches were available, which caches are still available, and then slowly transition into looking at Redis and how we can use Redis, what the data types are, see there are transactions, there's PubSub and things like that. And I have a lot of demos. In fact, if you look at my desktop here and you look at the amount of visual studios I have open, I have quite a few of those and I will probably not be able to tackle all of these in this one hour we have here. So once we end up with the patterns, we'll do a quick photo of hands on which patterns you actually want to see because there's a lot of interesting things you can do with Redis. Like for example, doing the session voting buttons here would be a nice example of using Redis in one of those patterns. Right. So the Azure cash. Caching on Azure is a bit of an interesting story. So back in 2009, Microsoft released the Windows Server app fabric, which was on Windows Server. If you've been using this, you may remember the code name Velocity. That was actually a cache you could use on your Windows Server with your ASP.NET applications with whatever you wanted to store in there. And then Azure came about and Microsoft thought, okay, we need a cache that people can leverage in their cloud solutions. So let's make a port of this Windows Server app fabric and put it on Azure as well. And they did that and they came up with a shared cache, which is an interesting thing as a vendor to just provide a service that a lot of people can use and a lot of people can share because it's very cost effective and so on. It had one little issue though. The fact that it was shared and the fact that Microsoft wanted to guarantee people got some throughput and people got their data back from the cache. The reason that was done is that they also introduced Trotling on the cache, which is an insane concept if you think about it for a cache because when you need to cache, why would you use a cache to offload your database server, to offload your storage, to offload whatever you want to store in your cache. But if you have Trotling on your cache, that's an insane thing to have because your application would crash once you have loads. So the shared cache was not really popular for that reason, but again, it was still based on Windows Server app fabric. Good thing was that Microsoft noticed that this Trotling idea was probably not too good. So there came a self-hosted dedicated cache where you could basically just deploy your cloud services on Azure and install a NuGet package or install an extension there and the cache would be co-located with your web server or your worker role that you would deploy. Interesting solution and that's still out there today. You can actually still use that. But it's not ideal because you're sharing the memory on your web server and if you have a lot of traffic, your web server may need that memory to serve actual connections and clients. So there was a different option as well, which is the managed cache, which is basically the same thing, except this time instead of having a shared cache, Microsoft would just provision a VM for you or a cluster of VMs and then have the cache running in there. Quite popular, a lot of people actually use it, but yeah, there's something else. People started to develop their applications differently. So one of the things that happened is that a lot of different technologies came to Azure. So for example, PHP, Java, Node.js and so on. So those people also wanted to use caching in their solutions. And on Azure, Microsoft thought, okay, so now we will have to write clients for pretty much any language that's there. Interesting thing and a lot of work. So Microsoft thought, okay, let's look at a different solution. Let's look at some server that already exists. Let's look at an ecosystem where there's clients for virtually any language and just make it run on Azure and provide that thing on Azure. And that's what Microsoft did and the reason why Redis is now running on Azure. If you go to the Redis website and you look at the clients for all the various languages, I wouldn't be surprised if there's an actual cobalt Redis client or something like that. So pretty much any language you can run on Azure or pretty much any language you want to run on your VMs, you can work with Redis from that. So that's the reason why Microsoft decided, okay, let's go with Redis. Let's make it run on Windows so we can actually host it and make it available. And the clients are already there. We don't have to maintain them. There's alternatives in open source. Like for every language, there's like five or six clients. So you can basically pick the best one for your goal. Another reason is that the f-fabric cache was actually just a cache. And people started to think differently about how they developed their applications. If you look at Twitter, for example, they basically have the MySQL databases in the back where they store your tweets and user data and so on. But virtually anything you access on the Twitter website, on the Twitter apps on your mobile phone, for example, is stored in Redis, in memory. So people actually start using caches as their main persistent store or their main go-to database to store their application data while it's running. Of course, there's always still databases in the back to persist it. And in case the cache goes down, you can still reach out to the database and repopulate everything. But people started to actually use the cache as the main data source. And that's a perfect thing to do, especially with Redis, because Redis has different options, as we will see, to store that data and make very effective use of that. To give that Twitter example a little bit more context, if you would build Twitter, you could do like if you want to show the timeline of people. I take my phone and I want to watch the stream of tweets that I have there. You could probably map it relationally and have like one tweet, mapping to the users that you're following and so on. But that's in the relational world. What Twitter actually does is they say, memory is cheap, memory is really fast, let's store everything in Redis. There's this interesting website called highscalability.com. There's really interesting articles there from Twitter from like all the big cloud guys. And apparently Twitter is actually storing your timeline in duplicates format in memory. So if you access your timeline on Twitter, you actually get your version of the cache. So for every tweet I send, all my followers will get my tweet in their cache, which is an insane amount of data. But just to prove the example that people start to think about their applications differently and start to store all that data in one store. So that's two good reasons to actually move to a different caching system, which is Redis. So let's talk about Redis for a while. Redis is a very simple name. It's a remote dictionary server. That's a very interesting concept to remember. This is the actual, yeah, what are we that you can find on their websites, which is kind of complicated, but there's a couple of things we want to remember from Redis. One thing is it's a remote dictionary server, which means it's a dictionary and it's a server. And in a dictionary, you typically store key value pairs. You store a value and you can access that value by asking for a specific key. So it's basically like in.NET where you could have a string dictionary or a dictionary of objects or whatever. You could do the same in Redis. Redis is actually nothing more than that. It's a dictionary that you can access via TCP. Another interesting thing is that Redis is an in-memory remote dictionary server. There's no persistence. So if you want to use Redis as your main data store, you probably want to cluster it or set it up in a way that there's always at least one copy of your data still available because there's no persistence out of the box. Well, there is persistence out of the box and you can actually store your cache data to disk to repopulate the data if your server crashes or has to reboot, for example. But you have to keep in mind that Redis is memory based. All your data has to fit in memory and, yeah, you don't want to actually use that persistence because it could slow down your caching. It's single-treaded, which is a good thing and probably also a batting. But the good thing about being single-treaded as Redis is that there's always one client being served, which also means that you can do interesting things like transaction style work. So for example, if you want to store a value and you want to increment that value in Redis, you know that only one concurrent user will actually be increasing that value at the same time, which means your count, for example, will be correct. It will be the actual count of increments. All your different users have been doing because it's a single-tread and it's just making sure that there's only one person or one client actually accessing that data. It's networked, so you cannot embed it in your application. It's like a standalone server you have to run and connect to. And it's basically exploiting the fact that it's a dictionary because dictionaries are very fast to work with. It's very fast to search for things by ID. So the author of Redis actually thought of some interesting things to do with those dictionaries, of which there's running scripts, for example. There's a PubSub mechanism. You can do snapshotting of data and so on. So there's just much more to this simple key value store than you would think. So I talked about persistence. Is there no persistence? Well, you can actually persist to disk if you want to. There's two ways of doing that. There's append-only files and there's Redis database files. You can configure Redis to store your cache data to disk. Perfectly possible if you want to run Redis on your own, if you want to run your own Redis cluster. It probably makes sense to do this because if your cache goes down or the server has to reboot, it can actually repopulate the entire memory database from disk. Whether you want that or not depends probably a little bit on how volatile your data is that you're storing. Like if the server has to reboot for 10 minutes, maybe you don't just want to restore the data because the data is still, but still you can do it. One thing you must keep in mind though is that everything must fit in memory. So if you want to store one terabyte of data in Redis, you will have to have one terabyte of memory. Even if you're using persistence and maybe even if you're having all the disk space in the world, it has to fit in memory. And that's a very important notion to have because you cannot just keep storing stuff in Redis. Eventually memory will run out and you'll have to think about having multiple Redis servers and do some partitioning and so on. All right. So let's give you a quick Redis 101 just to give you a feel of what Redis actually is. So I have a Redis download here on my system. And as you can see, if you download the Windows version, there's a couple of files in there. There's Redis benchmark. There's the Redis client and there's the Redis server. I'm actually going to run the Redis server in a bit. So I actually have a locally running Redis server and everything on this Wi-Fi can probably connect to it. There's no authentication. There's no security. Anyone can just connect to that thing. Reason for that is Redis wants to be very light. Redis doesn't want to have the additional overhead of doing authentication and so on. But also keep that in mind if you want to deploy Redis manually, there's no real security on Redis. So you're basically opening up your cache to the world if you run it on a public IP, for example. Some interesting things in there are that this Redis benchmark file is in there as well. It's actually an executable that you can use to run against your Redis data store and you can give it a lot of parameters like how random do you want the data to be, how many cache hits and misses do you want to generate. But you can also specify the object size of the things that you want to test and benchmark Redis with. And that's a very interesting thing because Redis is perfect for a lot of situations but it's also a bad thing to use for some situations as we will see later. So whenever you want to start with Redis and you know pretty much what type of data you want to store in Redis, run the benchmark, see how your server is performing and see if Redis is actually the right choice at that moment. Very useful tool for that. So let's run Redis and I have a nice batch file there which basically uses a default configuration and starts Redis. As you can see, one of the main advantages of Redis is there's ASCII art involved. But we now have a Redis server running. So you can connect to this one and I will actually connect to that one as well using the Redis Clients and I made it a little bit bigger. I decided not to do this with this thing because nothing interesting happens in this window except an occasional error every now and then. So let's minimize that one. So let's use the Redis Clients. By default it will just connect to the server you're running on your local host so you don't have to configure it and so on. It will just connect. Of course you can also, I mean PowerShell of course. Let's try that one more time. There we are. That seems to work. Okay, so my client is now connected to my local server. If you want to connect to a remote Redis server or Azure for example, you can also use this tool for that. So let's store something and let's store my name and let's say I'm storing Martin. Very simple, I stored something in the cache. I can get that name and I will retrieve that name from cache which is very useful and very interesting and this is what you expect the cache would do. Now there's a couple of more interesting things we can do with Redis. Like I said, it's a simple dictionary so an operation like this where you store something and retrieve something is very easy but Redis has some additional functions that you can use to for example get a range from a specific key. So from my name I can get a substring from character zero to character four and get a part of that thing. Why is that interesting? Well probably not very interesting in this example but imagine you are storing a lot of data and you know pretty much what the chunk size of that data is and you know you want to retrieve a specific part of that data. Why would you block this single threaded server with sending you back like a megabyte of information if you could just ask the server to give you back just that piece of data you're interested in from that specific key in the dictionary. So a very interesting function, not really in this case but you can use it and leverage it to minimize bandwidth first of all and minimize the time Redis has to spend on blocking you because it's single threaded and serving your data back. So use those functions. All right, another thing you would expect from a cache is that you can expire keys. So let's do that. Let's say my name should expire in five seconds and let's do a get name. It's still there, still there, still there and now it's gone. So you can expire your keys as well which is interesting. You can just use it as a basic cache. So if you're just interested in storing something and retrieving something, thank you for being here. You may now leave. That's the simple cache of Redis. Now let's dive into some more interesting things. One thing I still want to show you here is that there's the info commands which gives you back the configuration first of all but also the number of cache hits, the cache misses and so on. So you can actually use this info commands to get some details on how your server is operating to see if you're hitting the master or the slave for example if you connect to Azure and so on. So remember the info commands as well. All right, if you want to start with Redis, if you leave this talk and you want to really start with Redis, remember this URL. I have it again at the end of the talk and I will probably tweet it again as well. Just search for the little Redis book. It's actually a very little book. It's like 40 pages and it contains pretty much anything you want to know about Redis. It gives you a lot of examples, a lot of use cases and scenarios and so on. So if you're really keen on getting started with this, remember this book. It's a really good thing. All right, so I told you Redis is much more than a cache and much more than something where you can store a simple value and retrieve a simple value. So let's go over some of the data types that are available. We've already seen the string data type which is basically just get and sets. It's the string data type because internally Redis stores it as a stream, a byte string or some bytes but you can actually store a lot of things in there. If you want to store integers, you can just store an integer, increment the integer and so on. If you want to store a byte array or a JSON document or HTML for output caching, for example, you can do that. It's a very simple, it's the basic premise of Redis where you have keys and values that you want to store. More interesting is the hash data type. The hash data type is like you would expect, it's a hash of fields and values. So if you want to store, for example, in a web shop, if you want to store a category that you want to retrieve, like for example, books and the amount of books that are in this specific category and so on, you can actually use the hash for that. And why is that interesting? Well, because I can ask Redis to retrieve the entire hash to retrieve all the fields and all the values that are in there, but I can also ask Redis to give me just the name of this specific hash. Again, to make the communication with the cache very fast and to make it very focused on just the data you need to retrieve from Redis. So if I have a page where I only want to show the category name or the number of products, I can just tell Redis, give me just the name from this hash. There's also sets. It's basically like you would have arrays of integers in.NET. It's basically just that. So you can have a set, you can give it a key, of course, to retrieve the sets and you can push some values into that thing. So for example, if you're building, again, an e-commerce website, for example, you could use this set to store product IDs for a specific category. So if you're a category one is the books, you can store the product IDs in a set. Why is that useful? Why just store those product IDs? Well, because Redis can actually quickly retrieve the IDs for a specific category. You can also do things like intersecting data. So if you want to see which books are in thrillers and in nonfiction, it could be a book on JavaScript, for example. You could actually intersect the categories there and look at the different books that are available in both categories or the books that are available in not that category but that are in that category and so on. You can have lists. Lists is pretty much similar to the sets except lists can contain anything. So not just numbers. You can push strings in there. You can push objects in there and so on. So if you would think about building a common system, for example, for your e-commerce websites, the comments could be stored in a list and you can tell Redis to give me the first five items of the list or give me the last item of the list and so on. So you can, again, get very focused data and very fast retrieval from Redis. There are sorted sets as well and that's sort of a weird thing in terms of what it stores because it says it has fields and scores. It's actually just you store a value and you can give it a number and that number is used to do scoring on that sorted sets. It will actually sort the entire thing by the score that you have there and it does things like, for example, give me the top five products or give me the top five whatever is in there. So imagine you are building a Twitter clone, for example. You could store in this sorted set the different hashtags that people are using and whenever a hashtag is being used, you could increment the score and that makes it very easy to do things like what is trending in this specific country because you just ask Redis, give me the top ten hashtags that are being used currently in the system and you will get that information. There's a couple more data types. There's the bitmap which is basically just a stream of Booleans where you can do things like, is this user logged in or not and just store one bit basically. There's also the hyperlog log which is a very specialized data type. Read the little Redis book if you really need those but make sure you know these things because they are really important to build stuff on Redis and make it fast and scalable and so on. So let's go over a bit of the commands to get you a little bit more familiar with what these data types can all do. Let's go back to our PowerShell. Let's clear this thing. So we had the strings where we could store data. I stored my name, I expired my name, I got a range from this name and so on. I told you you can also store integers. So let's say we have an attendee count and let's say it's 2-0. What I can do is say increment attendees and it will basically increment my integer. You always get back the current value of this integer and am I now crashing my Redis server? It almost looks like it. There we go. So my integer has been increased and remember because it's single-threaded a lot of clients can actually increment this integer and you will never have conflicts. It will be the actual value of this integer. Just locking because it's only single-threads. I can increment by a number as well. So if I want to say my attendee count goes up with 200, I could do that as well. So it's not just by one. You can do whatever you want with that. So that's a number of the things you can do there and it's really useful to have. All right. Let's go over the hashes and let's build an e-commerce web store and we want to store our product categories in Redis. So let's do that. Let's do a hash set for categories one. I'll shorten this to C1 just to have a little typing and let's say the name of that thing is books, description, good books, number of products, five. And what did I miss? I didn't miss a thing. One, two, three, four, five. Okay. I obviously gave one parameter too much because I didn't use a proper string there. So let's make that good underscore books or you could use quotes as well to store this thing. So this has been stored and I can get all the values from this specific category and just say give me all those things and you will get back a list of properties and their values. More interesting thing is that I can also get just one thing and I could say get me from category one just the name and I'll just get the name property of that thing. Again, why would you do this? Just to make things very fast, to just get the information you actually need in your application. I could also do things like increments. There's an h increment by so if the number of products goes up, I can just up the number of the products without retrieving everything and then restoring everything. But that's basically the things you can do with the hashes. All right. Let's work with a set. So if I say in category one, we have some products, bear with me, C1 products. I'm going to store some products and let's say product 20s in there, product 30s in there, 99, 80, 70 and so on. I've done that. I stored the product IDs for this specific category in my bookstore in a set. I can do increments. I could do things like adding or just getting the top value from this list, but I could also do things like create a second category which has products 20 and 11. Let's say 30, 99, 70 and 80. So this thing now has four items. Again, very simple array and I can put integers in there. But the interesting thing comes when I do set intersects and so on. So I can do an intersect between my first category, the products in there and my second category, the products in there. And I will get back the common items in those two sets. I could also do a diff between both and I could say show me all the items that are in category one, but not in category two. Obviously, you have to type it correctly. I will get back these two numbers. So I can easily do intersects and so on. Why would you do this? Can my application do this? Well, again, just retrieve the data you need and ask Redis to do this. Also, since Redis uses memory to store its data and if you have a lot of memory, you probably also have a quite good CPU in there. So why not just upload these simple operations or offload these simple operations to the Redis cache in there? Alright, let's go with the list. I can push items in my list and say for my product one, I have some comments and one of those comments is good product. I can push some more in there. Let's say awesome stuff. And I now have a couple of items on this list. I can get all those things. I can get the number of items in there. So I want to get the number of items I can get that. I should be able to get that thing. Okay, yeah, give me item one, give me item two and so on. So I can get all those things separately from this index from this list. Other things I can do is ask for the length of this list. But a more interesting thing is that I can also do pops from this list. So just like you can do with an area, you can shift and pop items from that list. I could do for example a pop on my product one comments and say I just want one item. I'll get the item. I'll get the second item. I'll get the third item. They're all removed from this list. But an interesting thing is if I reach the end of the list and I still pop items from this list, these will just put me in a waiting state until new items are in that list. So if I have a second client and I would do a R push product one comments, great. It would eventually show up in my first client and I can just retrieve items from that list while they are being added. I can do for each is over that and just wait until I reach the end of the list and even wait until new data arrives in there. And then there's the sorted set. And the sorted set is a very interesting beast because it sort of looks funky where you just store like the best sellers and say the score is 100 for book. Let's say the score is one to three for car. From my set, I can then do things like scoring and say give me from my best sellers for car. What's the score of that thing? It's not that. It's Z score. And I think that's because I started with a string. Anyway, you can score the items in there. It's really interesting to do things like sorting, calculating top lists and so on. So I hope you see that by storing your data in an efficient way in the type of data that you're actually storing, not just the raw object and get back the objects, you can actually make Redis very, very fast by just retrieving the items you want to retrieve from Redis. So let's go over them one more time just to make sure you know these things. Maybe a little bit boring, maybe not what you expect from this talk, but I think it's really important to know all these data types by heart. So instead of a store, let's build Twitter this time. I could use strings for storing things like output caching. I could use strings for storing just the raw HTML of my pages and serve that to my clients. But I could also use strings to store numbers. And what I could do is, for example, when a user registers on my website, I could store the user ID in Redis and have Redis increment that user ID every time. Very safe because Redis is thread safe in this regard. It will always give me the new ID that I can use for my user. In the hashes, I could store the data for my user. So for example, in this case, I stored myself and my Twitter handle in a hash. I have my name in there. There's my Twitter handle in there. There could be my date of birth. My password could be in there. So basically, all the different properties of my current user can be in a hash. Very interesting because I can retrieve this user by ID. I can just say, Redis, give me back the entire list of properties and their values for user one. But I could also tell Redis to just give me the Twitter handle, for example, if I just want to render a page that doesn't have all these additional properties. Very fast to retrieve and very interesting to use this approach to just get the things you need while you're still storing a lot of data about this specific user. I could use sets to do things like who's following who. I could use a set to store the followers that I have on Twitter so that Twitter has, or our Twitter clone actually has a list of, or a set of data of the user that I'm following so it knows where to store the data, where to write the data, who to send notifications to and things like that. It's also interesting because I can do these things like intersecting and so on. So if I want to boost my revenue and make sure users come, keep coming back and users discover other users to follow and so on, what I could do is simply for my current user, loop over the sets of all the users that are following me and do suggestions based on intersections and differences between those sets. Like for example, you should follow this user because somebody you are following follows this user and it's probably interesting for you. So you can use sets to do those types of operations. There's lists of course, could be your timeline, could be any user's timeline, just whenever I tweet, push something in that list and it will show up in the application. And then we can use sorted sets to do things like which hashtags are trending in a specific country and so on. So for example, for no way I could say NDC Oslo has this specific score or this specific number of mentions and I can do very simple things like give me the top five hashtags that are happening today for example. All right, data types. You may have noticed from both the examples that I was typing as well as the tables I had previously that I have used a pattern for storing my different keys. There's a good reason I did that. It's actually one I have a slight form of OCD where I want to know what a specific thing is so I make sure that I name things correctly so I know this cache key will hold a user and so on. Another interesting thing is that by doing things like storing users one Martin, I could find users by ID or I could find users by name by simply asking Redis to query for the keys that look similar like users colon one colon asterisk. I don't know the name up front but I still want to get that user. I can ask Redis to give me back that information. One word of warning though, if you have a lot of little or big objects in your Redis database, if you have a lot of objects basically, you probably don't want to run these keys commands all the time because the keys command actually loops the entire store in Redis. If you have a lot of keys this may actually take a while to fetch. It's interesting to be able to do this, to be able to maybe store parts of your data like this and get the correct data by just querying for it based on what the key is. All right, so I've been doing a lot of Redis so far. What about.NET? How can I use Redis from.NET? Well, there's actually a lot of options. Like I started, I said pretty much any language has at least five clients for Redis. It's the same thing with C-Sharp. There's I think five or six right now. So if you want to work with that thing, you can just install a NuGet package and start working with NuGet. There's a couple for.NET. I personally very much like the Stack Exchange one. Why? It's a very simple one to use. All the different operations in there actually map to the different data types and so on. So it's really easy to know what data types you're working with. And of course it powers Stack Exchange, one of the very big sites on the Internet. So I know it's probably stable and I know it's probably been well tested and so on. There's definitely a lot more and you can basically find your preference in there. Next to the pure clients for Redis, there's also a number of packages you will find that do things like provide providers for ASP.NET. So for example, if you want to store session state or you want to store user profiles and roles and that kind of thing in Redis, you can actually install those NuGet packages as well and basically your configuration will point everything suddenly to Redis. So let's do that. Let's connect to Redis from.NET. In one of my Visual Studios, yes, let's open up a new console application. Let's make a quick Windows desktop console application. I don't want to log in and let's put it somewhere there. I'm happy with that. What I can do is, first thing I could do is install my Redis client from NuGet. This is probably because I have way too many Visual Studios open. So let's take one of the demos I prepared. I actually have the code that I was about to write. So what I would do is install the Stack Exchange Redis client, for example, and then in my program, I would just create a connection multiplexer, connect to Redis, get a database from Redis, second line there, and then set strings, get strings, add items to lists, add items to whatever. Probably all the commands that I showed you that you can use in the Redis client, you can also use those in your.NET codes. Couple of interesting things to note in this slice of code. So you have connection strings, of course, to make your connection. For your local host, it's as simple as this. You just give it the host name and the port number. For Azure, you will probably be using a password as well because Azure did add an authentication layer to this thing. So you want to update the connection string to also have the authentication there. Very important thing is don't create this connection multiplexer object for every call you make in your application. Why? Because this thing will actually set up the connection to Redis and keep that connection open. And of course, when you dispose of this object, it will close the connection, but it's making a connection every time. And it's creating a pool of connection and so on. So if you create this little object, every time you want to access Redis, you're losing performance. So just make sure this is a static or a singleton or whatever that you only do it once to connect to Redis. Second line of code there is also interesting. Redis has this concept or the notion of multiple databases. It's not like Redis will store data in different files and so on because, again, it's just an in-memory database or an in-memory key value store. But the interesting thing is that you can sort of partition the data that you are storing in databases. You can have, I think, up to nine. And you could do, for example, you could store your users in database zero, your products in database one, your whatever in database two. And the reason that is there is basically, again, for OCD people to just partition their data and make sure the data is in the database they expect it to be. An advantage with that is I showed you that you can do this keys command and I told you it's actually a bad command because it loops all the keys in Redis. If you store the data that you want to retrieve by using this keys command in a different database, it's actually a shorter list to loop over and it's more efficient to run this keys command. So keep in mind you can use multiple databases. In this case, I just got the default but I could also do something like give me back database zero or database one and so on. So what I'm doing here is just storing a test string and then getting the test string. So let's run this on the local machine here. I have to press the right button, of course. And this will basically do a very simple string set and a very simple string get. So as you can see, I just retrieved the value that I previously stored in Redis. An interesting thing, though, is that in my command line here, I can just say get test and I will get the same value. So even if I have.NET code and a PHP application, for example, I could share all these things and retrieve all that data. So very simple example but just to get you familiar with the concept of connecting to it from.NET. All right, let's go back to slides for a bit. How about Azure? Because the talk is called Redis on Azure and so on. What about Azure? Well, the reason I'm not talking that much about Redis on Azure is that Redis on Azure is just Redis. With one difference or two differences actually, there's SSL support in there and there's an authentication layer in there. But apart from that, it's just Redis. So often when I give the stock people ask me, can I have like the local storage emulator? I have for Azure storage. Can I have the same thing for Redis? Well, yes and no. There's no specific Azure Redis that you have to install in your system to run it locally. You can just get it from GitHub or NuGet and run Redis on your machine because again, Azure Redis is just Redis. The only thing I dislike a little bit is that for downloading Redis, you actually have to go to NuGet, download the NuGet package and then in there, the execute to police. It can be found. So if you search for Redis-32 or Redis-64, you can get 32 or 64-bit version of Redis and a NuGet package is just a zip file. So if you extract it, Redis is in there and you can find all the files in there. Right. So let's give you a quick demo on how to set up Redis on Azure. It's only available from the new portal. So if this is the only portal you're using, you'll be fine. If you're still on the old portal, you have to switch to this one to manage your Redis cache. You can simply create a Redis cache. I can easily create a new Redis cache. It should be somewhere in there. There's too many services. Yep, there it is. And when you create a Redis cache, there's a couple of options you can give. First of all, there's the DNS name. So your Redis cache will live at something, whatever you pick, dot cache, dot score, dot windows, dot net. So that's one thing you can pick. Second thing you can pick is of course, where does this cache run. But the third thing you can pick is the pricing tier. And that's an interesting thing. So what Azure offers is a number of dedicated deployments of the Redis cache. But what you will have to do is pick the right version of the Redis cache. There's a standard version where you basically have two instances of Redis running. You get a master and a slave. So whenever the master goes down, it will basically fail over to the slave and your data is still available in memory. If you don't care about that happening, and it can actually happen because Azure does updates in the data center and so on. So every now and then there must be a reboot of this machine. But if you don't care about that, you could also use the basic version, which is just a single instance, and whatever the machine reboots your cache data will become. So once you pick that, you can create one. And it takes a while because it's actually spinning up a new virtual machine for you. So I'm not going to do that here. I actually created one previously. So I have a cache here, which is martinba.redis.core.windows.net. And what you get on Azure is some graphs on your hits and misses, for example. So Azure is basically running this info command a number of times against your Redis instance to just show you some nice charts on what's happening in there. Those are actually very interesting because if you look at the hits and misses, you can actually see how your application is behaving in terms of am I caching the right data, am I retrieving the right data, and so on. Another interesting thing is that you can add alerts to some of these values. So if you say that the number of misses, for example, goes up by a certain percentage, you can get an alert when that happens. If you get, for example, a lot of hits, then you may want to send an email to your developer saying you're doing good because you're fetching everything from cache there. So there's some basic monitoring. And apart from that, as I said, it's basically just Redis running on Azure. The only thing that's different is that you have some access keys to connect to it and that you have access ports where you can enable SSL as well. So if I want to connect to Azure, the only thing I have to do in my connection string is basically add this thing and say I have a password which is this specific key and, of course, set the correct host name there. So that's the only difference between Redis and Azure and Redis on your local machine. So that was the Azure part. Oh, there's one more thing I want to show you. I showed you that in the Azure portal there's a lot of statistics and so on on hits and misses. This is actually from someone who's very well known in the Redis community, Redsmin. It's an online tool that you can also connect to your Redis server and gives you much more insights even in the data that you may want to retrieve from Redis. So I actually added my Redis cache in there and because the Internet doesn't seem to be too fast, I'll quickly go over it. So you get a dashboard of whatever is happening on your Redis instance, but you also get a console where you can run get sets and do basically whatever you want to do without having to run the local version of this thing. And you can add keys, you can retrieve keys and so on. Why is this important? Because this is like the useful version of the info command with history. So you can see the number of hits and misses. If you do a new deployment that should increase your hit ratio on Redis, you could determine the difference between how it was before the deployment and after the deployment and so on. So very useful too. One thing you will see, there's always some clients connected to your Redis instance. If you go to that list, you will see it's actually of course this tool because it's fetching the same data from my Redis cache, but there's also Azure running this info command to be able to populate all the different metrics in the portal. Right. So what else is in Redis? Well, there's transactions. Like I said, it's a single-traded thing, so you can do things like save increments and so on. But because it's single-traded, you can actually block other users by using some transaction mechanism. Well, you cannot actually block them, but you can ensure that the data that you are storing is the actual data and it's not being overwritten and so on. So if you want to do transactions, that's actually very simple. I'm going to quickly do it from the console. What you can do is say, I want to watch my attendees variable, for example. Then I'm saying, I'm starting a transaction by saying multi. And then I'm going to say set these two, say 99. Nothing happens. Redis said it's queued right now, which is a good thing. I may want to do multiple things in one transaction. I may want to watch multiple variables, set multiple variables and make sure they are all stored and not just one of them. So it's queued. And what I can do now is in a different window, set my attendees, for example, to a thousand. It happened. It has been stored. If I now say, okay, run my transaction, Redis will say no. Why does it do that? Well, because the value changed while I was in this transaction. And it gives me the opportunity to just reissue my commands, restart my transaction and make sure that the data I want to store is actually the data that will be stored in this case. So it works sort of in the inverse way of what you expect from a transaction, but it's actually very useful because you get basically a notification that the variables that you are watching have been changed and you may want to run your logic again on this thing. Right. What else do we have? We have publish and subscribe. That's actually a very, very useful thing in Redis because it's a very simple queuing mechanism where you can just send messages and retrieve messages on the other end. So how do you work with queues or pups up? Well, it's very simple. You just say subscribe to say news and you're subscribed. It keeps waiting because it's just waiting for messages. What I can say then is in a different client, publish news and say, hello, publish news. How are you? And you get all these messages in this queue. Again, I failed on the string thing, but it sends the messages to my queue. I can have multiple publishers. I can have multiple subscribers and so on. And I can also do this from.NET code. So if you want to do pups up from.NET, you basically say I want to add a subscriber to the news channel and just show whatever message ends up on there. So if I run this thing on my local Redis instance, you will see that it's actually publishing data to my news queue and retrieving data from that thing as well. Same thing. I'm publishing from.NET in this case, but I will retrieve the same messages in my Redis client. So even if I'm running PHP and.NET and Java and so on, I will get all these messages. Very interesting for very simple queuing scenarios. Probably not the best thing you want to run your entire backend on and make sure that every message is definitely being processed and so on. It's a very simple way of communicating between different services over queues. And the best thing is it actually, yeah, it makes sure it delivers them to all the clients that are currently listening on that thing as well. Other things in Redis. Oh, yeah. There's one more. There's a special queue in Redis that you can configure and enable, which is the queue that gives you key space notifications. And you can, for example, subscribe to the event when an item is expired in your cache. You may want to be notified of those types of events to make sure that you can actually publish the new data to Redis and make sure the data is actually cached again. One little downside there is you only get the name of the variable. You don't get things like the actual value when they did expire and so on. So it's a very, very, it's like the simplest way of notifying the users that something has changed in the cache. All right. One more thing in there. You can do scripting. So Redis has Lua support where you can basically run Lua language type of scripts. Lua is basically, it looks very similar to JavaScript and it can do a lot of things that you can also do with JavaScript. So like I said, if you run a Redis cluster or Redis server, you're probably using a lot of memory while you may actually want to use some of the processing power in there as well. And one of the things you can do is do things like JSON encoding and decoding, for example, on your Redis server. So to give you an example, I created a number of scripts or a couple of scripts. It's in this one where I'm basically just evaluating a hello world. And if I run that one, what I will get is a very simple hello world back. So nothing very special there. But just to give you the example that you can actually run scripts in there, and if you look at the results, you'll actually see that there's a hello world in there. So let's run this thing and step through it. And you will see hello world, which has been returned by a script. Not very interesting. What could be more interesting is if you store JSON objects in a string. So let's store an apple in our string. And if you retrieve that apple from Redis, I get back my raw JSON. So I just stored this JSON data in Redis, which is quite awesome. An interesting thing is that I can also do use a script, for example, to just give me one property of this JSON object. So imagine for my key apple, I'm only interested in the color. What I can actually do is run this script on Redis, which will basically check does the key exist? If it does exist, decode it from JSON and then return this specific property that you're interested in. Well, if we run this, we will just get the color of the apple. So the color is red. So what I just did is use Redis to basically decode my JSON data and just return one of the properties that are in there. One downside, one little downside is that you always have to send the script with the call that you are doing. So yeah, don't put scripts of a few hundred megabytes in there because you will lose your performance again. So it's to do very simple things. But you could do things like store, like in this case, I just stored JSON data as a string. But you could probably also store your objects as a hash set with keys and values and then properties and values. And then have a function or have a script that returns that entire thing as a JSON encoded object that you can use in your application. So it would make a lot of sense actually to do that. Right. Oh, there's a lot of helper functions available. So there's JSON encodings, there's working with strings doing things like substring and so on. There's some debugging in there. You can do even message pack, which is one of those wire formats encodings that you can use to even further reduce the data for an object that you're sending over the wire. If you look at the documentation in Redis, you can actually import additional libraries that are written in Lua to use in your scripts. So if you need specific things like connecting to a database, I wouldn't really do it in Redis, but you could in theory do all those things. All right. So I told you at the beginning that Redis is sort of constrained in that all your data has to fit in memory. So what do you do if you need more memory? What if you go beyond what your server or your subscription allows you to do? Well, the traditional answer in Cloud World is use sharding. Have multiple Redis servers do some consistent hashing where you know which key will be on which server and you can then use that to connect to your various Redis instances and get the data from the one that you are interested in. If you are lazy or you just have a massive deployment and there's no really good way of doing the sharding manually, what you could do is install twin proxy. That's actually a proxy for Redis written by Twitter and the proxy will actually do the hashing for you and you connect to the proxy and it will in itself pick the right server on the back end to get all those keys. And you can use that in the.NET world as well. The only thing you have to do is say there's a proxy involved so the client knows the proxy is there and it cannot do certain operations because of that. Alright, how much time do I have left? 10 minutes. Okay, let's do some patterns. I have a couple of them and I'll quickly get my visual studio just to see which ones we have. We have counting in Redis. We have showing the latest items from a list. We have rate limiting an API. I actually built a clone of WhatsApp, a messaging application. I have alter completes. I have discovering friends in a social application and I have sorting. Which ones do you want to see in these 10 minutes we have left? Counting. That's one. Latest items. No one. Perfect. Rate limiting in an API. That's a little bit more so let's make that the top one now. The WhatsApp clone, the messaging application. Let's see even more. Alter completion. Okay. Then there's friends and sorting but I think since the reaction was on those topics let's start with something like the rate limiting. I think that's an interesting thing to do with caching. A lot of APIs that you see on the internet actually have rate limiting involved. Let me also get the slide for that thing. Here we are. A lot of APIs have this concept of doing rate limiting where you for example say that a specific user can only access my API once a minute or a thousand times per hour and things like that. How would you build something like that? You could use a database for that but that means that just to make sure that you are blocking or unblocking your users from doing certain things you actually have to do the work on the database. You have to maintain counters. You have to do a lot of things there. So why not use Redis to do this rate limiting? So imagine you want to only allow five requests per 60 seconds to a specific API. What you could do is for every user or for every client or for every API key that you have, store a list in Redis. In that list you would record the timestamp of the latest request and whenever you hit the fifth item, whenever you only have like three items or four items you're okay because there's only four requests being made at that moment but if you have more, if you have this limit of five items in this case, what you want to do is check the leftmost item in that list, see if it has expired or not, is it still within this 60 second boundary or not? If it's not, perfect. Let's remove it and add a new one and you're not being limited. If it's still within this 60 second boundary, you just block the user because the list is full and there's no item that can be removed in that case. Very simple approach and to demonstrate, I actually have that thing in an API. Let's run it. I have a very simple Hello World API where I can give an API key header to my server and I can have it say Hello World. So this would be my request. I would just reach out to my API with a specific user agent and an API key and I just execute the request against this server and you'll see it will work. It will not work. Did I just start the wrong instance? Yes, I did start the wrong instance. Let's start our rate limiting. All right, rate limiting. There we are. So let's try that once more. Let's hit our Hello World application and you get back a response. 200, okay, 200, okay. Now let's do a lot of requests and you'll see the requests are being blocked because I hit my rate limits because I did more than five requests per, I think in this case I have it in a couple of seconds just for demo purposes, but it's easy to do. I have rate limiting. How did I build this thing? Quite easy. In my application, I actually have added a Trottling HTTP handler or they are a delegating handler where I'm connecting to Redis. Whenever an API key header is present, I'm actually getting my database and I'm getting the rate limit for a specific API key. So per client, I would maintain a list. Then I would say, okay, what's the length of this list? If it's lower than five, which will be my limit, no problem. Just let the request go through but just add a new item on the left, which is the current timestamp or the current ticks that I want to store. If I do have five requests or five items in this list, I get the first item from that list or the last item of that list in this case. See if it has expired or not and if it expired, I'm being trottled. If it hasn't, or if it did expire, I'm just removing it from the right of my list to make sure my list is actually shrinking and I don't have to do a lot of complicated logic to block my users from using this API key or using this API. So that's a very simple approach on how you can use Redis for something that you probably wouldn't use a cache in other scenarios. All right, let's do the autocomplete because that was one of the very interesting things you were saying as well. So autocomplete, how would you do autocompletion for a large set of products or dictionary or whatever? There's a couple of ways you could do this. Just store them in a database and run a like query on those and that will probably just work fine. But yeah, it's a bit why would you spend your database on just storing that data and querying for that data? Another thing could be using Lucene or Elasticsearch to do that, which may be overkill because it's a full text indexing search thingy and not something to just autocomplete words. So let's go with Redis. How would you store this thing in Redis? It could be a list, it could be individual keys for every item you want to store. But I think one of the interesting data types to use for this scenario is using assorted sets where you basically store the words or the term that you want to autocomplete and a score of zero for every item. What's the use of having a score of zero for every item if you're using this sorted set where I want to do scoring for retrieving the data? Well, the second property or the second item Redis sorts these items on is the actual string value of the key that you are storing, which means if you want to do things like autocompletion, you could just say Redis, give me something that looks like this. It will sort the items for you and get the topmost items or the best items to store that data. So to give you a quick example of my autocompletes, I have stored the entire English dictionary in a cache on Azure and I can search for things like Apple and it will just give me all the words that I can complete and so on. I can do pirates, I can do piranha, things like that. So you can see it's actually really fast to search this entire English dictionary. How is it stored? Well, like I said, in assorted sets. So let me quickly go over the codes. So to store the data, I have this populate method, it's quick and dirty, but what I'm doing there is basically parsing a dictionary.txt file and adding it to assorted sets. My assorted set is called autocomplete words. I'm storing the word in there with a score of zero. Again, why the score of zero? Because Redis will actually score the items based on the string value and not on the score in this case. So to provide autocompletion, what I'm doing is I'm basically getting a range from my assorted sets where my query is my actual words that I'm autocompleting or the substring that I have already autocompleted. So I want to search for Apple. I'll just have a query which is APP, for example. And I'm saying give me 20 items that are closest to that words and Redis will just give me these 20 items if those items are discovered in the dictionary. So it's really easy to do. It's very efficient and Redis is really good at this sort of thing where you're not retrieving the entire English dictionary to search in the list for the autocomplete words but you're making Redis do this heavy lifting of the searching for you. Of course, it's not Lucene, it's not Elasticsearch, but for autocompletion, for example, it's a very nice scenario. And I think I'm pretty much done now. So if you're interested in one of the other patterns, come and find me and I'll be happy to show them to you. With that, I want to thank you for being here and remember one thing which is the little Redis book. That's the only takeaway you must have if you leave this room. If you want to start with Redis, you think it's interesting, read that little book. Thank you. Thank you. You
 All right, let's wait for the screen to clear and there we are. Welcome this morning in  this session on get more than a cash back on the Redis cash on Azure. Now, before we  start, a quick word of warning, this talk is going to be mostly about Redis, a little  bit about Azure. And the reason for that is that the Redis cash on Azure is actually just  the Redis cash. So if you want to use Redis and you want to start with Redis, you can  simply download it and start working with it. So the goal of this talk is actually to  make you more familiar with Redis. To see all the things that are in there, to see that  it's more than just storing a value, retrieving a value like you would use a cash normally.  And then if you're interested, you can always start using it on Azure. But it's basically  the same. So whatever reason you're here, if it's for Azure, perfect. If it's for Redis,  even better. It's just going to be about Redis mostly. Let's quickly introduce myself.  I'm Martin Balliao. I flew in from Antwerp in Belgium. I'm a software engineer at Microsoft  and also work on my gets in my evening hours. I'm the co-founder of the Azure user group  in Belgium. So I've been doing a lot of Azure in the past and I continue on doing that.  And if you want to follow me on Twitter or read my blog on all these topics, you can  find them there. Oh, and if you want to buy me a beer, especially here in Norway, buy  my book by twice or at least with the beer prices here by three times, I would really  appreciate that. Thank you. All right. So the agenda for today, like I said, is going  to be mostly about Redis, but we'll start a little bit with the Azure cash story, the  things that happened there, which caches were available, which caches are still available,  and then slowly transition into looking at Redis and how we can use Redis, what the data  types are, see there are transactions, there's PubSub and things like that. And I have a  lot of demos. In fact, if you look at my desktop here and you look at the amount of  visual studios I have open, I have quite a few of those and I will probably not be able  to tackle all of these in this one hour we have here. So once we end up with the patterns,  we'll do a quick photo of hands on which patterns you actually want to see because there's a  lot of interesting things you can do with Redis. Like for example, doing the session  voting buttons here would be a nice example of using Redis in one of those patterns.  Right. So the Azure cash. Caching on Azure is a bit of an interesting story. So back  in 2009, Microsoft released the Windows Server app fabric, which was on Windows Server. If  you've been using this, you may remember the code name Velocity. That was actually a cache  you could use on your Windows Server with your ASP.NET applications with whatever you  wanted to store in there. And then Azure came about and Microsoft thought, okay, we need  a cache that people can leverage in their cloud solutions. So let's make a port of this  Windows Server app fabric and put it on Azure as well. And they did that and they came up  with a shared cache, which is an interesting thing as a vendor to just provide a service  that a lot of people can use and a lot of people can share because it's very cost effective  and so on. It had one little issue though. The fact that it was shared and the fact that  Microsoft wanted to guarantee people got some throughput and people got their data back from  the cache. The reason that was done is that they also introduced Trotling on the cache,  which is an insane concept if you think about it for a cache because when you need to cache,  why would you use a cache to offload your database server, to offload your storage,  to offload whatever you want to store in your cache. But if you have Trotling on your  cache, that's an insane thing to have because your application would crash once you have  loads. So the shared cache was not really popular for that reason, but again, it was  still based on Windows Server app fabric. Good thing was that Microsoft noticed that  this Trotling idea was probably not too good. So there came a self-hosted dedicated cache  where you could basically just deploy your cloud services on Azure and install a NuGet  package or install an extension there and the cache would be co-located with your web  server or your worker role that you would deploy. Interesting solution and that's still  out there today. You can actually still use that. But it's not ideal because you're sharing  the memory on your web server and if you have a lot of traffic, your web server may need  that memory to serve actual connections and clients. So there was a different option as  well, which is the managed cache, which is basically the same thing, except this time  instead of having a shared cache, Microsoft would just provision a VM for you or a cluster  of VMs and then have the cache running in there. Quite popular, a lot of people actually  use it, but yeah, there's something else. People started to develop their applications  differently. So one of the things that happened is that a lot of different technologies came  to Azure. So for example, PHP, Java, Node.js and so on. So those people also wanted to  use caching in their solutions. And on Azure, Microsoft thought, okay, so now we will have  to write clients for pretty much any language that's there. Interesting thing and a lot of  work. So Microsoft thought, okay, let's look at a different solution. Let's look at some  server that already exists. Let's look at an ecosystem where there's clients for virtually  any language and just make it run on Azure and provide that thing on Azure. And that's  what Microsoft did and the reason why Redis is now running on Azure. If you go to the  Redis website and you look at the clients for all the various languages, I wouldn't  be surprised if there's an actual cobalt Redis client or something like that. So pretty much  any language you can run on Azure or pretty much any language you want to run on your  VMs, you can work with Redis from that. So that's the reason why Microsoft decided, okay,  let's go with Redis. Let's make it run on Windows so we can actually host it and make  it available. And the clients are already there. We don't have to maintain them. There's alternatives  in open source. Like for every language, there's like five or six clients. So you can basically  pick the best one for your goal. Another reason is that the f-fabric cache was actually just  a cache. And people started to think differently about how they developed their applications.  If you look at Twitter, for example, they basically have the MySQL databases in the  back where they store your tweets and user data and so on. But virtually anything you  access on the Twitter website, on the Twitter apps on your mobile phone, for example, is  stored in Redis, in memory. So people actually start using caches as their main persistent  store or their main go-to database to store their application data while it's running.  Of course, there's always still databases in the back to persist it. And in case the  cache goes down, you can still reach out to the database and repopulate everything. But  people started to actually use the cache as the main data source. And that's a perfect  thing to do, especially with Redis, because Redis has different options, as we will see,  to store that data and make very effective use of that.  To give that Twitter example a little bit more context, if you would build Twitter, you  could do like if you want to show the timeline of people. I take my phone and I want to watch  the stream of tweets that I have there. You could probably map it relationally and have  like one tweet, mapping to the users that you're following and so on. But that's in  the relational world. What Twitter actually does is they say, memory is cheap, memory  is really fast, let's store everything in Redis. There's this interesting website called  highscalability.com. There's really interesting articles there from Twitter from like all  the big cloud guys. And apparently Twitter is actually storing your timeline in duplicates  format in memory. So if you access your timeline on Twitter, you actually get your version  of the cache. So for every tweet I send, all my followers will get my tweet in their cache,  which is an insane amount of data. But just to prove the example that people start to  think about their applications differently and start to store all that data in one store.  So that's two good reasons to actually move to a different caching system, which is Redis.  So let's talk about Redis for a while. Redis is a very simple name. It's a remote dictionary  server. That's a very interesting concept to remember. This is the actual, yeah, what  are we that you can find on their websites, which is kind of complicated, but there's  a couple of things we want to remember from Redis. One thing is it's a remote dictionary  server, which means it's a dictionary and it's a server. And in a dictionary, you typically  store key value pairs. You store a value and you can access that value by asking for a  specific key. So it's basically like in.NET where you could have a string dictionary  or a dictionary of objects or whatever. You could do the same in Redis. Redis is actually  nothing more than that. It's a dictionary that you can access via TCP. Another interesting  thing is that Redis is an in-memory remote dictionary server. There's no persistence.  So if you want to use Redis as your main data store, you probably want to cluster it or  set it up in a way that there's always at least one copy of your data still available  because there's no persistence out of the box. Well, there is persistence out of the box  and you can actually store your cache data to disk to repopulate the data if your server  crashes or has to reboot, for example. But you have to keep in mind that Redis is memory  based. All your data has to fit in memory and, yeah, you don't want to actually use  that persistence because it could slow down your caching. It's single-treaded, which is  a good thing and probably also a batting. But the good thing about being single-treaded  as Redis is that there's always one client being served, which also means that you can  do interesting things like transaction style work. So for example, if you want to store  a value and you want to increment that value in Redis, you know that only one concurrent  user will actually be increasing that value at the same time, which means your count,  for example, will be correct. It will be the actual count of increments. All your different  users have been doing because it's a single-tread and it's just making sure that there's only  one person or one client actually accessing that data. It's networked, so you cannot embed  it in your application. It's like a standalone server you have to run and connect to. And  it's basically exploiting the fact that it's a dictionary because dictionaries are very  fast to work with. It's very fast to search for things by ID. So the author of Redis actually  thought of some interesting things to do with those dictionaries, of which there's running  scripts, for example. There's a PubSub mechanism. You can do snapshotting of data and so on.  So there's just much more to this simple key value store than you would think.  So I talked about persistence. Is there no persistence? Well, you can actually persist  to disk if you want to. There's two ways of doing that. There's append-only files and  there's Redis database files. You can configure Redis to store your cache data to disk. Perfectly  possible if you want to run Redis on your own, if you want to run your own Redis cluster.  It probably makes sense to do this because if your cache goes down or the server has  to reboot, it can actually repopulate the entire memory database from disk. Whether  you want that or not depends probably a little bit on how volatile your data is that you're  storing. Like if the server has to reboot for 10 minutes, maybe you don't just want  to restore the data because the data is still, but still you can do it.  One thing you must keep in mind though is that everything must fit in memory. So if  you want to store one terabyte of data in Redis, you will have to have one terabyte  of memory. Even if you're using persistence and maybe even if you're having all the disk  space in the world, it has to fit in memory. And that's a very important notion to have  because you cannot just keep storing stuff in Redis. Eventually memory will run out and  you'll have to think about having multiple Redis servers and do some partitioning and  so on. All right. So let's give you a quick Redis 101 just to give you a feel of what  Redis actually is. So I have a Redis download here on my system. And as you can see, if  you download the Windows version, there's a couple of files in there. There's Redis  benchmark. There's the Redis client and there's the Redis server. I'm actually going to run  the Redis server in a bit. So I actually have a locally running Redis server and everything  on this Wi-Fi can probably connect to it. There's no authentication. There's no security.  Anyone can just connect to that thing. Reason for that is Redis wants to be very light.  Redis doesn't want to have the additional overhead of doing authentication and so on.  But also keep that in mind if you want to deploy Redis manually, there's no real security  on Redis. So you're basically opening up your cache to the world if you run it on a public  IP, for example. Some interesting things in there are that this Redis benchmark file is  in there as well. It's actually an executable that you can use to run against your Redis  data store and you can give it a lot of parameters like how random do you want the data to be,  how many cache hits and misses do you want to generate. But you can also specify the  object size of the things that you want to test and benchmark Redis with. And that's  a very interesting thing because Redis is perfect for a lot of situations but it's also  a bad thing to use for some situations as we will see later. So whenever you want to  start with Redis and you know pretty much what type of data you want to store in Redis,  run the benchmark, see how your server is performing and see if Redis is actually the  right choice at that moment. Very useful tool for that. So let's run Redis and I have a  nice batch file there which basically uses a default configuration and starts Redis.  As you can see, one of the main advantages of Redis is there's ASCII art involved. But  we now have a Redis server running. So you can connect to this one and I will actually  connect to that one as well using the Redis Clients and I made it a little bit bigger.  I decided not to do this with this thing because nothing interesting happens in this window  except an occasional error every now and then. So let's minimize that one. So let's use the  Redis Clients. By default it will just connect to the server you're running on your local  host so you don't have to configure it and so on. It will just connect. Of course you  can also, I mean PowerShell of course. Let's try that one more time. There we are. That  seems to work. Okay, so my client is now connected to my local server. If you want to connect  to a remote Redis server or Azure for example, you can also use this tool for that. So let's  store something and let's store my name and let's say I'm storing Martin. Very simple,  I stored something in the cache. I can get that name and I will retrieve that name from  cache which is very useful and very interesting and this is what you expect the cache would  do. Now there's a couple of more interesting things we can do with Redis. Like I said,  it's a simple dictionary so an operation like this where you store something and retrieve  something is very easy but Redis has some additional functions that you can use to for  example get a range from a specific key. So from my name I can get a substring from character  zero to character four and get a part of that thing. Why is that interesting? Well probably  not very interesting in this example but imagine you are storing a lot of data and you know  pretty much what the chunk size of that data is and you know you want to retrieve a specific  part of that data. Why would you block this single threaded server with sending you back  like a megabyte of information if you could just ask the server to give you back just  that piece of data you're interested in from that specific key in the dictionary. So a  very interesting function, not really in this case but you can use it and leverage it to  minimize bandwidth first of all and minimize the time Redis has to spend on blocking you  because it's single threaded and serving your data back. So use those functions. All right,  another thing you would expect from a cache is that you can expire keys. So let's do that.  Let's say my name should expire in five seconds and let's do a get name. It's still there,  still there, still there and now it's gone. So you can expire your keys as well which  is interesting. You can just use it as a basic cache. So if you're just interested in storing  something and retrieving something, thank you for being here. You may now leave. That's  the simple cache of Redis. Now let's dive into some more interesting things. One thing  I still want to show you here is that there's the info commands which gives you back the  configuration first of all but also the number of cache hits, the cache misses and so on.  So you can actually use this info commands to get some details on how your server is  operating to see if you're hitting the master or the slave for example if you connect to  Azure and so on. So remember the info commands as well. All right, if you want to start with  Redis, if you leave this talk and you want to really start with Redis, remember this  URL. I have it again at the end of the talk and I will probably tweet it again as well.  Just search for the little Redis book. It's actually a very little book. It's like 40  pages and it contains pretty much anything you want to know about Redis. It gives you  a lot of examples, a lot of use cases and scenarios and so on. So if you're really keen  on getting started with this, remember this book. It's a really good thing. All right,  so I told you Redis is much more than a cache and much more than something where you can  store a simple value and retrieve a simple value. So let's go over some of the data types  that are available. We've already seen the string data type which is basically just get  and sets. It's the string data type because internally Redis stores it as a stream, a  byte string or some bytes but you can actually store a lot of things in there. If you want  to store integers, you can just store an integer, increment the integer and so on. If you want  to store a byte array or a JSON document or HTML for output caching, for example, you  can do that. It's a very simple, it's the basic premise of Redis where you have keys  and values that you want to store. More interesting is the hash data type. The hash data type  is like you would expect, it's a hash of fields and values. So if you want to store, for example,  in a web shop, if you want to store a category that you want to retrieve, like for example,  books and the amount of books that are in this specific category and so on, you can  actually use the hash for that. And why is that interesting? Well, because I can ask  Redis to retrieve the entire hash to retrieve all the fields and all the values that are  in there, but I can also ask Redis to give me just the name of this specific hash. Again,  to make the communication with the cache very fast and to make it very focused on just the  data you need to retrieve from Redis. So if I have a page where I only want to show the  category name or the number of products, I can just tell Redis, give me just the name  from this hash. There's also sets. It's basically like you would have arrays of integers in.NET.  It's basically just that. So you can have a set, you can give it a key, of course, to  retrieve the sets and you can push some values into that thing. So for example, if you're  building, again, an e-commerce website, for example, you could use this set to store product  IDs for a specific category. So if you're a category one is the books, you can store  the product IDs in a set. Why is that useful? Why just store those product IDs? Well, because  Redis can actually quickly retrieve the IDs for a specific category. You can also do things  like intersecting data. So if you want to see which books are in thrillers and in nonfiction,  it could be a book on JavaScript, for example. You could actually intersect the categories  there and look at the different books that are available in both categories or the books  that are available in not that category but that are in that category and so on. You can  have lists. Lists is pretty much similar to the sets except lists can contain anything.  So not just numbers. You can push strings in there. You can push objects in there and  so on. So if you would think about building a common system, for example, for your e-commerce  websites, the comments could be stored in a list and you can tell Redis to give me the  first five items of the list or give me the last item of the list and so on. So you can,  again, get very focused data and very fast retrieval from Redis. There are sorted sets  as well and that's sort of a weird thing in terms of what it stores because it says it  has fields and scores. It's actually just you store a value and you can give it a number  and that number is used to do scoring on that sorted sets. It will actually sort the entire  thing by the score that you have there and it does things like, for example, give me  the top five products or give me the top five whatever is in there. So imagine you are building  a Twitter clone, for example. You could store in this sorted set the different hashtags  that people are using and whenever a hashtag is being used, you could increment the score  and that makes it very easy to do things like what is trending in this specific country  because you just ask Redis, give me the top ten hashtags that are being used currently  in the system and you will get that information. There's a couple more data types. There's  the bitmap which is basically just a stream of Booleans where you can do things like, is  this user logged in or not and just store one bit basically. There's also the hyperlog  log which is a very specialized data type. Read the little Redis book if you really  need those but make sure you know these things because they are really important to build  stuff on Redis and make it fast and scalable and so on. So let's go over a bit of the commands  to get you a little bit more familiar with what these data types can all do. Let's go  back to our PowerShell. Let's clear this thing. So we had the strings where we could store  data. I stored my name, I expired my name, I got a range from this name and so on. I  told you you can also store integers. So let's say we have an attendee count and let's  say it's 2-0. What I can do is say increment attendees and it will basically increment  my integer. You always get back the current value of this integer and am I now crashing  my Redis server? It almost looks like it. There we go. So my integer has been increased  and remember because it's single-threaded a lot of clients can actually increment this  integer and you will never have conflicts. It will be the actual value of this integer.  Just locking because it's only single-threads. I can increment by a number as well. So if  I want to say my attendee count goes up with 200, I could do that as well. So it's not  just by one. You can do whatever you want with that. So that's a number of the things  you can do there and it's really useful to have. All right. Let's go over the hashes  and let's build an e-commerce web store and we want to store our product categories in  Redis. So let's do that. Let's do a hash set for categories one. I'll shorten this to  C1 just to have a little typing and let's say the name of that thing is books, description,  good books, number of products, five. And what did I miss? I didn't miss a thing. One,  two, three, four, five. Okay. I obviously gave one parameter too much because I didn't  use a proper string there. So let's make that good underscore books or you could use quotes  as well to store this thing. So this has been stored and I can get all the values from this  specific category and just say give me all those things and you will get back a list  of properties and their values. More interesting thing is that I can also get just one thing  and I could say get me from category one just the name and I'll just get the name property  of that thing. Again, why would you do this? Just to make things very fast, to just get  the information you actually need in your application. I could also do things like increments.  There's an h increment by so if the number of products goes up, I can just up the number  of the products without retrieving everything and then restoring everything. But that's  basically the things you can do with the hashes. All right. Let's work with a set. So if I  say in category one, we have some products, bear with me, C1 products. I'm going to store  some products and let's say product 20s in there, product 30s in there, 99, 80, 70 and  so on. I've done that. I stored the product IDs for this specific category in my bookstore  in a set. I can do increments. I could do things like adding or just getting the top  value from this list, but I could also do things like create a second category which  has products 20 and 11. Let's say 30, 99, 70 and 80. So this thing now has four items.  Again, very simple array and I can put integers in there. But the interesting thing comes  when I do set intersects and so on. So I can do an intersect between my first category,  the products in there and my second category, the products in there. And I will get back  the common items in those two sets. I could also do a diff between both and I could say  show me all the items that are in category one, but not in category two. Obviously, you  have to type it correctly. I will get back these two numbers. So I can easily do intersects  and so on. Why would you do this? Can my application do this? Well, again, just retrieve the data  you need and ask Redis to do this. Also, since Redis uses memory to store its data and if  you have a lot of memory, you probably also have a quite good CPU in there. So why not  just upload these simple operations or offload these simple operations to the Redis cache  in there? Alright, let's go with the list. I can push items in my list and say for my  product one, I have some comments and one of those comments is good product. I can push  some more in there. Let's say awesome stuff. And I now have a couple of items on this list.  I can get all those things. I can get the number of items in there. So I want to get  the number of items I can get that. I should be able to get that thing. Okay, yeah, give  me item one, give me item two and so on. So I can get all those things separately from  this index from this list. Other things I can do is ask for the length of this list.  But a more interesting thing is that I can also do pops from this list. So just like  you can do with an area, you can shift and pop items from that list. I could do for example  a pop on my product one comments and say I just want one item. I'll get the item. I'll  get the second item. I'll get the third item. They're all removed from this list. But an  interesting thing is if I reach the end of the list and I still pop items from this list,  these will just put me in a waiting state until new items are in that list. So if I have  a second client and I would do a R push product one comments, great. It would eventually show  up in my first client and I can just retrieve items from that list while they are being  added. I can do for each is over that and just wait until I reach the end of the list  and even wait until new data arrives in there. And then there's the sorted set. And the sorted  set is a very interesting beast because it sort of looks funky where you just store like  the best sellers and say the score is 100 for book. Let's say the score is one to three  for car. From my set, I can then do things like scoring and say give me from my best sellers  for car. What's the score of that thing? It's not that. It's Z score. And I think that's  because I started with a string. Anyway, you can score the items in there. It's really  interesting to do things like sorting, calculating top lists and so on. So I hope you see that  by storing your data in an efficient way in the type of data that you're actually storing,  not just the raw object and get back the objects, you can actually make Redis very, very fast  by just retrieving the items you want to retrieve from Redis. So let's go over them one more  time just to make sure you know these things. Maybe a little bit boring, maybe not what  you expect from this talk, but I think it's really important to know all these data types  by heart. So instead of a store, let's build Twitter this time. I could use strings for  storing things like output caching. I could use strings for storing just the raw HTML  of my pages and serve that to my clients. But I could also use strings to store numbers.  And what I could do is, for example, when a user registers on my website, I could store  the user ID in Redis and have Redis increment that user ID every time. Very safe because  Redis is thread safe in this regard. It will always give me the new ID that I can use for  my user. In the hashes, I could store the data for my user. So for example, in this case,  I stored myself and my Twitter handle in a hash. I have my name in there. There's my  Twitter handle in there. There could be my date of birth. My password could be in there.  So basically, all the different properties of my current user can be in a hash. Very  interesting because I can retrieve this user by ID. I can just say, Redis, give me back  the entire list of properties and their values for user one. But I could also tell Redis  to just give me the Twitter handle, for example, if I just want to render a page that doesn't  have all these additional properties. Very fast to retrieve and very interesting to use  this approach to just get the things you need while you're still storing a lot of data about  this specific user. I could use sets to do things like who's following who. I could use  a set to store the followers that I have on Twitter so that Twitter has, or our Twitter  clone actually has a list of, or a set of data of the user that I'm following so it  knows where to store the data, where to write the data, who to send notifications to and  things like that. It's also interesting because I can do these things like intersecting and  so on. So if I want to boost my revenue and make sure users come, keep coming back and  users discover other users to follow and so on, what I could do is simply for my current  user, loop over the sets of all the users that are following me and do suggestions based  on intersections and differences between those sets. Like for example, you should follow this  user because somebody you are following follows this user and it's probably interesting for  you. So you can use sets to do those types of operations. There's lists of course, could  be your timeline, could be any user's timeline, just whenever I tweet, push something in that  list and it will show up in the application. And then we can use sorted sets to do things  like which hashtags are trending in a specific country and so on. So for example, for no  way I could say NDC Oslo has this specific score or this specific number of mentions  and I can do very simple things like give me the top five hashtags that are happening  today for example. All right, data types. You may have noticed from both the examples  that I was typing as well as the tables I had previously that I have used a pattern for  storing my different keys. There's a good reason I did that. It's actually one I have  a slight form of OCD where I want to know what a specific thing is so I make sure that  I name things correctly so I know this cache key will hold a user and so on. Another interesting  thing is that by doing things like storing users one Martin, I could find users by ID  or I could find users by name by simply asking Redis to query for the keys that look similar  like users colon one colon asterisk. I don't know the name up front but I still want to  get that user. I can ask Redis to give me back that information. One word of warning  though, if you have a lot of little or big objects in your Redis database, if you have  a lot of objects basically, you probably don't want to run these keys commands all the time  because the keys command actually loops the entire store in Redis. If you have a lot of  keys this may actually take a while to fetch. It's interesting to be able to do this, to  be able to maybe store parts of your data like this and get the correct data by just  querying for it based on what the key is. All right, so I've been doing a lot of Redis  so far. What about.NET? How can I use Redis from.NET? Well, there's actually a lot of  options. Like I started, I said pretty much any language has at least five clients for  Redis. It's the same thing with C-Sharp. There's I think five or six right now. So if you want  to work with that thing, you can just install a NuGet package and start working with NuGet.  There's a couple for.NET. I personally very much like the Stack Exchange one. Why? It's  a very simple one to use. All the different operations in there actually map to the different  data types and so on. So it's really easy to know what data types you're working with.  And of course it powers Stack Exchange, one of the very big sites on the Internet. So  I know it's probably stable and I know it's probably been well tested and so on. There's  definitely a lot more and you can basically find your preference in there. Next to the  pure clients for Redis, there's also a number of packages you will find that do things like  provide providers for ASP.NET. So for example, if you want to store session state or you  want to store user profiles and roles and that kind of thing in Redis, you can actually  install those NuGet packages as well and basically your configuration will point everything suddenly  to Redis. So let's do that. Let's connect to Redis from.NET. In one of my Visual Studios,  yes, let's open up a new console application. Let's make a quick Windows desktop console  application. I don't want to log in and let's put it somewhere there. I'm happy with that.  What I can do is, first thing I could do is install my Redis client from NuGet. This is  probably because I have way too many Visual Studios open. So let's take one of the demos  I prepared. I actually have the code that I was about to write. So what I would do is  install the Stack Exchange Redis client, for example, and then in my program, I would just  create a connection multiplexer, connect to Redis, get a database from Redis, second  line there, and then set strings, get strings, add items to lists, add items to whatever.  Probably all the commands that I showed you that you can use in the Redis client, you  can also use those in your.NET codes. Couple of interesting things to note in this slice  of code. So you have connection strings, of course, to make your connection. For your  local host, it's as simple as this. You just give it the host name and the port number.  For Azure, you will probably be using a password as well because Azure did add an authentication  layer to this thing. So you want to update the connection string to also have the authentication  there. Very important thing is don't create this connection multiplexer object for every  call you make in your application. Why? Because this thing will actually set up the connection  to Redis and keep that connection open. And of course, when you dispose of this object,  it will close the connection, but it's making a connection every time. And it's creating  a pool of connection and so on. So if you create this little object, every time you  want to access Redis, you're losing performance. So just make sure this is a static or a singleton  or whatever that you only do it once to connect to Redis. Second line of code there is also  interesting. Redis has this concept or the notion of multiple databases. It's not like  Redis will store data in different files and so on because, again, it's just an in-memory  database or an in-memory key value store. But the interesting thing is that you can  sort of partition the data that you are storing in databases. You can have, I think, up to  nine. And you could do, for example, you could store your users in database zero, your products  in database one, your whatever in database two. And the reason that is there is basically,  again, for OCD people to just partition their data and make sure the data is in the database  they expect it to be. An advantage with that is I showed you that you can do this keys  command and I told you it's actually a bad command because it loops all the keys in Redis.  If you store the data that you want to retrieve by using this keys command in a different  database, it's actually a shorter list to loop over and it's more efficient to run  this keys command. So keep in mind you can use multiple databases. In this case, I just  got the default but I could also do something like give me back database zero or database  one and so on. So what I'm doing here is just storing a test string and then getting the  test string. So let's run this on the local machine here. I have to press the right button,  of course. And this will basically do a very simple string set and a very simple string  get. So as you can see, I just retrieved the value that I previously stored in Redis.  An interesting thing, though, is that in my command line here, I can just say get test  and I will get the same value. So even if I have.NET code and a PHP application, for  example, I could share all these things and retrieve all that data. So very simple example  but just to get you familiar with the concept of connecting to it from.NET. All right,  let's go back to slides for a bit. How about Azure? Because the talk is called Redis on  Azure and so on. What about Azure? Well, the reason I'm not talking that much about Redis  on Azure is that Redis on Azure is just Redis. With one difference or two differences actually,  there's SSL support in there and there's an authentication layer in there. But apart from  that, it's just Redis. So often when I give the stock people ask me, can I have like the  local storage emulator? I have for Azure storage. Can I have the same thing for Redis? Well,  yes and no. There's no specific Azure Redis that you have to install in your system to  run it locally. You can just get it from GitHub or NuGet and run Redis on your machine because  again, Azure Redis is just Redis. The only thing I dislike a little bit is that for downloading  Redis, you actually have to go to NuGet, download the NuGet package and then in there, the execute  to police. It can be found. So if you search for Redis-32 or Redis-64, you can get 32  or 64-bit version of Redis and a NuGet package is just a zip file. So if you extract it,  Redis is in there and you can find all the files in there. Right. So let's give you a  quick demo on how to set up Redis on Azure. It's only available from the new portal. So  if this is the only portal you're using, you'll be fine. If you're still on the old portal,  you have to switch to this one to manage your Redis cache. You can simply create a Redis  cache. I can easily create a new Redis cache. It should be somewhere in there. There's too  many services. Yep, there it is. And when you create a Redis cache, there's a couple  of options you can give. First of all, there's the DNS name. So your Redis cache will live  at something, whatever you pick, dot cache, dot score, dot windows, dot net. So that's  one thing you can pick. Second thing you can pick is of course, where does this cache run.  But the third thing you can pick is the pricing tier. And that's an interesting thing. So  what Azure offers is a number of dedicated deployments of the Redis cache. But what you  will have to do is pick the right version of the Redis cache. There's a standard version  where you basically have two instances of Redis running. You get a master and a slave.  So whenever the master goes down, it will basically fail over to the slave and your  data is still available in memory. If you don't care about that happening, and it can  actually happen because Azure does updates in the data center and so on. So every now  and then there must be a reboot of this machine. But if you don't care about that, you could  also use the basic version, which is just a single instance, and whatever the machine  reboots your cache data will become. So once you pick that, you can create one. And it  takes a while because it's actually spinning up a new virtual machine for you. So I'm not  going to do that here. I actually created one previously. So I have a cache here, which  is martinba.redis.core.windows.net. And what you get on Azure is some graphs on your hits  and misses, for example. So Azure is basically running this info command a number of times  against your Redis instance to just show you some nice charts on what's happening in there.  Those are actually very interesting because if you look at the hits and misses, you can  actually see how your application is behaving in terms of am I caching the right data, am  I retrieving the right data, and so on. Another interesting thing is that you can add alerts  to some of these values. So if you say that the number of misses, for example, goes up  by a certain percentage, you can get an alert when that happens. If you get, for example,  a lot of hits, then you may want to send an email to your developer saying you're doing  good because you're fetching everything from cache there. So there's some basic monitoring.  And apart from that, as I said, it's basically just Redis running on Azure. The only thing  that's different is that you have some access keys to connect to it and that you have access  ports where you can enable SSL as well. So if I want to connect to Azure, the only thing  I have to do in my connection string is basically add this thing and say I have a password which  is this specific key and, of course, set the correct host name there. So that's the only  difference between Redis and Azure and Redis on your local machine. So that was the Azure  part. Oh, there's one more thing I want to show you. I showed you that in the Azure portal  there's a lot of statistics and so on on hits and misses. This is actually from someone  who's very well known in the Redis community, Redsmin. It's an online tool that you can  also connect to your Redis server and gives you much more insights even in the data that  you may want to retrieve from Redis. So I actually added my Redis cache in there and  because the Internet doesn't seem to be too fast, I'll quickly go over it. So you get  a dashboard of whatever is happening on your Redis instance, but you also get a console  where you can run get sets and do basically whatever you want to do without having to  run the local version of this thing. And you can add keys, you can retrieve keys and so  on. Why is this important? Because this is like the useful version of the info command  with history. So you can see the number of hits and misses. If you do a new deployment  that should increase your hit ratio on Redis, you could determine the difference between  how it was before the deployment and after the deployment and so on. So very useful too.  One thing you will see, there's always some clients connected to your Redis instance.  If you go to that list, you will see it's actually of course this tool because it's  fetching the same data from my Redis cache, but there's also Azure running this info command  to be able to populate all the different metrics in the portal. Right. So what else  is in Redis? Well, there's transactions. Like I said, it's a single-traded thing, so you  can do things like save increments and so on. But because it's single-traded, you can  actually block other users by using some transaction mechanism. Well, you cannot actually block  them, but you can ensure that the data that you are storing is the actual data and it's  not being overwritten and so on. So if you want to do transactions, that's actually very  simple. I'm going to quickly do it from the console. What you can do is say, I want to  watch my attendees variable, for example. Then I'm saying, I'm starting a transaction  by saying multi. And then I'm going to say set these two, say 99. Nothing happens. Redis  said it's queued right now, which is a good thing. I may want to do multiple things in  one transaction. I may want to watch multiple variables, set multiple variables and make  sure they are all stored and not just one of them. So it's queued. And what I can do  now is in a different window, set my attendees, for example, to a thousand. It happened. It  has been stored. If I now say, okay, run my transaction, Redis will say no. Why does it  do that? Well, because the value changed while I was in this transaction. And it gives me  the opportunity to just reissue my commands, restart my transaction and make sure that  the data I want to store is actually the data that will be stored in this case. So it works  sort of in the inverse way of what you expect from a transaction, but it's actually very  useful because you get basically a notification that the variables that you are watching have  been changed and you may want to run your logic again on this thing.  Right. What else do we have? We have publish and subscribe. That's actually a very, very  useful thing in Redis because it's a very simple queuing mechanism where you can just  send messages and retrieve messages on the other end. So how do you work with queues  or pups up? Well, it's very simple. You just say subscribe to say news and you're subscribed.  It keeps waiting because it's just waiting for messages. What I can say then is in a  different client, publish news and say, hello, publish news. How are you? And you get all  these messages in this queue. Again, I failed on the string thing, but it sends the messages  to my queue. I can have multiple publishers. I can have multiple subscribers and so on.  And I can also do this from.NET code. So if you want to do pups up from.NET, you basically  say I want to add a subscriber to the news channel and just show whatever message ends  up on there. So if I run this thing on my local Redis instance, you will see that it's  actually publishing data to my news queue and retrieving data from that thing as well.  Same thing. I'm publishing from.NET in this case, but I will retrieve the same messages  in my Redis client. So even if I'm running PHP and.NET and Java and so on, I will get  all these messages. Very interesting for very simple queuing scenarios. Probably not the  best thing you want to run your entire backend on and make sure that every message is definitely  being processed and so on. It's a very simple way of communicating between different services  over queues. And the best thing is it actually, yeah, it makes sure it delivers them to all  the clients that are currently listening on that thing as well.  Other things in Redis. Oh, yeah. There's one more. There's a special queue in Redis that  you can configure and enable, which is the queue that gives you key space notifications.  And you can, for example, subscribe to the event when an item is expired in your cache.  You may want to be notified of those types of events to make sure that you can actually  publish the new data to Redis and make sure the data is actually cached again. One little  downside there is you only get the name of the variable. You don't get things like the  actual value when they did expire and so on. So it's a very, very, it's like the simplest  way of notifying the users that something has changed in the cache.  All right. One more thing in there. You can do scripting. So Redis has Lua support where  you can basically run Lua language type of scripts. Lua is basically, it looks very similar  to JavaScript and it can do a lot of things that you can also do with JavaScript. So like  I said, if you run a Redis cluster or Redis server, you're probably using a lot of memory  while you may actually want to use some of the processing power in there as well. And  one of the things you can do is do things like JSON encoding and decoding, for example,  on your Redis server. So to give you an example, I created a number of scripts or a couple  of scripts. It's in this one where I'm basically just evaluating a hello world. And if I run  that one, what I will get is a very simple hello world back. So nothing very special  there. But just to give you the example that you can actually run scripts in there, and  if you look at the results, you'll actually see that there's a hello world in there. So  let's run this thing and step through it. And you will see hello world, which has been  returned by a script. Not very interesting. What could be more interesting is if you store  JSON objects in a string. So let's store an apple in our string. And if you retrieve  that apple from Redis, I get back my raw JSON. So I just stored this JSON data in Redis,  which is quite awesome. An interesting thing is that I can also do use a script, for example,  to just give me one property of this JSON object. So imagine for my key apple, I'm only interested  in the color. What I can actually do is run this script on Redis, which will basically  check does the key exist? If it does exist, decode it from JSON and then return this specific  property that you're interested in. Well, if we run this, we will just get the color  of the apple. So the color is red. So what I just did is use Redis to basically decode  my JSON data and just return one of the properties that are in there. One downside, one little  downside is that you always have to send the script with the call that you are doing. So  yeah, don't put scripts of a few hundred megabytes in there because you will lose your performance  again. So it's to do very simple things. But you could do things like store, like in this  case, I just stored JSON data as a string. But you could probably also store your objects  as a hash set with keys and values and then properties and values. And then have a function  or have a script that returns that entire thing as a JSON encoded object that you can  use in your application. So it would make a lot of sense actually to do that. Right.  Oh, there's a lot of helper functions available. So there's JSON encodings, there's working  with strings doing things like substring and so on. There's some debugging in there. You  can do even message pack, which is one of those wire formats encodings that you can  use to even further reduce the data for an object that you're sending over the wire.  If you look at the documentation in Redis, you can actually import additional libraries  that are written in Lua to use in your scripts. So if you need specific things like connecting  to a database, I wouldn't really do it in Redis, but you could in theory do all those  things. All right. So I told you at the beginning that Redis is sort of constrained in that  all your data has to fit in memory. So what do you do if you need more memory? What if  you go beyond what your server or your subscription allows you to do? Well, the traditional answer  in Cloud World is use sharding. Have multiple Redis servers do some consistent hashing where  you know which key will be on which server and you can then use that to connect to your  various Redis instances and get the data from the one that you are interested in. If you  are lazy or you just have a massive deployment and there's no really good way of doing the  sharding manually, what you could do is install twin proxy. That's actually a proxy for Redis  written by Twitter and the proxy will actually do the hashing for you and you connect to  the proxy and it will in itself pick the right server on the back end to get all those keys.  And you can use that in the.NET world as well. The only thing you have to do is say  there's a proxy involved so the client knows the proxy is there and it cannot do certain  operations because of that. Alright, how much time do I have left? 10 minutes. Okay, let's  do some patterns. I have a couple of them and I'll quickly get my visual studio just  to see which ones we have. We have counting in Redis. We have showing the latest items  from a list. We have rate limiting an API. I actually built a clone of WhatsApp, a messaging  application. I have alter completes. I have discovering friends in a social application  and I have sorting. Which ones do you want to see in these 10 minutes we have left? Counting.  That's one. Latest items. No one. Perfect. Rate limiting in an API. That's a little bit  more so let's make that the top one now. The WhatsApp clone, the messaging application.  Let's see even more. Alter completion. Okay. Then there's friends and sorting but I think  since the reaction was on those topics let's start with something like the rate limiting.  I think that's an interesting thing to do with caching. A lot of APIs that you see on  the internet actually have rate limiting involved. Let me also get the slide for that thing.  Here we are. A lot of APIs have this concept of doing rate limiting where you for example  say that a specific user can only access my API once a minute or a thousand times per  hour and things like that. How would you build something like that? You could use a database  for that but that means that just to make sure that you are blocking or unblocking your  users from doing certain things you actually have to do the work on the database. You have  to maintain counters. You have to do a lot of things there. So why not use Redis to do  this rate limiting? So imagine you want to only allow five requests per 60 seconds to  a specific API. What you could do is for every user or for every client or for every API  key that you have, store a list in Redis. In that list you would record the timestamp  of the latest request and whenever you hit the fifth item, whenever you only have like  three items or four items you're okay because there's only four requests being made at that  moment but if you have more, if you have this limit of five items in this case, what you  want to do is check the leftmost item in that list, see if it has expired or not, is it  still within this 60 second boundary or not? If it's not, perfect. Let's remove it and  add a new one and you're not being limited. If it's still within this 60 second boundary,  you just block the user because the list is full and there's no item that can be removed  in that case. Very simple approach and to demonstrate, I actually have that thing in  an API. Let's run it. I have a very simple Hello World API where I can give an API key  header to my server and I can have it say Hello World. So this would be my request. I would  just reach out to my API with a specific user agent and an API key and I just execute the  request against this server and you'll see it will work. It will not work. Did I just  start the wrong instance? Yes, I did start the wrong instance. Let's start our rate limiting.  All right, rate limiting. There we are. So let's try that once more. Let's hit our Hello  World application and you get back a response. 200, okay, 200, okay. Now let's do a lot of  requests and you'll see the requests are being blocked because I hit my rate limits because  I did more than five requests per, I think in this case I have it in a couple of seconds  just for demo purposes, but it's easy to do. I have rate limiting. How did I build this  thing? Quite easy. In my application, I actually have added a Trottling HTTP handler or they  are a delegating handler where I'm connecting to Redis. Whenever an API key header is present,  I'm actually getting my database and I'm getting the rate limit for a specific API key. So  per client, I would maintain a list. Then I would say, okay, what's the length of this  list? If it's lower than five, which will be my limit, no problem. Just let the request  go through but just add a new item on the left, which is the current timestamp or the  current ticks that I want to store. If I do have five requests or five items in this list,  I get the first item from that list or the last item of that list in this case. See if  it has expired or not and if it expired, I'm being trottled. If it hasn't, or if it did  expire, I'm just removing it from the right of my list to make sure my list is actually  shrinking and I don't have to do a lot of complicated logic to block my users from using this API  key or using this API. So that's a very simple approach on how you can use Redis for something  that you probably wouldn't use a cache in other scenarios. All right, let's do the autocomplete  because that was one of the very interesting things you were saying as well. So autocomplete,  how would you do autocompletion for a large set of products or dictionary or whatever?  There's a couple of ways you could do this. Just store them in a database and run a like  query on those and that will probably just work fine. But yeah, it's a bit why would  you spend your database on just storing that data and querying for that data? Another thing  could be using Lucene or Elasticsearch to do that, which may be overkill because it's  a full text indexing search thingy and not something to just autocomplete words. So let's  go with Redis. How would you store this thing in Redis? It could be a list, it could be  individual keys for every item you want to store. But I think one of the interesting data  types to use for this scenario is using assorted sets where you basically store the words or  the term that you want to autocomplete and a score of zero for every item. What's the  use of having a score of zero for every item if you're using this sorted set where I want  to do scoring for retrieving the data? Well, the second property or the second item Redis  sorts these items on is the actual string value of the key that you are storing, which  means if you want to do things like autocompletion, you could just say Redis, give me something  that looks like this. It will sort the items for you and get the topmost items or the best  items to store that data. So to give you a quick example of my autocompletes, I have  stored the entire English dictionary in a cache on Azure and I can search for things  like Apple and it will just give me all the words that I can complete and so on. I can  do pirates, I can do piranha, things like that. So you can see it's actually really  fast to search this entire English dictionary. How is it stored? Well, like I said, in assorted  sets. So let me quickly go over the codes. So to store the data, I have this populate  method, it's quick and dirty, but what I'm doing there is basically parsing a dictionary.txt  file and adding it to assorted sets. My assorted set is called autocomplete words. I'm storing  the word in there with a score of zero. Again, why the score of zero? Because Redis will  actually score the items based on the string value and not on the score in this case. So  to provide autocompletion, what I'm doing is I'm basically getting a range from my assorted  sets where my query is my actual words that I'm autocompleting or the substring that I  have already autocompleted. So I want to search for Apple. I'll just have a query which is  APP, for example. And I'm saying give me 20 items that are closest to that words and  Redis will just give me these 20 items if those items are discovered in the dictionary.  So it's really easy to do. It's very efficient and Redis is really good at this sort of thing  where you're not retrieving the entire English dictionary to search in the list for the autocomplete  words but you're making Redis do this heavy lifting of the searching for you. Of course,  it's not Lucene, it's not Elasticsearch, but for autocompletion, for example, it's a very  nice scenario. And I think I'm pretty much done now. So if you're interested in one of  the other patterns, come and find me and I'll be happy to show them to you. With that, I  want to thank you for being here and remember one thing which is the little Redis book.  That's the only takeaway you must have if you leave this room. If you want to start with  Redis, you think it's interesting, read that little book. Thank you.  Thank you.  You
Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_0_83.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_84_1379.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_1380_2231.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_2232_2915.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_2916_3131.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_3132_3191.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_3192_3575.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_3576_3623.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_3624_7211.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_7212_12359.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_12360_12479.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_12480_16415.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_16416_18251.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_18252_18491.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_18492_18575.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_18576_20711.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_20712_20903.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_20904_21179.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_21180_25727.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_25728_25775.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_25776_26519.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_26520_33131.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_33132_44615.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_44616_48851.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_48852_50795.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_50796_52727.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_52728_53687.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_53688_58115.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_58116_58199.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_58200_58475.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_58476_58727.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_58728_59843.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_59844_59903.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_59904_60539.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_60540_60635.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_60636_61715.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_61716_62795.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_62796_63863.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_63864_64811.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_64812_65231.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_65232_65747.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_65748_67307.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_67308_67919.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_67920_67979.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_67980_69983.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_69984_70055.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_70056_70379.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_70380_70427.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_70428_71375.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_71376_72011.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_72012_72779.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_72780_73775.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_73776_73835.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_73836_73883.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_73884_74663.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_74664_74723.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_74724_74771.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_74772_75503.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_75504_75779.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_75780_76067.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_76068_76775.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_76776_77027.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_77028_78011.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_78012_78059.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_78060_78983.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_78984_80555.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_80556_80675.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_80676_80891.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_80892_82175.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_82176_82235.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_82236_82367.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_82368_84539.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_84540_84587.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_84588_84671.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_84672_84899.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_84900_84983.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_84984_85139.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_85140_85427.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_85428_85739.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_85740_85823.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_85824_86051.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_86052_86375.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_86376_88223.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_88224_90383.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_90384_91091.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_91092_93251.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_93252_93311.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_93312_93359.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_93360_93407.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_93408_93803.png Get_more_than_a_cache_back__The_Microsoft_Azure__Redis__Cache_-_Maarten_Balliauw_93804_97014.png