Key-Value Stores

Notes from my learning about simple NoSQL storage solutions.
  • 10


    Tokyo Cabinet's Key-Value Database Types


    The B+Tree Database



    Read more…

  • 1


    Tokyo Cabinet as a Key-Value Store


    Getting and Setting Keys


    #!/usr/bin/env ruby -KU
    require "oklahoma_mixer""data.tch") do |db|
        db[:one] = 1
        db[:two] = 2
        db["users:1"] = "James"
        db["users:2"] = "Ruby"
        puts "Reading data."
        %w[ db[:one]
            db.keys(:limit\ =>\ 2)

    Read more…

  • 17


    Where Redis is a Good Fit

    Like any system, Redis has strengths and weaknesses. Some of the biggest positives with Redis are:

    • It's wicked fast. In fact, it may just be the fastest key-value store.
    • The collection types and the atomic operations that work on them allow you to model some moderately complex data scenarios. This makes Redis fit some higher order problems where a simple key-value store wouldn't quite be enough.
    • The snapshot data dumping model can be an asset. You get persistence with Redis, but you pay a minimal penalty for it.

    Of course, there are always some minuses. These are the two I consider the most important:

    • Redis is an in-memory data store, first and foremost. That means your entire dataset must fit completely in RAM and leave enough breathing room for anything else the server must do.
    • Snapshot backups are not perfect. If Redis fails between snapshots, you can lose data. You need to make sure that's acceptable for any application you use it in.

    It may seem weird to call snapshots both a pro and a con, but it does work for you in some ways and against you in others. You have to decide where the trade-off is worth it.

    Read more…

  • 16


    Lists and Sets in Redis

    [Update: though all of the techniques I show here still apply, many methods of the Redis gem have changed names to match the actual Redis commands they call. There are also easier and more powerful ways to do some of what I show in here, thanks to additions to Redis.]

    Redis adds one huge twist to traditional key-value storage: collections. Supporting both lists and sets through some very powerful atomic operations allows for advanced key-value usage.


    Redis allows a single key to hold a list of values. This is your typical ordered list with the operations you would expect: appending, indexed access, and access to a range of values.

    This has many potential uses. I'll cover two that I think will be very common. First, if you are going to use Redis as a full database, you store things that are naturally a list of items, like comments, in a real list. Let's look at some code:

    #!/usr/bin/env ruby -wKU
    require "redis"
    CLEAR = `clear`
    # create an article to comment on
    db                =
    article_id        = db.incr("global:next_article_id")
    article           = "article:#{article_id}"
    class << article
      def method_missing(field, *args, &blk)
        return super unless field.to_s !~ /[!?=]\z/ && args.empty? && blk.nil?
    db[article.title] = "My Favorite Language"
    db[article.body]  = "I love Ruby!"
    # initialize some session details
    comments_per_page = 2
    comment_page      = 1
    login             = ARGV.shift || "JEG2"
    loop do
      # show article
      print CLEAR
      puts "#{db[article.title]}:"
      puts "  #{db[article.body]}"
      # paginate comments
      start      =  comments_per_page * (comment_page - 1)
      finish     =  start + comments_per_page - 1
      comments   =  db.list_range(article.comments, start, finish)
      pagination =  Array( ? nil : "(p)revious")
      pagination << "(n)ext" if db.list_length(article.comments) - 1 > finish
      # show comments
      comments.each do |comment|
        posted, user, body = comment.split("|", 3)
        puts "----"
        puts "  #{body}"
        puts "  posted by #{user} on #{posted}"
      # handle commands
      print "Command? [#{(%w[(c)omment (q)uit] + pagination).join(', ')}]  "
      case (command = gets)
      when /\Ac(?:omment)?\Z/i  # add a comment
        print "Your comment?  "
        comment = gets or break
        posted  ='%m/%d/%Y at %H:%M:%S')
        db.push_tail(article.comments, "#{posted}|#{login}|#{comment.strip}")
      when /\Ap(?:revious)?\Z/i  # view previous page of comments
        if pagination.first =~ /\A\(p\)/
          comment_page -= 1
          puts "You are on the first page of comments."
          gets or break
      when /\An(?:ext)?\Z/i  # view next page of comments
        if pagination.last =~ /\A\(n\)/
          comment_page += 1
          puts "You are on the last page of comments."
          gets or break
      when /\Aq(?:uit)?\Z/i, nil  # exit program

    Read more…

  • 15


    Using Redis as a Key-Value Store


    Redis is a first and foremost a server providing key-value storage. As such, the primary features of any client library are for connecting to the server and manipulating those key-value pairs.

    Connecting to the Server

    Connecting to the Redis server can be as simple as, thanks to some defaults in both the server and Ezra's Ruby client library for talking to that server. I won't pass any options to the constructor calls below, but you can use any of the following as needed:

    • :host if you need to connect to an external host instead of the default
    • :port if you need to use something other than the default port of 6379
    • :password if you configured Redis to require a password on connection
    • :db if you want to select one of the multiple configured databases, other than the default of 0 (databases are identified by a zero-based index)
    • :timeoeut if you want a different timeout for Redis communication than the default of 5 seconds
    • :logger if you want the library to log activity as it works

    Read more…

  • 14


    Setting up the Redis Server

    Before we can play with Redis, you will need to get the server running locally. Luckily, that's very easy.

    Installing Redis

    Building Redis is a simple matter of grabbing the code and compiling it. Once built, you can place the executables in a convenient location in your PATH. On my box, I can do all of that with these commands:

    curl -O
    tar xzvf redis-1.0.tar.gz 
    cd redis-1.0
    sudo cp redis-server redis-cli redis-benchmark /usr/local/bin

    Those commands build version 1.0 of the server, which is the current stable release as of this writing. You may need to adjust the version numbers down the road to get the latest releases though.

    I also copied the executables to where I prefer to have them: /usr/local/bin. Feel free to change that directory in the last command to whatever you prefer.

    If you will be talking to Redis from Ruby, as I will show in all of my examples, you are going to need a client library. I recommend Ezra Zygmuntowicz's redis-rb. You can install that gem with:

    Read more…

  • 14


    Using Key-Value Stores From Ruby



    1. Setting up the Redis Server
    2. Using Redis as a Key-Value Store
    3. Lists and Sets in Redis
    4. Where Redis is a Good Fit

    Tokyo Cabinet, Tokyo Tyrant, and Tokyo Dystopia

    1. Installing the Tokyo Software
    2. Tokyo Cabinet as a Key-Value Store
    3. Tokyo Cabinet's Key-Value Database Types
    4. Tokyo Cabinet's Tables
    5. Threads and Multiprocessing With Tokyo Cabinet
    6. Tokyo Tyrant as a Network Interface
    7. The Strengths of Tokyo Cabinet