Build Your Own Redis: Barebones TCP Server [1/4]

This is the first article in a series where we’ll build a toy Redis clone in Ruby. If you’d like to code-along, try the Build your own Redis challenge!

Previous article: Introduction

Next article: Ping <-> Pong

Sections in this article:

Introduction to TCP

As mentioned in the previous article, Redis clients talk to Redis by issuing commands over TCP, encoded using the Redis Protocol.

If you’re familiar with HTTP, you’re used to dealing with requests and responses. When your app receives a request - it is interpreted according the HTTP spec, and you get access to data in a parsed form. Headers is a dictionary, body is a stream of bytes etc.

HTTP is an application layer protocol. When it comes to TCP, think of it as though we’re operating at one layer beneath - the ‘transport’ layer. At this layer, the idea of a request or response doesn’t exist. There’s just bytes being sent from one program to another, and back.

The easiest way to experience this locally is to run a TCP server using netcat:

$ netcat -l -p 4000 # Accepting TCP connections on port 4000

While that is running, simulate a TCP client using netcat again in a separate tab:

$ netcat localhost 4000 # TCP client

You’ve now got a TCP client and server that are connected to each other. Write something in the first window, it’ll pop up in the other. And vice-versa.

Inspecting TCP sockets using lsof

You can use lsof to view a list of programs that are listening/connected to a specific port.

When I’ve got netcat running in listen mode, here’s what I see:

➜  rohitpaulk.com git:(master) ✗ lsof -i :5000
COMMAND   PID       USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
netcat  35237 rohitpaulk    3u  IPv4 441826      0t0  TCP *:commplex-main (LISTEN)

Once I’ve established a connection, here’s what I see:

➜  rohitpaulk.com git:(master) ✗ lsof -i :5000
COMMAND   PID       USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
netcat  35237 rohitpaulk    4u  IPv4 443259      0t0  TCP localhost:commplex-main->localhost:49966 (ESTABLISHED)
netcat  36058 rohitpaulk    3u  IPv4 446736      0t0  TCP localhost:49966->localhost:commplex-main (ESTABLISHED)

(Note that LISTEN has turned into ESTABLISHED)

Ruby’s TCP primitives

Now that we’ve seen how TCP connections work and are able to inspect them using lsof, let’s try to do this in Ruby.

require "socket"

server = TCPServer.new(5000)
client = server.accept # Blocking call, waits for a connection

When I run the above in IRB, it blocks at the server.accept line. Our script is now listening on port 5000!

Let’s try connecting to it in a separate irb session:

require "socket"

sock = TCPSocket.new(5000)

We’re now able to communicate over TCP in Ruby!

The TCPServer we created is capable of handling multiple clients at a time. Every call to server.accept will yield a new client that is ready to connect (or block until a client is available).

Implementing the TCP server

Let’s now boil down what we learnt into a RedisServer class.

require "socket"

class RedisServer
  def initialize(port)
    @server = TCPServer.new(port)
  end

  def listen
    loop do
      client = @server.accept
      # TODO: Serve client
    end
  end
end

Our server is capable of accepting clients on a specific port, but it doesn’t actually talk to clients yet. Let’s change that in the next article: Build Your Own Redis: PING <-> PONG!