Build Your Own Redis: Ping <-> Pong [2/4]
This is the second article in a series where we’ll build a toy Redis clone in Ruby. If you’d like to code-along, try the Build your own Redis challenge!
Previous article: Barebones TCP Server
Next article: Concurrent Clients
Sections in this article:
- The Redis Protocol
- Redis Commands via RESP
- Integrated Test
- Implementing PING
The Redis Protocol
In the previous article, we look at how to create a TCP server that can accept and handle clients. Redis clients talk to Redis using the Redis Protocol.
The Redis Protocol (we’ll refer to it as RESP from now on) is a serialization format that supports multiple data types like strings, integers and arrays.
The first byte in a reply denotes what data type we’re dealing with.
In this tutorial, we’ll only look at the following data types:
- Simple Strings (starting with
- Bulk Strings (starting with
- Arrays (starting with
For simple strings, the first byte is sent as
+. This is followed by the
actual string, and then a CLRF (
As an example, here’s what the string ‘OK’ looks like when encoded in RESP:
+- signifies that the data type is a simple string
OK- this is the string being sent
\r\n(CLRF) - this denotes the end of the string
Similarly, “PONG” would be encoded as:
One problem with simple strings is that they aren’t binary-safe. Simple strings use CLRF to denote the end of the string. What if the string you meant to send contained a CLRF?
+THIS HAS A \r\n INSIDE IT\r\n
RESP will decode the above as ‘THIS HAS A ‘, instead of ‘THIS HAS A \r\n INSIDE
IT’. When RESP encounters the
\r\n, it thinks the string has ended!
This is where bulk strings come in handy. In bulk strings, the caller specifies the number of bytes they intend to send before sending the string. This way, we just read a fixed number of bytes, instead of searching for a CLRF.
Here’s the string “HEY” encoded as a bulk string:
$- denotes that the data type is a bulk string
3- the number of bytes in the string
\r\n- delimiter, the actual string starts after this
HEY- the actual string
\r\n- final delimiter
Because we’re sending the number of bytes upfront, we can embed a CLRF in the response too, if needed.
The string “THIS CONTAINS A \r\n INSIDE IT” (28 bytes in length) can be encoded as so:
$30\r\nTHIS CONTAINS A \r\n INSIDE IT\r\n
Arrays in RESP can be arrays of any data type RESP supports, including arrays themselves. You can encode arrays of strings, arrays of arrays etc.
Arrays start with
*, followed by the number of elements in the array.
Here’s how the array
["hey", "there"] is encoded in RESP:
*- denotes that the data type is an array
2- the number of elements in the array
\r\n- delimiter, the actual elements start after this
"hey", encoded as a Bulk String
"there", encoded as a Bulk String
Redis Commands via RESP
Redis commands are sent as arrays of bulk strings in RESP.
PING is sent as an array with one element:
GET key is sent as so:
Encoded in RESP, this’ll look like:
The Redis server responds with any valid RESP data type.
Let’s take the
PING command as an example.
Here’s what it looks like when you execute
➜ redis-cli 127.0.0.1:6379> PING PONG
Under the hood, here’s what the client sent to the server:
["PING"], encoded as an array of Bulk Strings in RESP)
And here’s what the server sent back:
"PONG", encoded as a Simple String in RESP)
Let’s start out with an integrated test for the
PING command. We’ll use Ruby’s
redis-rb for this, since it already knows
to speak RESP.
require "redis" require "minitest/autorun" # 6379 for official redis, 6380 for ours SERVER_PORT = ENV["SERVER_PORT"] class TestRedisServer < Minitest::Test def test_responds_to_ping r = Redis.new(port: SERVER_PORT) assert_equal "PONG", r.ping end end
When I run this against the official Redis, tests pass:
➜ SERVER_PORT=6379 ruby tests/test_main.rb Run options: --seed 20154 # Running: . 1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
When I run this against our Redis (switched the port from 6379 to 6380), tests fail:
➜ SERVER_PORT=6380 ruby tests/test_main.rb Run options: --seed 17862 # Running: E 1) Error: TestMain#test_ping: Redis::TimeoutError: Connection timed out /home/rohitpaulk/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:71:in `rescue in _read_from_socket' ... /home/rohitpaulk/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/redis-4.1.3/lib/redis.rb:152:in `ping' tests/test_main.rb:11:in `test_ping' 1 runs, 0 assertions, 0 failures, 1 errors, 0 skips
The failure makes sense! Our server accepts connections, but it doesn’t send back replies to the client. The redis client hit a time-out when waiting for a reply.
Now that we have a failing test setup, it’s time to change our server implementation so that the test passes.
(For the uninitiated, we’re following TDD here)
We need to send
PONG back to a client when they send us the
To check what command the user has sent us, we’ll need to parse RESP and check
if the command we’ve received is indeed
Do we need to parse RESP though? To make our current set of tests pass, we
could just ignore what the user sent us, and send back
require "socket"class RedisServerdef initialize(port)@server = TCPServer.new(port)enddef listenloop doclient = @server.accept- # TODO: Serve client+ # TODO: Handle commands other than PING+ client.write("+PONG\r\n")endendend
Running the tests to check, and…
➜ SERVER_PORT=6380 ruby tests/test_main.rb Run options: --seed 20154 # Running: . 1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
Since our server only understands
PING at the moment, it was easy to just
ignore what the client sent and always send
If a client sends us
ECHO hey, we’ll still send
➜ redis-cli 127.0.0.1:6379> PING PONG 127.0.0.1:6379> ECHO hey PONG
This’ll work for now (yes, we cheated on the test). We’ll clean
this up when we work on handling other commands like
In the next article, we’ll look at how our current implementation is flawed when it comes to handling multiple clients.