memcached SASL support
After the initial announcement
by Dustin Sallings that the memcached server now has SASL support, (which is a feature in the newly released 1.4.3, announced just today) this
past week has also resulted in a flurry of other developments, particularly in the client realm: A
couple weeks ago Dustin added support for SASL in spymemcached (Java)
and as of this week, Trond Norbye added SASL support to libmemcached,
now making it possible to take advantage of the memcached server's SASL
support for your applications that use memcached! So now you probably want to know: "How can I use
these new features?". This post will attempt to show you how you can
get SASL working with memcached, as well as explain the mechanics of
just what SASL is and how it works with memcached to provide authentication that your applications can take advantage of.
What is SASL?
SASL stands for Simple Authentication and Security Layer, which
provides you a means of adding authentication support to
connection-based protocols, such as the memcached client binary
protocol in this case. The way SASL works is that there is a function
(API) call that is added to the client that provides a functionality
for identifying and authenticating a given user. SASL also provides the
means to implement protection throughout the entire connection. Once a
user is identified and authenticated, SASL provides a security layer
between the client protocol and connection.
How is SASL implemented for memcached?
Until the the binary protocol was added as well as the SASL
implementation, memcached had no authentication layer whatsoever. Any
client could connect to memcached perform any operation at will. It
was up to the memcached user (usually a web developer) to make sure
that their network was locked down and that their application was designed
securely. Then the binary protocol was added to memcached which made for
a more efficient and compact client-server connection. This made it
possible to also add SASL support to memcached. With SASL support,
functions were added to the server that made is so a client connection
was forced to authenticate before the complete connection could
continue. This of course required a client that sent the authentication
information using SASL. At first, only spymemcached client (java) was
support, but as already mentioned in the introduction, libmemcached now
also has SASL support.
Figure 1: Basic diagram of memcached SASL
Basically, SASL uses some sort of storage for user credentials-- it can be LDAP or SQL database. In this example, the user credentials are stored in a database file on the server that memcached will be running on. In order for a user to be able to connect to, memcached server won't allow the user to connect unless they provide their credentials, and those credentials must match the user credentials stored in the SASL database. More information on the SASL specifics of the implementation for memcached can be found here.
How do I install SASL with memcached?
Well, the first thing you need to do is get the latest memcached server. One place you'll find this is the memcached github repository or memcached.org, explained in the next step. You also will
need a few prerequisites on your server such as sasl development libraries and as well as SASL utilities such as saslpasswd. On
Ubuntu, for instance, the packages are installed as:
sudo apt-get -f install libsasl2-2 sasl2-bin libsasl2-2 libsasl2-dev libsasl2-modules
Not to forget, the usual requirements for memcached: libevent
Just search accordingly for your packages according to the OS you are using.
How do I set up SASL memcached?
Now you will need to obtain SASL memcached. The code is now officially released from memcached.org
wget http://memcached.googlecode.com/files/memcached-1.4.3.tar.gz
tar xvzf memcached-1.4.3.tar.gz
Or as always it is available via Github
git clone git://github.com/memcached/memcached.git
Now compile and install memcached with SASL enabled:
cd memcached-1.4.3
./configure --enable-sasl
make
make test
sudo make install
Next, you will set up the SASL database and memcached application
configuration file. There are two ways to do this: system-wide or for a
specific user.
System-wide SASL database file:
For a system-wide SASL set-up, you will need set up a memcached.conf
file to point to the SASL database file which contains the user and
password information in /usr/lib/sasl2/memcached.conf, named per application. An example of this is:
mech_list: plain
log_level: 5
Note: on Ubuntu, this database file is already in existence. If you
have to have one created, make sure whatever directory you specify for
the location of the sasl database exists prior to running the next
command)
To add a user to the SASL database file:
sudo saslpasswd2 -c -a memcached
As you can see, -a specifies the application,
'memcached', which needs to match the name you gave the conf file in
the previous step 'memcached.conf'. When you run this saslpasswd2
command, you will be prompted for a password and password verification
as shown below:
patg@ishvara:~/code-dev/memcached-dustin$ sudo saslpasswd2 -c -a memcached capttofu
Password:
Again (for verification):
You can verify that the user you created now exists. To do so, run:
sudo sasldblistusers2
-or-
sudo sasldblistusers2 -f /etc/sasldb2
You'll see an output such as:
patg@ishvara:~/code-dev/memcached-dustin$ sudo sasldblistusers2
test1@ishvara: userPassword
User-specific SASL database file:
The first thing you need to ensure is that you set a very important environment variable, SASL_CONF_PATH whenever you run memcached. In this example, this path will be set to /home/patg/sasl.
export SASL_CONF_PATH=/home/patg/sasl
Make sure to create whatever directory path you specify!
Next, you will set up the memcached.conf file for SASL (named per application). In this example, the contents are:
mech_list: plain
log_level: 5
sasldb_path: /home/patg/sasl/sasldb2
Next you will have to create the database file that you just specified in the previous step in your memcached.conf file.
sudo saslpasswd2 -c -a memcached -f /home/patg/sasl/sasldb2 capttofu
Notice that the -a flag specifies the name of the application memcached, which must match to the name of the config file you
specified in the previous example above, memcached.conf. Also notice
that this differs from the system-wide example because you have to
specify the SASL database file in this example with the -f flag. When
you run saslpasswd2, you will be prompted to enter the password and
password verification, as shown in the example below:
patg@ishvara:~$ saslpasswd2 -c -a memcached -f /home/patg/sasl/sasldb2 capttofu
Password:
Again (for verification):
Now to run sasldblistusers2 to verify that you in fact added the
user. As with saslpasswd2, you will have to specify the SASL database
file
patg@ishvara:~$ sasldblistusers2 -f /home/patg/sasl/sasldb2
capttofu@ishvara: userPassword
Running SASL-enabled memcached
Using either a system-wide sasl database file or user-level SASL
database file, you will need to start memcached with the appropriate
flags. When memcached runs with SASL enabled, the binary protocol is
used, text protocol turned off. In this example, a user-level SASL
database file is used. As the user 'patg', the following was done:
export SASL_CONF_PATH=/home/patg/sasl
/usr/local/bin/memcached -S -vvv
Now to use memcached, you will need a SASL-enabled client, which as
announced in this post, spymemcached (Java) and libmemcached support
now. For simplicity (at least for me!) I hacked up one of the tests
that come with memcached, binary-sasl.t, and renamed it patg.t (Reminds
me I need to add SASL support to Cache::Memcached!). I also had to
export an environment variable that the test suite code uses to state
that you use an existing running memcached server:
export T_MEMD_USE_DAEMON=localhost:11211
Then I ran my test:
patg@ishvara:~/code-dev/memcached-dustin$ perl t/patg.t
1..20
returning handle
ok 1 - started the server
ok 2 - Proper version: 0
ok 3 - list_mechs CRAM-MD5 PLAIN
ok 4 - this fails to authenticate
ok 5 - error code matches
ok 6 - this fails to authenticate
ok 7 - error code matches
ok 8 - this fails to authenticate
ok 9 - error code matches
ok 10 - this fails to authenticate
ok 11 - error code matches
ok 12 - bad mech
ok 13 - bad auth
ok 14 - authenticated
ok 15
ok 16 - somevalue = somevalue
ok 17
ok 18
ok 19 - somevalue = somevalue
ok 20
Of interest and worth pointing out, notice the list_mechs test. This
test lists what your SASL authentication mechanisms your server is
configured for. What is of interest to me here is that I see my running
server authenticating, since I started the test with -vvv.
A failed authentication:
<30 0x00 0x00 0x00 0x00
authenticated() in cmd 0x21 is true
30: going from conn_parse_cmd to conn_nread
mech: ``PLAIN'' with 23 bytes of data
sasl result code: -13
Unknown sasl response: -13
>30 Writing an error: Auth failure.
A successful authentication:
authenticated() in cmd 0x21 is true
30: going from conn_parse_cmd to conn_nread
mech: ``PLAIN'' with 16 bytes of data
sasl result code: 0
A successful GET using authentication:
authenticated() in cmd 0x00 is true
30: going from conn_parse_cmd to conn_nread
<30 GET x
> FOUND KEY xL 0 9
somevalue
ofu
>30 Writing bin response:
The server is working!
Now to try out SASL memcached using libmemcached with SASL support.
Installing libmemcached
The first thing to install libmemcached with Trond's new SASL
functionality is to clone his tree from Launchpad. You will need the
bazaar revision control system for this. For Ubuntu, it is:
apt-get install bzr
Next, clone the tree:
bzr clone lp:~trond-norbye/libmemcached/sasl_rfe_462250
cd sasl_rfe_462250/
sh config/bootstrap
./configure # note, you do not have to use --enable-sasl because SASL will be compiled by default
make
make test
sudo make install
This installs libmemcached. Now, how do you use it? Well, you will
have to write programs to utilize it. For my enjoyment, I wrote a
simple C program that uses the information Trond provided on his post.
I called it sasl_test.c which you can find on Northscale's downloads. The crux of what it contains is shown below.
I have a main function that contains these important lines:
/* initialize the client connection to use SASL */
if (sasl_client_init(NULL) != SASL_OK)
{
fprintf(stderr, "Failed to initialize sasl library!\n");
return 1;
}
memcached_st *memc = memcached_create(NULL);
/* set the sasl callbacks for sasl auth */
memcached_set_sasl_callbacks(memc, sasl_callbacks);
memcached_server_st *servers = memcached_servers_parse(servers_list);
memcached_server_push(memc, servers);
memcached_server_list_free(servers);
/* you have to use binary protocol to use SASL */
memcached_behavior_set(memc, MEMCACHED_BEHAVIOR_BINARY_PROTOCOL, 1);
/* set a value */
rc= memcached_set(memc, key, keylen, value, vallen, 0, 0);
if (rc == MEMCACHED_SUCCESS) printf("setting key %s value %s\n", key, value);
assert(rc == MEMCACHED_SUCCESS);
/* get a value */
retval= memcached_get(memc, key, keylen, &retlen, (uint32_t)0, &rc);
if (rc == MEMCACHED_SUCCESS) printf("fetched key %s value %s\n", key, retval);
assert(rc == MEMCACHED_SUCCESS);
/* free the connection */
memcached_free(memc);
sasl_done();
Then, per Trond's instructions (lifted from his page), added callbacks, which I declare in another file I have sasl_test.h
static int get_username(void *context, int id, const char **result,
unsigned int *len);
static int get_password(sasl_conn_t *conn, void *context, int id,
sasl_secret_t **psecret);
static sasl_callback_t sasl_callbacks[] = {
{
SASL_CB_USER, &get_username, NULL
}, {
SASL_CB_AUTHNAME, &get_username, NULL
}, {
SASL_CB_PASS, &get_password, NULL
}, {
SASL_CB_LIST_END, NULL, NULL
}
};
Then I defined these in callbacks sasl_test.c (notice the password is set according to what was added to the sasl db file):
static char *username = "capttofu";
static char *passwd = "s3kr1t";
static int get_username(void *context, int id, const char **result,
unsigned int *len)
{
if (!result || (id != SASL_CB_USER && id != SASL_CB_AUTHNAME)) {
return SASL_BADPARAM;
}
*result= username;
if (len) {
*len= (username == NULL) ? 0 : (unsigned int)strlen(username);
}
return SASL_OK;
}
static int get_password(sasl_conn_t *conn, void *context, int id,
sasl_secret_t **psecret)
{
static sasl_secret_t* x;
if (!conn || ! psecret || id != SASL_CB_PASS) {
return SASL_BADPARAM;
}
if (passwd == NULL) {
*psecret = NULL;
return SASL_OK;
}
size_t len = strlen(passwd);
x = realloc(x, sizeof(sasl_secret_t) + len);
if (!x) {
return SASL_NOMEM;
}
x->len = len;
strcpy((void *)x->data, passwd);
*psecret = x;
return SASL_OK;
}
I compile this with the following command:
gcc -g -O0 -I/usr/local/include/libmemcached -lmemcached -lmemcachedutil -o sasl_test sasl_test.c
Then I can run the test!
./sasl_test localhost:11211 testkey testvalue
If I observe my SASL-enabled memcached server, I see that SASL authentication works.
Connection:
<30 new binary client connection.
30: going from conn_new_cmd to conn_waiting
30: going from conn_waiting to conn_read
30: going from conn_read to conn_parse_cmd
Authentication:
<30 Read binary protocol data:
<30 0x80 0x20 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
authenticated() in cmd 0x20 is true
Set, authenticated:
30: going from conn_read to conn_parse_cmd
<30 Read binary protocol data:
<30 0x80 0x01 0x00 0x07
<30 0x08 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x18
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
authenticated() in cmd 0x01 is true
30: going from conn_parse_cmd to conn_nread
<30 SET testkey Value len is 9
> FOUND KEY testkey0 0 10
test value
ofu
Get, authenticated:
30: going from conn_read to conn_parse_cmd
<30 Read binary protocol data:
<30 0x80 0x0c 0x00 0x07
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x07
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
<30 0x00 0x00 0x00 0x00
authenticated() in cmd 0x0c is true
30: going from conn_parse_cmd to conn_nread
<30 GET testkey
> FOUND KEY testkey0 0 9
testvalue
ttofu
>30 Writing bin response:
And it works! Now I can modify other libmemcached-based programs to use SASL.
Of course, you could also use spymemcached for using SASL as well.
Summary
With this post, you should come away having a better understanding
of what SASL is, and how you can use it with memcached. You should be
fully able to now obtain, compile and install both the memcached server
as well as the client, libmemcached with SASL enabled, as well as be
able to start writing memcached applications that use libmemcached to
use SASL.
Have fun!
