Lua 脚本功能是 Reids 2.6 版本的最大亮点， 通过内嵌对 Lua 环境的支持， Redis 解 决了长久以来不能高效地处理 CAS （check-and-set）命令的缺点， 并且可以通过组合使 用多个命令， 轻松实现以前很难实现或者不能高效实现的模式。
上文基本就是对 Redis Lua scripting 的蹩脚翻译。
本篇先介绍一下服务端脚本是如何使用的，然后对 redis 用于脚本支持的代码进行分析， 以了解如何在服务器中嵌入 lua 虚拟机和如何和其进行交互。
Usage
运行脚本的命令：
-
EVAL- used to evaluate scripts using the Lua intepreter built into Redis.-
The first argument is a Lua 5.1 script. The script does not need to define a Lua function (and should not). A
chunk, in Lua terminology.eval "local function f() return 1 end return f()" 0 (integer) 1
-
The second argument is the number of arguments that follows the script. The arguments can be accessed by Lua using the global variable
KEYS. -
The rest of the arguments can be accessed by Lua using the global variable
ARGV. -
Using
redis.call()andredis.pcallto call Redis commands from a Lua script. The arguments of theredis.call()andredis.pcall()functions are simply all the arguments of a well formed Redis command.eval "return redis.call('set', KEYS[1], ARGV[1])" 1 foo bar OK
-
Lua scripts can return a value, that is converted from the Lua type to the Redis protocol using a set of conversion rules. If a Redis type is converted into a Lua type, and then the result is converted back into a Redis type, the result is the same as of the initial value.
- There is not simple way to have nils inside Lua arrays, this is a result of Lua table semantics, so when Redis converts a Lua array into Redis protocol the conversion is stopped if a nil is encourtered.
-
-
EVALSHA- used to evaluate scripts using the Lua interpreter built into Redis.EVALSHAworks exactly likeEVAL, but instead of having a script as the first argument it has the SHA` digest of a script.-
If the server still remembers a script with a matching SHA1 digest, the script is executed.
-
If the server does not remember a script with this SHA1 digest, a special error is returned telling the client to use
EVALinstead.
-
Redis 使用唯一 Lua 解释器运行所有脚本，并且保证脚本执行的原子性：脚本正在运行期 间，Redis 不会执行任何其它命令。
另外一方面，对脚本原子性的保证，在执行较慢的脚本时，会降低整体的吞吐率。
管理脚本的命令：
SCRIPT FLUSH- only way to force Redis to flush the script cache.SCRIPT EXISTS sha1- check whether the scripts are still in Redis's cache.SCRIPT LOAD script- register the specified script without executing it.SCRIPT KILL- only way to interrupt a long-running script that reach the configured maximum execution time for scripts.
如果脚本运行时间超出设置的最大运行时长后，Redis 开始接收并处理 SCRIPT KILL 和
SHUTDOWN NOSAVE 命令。
SCRIPT KILL 只能用来停止只执行了 读操作 的脚本。如果脚本已经执行了写操作，客
户端只能使用 SHUTDOWN NOSAVE 命令来关闭 Redis 服务端进程以保证磁盘数据的一致
性。
Limitation
Redis 中的脚本会被复制到其它 slave 上，同时也会被原样写入 AOF 文件。这样做的好处 是节省了带宽 (直接传输脚本本身比传输脚本生成的命令开销要小)。但是，这样的做法带 来的问题是 Redis 需要对脚本进行一定约束：
The script always evaluates the same Redis write commands with the same arguments given the same input data set. Operations performed by the scripts cannot depend on any hidden information or state that may change as script execution proceeds or between different exectuions of the script, nor can it depend on any external input from I/O devices.
为了实现上述约束，Redis 对 Lua 运行环境和脚本的行为做了以下限制：
-
Lua does not export commands to access the system time or other external state.
-
Redis will block the script with an error if a script calls a Redis command able to alter the data set after a Redis random command like RANDOMKEY, SRANDMEMBER, TIME. This means that if a script is read-only and does not modify the data set it is free to call those commands. Note that a random command does not necessarily mean a command that uses random numbers: any non-deterministic command is considered a random command (the best example in this regard is the TIME command).
-
Redis commands that may return elements in random order, like SMEMBERS (because Redis Sets are unordered) have a different behavior when called from Lua, and undergo a silent lexicographical sorting filter before returning data to Lua scripts. So redis.call("smembers",KEYS[1]) will always return the Set elements in the same order, while the same command invoked from normal clients may return different results even if the key contains exactly the same elements.
-
Lua pseudo random number generation functions math.random and math.randomseed are modified in order to always have the same seed every time a new script is executed. This means that calling math.random will always generate the same sequence of numbers every time a script is executed if math.randomseed is not used.
-
Redis scripts are not allowed to create global variables, in order to avoid leaking data into the Lua state. If a script needs to maintain state between calls (a pretty uncommon need) it should use Redis keys instead. In order to avoid using globals variables in your scripts simply declare every variable you are going to use using the local keyword.
同时，Redis 的 Lua 运行环境，提供了有限的模块支持。同时，Redis 保证这些模块在 各个 Redis 实例中都是一样的。这样就保证了脚本代码在各个 Redis 实例中行为的一致 性。
Redis Lua 运行环境提供的模块有: base, table, string, math, debug,
cjson, struct, cmsgpack。
Implementation
了解了 Redis 脚本相关操作和脚本限制后，再来分析一下 Redis 是如何实现上面提到的这 些特性的。
Lua API 的设用方法和与Lua交互的规范 (virtual stack 是如何使用的等)，可参阅 Programming in Lua；Lua API 的详细说明， 请参阅 Lua API。
同时，需要注意的是，Redis 编译时会链接自带的 lua 代码编译出的静态库。同时，Redis
对 lua 源代码进行了扩展，它将 cjson，struct，cmsgpack 变成了内置模块。
Lua 运行环境的初始化函数是 scriptingInit，在 Redis 启动时，被 initServer
函数调用。
----------------scripting.c:527-----------------
void scriptingInit(void) {
lua_State *lua = lua_open();
luaLoadLibraries(lua);
luaRemoveUnsupportedFunctions(lua);
...
lua_newtable(lua);
/* redis.call */
lua_pushstring(lua, "call");
lua_pushcfunction(lua, luaRedisCallCommand);
lua_settable(lua, -3);
...
/* Finally set the table as 'redis' global var. */
lua_setglobal(lua, "redis");
/* Replace math.random and math.randomseed with our implementaions. */
lua_getglobal(lua, "math");
lua_pushstring(lua, "random");
lua_pushcfunction(lua, redis_math_random);
lua_settable(lua, -3);
lua_pushstring(lua, "randomseed");
lua_pushcfunction(lua, redis_math_randomseed);
lua_settable(lua, -3);
lua_setglobal(lua, "math");
/* Add a helper function that we use to sort the multi bulk output
* of non deterministic commands, when containing 'false' elements. */
{
char *compare_func = "function __redis__compare_helper(a, b)\n"
...;
luaL_loadbuffer(lua, compare_func, strlen(compare_func), "@cmp_func_def");
lua_pcall(lua, 0, 0, 0);
}
/* Create the (non connected) client that we use to execute Redis commands
* inside the Lua interpreter.
* Note: there is no need to create it again when this function is called
* by scriptingReset(). */
if (server.lua_client == NULL) {
server.lua_client = createClient(-1);
server.lua_client->flags |= REDIS_LUA_CLIENT;
}
/* Lua beginners often don't use "local", this is likely to introduce
* subtle bugs in their code. To prevent problems we protect accesses
* to global variables. */
scriptingEnableGlobalsProtection(lua);
server.lua = lua;
}
对上述代码的补充说明：
- 关于 Lua API 的调用规范和细节说明，参阅上面的两个链接。
- 上面代码完成的工作有：
- 创建一个新的 Lua 解释器 (
lua_State) - 加载类库
base,table,string,math,debug,cjson,struct,cmsgpack。上文说过，cjson,struct和cmsgpack是 Redis 添加到 lua 核心代码中的类库。 - 禁用可能带来安全隐患的函数，比如
loadfile等。 - 在全局空间创建包含有
call,pcall,log等 Redis 自定义函数的table， 并命名为redis。这样，在EVAL指令中的脚本就可以直接完成像redis.call这样的调用了。 - 重新定义对 Redis 来讲不安全的函数
random和randomseed。 - 在全局空间，定义函数
__redis__compare_helper。 - 创建 fake client，这样脚本使用 Redis 命令时就能复用普通连接的命令执行 逻辑了。
- 开启"保护模式" - 禁止脚本声明全局变量。
- 创建一个新的 Lua 解释器 (
Redis 初始化完成后，开始监听客户端请求。当客户端调用 EVAL, EVALSHA 等命令时，
Redis 才会调用脚本模块进行处理。
对客户端请求的接收、命令解析等，本篇不作讨论。下面只将函数调用链罗列如下：
/* redis initialization */
acceptTcpHandler
anetTcpAccept
acceptCommonHandler
createClient
readQueryFromClient
/* wait read event */
/* when data arrives */
readQueryFromClient
processInputBuffer
processCommand
lookupCommand
call
proc /* function pointer */
/* for command `EVAL`, `proc` points to `evalCommand` */
evalCommand
evalGenericCommand
/* for command `EVALSHA`, `proc` points to `evalShaCommand` */
evalShaCommand
evalGenericCommand
最后，evalGenericCommand 就是我们关心的在 lua 解释器上执行命令上传的脚本的
入口函数。
-----------scripting.c:787-----------------
void evalGenericCommand(redisClient *c, int evalsha) {
lua_State *lua = server.lua;
...
funcname[0] = 'f';
funcname[1] = '_';
if (!evalsha) {
/* Hash the code if this is an EVAL call */
sha1hex(funcname + 2, c->argv[1]->ptr, sdslen(c->argv[1]->ptr));
} else {
/* We already have the SHA if it is a EVALSHA */
...
}
/* Try to lookup the Lua function */
lua_getglobal(lua, funcname);
if (lua_isnil(lua, 1)) {
lua_pop(lua, 1); /* remove the nil from the stack */
/* Function not defined... let's define it if we have the
* body of the function. If this is an EVALSHA call we can just
* return an error. */
if (evalsha) {
addReply(c, shared.noscripterr);
return;
}
if (luaCreateFunction(c, lua, funcname, c->argv[1]) == REDIS_ERR) return;
lua_getglobal(lua, funcname);
redisAssert(!lua_isnil(lua, 1));
}
/* Populate the argv and keys table accordingly to the arguments that
* EVAL received. */
luaSetGlobalArray(lua, "KEYS", c->argv + 3, numkeys);
luaSetGlobalArray(lua, "ARGV", c->argv + 3 + numkeys, c->argc - 3 - numkeys);
...
/* At this point whatever this script was never seen before or if it was
* already defined, we can call it. We have zero arguments and expect
* a single return value. */
err = lua_pcall(lua, 0, 1, 0);
...
lua_gc(lua, LUA_GCSTEP, 1);
if (err) {
addReplyErrorFormat(c, "Error running script (call to %s): %s\n",
funcname, lua_tostring(lua, -1));
lua_pop(lua, 1); /* Consume the Lua reply. */
} else {
luaReplyToRedisReply(c, lua);
}
...
}
对上述代码的补充说明：
- Redis 针对整个脚本字符计算 sha1。
- Redis 由客户端上传的脚本在 lua 全局作用域中创建
lua函数，函数名为f_sha1。 此函数不接收参数，并且只能有一个返回值。 KEYS和ARGV的个数并不需要相等。Redis 根据其值在 lua 全局作用域中创建 table。luaReplyToRedisReply将 lua 函数的返回值，转换成 Redis 类型。
Redis 脚本的处理和调用逻辑到此就算完成了。
如果脚本需要使用 Redis 命令 (大部分应用场景都需要脚本和 Redis 进行交互) 的话，就
需要使用 redis.call 和 redis.pcall 等 (还有 redis.log 等等的函数) 在初始化
环节注册到 lua 环境中的函数。
下面以命令 EVAL "return redis.call('set', KEYS[1], ARGV[1])" 1 foo bar 为线索
对 Redis 提供的 redis.call 函数的执行逻辑和如何从 lua 环境调用 C 函数进行分析
(上面分析的过程，相当于是在 C 函数中如何执行 lua 代码)。这样一来，lua 和宿主语言
的两个调用方向才算是完整了。
These two views of Lua (as an extension language and as an extensible language) correspond to two kinds of interaction between C and Lua. In the first kind, C has the control and Lua is the library. The C code in this kind of interaction is what we call application code. In the second kind, Lua has the control and C is the library. Here, the C code is called library code. Both application code and library code use the same API to communicate with Lua, the so called C API.
在 lua 环境的初始化环节，Redis 将 luaRedisCallCommand 注册为 lua 环境的
redis.call 函数。
lua_pushstring(lua, "call");
lua_pushcfunction(lua, luaRedisCallCommand);
lua_settable(lua, -3);
luaRedisCallCommand 函数实现如下：
-------------scripting.c:338-------------
int luaRedisCallCommand(lua_State *lua) {
return luaRedisGenericCommand(lua, 1);
}
-------------scripting.c:192-------------
int luaRedisGenericCommand(lua_State *lua, int raise_error) {
int j, argc = lua_gettop(lua);
struct redisCommand *cmd;
robj **argv;
redisClient *c = server.lua_client;
...
/* Build the arguments vector */
argv = zmalloc(sizeof(robj *) * argc);
for (j = 0; j < argc; j++) {
if (!lua_isstring(lua, j+1)) break;
argv[j] = createStringObject((char *) lua_tostring(lua, j + 1),
lua_strlen(lua, j + 1);
}
...
/* Setup our fake client for command execution */
c->argv = argv;
c->argc = argc;
/* Command lookup */
cmd = lookupCommand(argv[0]->ptr);
...
/* Run the command */
c->cmd = cmd;
call(c, REDIS_CALL_SLOWLOG | REDIS_CALL_STATS);
...
redisProtocolToLuaType(lua, reply);
...
return 1;
}
对以上代码的补充说明：
-
Redis 接收到
EVAL命令后，调用evalGenericCommand将脚本交给 Lua 解释器执 行。Lua 解释器执行return redis.call('set' KEYS[1], ARGV[1])。由于，redis.call又由luaRedisCallCommand，这时，Lua 解释器再调用此函数交命令交由 Redis 执行。 -
redis.call中使用的命令SET foo bar由 fake client 作为载体执行。 -
命令执行结束后，Redis 将执行结果使用
redisProtocolToLuaType封装成 Lua 类型 并通过 Lua 调用协议写入 Lua stack。
Redis 脚本的管理命令基本和 Lua 运行环境关系不大，在此就不再赘述了。
Comments
不要轻轻地离开我，请留下点什么...