[聚合文章] 使用varnish 4代理处理jsonp接口

背景

使用varnish是一个很不错的HTTP加速方案，挪威最大的在线报纸 Verdens Gang 使用3台Varnish代替了原来的12台Squid，性能比以前更好。然而varnish默认情况下是以url进行hash，来标识缓存，所以对于jsonp这种带有callback参数的请求，每一次callback都不一样，很可能会生成大量重复数据，占用内存空间，浪费资源。最近就遇到了这个问题，好在这个还是有解决办法的。

实现原理

其实jsonp很简单，就是json数据加一个callback和一对括号就可以了，所以只要我们取到没有callback的json数据，并进行缓存，再把数据用标签包起来就可以了。是的，就是这么简单，但是如何实现呢？ 其实实现起来也很简单，在varnish 4的VCL里面其实可以使用synthetic来组合数据，但是这个函数又只能在vcl_synth和vcl_backend_error内使用，其中vcl_synth是用来处理错误的，而vcl_backend_error是用来处理后端服务器错误，所以我们就必须先抛出错误，然后读取json数据，再进行拼接，然后返回。 这里拼接数据时还需要用到varnish的Edge Side Includes(ESI)。

代码示例

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and https://www.varnish-cache.org/trac/wiki/VCLExamples for more examples.
 
# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
 
# Default backend definition. Set this to point to your content server.
 
# 后端服务器
backend default {
    .host = "127.0.0.1";
    .port = "8000";
    .connect_timeout = 8s;
    .first_byte_timeout = 8s;
    .between_bytes_timeout = 5s;
}
 
# varnish服务器
backend jsonp_template_backend {
  .host = "127.0.0.1";
  .port = "80";
}
 
sub vcl_recv {
    # 当地址是/JSONP-ESI-TEMPLATE时，则抛出760错误
    if (req.url == "/JSONP-ESI-TEMPLATE") {
        return (synth(760, "Json"));
    }
}
 
sub vcl_recv {
    # Happens before we check if we have this in cache already.
    #
    # Typically you clean up the request here, removing cookies you don't need,
    # rewriting the request, etc.
    if (req.method != "GET") {
        return (pass);
    }
    if (req.url ~ "callback=") {
        # 保存callback参数，后续拼装数据时会使用到
        setreq.http.X-Callback = regsub(
            req.url, ".*[\?&]callback=([\.A-Za-z0-9_]+).*", "\1"
        );
        # 去除callback和_参数
        setreq.http.X-ESI-Url = regsub(req.url, "&?callback=[\.A-Za-z0-9_]+", "");
        setreq.http.X-ESI-Url = regsub(req.http.X-ESI-Url, "&?_=[\.A-Za-z0-9_]+", "");
        setreq.http.X-ESI-Url = regsub(req.http.X-ESI-Url, "\?$", "");
        setreq.http.X-ESI-Url = regsub(req.http.X-ESI-Url, "\?&", "?");
        # 设置后端请求地址
        setreq.url = "/JSONP-ESI-TEMPLATE";
        # 设置请求后端服务器
        setreq.backend_hint = jsonp_template_backend;
        return (pass);
    }
    return (hash);
}
 
sub vcl_backend_response {
    # 如果后端请求包含有X-ESI则启用X-ESI
    if (beresp.http.X-ESI) {
        unsetberesp.http.X-ESI;
        setberesp.do_esi = true;
    }
}
 
sub vcl_backend_response {
    # X-JSONP-Server means we need to clean up the response a bit
    if (beresp.http.X-JSONP-Server) {
        unsetberesp.http.X-JSONP-Server;
        setberesp.http.Server = "JSONP-Server";
    }
}
 
sub vcl_synth {
    # 处理760错误，这里设置相关参数，后续请求后端要用到
    if (resp.status == 760) {
        setresp.http.X-ESI = "1";
        setresp.http.X-JSONP-Server = "1";
        
        # 设置状态码为200
        setresp.status = 200;
        
        # 数据拼接，拼接后直接返回
        synthetic({"<esi:include />"} + {"/**/"} + req.http.X-Callback + {"(<esi:include src="} +  req.http.X-ESI-Url + {" />)"});
        return(deliver);
    }
}
 
sub vcl_backend_response {
    # Happens after we have read the response headers from the backend.
    #
    # Here you clean the response headers, removing silly Set-Cookie headers
    # and other mistakes your backend does.
    unsetberesp.http.set-cookie;
    if (beresp.ttl <= 0s) {
        setberesp.ttl = 120s;
    }
    setberesp.do_gzip = true;
    return (deliver);
}
 
sub vcl_deliver {
    # Happens when we have all the pieces we need, and are about to send the
    # response to the client.
    #
    # You can do accounting or modifying the final object here.
}

注意

这里的代码是varnish 4.*版本的代码示例，如果使用了其他版本的varnish配置可能有所不同。如果使用较低版本，varnish 4里面有些语法已经变更。使用时可能需要修改一下。