My Octopress Blog

life and I

Thrift

| Comments

简介

[Apache Thrift][at]是一个跨平台的可伸缩的网络服务开发平台。通过其代码生成器 生成的不同语言的代码可以用类似的接口进行调用。支持语言如下

  • C++
  • Java
  • Python
  • PHP
  • Ruby
  • Erlang
  • Perl
  • Haskell
  • C#
  • Cocoa
  • JavaScript
  • Node.js
  • Smalltalk
  • OCaml
  • Delphhi
  • other…

编译安装

参见Debian or Ubuntu setup

创建Makefile

默认

1
./configure

但是生成的在本机遇到两个问题

erlang

通过git下载jsx失败

解决:通过配置项去掉erlang支持

1
2
3
4
5
6
./configure --without=erlang

**ruby**
ruby_noexec_wrapper错误

解决:通过配置去掉ruby支持

bash ./configure –without=ruby

1
2
3
4
**依赖库**
libboost

如果安装了libboost,需要指定其库位置

bash ./configure –with-boost=/usr/local

1
javac

bash ./configure JAVAC=/usr/bin/javac

1
###编译

bash make

1
###安装

bash make install

1
2
3
4
5
6
7
8
###可能问题

**compiler/cpp/thriftl.cc:2190: undefined reference to yywrap'”**
需要安装 Flex library 

**mv: cannot stat “'.deps/TBinaryProtocol.Tpo': No such file or directory”**

重新配置

bash –enable-libtool-lock

1
###安装

bash make install

1
2
3
4
5
6
7
##自带教程


###C++

自带教程在根目录的tutorial目录中,包含了一些常用语言的例子。以c++为例,
进行如下操作

bash $ thrift -r –gen cpp tutorial.thrift $ cd Cpp $ Make

1
2
3
4
5
6
7
8
9
10
**htons, htonl找不到**
解决:需要添加宏定义-DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H

**undefined reference to `apache::thrift::transport::TServerSocket::TServerSocket(int)'**
解决:将-lthrift移动到最后

**修改boost包含目录**
因为boost 1.49以后安装目录和1.33不同,需要确定后修改相关定义

最后Makefile修改如下

Makefile BOOST_DIR = /usr/local/include/boost/ THRIFT_DIR = /usr/local/include/thrift LIB_DIR = /usr/local/lib

GEN_SRC = ../gen-cpp/SharedService.cpp ../gen-cpp/shared_types.cpp ../gen-cpp/tutorial_types.cpp ../gen-cpp/Calculator.cpp

default: server client

server: CppServer.cpp

g++ -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H -o  CppServer -I${THRIFT_DIR} -I${BOOST_DIR} -I../gen-cpp -L${LIB_DIR}  CppServer.cp

client: CppClient.cpp

g++  -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H -o CppClient -I${THRIFT_DIR} -I${BOOST_DIR} -I../gen-cpp -L${LIB_DIR} CppClient.cpp

clean:

$(RM) -r CppClient CppServer
1
###Java

bash $ thrift -r –gen java tutorial.thrift $ cd java $ ant

1
运行的话,直接运行

bash $./JavaServer & $./JavaClient

1
###Python

bash $ thrift -r –gen py tutorial.thrift

1
2
3
4
5
6
**import Thrift error**
没找到包路径,需要在~/.bashrc中添加export PYTHONPATH=/usr/lib/python2.7/site-packages/
然后重启terminal

**TypeError: getaddrinfo() argument 1 must be string or None**
修改如下

python transport = TSocket.TServerSocket(port=port)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
##教程

###The Missing Guide

####语法类型

#####Type

包含基本类型,用户定义结构,容器类型,异常以及服务定义

**基本类型**

没有无符号类型,因为很多生成语言不支持该类型,比如java

- bool: A boolean value (true or false), one byte

- byte: A signed byte

- i16: A 16-bit signed integer

- i32: A 32-bit signed integer

- i64: A 64-bit signed integer

- double: A 64-bit floating point number

- string: Encoding agnostic text or binary string

**容器**

因为容器的复杂性,thrift没有消息长度,但是过于复杂的容器会阻碍消息的编解码

- list<t1>: An ordered list of elements of type t1. May contain duplicates.

- set<t1>: An unordered set of unique elements of type t1.

- map<t1,t2>: A map of strictly unique keys of type t1 to values of type t2.

**自定义结构和异常**

自定义结构类似于c的语法,而异常类似于把结构定义中的struct变为exception

**服务**

和定义接口或者纯虚类一样

#####Typedef

类似c/c++

c typedef i32 MyInteger typedef Tweet ReTweet

1
#####枚举

c enum TweetType {

TWEET,       // 1
RETWEET = 2, // 2
DM = 0xa,    // 3
REPLY

} // 4

struct Tweet { 1: required i32 userId; 2: required string userName; 3: required string text; 4: optional Location loc; 5: optional TweetType tweetType = TweetType.TWEET // 5

   16: optional string language = "english"

}

1
2
3
4
5
和protobuf不同的是thrift不支持包含枚举(包含结构)。枚举值是32位的正数。

#####注释

支持shell, c/c++, java风格注释

This is a valid comment.

/ * This is a multi-line comment. * Just like in C. /

// C++/Java style single-line comments work just as well.

1
#####命名空间

c namespace cpp com.example.project // 1 namespace java com.example.project // 2

1
2
3
4
5
6
1. 在cpp中展开为 *namespace com { namespace example { namespace project {*
2. 在java中展开为 *package com.example.project*

#####引用外部文件

可以引用外部其他thrift文件,搜索目录为当前目录,需要注意的是,thrift文件后缀名为thrift,使用其他文件中结构时,需要加前缀

c include “tweet.thrift” // 1 … struct TweetSearchResult {

    1: list<tweet.Tweet> tweets; // 2

}

1
2
3
4
1. 后缀名为".thrift"
2. 使用Tweet结构需要添加tweet前缀

#####常量

c const i32 INT_CONST = 1234; // 1 const map<string,string> MAP_CONST = {“hello”: “world”, “goodnight”: “moon”}

1
2
3
4
#####定义结构

结构是thrift IDL的基础单元,每个结构由不同区域组成,每个区域有唯一的id, 一个类型说明, 一个变量名以及一个可选的默认值
下面是一个需要完成类似Tweeter的结构Tweet

c

struct Tweet { 1: required i32 userId; // 1 2: required string userName; // 2 3: required string text; 4: optional Location loc; // 3 16: optional string language = “english” // 4 }

struct Location { // 5 1: required double latitude; 2: required double longitude; }

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
1. 所有区域id必须唯一且为正数
2. 区域需要标识为 *required* 或者 *optional*
3. 结构可以包含其他结构
4. 可以为变量选择默认值
5. 可以在一个thrift文件中定义多个结构

区域id为解析器标识不同的变量,一旦定义就不要修改。标识了required的变量但是没有赋值,thrift序列化时会报异常。如果标识了optional
且赋了默认值的变量解析消息时不存在,那么解析后的结构里面会对其赋予默认值。

对于将变量设置为required需要非常小心,一旦赋予后,改变它为optional会引起各种问题,老的解析器会因此报异常。部分使用者认为required
这种设置带来的问题比好处多,因此只使用optional或者repeat,但是这个观点并未被普遍承认。


#####定义服务

thrift定义了一个跨语言的RPC框架。可以帮助用户快速构建网络应用,这是thrift的一个主要特点。
其定义类似于java的接口定义,thrift会根据你选择的语言类型默认生成客户端和服务端代码。

c service Twitter {

// A method definition looks like C code. It has a return type, arguments,
// and optionally a list of exceptions that it may throw. Note that argument
// lists and exception list are specified using the exact same syntax as
// field lists in structs.
void ping(),                                    // 1
     bool postTweet(1:Tweet tweet);                  // 2
TweetSearchResult searchTweets(1:string query); // 3

// The 'oneway' modifier indicates that the client only makes a request and
// does not wait for any response at all. Oneway methods MUST be void.
oneway void zip()                               // 4

}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
1. 逗号和分号都可以作为结束
2. 参数必须带类型和id
3. 返回值必须带类型
4. oneway返回值必须为void

####代码生成

下面图标识的thrift的网络模型

{% img data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAeoAAAEmCAIAAAAiJUSiAAAqT0lEQVR42u2dC1hVVcK/A8EQEy8hKmlYXtJKSpOyTMdLhikZ6ieiGaiDiqFmXjPvOhoKIV5QzG4zX9lM9U0+TuNYWWNFVpZFUXntQjHRTDfMpCj7/v/fafntjueyzwGMDvL+nvfhOXvt29r7nPPuddbZZ3HW/yOEEFILcxangBBC0DchhJAa1/f/EkIICfigb0IIQd+EEELQNyGEEPRNCCHomxBCCPomhBCCvgkhBH0TQghB34QQQtA3IYQQ9E0IIeibEEII+iaEEIK+CSEEfRNCCEHfhBBC0DchhKBvTgohhKBvQggh6JsQQgj6JoQQ9E0IIQR9E0IIQd+EEIK+CSGEoG9CCCHomxBC0Df6JoQQ9E0IIQR9E0IIQd+EEIK+PeUsQgghvhKg+t78/mYAAPCGPFlWVnbs2LHy8vKKiooTJ06gbwCA2qHv4uLi0tLSL7/8UhKXwdE3AEDt0HdRUdHhw4dLSkpkcLXB0TcAQO3Qd0FBQWFhoQyuNrga4OgbAKB26HvHjh0yuNrgxcXFZWVl6BsAoHbo+5FHHtm5c+fevXvVAP/yyy/RNwAA+kbfAADoGwAA0DcAAPpG3wAA6Bt9AwCgbwAA9I2+AQDQN/oGAEDfAADoG30DAKBv9A0AgL4BAAB9AwCgb/QNAIC+q67v/CP5cx+fOzB94GXXXdY3pW/SgiSeWgBA34Gub7lb1tYqYeeEdYjrEB4RHhQUxFMLAOg70PW9eOdiLR96dmjWq1kOmx/OX/j3hTy1AIC+A13fS55aouWDgoPmb5/v3jBPWpAU0yWmcVTjTtd0WvC3BaY8flL8hV0vnJw/OXFWYqv2rWZunTn8juEquWnGTda60/80XSUrdq+o1HZ4SQEA+q7EYbTu1FqrhNQPSZyZmLc/zyrvP7a/ytt3bz94yuCwc8LCGoblFuaq3HS2xFwaI+lHxUTNeHiGzKuS8MbhG97bYNa9fMDl0R2jK7sdXlIAgL4rcRhqI5930Xln/ZwWF7SY+/hcFd714l3B9YJFzr4cTfZK7qW5Y1aMsbSrJVftWWW10yPbRKrw92t+r8msV7O04oj5Iyq7HQAA9F25I9l4cOOwOcPqN6jv6AcPC1369NKMezIc32c2DIu7MU60ubiNJuMnxlvavXn5zc5bGHL7EBV2urqTHg+/Y3hIaMjdr9+tx5XdDgAA+q40y55ZFtE8QqsPnjI4fVO6UXn8pPiB6QMNk/Im/aLdP5yi3cyCzKDgIEVt+ZbtWnYb2M2UV3Y7AADouyp0vb6rVr9yyJXLn1tuulNm/2W2yzLetHtxr4tV3rlnZ/2dev9UU1iF7QAAoG/fyKqx/WKH3zF85taZSQuTQkJDtHpabppm9RjaQ48j20QOuX3I5PzJCdMSkhcn22g3bW2aMXXTlk03HdpklVd2OwAA6Ns3Mx6eEVwv+Kz/S3jjcKnczFr/zvoBaQNCw0LNrEbNGiVMTbDRbt7+PK2uWYMyBjmXV3Y7AADo2y/UUl75wso5j81Z+vRS5xsHrbtKNHfNG2uqebJO13YAANA3AAD6PiP0va5oXc2cODXDF+1YZO4ED4TtVIrlzy3XTte+tbYG1goozoBD+K1e8IC+f119p65O7TG0h80CfVP6tmrfasK6CdU/caer1/s36T2PuTRGO7118601sFZA4c8hnMYXSQ2QsSWj++Dua99ei8vQdy3Wt95vWmvU0lE2y0jujZo1Gps19rfS95o31sx7Yt6yXcvQ92nB/XxW/xBO44ukBg5t9curw84J69K3S/6RfHSGvmulvlftWaUXcdvYtvmHa+hFXDXt9h7dW2uZu1bQd/VxP59nzAcI/w8teXGyljR3sgL6rn36TpiWoFVue/A2M7n2rbWJsxI79+wcERnR5uI21ivbGiDQZTLlrpSYLjHturWzZpm7WW6YfEOLC1q0uLDFsLnDrp9wvRa2frnjol1vQxI6Mz5nvOpjbiq3xjK0tuOxDh5HNPS2L20/7sa4lu1aasm+KX0X71zsfCyDMgbpQMyxmN/9WxbztkH7texv0XHfYDXHdHQ/Px7Pp/stSTaH4PH0Or9ITnud/TxX3g4tLTet41UdtWRU26jYfrHWMWqxyDaRNMDRd63Ud/Pzm2uV7L3ZZvKa/7pGk5f0vmT0stH9UvtZ/3zHxblmUm+Ps8PPju4Q7bhnPCI878DJ+w5/d/PvVNKgUYPBUwbrw6m549t657tsytuQhM7IDkYfej/HT4w3o5Pb18HjiIbe9nX5gMuFrlU9k3qaw7d2rZOgEu1CZjG/KXU+Fm8btF/LBo8brM6Yjh7Pj8fz6YL9IXg8vc7P7Gmvs//nyv3QdB1SSXC94LiEOB1Xhys7WBvRc61Zcx6dg9HQdy3T98aDG/X20zvEKtGLXluwfrzjrafCTEr9xvuNzm2kSfMeNiMOalKtLbOweR961LfNkIR+dp54rMNmTyMa+rOv/MP5DZs0DAoKMtvUkiH1Hb9EnfbANNPcU0vTOhZvG7RfywZvG6zOmI7ezo99D4PPQ/A4YKTzM/tr1Nmfc+V+aKqheUFOXD/RwzVgnOMakJKZgtHQdy3Td9YrWVreahOJW1beYtpZasKkrk61fv7uUd9qoTt3jGZsybCaXfVC6q1/Z73HblPnTdkMSeinvl3qoFbbWV5iZnnc17qidaOWjuoxtEfnazsbbZkP3bP/Mtu02qz7E5yPxVvl7deyuxfC+9mo8piOHp8jn/r2eQgeB4x0eZGc9joLNTXO8h6zEZdDm3rfVPMvpawXpDOqmOYOnT0Uo6HvWqZv2VlvUb2RnAtn/XlW29i25v3QNb6rjb6tSee32ZR7p5h3i5r2PvVtMyRhZb+6dHmru3+x6W1fMpQZ8Ty2X2zCtAQjCKNv1dl437qMOR+Ltw3ar2WDzdmo/piOldK3z0Pw+L2xS+Fpr7Of58rl0FRouvKsF6TLzY6aOy57HEZD37Wv79t8KHb/nwnpGx1vD733Ml/KrJS+lz691KjffJ019/G5ZiRxj+98myEJPXZ06qNulfXtbV9GBB2v6mgmm0U3s/S94G8LzCrmX4Cufnm1+ThvjsXbBu3Xyno1a/ofp3vsELA/G9Uc09Hl/LifT/8PwU99n/Y6+3muXA7NekFa3887o89bmjXviXkYDX3XPn0PmzvM8Q655+Q7pFdyrxHzR8z76zy1R1Qu85ofp/mvb9EhroP56H1h1wsjmkc0bdnU8e69b6rHdb0NSejCtAemaTG5VbZ1/urSf31729f87fNV2Diq8fic8VcMukINRuvzu9qeLS5ooUkdhaQQFRNlhuWyLOZxg/ZrmbvsVeLt1mlvZ6OaYzq6nB/38+nyscz+wP3U9+mts5/nyv3Q9CHSHEvizERtTQ1zM7xP3oG8iMgItWDQGfqulfrO2ZfTqFkjvYLNR0s1lEwfcUj9EDWdrK8fK6XvrFeyYvvF6uNqu27t1AbXm99qjLuv621IQnehdB/c3dTNSKQK+va2r2tHXqvjlbh7j+p9/YTrzVxzQu7cdmfTVk3NLRCpq1Nd+hC8bdBmLaPvJi2aeHw6bM5GNcd0dDk/7ufTBfsD91Pfp7fOfp4r90PLLcxVu8SMh+zweGTEhncd98PcNOMmek7Qd+3+1aVaK0FBQWqGW7fTZr6UWZ0RIZzv9DIfXa1OGJt7eP0ZklA6qP7/xvS4L73Dra/psvdmOw+l4jghBZnOjUd/Nmizlq4QcQlx1T8b1VzF5/n0eeC/xhNxujbifmg6kBW7V1jPstaS+q8cciUuQ9+1e8wTNVLUPDldpyZ+Uvy5552rz7ZqBJlvAk/jxms7atVGd4w2N+TBbzvmSZ8xfdwHSQb0XftGHPT4vXzVmHrf1KuHXd26c+uGTRq2ubiN2vXWbzdA7cTTeKqhyvAsoG/G+wYAQN/oGwAAfQMAAPoGAEDfv76+CSGE2CcQ9a2UlZUVFxcXFRUVFBTs2LHjEUIIIadGbpQh5UnZUs4MFH0fO3astLRUl5TCwkLVbychhJBTIzfKkPKkbClnBoq+y8vL9VmgpKRENdO1ZS8hhJBTIzfKkPKkbCln0vdNCCG1I8XFxWp3y91qeldUVASKvkeMeAYAALyBvgEA0Df6BgCoQX0HaN83zw0AgL2+A/HOE/QNAOBT34F43zf6BgDwqe8A/dE8zw0AAPoGAEDf6BsAAH2jbwAA9A0AgL7RNwAA+kbfAADoGwAA0DcAAPpG3wAA6Bt9AwCgbwAA9I2+AQDQdw3p++6739q69Yjh/vsPZGYWJifv4okHAPQd6Preu/c/qtKJE/8rTPU+/fR4auo/ee4BAH3XAn3/8Y8Hb7752ayst0yFN258l+ceANB37dC3mSwuPqbJ//7vQ3q8bduHBw+WrV5duHXr4ZKSb5csed0sI7kfOPB1WVnFBx9889e/fijvW1vLzS16992vvvrqezXhX3/9c1OYlPTMgw8ePHLkqMrffvvLOXNeMeUpKf/UllWiTX344Tf333/Avtxm196qCgDou07oe8aMPabCK1a8Yc16//2j2mNp6XHjxCee+FCF33zzw+OPf/DJJ9/qsbyZlOToLleJJn/66X8LCj7bsePj9977yuzi738vVvn+/V9rgfLyH8vLT5jOmeee+5fKCwu/2LJlv5aX4s3y3sptdu2xqgCAvs98favJ/NZbX/744096rGav2svWLLWjJ016wSysB6aLfMGCvZpMTt713XcnNJmb+7Y1KyfnbeftT578ooQuxo/frcldu0q0TH7+e3p84ECZ1dJ3xmO5za49VhUA0Hed0Pfx4z9+9ll5UdGXauqOHv2s86x77tlvLXzXXW+qpKLiJ+vuFElfJdu3f7RypWPWDz/8NGbMc87bz8wsVLla3AUFpeLDD7/R5LZtH2mWJG5OyAcffJOX9+7IkSe36bHcZtceqwoA6Ltu9X27z7rnnveskuzst1Ry7NgPpnkuXnnl3yrZufNjM0uXAZf7DrOy3jLa3bbtwyeeOIkWNnMXLXrtyJGj5rRoU9Za7uU2u/ZYVQBA3+j7FydOn77HHMLtt+8xJV988b0mt2zZb836wx/ecN7I1KkvmfKFC1/zVgejZp2r9PQXvJXb7Bp9AwD69qFvq8176FCZNP3005/o8RdffHfLLc9Zs77+umLr1iOZmW8+9tj7ph/m+ec/Vflnn5X/+c9HVq8ufPTR983NJLt2lTz44MF5815dv/4dLfD99yfMdryV2+wafQMA+vah79TUfz777L/Mt4iqyTvvfHXbbS9Zs2Re8/2nUlZWYW7sGzPmue3bP6qoOFl+9GiFzK7yt9/+Umo2PeaFhV8sW7bPbMdbuc2u0TcA1Dl9V43k5F1TphQ43/HtMisl5TmX8qSkZ2699cWxY3e7FKanv+DybadNuf2uAQDQNwAA+g4kfXtsz/6GqD0+c+bL48bt5qXmkalTX9L5SUn5ZwA+dwDou+b0nZf37vPPf/rbntB//OPjTz751vppD33W9rz/vuPuyVWrCn++g/7NPXs+c++VAoAzXN8ypnZ9772/8Q9bdu/+9OjRig0b3qlT+h47dvcdd7w6bdpL1dH3xIkvlJf/uG/f59ad7wBw5uvbvPOPHDlqRggJHOqIvs1tjuZ+myrrW9x//wFN3nffAd6ZAHVF348++r7LD2o8Dg04wteQfosXv75165GPP/720KEyfZYfPfrZHTs+/uyzcn2ot5qW1uB/Dz10+OOPj5WUfGuNOeU816O+vY1N6M+AgsnJu1RhHc6//nX8oYcObd/+kVaxfivkc8ubNr2ruQcO/FK3ytZn3bqigoJSTQqdlhkzTv68SOU6nz/fpf691srIKLDZ8siRu/7nfz7QIZijMGMMWPrWXG3k3/8upwEOUFf0LcNqv2lpz5tJb0MD+hzST9Y+erRC+jC3db/55helpcePH/9Rk9qOs5HVbNQCmmsOed06z70lLpPexib0Z0DBp576xPxqX+vu2/e52a8lPvst6zC/++6ELkt6/O23P44a9WwV6qNCoabxs8+eHDHRbETXPGNhXRu2bfvIPAvetqzTqHJVRlcIad3lKITO+c/jdr3GmxPgzNe3mqVaRVYyk96GBvRnSD/pW+3xsWN3m+My442oGa7HH310zNnIatprSan/5Zf/bTzlU982YxP6HFBQTjSVt37jY1xsxOdzy7q8GauaZrLRcZXro6M+duwHnSLreunSeeJty5r84YefrAF71cQ2F1FnfT/5ZDH/TwOgruh7woTntdOPPz6pV29DA/ozpJ811NS33/6gSWOf2293jEZSUvKtR0Gb4QO//rrCp75txib0OaCghGv+5Zt1UKfes+Fjy2bgFGstXZCqUJ9bbnnu3nv3P//8pzpvxsKmn8Rd3962vHDha+ZTkXV7iUvft/jTnw6p5OGHD/PmBDjz9T1y5C55TbIwk96GBvRnSL8q6HvDBse4Jf/5z3c+9W0zNqHPbzjNtUfStA7KWXz+b9lZ35Wqj4Rr/lHR669//uij7+uzjo2+vW1ZtTVjvFhj57rr+x//cPSuWLfuAMAZ3vdtPoNPnOj4pO9taEB/hvSrgr5Nz4AapD71bTM2oU99W5U3/R7z5+81o6YY8fm/ZWd9V6o+xsjvvHPyW4TPP//OWd+mp/vJJ4utH+N43PKcOa+Y8lmzXjb3C5nOHGd9m49Ed9zxKm9OgDqh74ceOvSzlQqdm9XuQwP6HNLPf32rua2FH3rosLNGfX516W1sQn/uL3zvva9N/8nBg2U6NHPtWbnyzUpt2VnflaqPMe9XX32/bl3Rnj2fmSfe6pPR+dSknC7Lmw5xj1tWo/vTT4+bp0bGLy09bs6zdfZGjXpWQtfFmHcmQF3Rtzx79KjjbW/6FrwNDehzSD//9V1Q8JkR6A8//CSJe2u3ukx6G5vQH31PmPD8669/fvz4jwcOOG5wNDe9WPeo+LllF31Xqj46pTpYnbenny7RWmYVc8Ll5Zde+sx8FWxuTPS2ZTWrdck0N8Dk5b3r0nnyyCNH6DkBqHO/ulQDUJVRM9z5jhSPQwNWc0g/S21miEGXHnZ/8Dg2oU+su/2svhT3//9QtS37v5auf9b5VCvbZSwXfcQx/Vf2W/55tMUXre5vCy0p3b/4YilvS4C6pW+hdp9aiGfqDym3bfvwP//57vnnP1U713xz+MwzJWfSa1GfCXbu/MT6T6QAUIf0bVrWv/Yp2779oyNHjlrdLDXGypVv7t796UcfffPNNz988ME3+pxxhpmuBp47APTNeN8AAOgbfQMAoG8AAPSNvgEA0Hfl9U0IIcQ+gahvpaysrLi4uKioqKCgYMeOHY8QQgg5NXKjDClPypZyZqDo+9ixY6WlpbqkFBYWqn47CSGEnBq5UYaUJ2VLOTNQ9F1eXq7PAiUlJaqZri17CSGEnBq5UYaUJ2VLOZO+b0IIqR0pLi5Wu1vuVtO7oqIiUPS9+f3NAADgDfQNAIC+0TcAQA3qO0D7vnluAADs9R2Id56gbwAAn/oOxPu+0TcAgE99B+iP5nluAADQNwAA+kbfAADoG30DAKBvAAD0jb4BANA3+gYAQN8AAIC+AQDQN/oGAEDf6BsAAH0DAKBv9F19Vr6wctGORTn7cnjxAQD63jxpw6TEmYmG5MXJGfdkbDy4MTDP+GXXXaaju/kPN/PiAwD0fdKJ9ULqBdcLNv/BM6ptVG5hbhXOyJo31sx7Yt6yXcvQNwCg7xrS94g7R2x4d0P6pvSgoCBNpq5KrcIZ6T26t9ZNmJqAvgEAfdecvs3keRedp8nhdwzX4/hJ8Rd2vXBy/uTEWYmt2reauXWmWUZyb3dFu4jIiDYXt7lh8g3yvgrH54xXidZt2rKp1lqxe4XNwoa03LSOV3VsHNVY7f3YfrFWubdV0DcAoG/P+l68c7FpfU97YJo1K+bSmKDgoKiYqBkPz1DhwPSBKmzYpOHgKYPldD2WrPMP50vusq0m23VrFz8xPuvVLJuFNWtQxiBNBtcLjkuI65far8OVHUx9bFZB3wCAvl31rVZw52s7h4SG6HHnnp3zj/yiyxYXtFi1Z5VZWA9MF/mcx+ZocuPBjWeHn63JtLVp7p0nNgtbsyaun+hcGfvto28AQN+u+m7QqEFkm8hOV3dKWpCUtz/vlM6K5b/ocsq9U1QSGhZq3Z0i6atkQNoAd33bLDz1vqmOWWeHrn9nvXNl7LePvgEAfXvt+7b/qnBS3iTTs2Ga56LbwG4q6XNLH3d92yxsZuma4XKTov320TcAoO8q6nvp00vNzYVLnlpiSpq2bKrJ0ctG63H/sf31uP+4/j4Xtmbd9uBtznu03z76BgD0XUV9i67xXc3XidMemGaa201bNV1XtE6zVKLJZtHN0jelm68ubRY2syKaRyTOTMzYkqE2u+m0sVkFfQMA+q66vnMLc3sm9awXUk+zgoKDLupx0bJnTv5OZ9OhTd0HdzdfNt66+Vb7hTWrV3Iv82Wpw+OREeYGQZtV0DcAoO/qIlOv2L3C+SZuCzWirTtVfC5sZq19e63/2wcAQN8AAOi79uvb9C9DYMKzA4C+PZO6OrXH0B7m8Z3b7kycmTj+7vFmMv9I/tzH5w5MH3jZdZf1TembtCDJecWFTy4ccvuQ7oO7x90Yd9OMm6w7Rk7eCPjzcIZaxrlQWxizYgwvoEqRsSVDJ9m9lwkA6rS+J6yboLVGLR1lJkcuGqlJ8yt2udt8YRh2TliHuA7hEeFBQUHWioMyBgXXCxbnX3J+686tg4KDQkJDhs4e6vLlZ1TbKOef5zRq1qhV+1Zn2KuhyqMt+rni6pdX6yno0reLdUc8ANR1fa/as0peaBvb1owr4qLvxTsXm59HmhsBtczCv59sSo/LHqdZ4Y3D1TY3Jbc/dLvzbSeWvhU1289sfVd5tEX/V0xenKwl9Zf3HgD6dpAwLcHl5zPO+l7y1BJzA9/87fNd7gyRha2xCZ3b4yo8t/W5zvrW6ooZ8apS+q7skIRmiMSZW2cmzkyM7hB9weUXZGzJyNufp4tH8/Obdx/c3WrkWoMpDpszLLpjtOrj3Ck0Pmd83I1xLdu1VLnW1TXMvlbeRlt0Rq1m7SKmS4xW7HRNpwV/W2CzoscD1znXYpFtImmAA6BvB/KaVsnem+1R36J1p9aaDKkfIidaw6Esf265aVa7qGrhkwtNudmg0fd1469zOP28c03XrZ/6rvKQhLK2diHN6XGjcxtd0vuSqJioBo0aOH8IsAZTlD0119TZ6u6/fMDlQu3cnkk9Va4t2NfK42iLLpifpLbv3l7V1sedsIZhuYW5Hlf0duBCNXGM5/XoHN5+AHVd3xsPblTTODwi3LnQRd8StBkE3Iw+aLpKzJhTalOrSejSk2uWnP2X2ZYlc/blGEldO/JaP/VdnSEJZXO1x1UTM/itGS9FzXA91qXIWd9q4WpJqb/bDd2MW12bzIfzdZ3QdszVyFutfPaB3PXiXeZLAvOPOnsl99LC5vtbb8M0uu/CcQ0Y57gGpGSm8PYDqOv6znolS8tHd4y20bex5LA5w+o3qG+GA1z69FJJ3Gh6zZtrnNdd+cJK50FLjCXXvrV20Y5F5geWU++f6o++qzMkYfrGdDMrvHG4Ju9+/W6rF6hlu5Yef8Apk5pf8Ft36Y1aOqrH0B6OQXTrh1gfMrzVyqe+M+5xXDzU4o67MU6Yi5ma2+4r2uxCDL9juOY6fzkMAHVU32o7q60nrdjr27DsmWUSnGbp47+sbTRt/QseZ09pm3kH8pz1rceJMxP1uEmLJtqdT31XZ0jCKuh7bNZY072jx2vfXms+bcT2i02YlqCPJpa+vdXKp77TN6Wbq078pPiB6QMN2tpmL8M0etyF6JvSV3PHZY/j7QdA3/dm033s/DN3b/oWXa93jCd15ZArTQexHl/6u0stk+pi0CGugwqtW8id9a25bWPbGukbfWe9mjX9j9OtrzSdqc6QhFXQt+nNUFvbUm3HqzqaWc2im1n69larzW6jLbpgfVVg+pRsVrTZhfVRY94T83j7AaDvzcPmDtMqajU7/6xGJZ2u7qTH0o0aofrMrlZ20sIk0wGSluvoaM4syDR+lNOnPTDt1s23mi/WGp3byPoi1FnfxqGhZ4da+jb3m2sjHitW5SEJ/de3mttq7Vr9QuZ+x/nb5+tx46jG43PGXzHoiqDgIOvyYFMr99EWXdAlTQtEtokccvuQyfmT1a43twB6G6bRfRf6QBMRGXHm3XMJgL6rqO+cfTmmM9r6tN7nlj5WC1pNY/NNmomE6HynoNrssf1jrQXqhdSLS4jLfi3b5Wc7lr6t3ltnfTdp0cRjxao8JKH/+lZtTeNdFxVJ3Nr1tSOvDakfoo33HtX7+gnXmwqY8+OtVu6jLbqw/p31A9IGhIaFmhV1zk2HicdhGj3u4qYZN9FzAoC+T0ENwKCgoKsSrxqYPlCtwoZNGjrfRSe/rHxh5ZzH5uhzvXXjoDMqXPC3BfKjx+5aeyRHOdS+d/7XGJLQ6jzJP5Kvo3OvuRxq7VQfJswdIz5r5T7aovvd39rdmjfW+FzRZRdaS+o33VYAgL5/Qe2+dle0Mz0bYeeExU+Mt36E+etx57Y7oztGm/9qX8PUuqHDM7Zk9BnTx+PlEwDqtL7N3YFqG6rVWWNnUI3QKjTYTwsD0gbEdImxulkCn9/qRAGgb8b7BgBA3+gbAAB9AwCgb/QNAIC+q6FvQggh9glEfStlZWXFxcVFRUUFBQU7dux4hBBCyKmRG2VIeVK2lDMDRd/Hjh0rLS3VJaWwsFD120kIIeTUyI0ypDwpW8qZgaLv8vJyfRYoKSlRzXRt2UsIIeTUyI0ypDwpW8qZ9H0TQkjtSHFxsdrdcrea3hUVFdx5AgBQO+48Qd8AAOgbfQMA1JS+A7Tvm+cGAMBe34F45wn6BgDwqe9AvO8bfQMAMOYJAAD6Rt8AAOgbfQMAoG8AAPSNvgEA0Df6BgBA3wAAgL4BANA3+gYAQN/oGwAAfQMAoG/0DQCAvtE3AAD69sTNf7g5cWaiRzSLJx4A0HeA6rvNxW3qN6gvQsNCzf/01ANT0rpz61r6bK15Y828J+Yt27WMFy4AnPmdJ1mvZhl9Z72SVdufrd6je+tAEqYm8MIFgLqr7/hJ8Rd2vXBy/uTEWYmt2reauXWmCsfnjI+7Ma5lu5Yq6ZvSd/HOxe7Lp9yVEtMlpl23dnpszV371lptp3PPzhGREWr4Jy9Odllr2Jxh0R2jtdmkBUnO1UhdldruinZmrRsm37Dh3Q3eqqe6aTEdSNOWTTVrxe4VvHwB0Hdd1Pdl112mwphLY4KCg6JiomY8PEOFlw+4XEi+PZN6au4lvS9xWV7ePDv87OgO0XocHhGedyDPzL3mv64xy49eNrpfaj/L0dZeZF7txdRk/N3jzdyB6QM12bBJw8FTBsvRZvv5h/M9Vk8Gl+JVqCtH/MR4HRcvXwD0XXf13eKCFqv2rHJfSw6VVYOCgrL3Zjsv3/z85qak0bmNNGmkL6RUTQ6/Y7jLdsxaHa/qqGa1ttnthm6abN+9vWZpv8H1gjU557E5mtx4cKMuDJpMW5vmrXp0ngAA+j7px5uXn3IXyrqidaOWjuoxtEfnazuH1A/RAlYfhVlejWszqXaxJjO2ZJjJW1beYvaiBnLq6tRNhzadspf/u9dlzIoxmoxoHqHHU+6dYr5QlbjNXO1UJQPSBnirHvoGAPTtKlZH//Xba8+76DwVxvaLTZiWEB4R7q5va3kXfYtZf57VNrat2VfX+K4e1xqbNVaT5553rh5Pyptkek7yj+Sbud0GOtrmfW7p47F66BsA0LdnfadvSjcdHWayWXSzSun75EY2OjYSFBSU+VKm+1q9kntpUq1sPV769FJTsSVPLTFzm7ZsajXwPeq7/9j+Kuw/rj8vXABA37/4cf72+SppHNV4fM74KwZdERQc5NxbYq9veXnE/BHz/jpvXPY4lddvUH9d0TprLTW31dYeNmeYyjV56+ZbzVpqpJuvK6c9MM20rJu2auq8oou+tZgKdV3RlYavLgHQN/r+pfDakdeG1A+RuHuP6n39hOvNiqZv2l7fnXt2Nl88avWLe108/U/TnfcSlxBnWtahZ4dK4tbucgtzeyb1rBdSz9FgDw66qMdFy55ZZlO9TYc2dR/c3ezIugYAAPpmzJOTSl379lrzOHtvds6+HD9XzD+Sn/lSpmk7u18kNHflCyutbyldpLxi9wrrjm+f5O3P83i3DACgb4asOm14bEQDAKBvV1wav785A9IGxHSJSd+YzkstYJ8jAPT92+s7dXVqj6E9aubE+Tmi4eJ/LFaJahXbL7b36N5DZw+d9edZjrsGN0yylk9enJxxT4bHfhUXzFqz/zLbKsl+LdtsxGeVfFa4alWqAhlbMroP7m71UAFAXdf3hHUTtNaopaNq5sT5HNEw/0i+mt7md5XNz2/e8aqO5zQ9R4+vuukqq1OlXkg9s4AS1TYqtzDXn66YIbcPsUoW7VhkVtfu7Kvks8JVq5Izfg6IuPrl1WHnhHXp28W64R0A6q6+V+1ZJSO0jW1rBgypSbzd2aImrQpVK+d7w1fsXrHgbwssV464c8SGdzekb0oPCnLcpJi6KrU6+vZZJX9uxalslar2qyK17rWkNbAXANRdfSdMS9Aqtz14m5ms1GiC0x6YpoWlLasHRpOWWZIWJsX2j62svjcd2mR+tKkK2IjY2qn5taf7kCm/ib59Vkn7SlqQFNMlpnFU407XdDIXJG8DIqblpumTh5ZUWz62X6x1frRYZJtIGuAAdV3fzc9vrlVchpfyczTBlS+s1GR0h2gzKddo8oLLLzCT0R2j7fvTPdpw+bPLTaG3YV2dXakLiWnq6kLij757Jfea+/hcw/i7x/8a+ravkvnxZ/vu7QdPGayPF2ENw3ILcz0OiDgoY5BKgusFxyXE9Uvt1+HKDtZGdP4dw3U9Ooc3IUDd1ffGgxulabV2XUzk/2iCLS5soeUzX8pc88YauSb07FD9XfPmGslX5RPWTaisvqfeN9X8dN7bt3/WSISOsbFCHWNjde7Z2WdT1KzlMadL3z6rdNeLd+nkCHNHvBkGYMyKMe6dJ9aQihPXT/RwDRjnuAakZKbwJgSou/qWhhzN547Rrndb+z2aoBqGDpXclaJP+vVC6l33++s0mb4pfeSikRKQ/dd3Hm0457E5pjD7tWwbETdo1CCyTWSnqzslLUjK25/n513kp7S+c05z69tnlTLuyXD06TcMi7sxTpgWt5rb7vo21zBdC9e/s959O8PvGK65Q2cP5U0IUHf1venQJklWQrH5sYz9aIJGNN1u6CYfdbqmkz7RO0b+G9NHzc+LelxUha8u1TI1he5jWnnsaK7Uj4Bqpu/bG2Zgr9Cw0PhJ8QPTBxom5U1y17cZUlHXA48fQfqm9NXccdnjeBMC1Om+b/OvaqyuksqOJqjmodrjcnp443C1OnU90MWgaaumaon71Jk3G5q+XV0zPN7g7M2V2tr0P063/gtEAOp7+XMnu/Wdbz937hO3BkS0hlS0vlJ2xgxuPu+JebwJAeq0vofNHeZo6t6T4U3f9qMJbv55zCnjGulJk2qkm0mfdzF7s6GuDboGqDyyTeSI+SPUDFclL/3dpaqbjSvN3eu6igSsvkWPoT3Mcakmk/Mn69OMuVHHfUBEM6RiRPOIxJmJOgNqmJsOmbwDeRGREbro8g4EqOv6ztmX06hZI+nA4+iABpvRBK2u2JbtWlr3C5rfrVT5vm+RWZCpmli/glFbvnXn1jfNuMmnvpu0aBLI+taHlQFpA6yf/+jMmw4T9wERcwtzeyX3Ml+EOjweGWFG5tJJoOcEAH1vtm7fDgoKUgvXZpkqjyZYHXSFWPLUkoVPLrT+r7E9urTEJcQF/kvHjKG45o01LuXuAyKaIRWtM6+1pP4rh1zJ2w8AfZ9ELT619Wr183HntjujO0b7vAG8VpOxJaPPmD7+3GkDAHVF36apW6ufDzVma/shnPHPEQD6rhPjfQMAoG8AAEDfAADoG30DAKBv9A0AgL4BANB3zeubEEKIfQJR30pZWVlxcXFRUVFBQcGOHTseIYQQcmrkRhlSnpQt5cxA0fexY8dKS0t1SSksLFT9dhJCCDk1cqMMKU/KlnJmoOi7vLxcnwVKSkpUM11b9hJCCDk1cqMMKU/KlnJmoOi7oqJCFxPVSVcVfS44TAgh5NTIjTKkPClbypmBou8TJ06oNrqeqFplZWVfEkIIOTVyowwpT8qWcmag6JsQQsivGvRNCCHomxBCCPomhBCCvgkhBH0TQghB34QQQtA3IYSgb0IIIeibEEII+iaEEIK+CSEEfRNCCEHfhBBC0DchhKBvQggh6JsQQgj6JoQQ9M1JIYQQ9E0IIQR9E0IIQd+EEIK+CSGEoG9CCCHomxBC0DchhJDA1jchhJBaFPRNCCHomxBCSE3l/wOElkIjUdGBzgAAAABJRU5ErkJggg==%}

#####Transport

Transport层提供了一个抽象的网络读写接口,解耦网络层和其他层之间的关系(比如序列化/反序列化)
Transport层接口如下

- open

- close

- read

- write

- flush

在Transport层上thrift还定义了ServerTransport层来提供接受或者创建transport对象的功能。顾名思义,该层用于为进入的链接创建新的
transport对象,其包含接口如下



- open

- listen

- accept

- close

对于thrift支持的语言来说,下面的接口都是可用的

- file: read/write to/from a file on disk

- http: as the name suggests

#####Protocol

负责内存中的数据结构和网络协议的转化

py writeMessageBegin(name, type, seq)

writeMessageEnd()
writeStructBegin(name)
writeStructEnd()
writeFieldBegin(name, type, id)
writeFieldEnd()
writeFieldStop()
writeMapBegin(ktype, vtype, size)
writeMapEnd()
writeListBegin(etype, size)
writeListEnd()
writeSetBegin(etype, size)
writeSetEnd()
writeBool(bool)
writeByte(byte)
writeI16(i16)
writeI32(i32)
writeI64(i64)
writeDouble(double)

writeString(string)

name, type, seq = readMessageBegin()
readMessageEnd()
name = readStructBegin()
readStructEnd()
name, type, id = readFieldBegin()
readFieldEnd()
k, v, size = readMapBegin()
readMapEnd()
etype, size = readListBegin()
readListEnd()
etype, size = readSetBegin()
readSetEnd()
bool = readBool()
byte = readByte()
i16 = readI16()
i32 = readI32()
i64 = readI64()
double = readDouble()

string = readString()

1
2
3
4
5
6
7
8
9
10
thrift的协议格式是流式的,不需要应用层进行显式的分页,因此对于thrift格式来说,复杂数据结构的大小没必要事先定义,比如string的长度
以及list内的个数。thrift支持三种编码方式

- binary: 简单的二进制编码
- compact: 压缩编码,参见[THRIFT-110](https://issues.apache.org/jira/browse/THRIFT-110)
- json:

#####Processor

Processor封装了从流中读取和写入的函数,读取和写入的都是Protocol对象,简单例子如下

java interface TProcessor {

    bool process(TProtocol in, TProtocol out) throws TException

}

1
2
3
4
5
6
7
8
9
10
11
12
由特定代码生成的服务中的Processor,在流中读取数据,将结构委托给相应的处理程序,然后将处理结果的对象写入流中

#####Server

一个服务具有上面描述的所有性质

- 创建一个transport
- 为每个transport创建一个protocol
- 根据读取/写入protocol创建processor
- 等待链接接入并进行处理

IDL例子

namespace cpp thrift.example namespace java thrift.example

enum TweetType {

TWEET,
RETWEET = 2,
DM = 0xa,
REPLY

}

struct Location { 1: required double latitude; 2: required double longitude; }

struct Tweet { 1: required i32 userId; 2: required string userName; 3: required string text; 4: optional Location loc; 5: optional TweetType tweetType = TweetType.TWEET; 16: optional string language = “english”; }

typedef list TweetList

struct TweetSearchResult { 1: TweetList tweets; }

const i32 MAX_RESULTS = 100;

service Twitter {

void ping(),
     bool postTweet(1:Tweet tweet);
TweetSearchResult searchTweets(1:string query);
oneway void zip()

} “`

备注

thrift序列化属于传值调用,它将IDL中的值全部分配空间,而不是给未赋值的元素设置null。这样可能会导致内存占用多。service定义的函数在动态语言里 无法返回null,因此需要空值时首先初始化一个空的结构然后进行返回。对于transport可能会因为调用不同参数的同名函数导致彼此通讯异常。比如 初始版本的service含有postTweet(1: Tweet tweet), 但是新版本中转变为postTweet(1: Tweet tweet, 2: string group),则老版本中client 端调用该函数,而server端调用新版本进行解析则会导致新定义的参数为空。

thrift可以保持前向和后向的兼容性,可以在老的message中添加新的字段不会影响彼此解析,但是需要注意的是

  • 不要修改tag值
  • 新添加的字段定义为optional
  • 不再需要的字段可以被移除,只要其tag值不再使用
  • 可以修改默认值,但是发送端是无法将修改的默认值通过网络传输给接收端的

资源

pt包含部分thrift序列化方法 Thrift: The Missing Guide这是一篇教程文档

Comments