Thrift介绍、编译参数以及如何和maven集成

什么是Thrift?

Thrift是一种接口描述语言和二进制通讯协议,它被用来定义和创建跨语言的服务。它被当作一个远程过程调用(RPC)框架来使用,是由Facebook为“大规模跨语言服务开发”而开发的。它通过一个代码生成引擎联合了一个软件栈,来创建不同程度的、无缝的跨平台高效服务,可以使用C#、C++(基于POSIX兼容系统)、Cappuccino、Cocoa、Delphi、Erlang、Go、Haskell、Java、Node.js、OCaml、Perl、PHP、Python、Ruby和Smalltalk。虽然它以前是由Facebook开发的,但它现在是Apache软件基金会的开源项目了。该实现被描述在2007年4月的一篇由Facebook发表的技术论文中,该论文现由Apache掌管。

目前流行的服务调用方式有很多种,例如基于 SOAP 消息格式的 Web Service,基于 JSON 消息格式的 RESTful 服务等。其中所用到的数据传输方式包括 XML,JSON 等,然而 XML 相对体积太大,传输效率低,JSON 体积较小,新颖,但还不够完善。本文将介绍由 Facebook 开发的远程服务调用框架 Apache Thrift,它采用接口描述语言定义并创建服务,支持可扩展的跨语言服务开发,所包含的代码生成引擎可以在多种语言中,如 C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk 等创建高效的、无缝的服务,其传输数据采用二进制格式,相对 XML 和 JSON 体积更小,对于高并发、大数据量和多语言的环境更有优势。本文将详细介绍 Thrift 的使用,并且提供丰富的实例代码加以解释说明,帮助使用者快速构建服务。Thrift的下载地址是:http://thrift.apache.org/download

thrift.exe支持的参数有哪些?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192

Microsoft Windows [版本 10.0.18362.657]
(c) 2019 Microsoft Corporation。保留所有权利。

~>thrift-0.13.0.exe -help
Usage: thrift [options] file
Options:
-version Print the compiler version
-o dir Set the output directory for gen-* packages
(default: current directory)
-out dir Set the ouput location for generated files.
(no gen-* folder will be created)
-I dir Add a directory to the list of directories
searched for include directives
-nowarn Suppress all compiler warnings (BAD!)
-strict Strict compiler warnings on
-v[erbose] Verbose mode
-r[ecurse] Also generate included files
-debug Parse debug trace to stdout
--allow-neg-keys Allow negative field keys (Used to preserve protocol
compatibility with older .thrift files)
--allow-64bit-consts Do not print warnings about using 64-bit constants
--gen STR Generate code with a dynamically-registered generator.
STR has the form language[:key1=val1[,key2[,key3=val3]]].
Keys and values are options passed to the generator.
Many options will not require values.

Options related to audit operation
--audit OldFile Old Thrift file to be audited with 'file'
-Iold dir Add a directory to the list of directories
searched for include directives for old thrift file
-Inew dir Add a directory to the list of directories
searched for include directives for new thrift file

Available generators (and options):
as3 (AS3):
bindable: Add [bindable] metadata to all the struct classes.
c_glib (C, using GLib):
cl (Common Lisp):
no_asd: Do not define ASDF systems for each generated Thrift program.
sys_pref= The prefix to give ASDF system names. Default: thrift-gen-
cpp (C++):
cob_style: Generate "Continuation OBject"-style classes.
no_client_completion:
Omit calls to completion__() in CobClient class.
no_default_operators:
Omits generation of default operators ==, != and <
templates: Generate templatized reader/writer methods.
pure_enums: Generate pure enums instead of wrapper classes.
include_prefix: Use full include paths in generated files.
moveable_types: Generate move constructors and assignment operators.
no_ostream_operators:
Omit generation of ostream definitions.
no_skeleton: Omits generation of skeleton.
csharp (C#):
async: Adds Async support using Task.Run.
wcf: Adds bindings for WCF to generated classes.
serial: Add serialization support to generated classes.
nullable: Use nullable types for properties.
hashcode: Generate a hashcode and equals implementation for classes.
union: Use new union typing, which includes a static read function for union types.
d (D):
dart (Dart):
library_name: Optional override for library name.
library_prefix: Generate code that can be used within an existing library.
Use a dot-separated string, e.g. "my_parent_lib.src.gen"
pubspec_lib: Optional override for thrift lib dependency in pubspec.yaml,
e.g. "thrift: 0.x.x". Use a pipe delimiter to separate lines,
e.g. "thrift:| git:| url: git@foo.com"
delphi (delphi):
ansistr_binary: Use AnsiString for binary datatype (default is TBytes).
register_types: Enable TypeRegistry, allows for creation of struct, union
and container instances by interface or TypeInfo()
constprefix: Name TConstants classes after IDL to reduce ambiguities
events: Enable and use processing events in the generated code.
xmldoc: Enable XMLDoc comments for Help Insight etc.
async: Generate IAsync interface to use Parallel Programming Library (XE7+ only).
erl (Erlang):
legacynames: Output files retain naming conventions of Thrift 0.9.1 and earlier.
maps: Generate maps instead of dicts.
otp16: Generate non-namespaced dict and set instead of dict:dict and sets:set.
go (Go):
package_prefix= Package prefix for generated files.
thrift_import= Override thrift package import path (default:github.com/apache/thrift/lib/go/thrift)
package= Package name (default: inferred from thrift file name)
ignore_initialisms
Disable automatic spelling correction of initialisms (e.g. "URL")
read_write_private
Make read/write methods private, default is public Read/Write
gv (Graphviz):
exceptions: Whether to draw arrows from functions to exception.
haxe (Haxe):
callbacks Use onError()/onSuccess() callbacks for service methods (like AS3)
rtti Enable @:rtti for generated classes and interfaces
buildmacro=my.macros.Class.method(args)
Add @:build macro calls to generated classes and interfaces
hs (Haskell):
html (HTML):
standalone: Self-contained mode, includes all CSS in the HTML files.
Generates no style.css file, but HTML files will be larger.
noescape: Do not escape html in doc text.
java (Java):
beans: Members will be private, and setter methods will return void.
private-members: Members will be private, but setter methods will return 'this' like usual.
nocamel: Do not use CamelCase field accessors with beans.
fullcamel: Convert underscored_accessor_or_service_names to camelCase.
android: Generated structures are Parcelable.
android_legacy: Do not use java.io.IOException(throwable) (available for Android 2.3 and above).
option_type: Wrap optional fields in an Option type.
rethrow_unhandled_exceptions:
Enable rethrow of unhandled exceptions and let them propagate futher. (Default behavior is to catch and log it.)
java5: Generate Java 1.5 compliant code (includes android_legacy flag).
reuse-objects: Data objects will not be allocated, but existing instances will be used (read and write).
sorted_containers:
Use TreeSet/TreeMap instead of HashSet/HashMap as a implementation of set/map.
generated_annotations=[undated|suppress]:
undated: suppress the date at @Generated annotations
suppress: suppress @Generated annotations entirely
unsafe_binaries: Do not copy ByteBuffers in constructors, getters, and setters.
javame (Java ME):
js (Javascript):
jquery: Generate jQuery compatible code.
node: Generate node.js compatible code.
ts: Generate TypeScript definition files.
with_ns: Create global namespace objects when using node.js
es6: Create ES6 code with Promises
thrift_package_output_directory=<path>:
Generate episode file and use the <path> as prefix
imports=<paths_to_modules>:
':' separated list of paths of modules that has episode files in their root
json (JSON):
merge: Generate output with included files merged
lua (Lua):
omit_requires: Suppress generation of require 'somefile'.
netcore (C#):
wcf: Adds bindings for WCF to generated classes.
serial: Add serialization support to generated classes.
nullable: Use nullable types for properties.
hashcode: Generate a hashcode and equals implementation for classes.
union: Use new union typing, which includes a static read function for union types.
netstd (C#):
wcf: Adds bindings for WCF to generated classes.
serial: Add serialization support to generated classes.
union: Use new union typing, which includes a static read function for union types.
ocaml (OCaml):
perl (Perl):
php (PHP):
inlined: Generate PHP inlined files
server: Generate PHP server stubs
oop: Generate PHP with object oriented subclasses
classmap: Generate old-style PHP files (use classmap autoloading)
rest: Generate PHP REST processors
nsglobal=NAME: Set global namespace
validate: Generate PHP validator methods
json: Generate JsonSerializable classes (requires PHP >= 5.4)
py (Python):
zope.interface: Generate code for use with zope.interface.
twisted: Generate Twisted-friendly RPC services.
tornado: Generate code for use with Tornado.
no_utf8strings: Do not Encode/decode strings using utf8 in the generated code. Basically no effect for Python 3.
coding=CODING: Add file encoding declare in generated file.
slots: Generate code using slots for instance members.
dynamic: Generate dynamic code, less code generated but slower.
dynbase=CLS Derive generated classes from class CLS instead of TBase.
dynfrozen=CLS Derive generated immutable classes from class CLS instead of TFrozenBase.
dynexc=CLS Derive generated exceptions from CLS instead of TExceptionBase.
dynimport='from foo.bar import CLS'
Add an import line to generated code to find the dynbase class.
package_prefix='top.package.'
Package prefix for generated files.
old_style: Deprecated. Generate old-style classes.
rb (Ruby):
rubygems: Add a "require 'rubygems'" line to the top of each generated file.
namespaced: Generate files in idiomatic namespaced directories.
rs (Rust):

st (Smalltalk):
swift (Swift 3.0):
log_unexpected: Log every time an unexpected field ID or type is encountered.
debug_descriptions:
Allow use of debugDescription so the app can add description via a cateogory/extension
async_clients: Generate clients which invoke asynchronously via block syntax.
namespaced: Generate source in Module scoped output directories for Swift Namespacing.
cocoa: Generate Swift 2.x code compatible with the Thrift/Cocoa library
promise_kit: Generate clients which invoke asynchronously via promises (only use with cocoa flag)
safe_enums: Generate enum types with an unknown case to handle unspecified values rather than throw a serialization error
xml (XML):
merge: Generate output with included files merged
no_default_ns: Omit default xmlns and add idl: prefix to all elements
no_namespaces: Do not add namespace definitions to the XML model
xsd (XSD):

基本用法:

thrift –gen 把一个thrift文件编译成对应语言的代码
thrift -r –gen 将所有 thrift所有引用的thrift全部编译成代码

常用做法是定义一个顶级的include.thrift,将所有其他的thrift文件全部包含进来,加上-r参数之后,这样所有的模型都能编译了。

常用参数解释:
-o dir gen-* 包的输出目录, 比如gen-java, 下面就是java的包
-out dir 生成文件的输出地址,gen-* 目录不会生成,和上面的命令互斥,用一个就行了。
-I dir 将目录加到引用文件的搜索路径列表中
–gen STR 用动态注册的生成器来产生代码。
这里的参数格式是这样的: language:[:key1=val1[,key2[,key3=val3]]]
冒号后的键和值是传递给生成器的参数。有的参数不需要值。

        以java为例:
        thrift.exe --gen java:beans,nocamel xxx.thrift
        的意思就是,生产java代码,传递给生成器的参数是beans和nocamel
        beans: 所有的成员变量私有,setter返回void
        nocamel:beans代码的accessor不用驼峰命名法。

如何支持maven集成

插件maven-thrift-plugin

插件的Github目录[2]上表示最小配置是这样的:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
  <build>
<plugins>
<plugin>
<groupId>org.apache.thrift.tools</groupId>
<artifactId>maven-thrift-plugin</artifactId>
<version>0.1.11</version>
<configuration>
<!--<thriftExecutable>/usr/local/bin/thrift</thriftExecutable>-->
<!--<thriftSourceRoot>src/main/thrift</thriftSourceRoot>-->
<!--<outputDirectory>src/main/java</outputDirectory>-->
</configuration>
<executions>
<execution>
<id>thrift-sources</id>
<phase>generate-sources</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
<!--<execution>-->
<!--<id>thrift-test-sources</id>-->
<!--<phase>generate-test-sources</phase>-->
<!--<goals>-->
<!--<goal>testCompile</goal>-->
<!--</goals>-->
<!--</execution>-->
</executions>
</plugin>
</plugins>
</build>

但是从我的使用来看,这个插件没有人维护,最后一次release的时间是2013年11月。插件使用起来有很多的问题。所以我建议是第二种方式。

maven-antrun-plugin

用这种方式虽然可能比较ugly,但是非常的灵活。几乎可以做你想用命令行或者脚本做到的任何事情。
我的配置是这样的:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>${maven-antrun-plugin.version}</version>
<executions>
<execution>
<phase>generate-sources</phase>
<configuration>
<target>
<property name="thrift.filename" value="thrift-${thrift.versoin}.exe"/>
<property name="thrift.filepath" value="${basedir}/../thrift/${thrift.filename}"/>
<property name="thrift.compile.result">0</property>

<mkdir dir="${thrift.output.directory}"/>
<!-- Define fileset of thrift files -->
<fileset id="thrift.src.files" dir="${thrift.input.directory}">
<include name="**/*.thrift"/>
</fileset>

<!-- Invoke thrift binary for each of these files -->
<apply executable="${thrift.filepath}" resultproperty="thrift.compile.result"
failifexecutionfails="true" failonerror="true"
searchpath="true" dir="${basedir}">
<arg value="-o"/>
<arg value="${thrift.output.directory}"/>
<arg value="--gen"/>
<arg value="java"/>
<srcfile/>
<fileset refid="thrift.src.files"/>
</apply>
<echo message="execution result is: ${thrift.compile.result}"></echo>
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>

这里重点介绍一下apply[3]的用法。 Apply是Ant的一个内置Task,用来执行一个系统命令。在windows上就是run, 在linux命令上类似bash。
解释一下这几个参数的用法:
executable 可执行文件的路径。
resultproperty 将命令的返回值保存到属性中。 实际上我们只有在命令执行报错的时候才会关注这个值。比如当我们将failonerror设置为true, 这个时候命令执行报错,但是构建不会停止,这个时候我们才需要将返回值保存下来留待后面检查。
failonerror 当命令返回值不是0的时候停止构建。默认false。
failifexecutionfails 如果无法启动程序的时候停止构建。默认true。

[4]就是一组文件,通过模式匹配或者其他参数来搜索到一组文件。比较简单。

也有兴趣也可以翻阅一下,其他一些ant任务的介绍[5]


  1. 1.https://www.ibm.com/developerworks/cn/java/j-lo-apachethrift/index.html
  2. 2.https://github.com/dtrott/maven-thrift-plugin
  3. 3.https://ant.apache.org/manual/Tasks/apply.html
  4. 4.https://ant.apache.org/manual/Types/fileset.html
  5. 5.https://blog.csdn.net/fanxiaobin577328725/article/details/53699735