
NSQ分布式消息队列
A realtime distributed messaging platform
一. 简介
开源社区可选的队列一搜一大把,断断续续 RabbitMQ/Celery/Kue 都有用过,但或多或少,因为语言以及维护、可扩展性方面,都不是很令人满意。
举例来说 RabbitMQ 语言不通,维护困难;Celery 总有奇奇怪怪的问题(我姿势不对?);Kue 依赖 Redis 就目前的使用来说,还是挺方便的,至少我的测试环境还没出个太大问题;唔,好像忘了 kafka ... (Java 万年黑表示不想多说)
想用 NSQ 的原因大概是:
- golang 开发
- 分布式设计
- 部署简单
- 客户端丰富
- http 接口简单易用
- 稳定性高?(官网文档写的,有待验证!)
二. 使用
据说部署简单,怎么个简单法呢?看下面:
# docker-compose.ymlversion: '3'services: nsqlookupd: image: nsqio/nsq command: /nsqlookupd # ports: # - 4160:4160 # - 4161:4161 nsqd: image: nsqio/nsq command: /nsqd --lookupd-tcp-address=nsqlookupd:4160 depends_on: - nsqlookupd # ports: # - 4150:4150 # - 4151:4151 # 这里是demo,不想要可以去掉 client: build: ./nsq-client command: npm run start depends_on: - nsqlookupd - nsqd nsqadmin: image: nsqio/nsq command: /nsqadmin --lookupd-http-address=nsqlookupd:4161 depends_on: - nsqlookupd ports: - 4171:4171
运行:
docker-compose up
输出:
➔ docker-compose upStarting nsqdemo_nsqlookupd_1 ...Starting nsqdemo_nsqlookupd_1 ... doneStarting nsqdemo_nsqadmin_1 ...Starting nsqdemo_nsqd_1 ...Starting nsqdemo_nsqd_1Starting nsqdemo_nsqd_1 ... doneStarting nsqdemo_client_1 ...Starting nsqdemo_client_1 ... doneAttaching to nsqdemo_nsqlookupd_1, nsqdemo_nsqadmin_1, nsqdemo_nsqd_1, nsqdemo_client_1nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:44.688524 nsqlookupd v1.0.0-compat (built w/go1.8)nsqadmin_1 | [nsqadmin] 2017/11/23 09:36:45.519141 nsqadmin v1.0.0-compat (built w/go1.8)nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:44.690764 TCP: listening on [::]:4160nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:44.690825 HTTP: listening on [::]:4161nsqadmin_1 | [nsqadmin] 2017/11/23 09:36:45.519747 HTTP: listening on [::]:4171nsqd_1 | [nsqd] 2017/11/23 09:36:45.626878 nsqd v1.0.0-compat (built w/go1.8)nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:45.639070 TCP: new client(192.168.0.4:34714)nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:45.639093 CLIENT(192.168.0.4:34714): desired protocol magic ' V1'nsqd_1 | [nsqd] 2017/11/23 09:36:45.627841 ID: 481nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:45.639255 CLIENT(192.168.0.4:34714): IDENTIFY Address:56f210cb639a TCP:4150 HTTP:4151 Version:1.0.0-compatnsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:45.639268 DB: client(192.168.0.4:34714) REGISTER category:client key: subkey:nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:45.645268 DB: client(192.168.0.4:34714) REGISTER category:topic key:sample_topic subkey:nsqd_1 | [nsqd] 2017/11/23 09:36:45.628718 TOPIC(sample_topic): creatednsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:45.648106 DB: client(192.168.0.4:34714) REGISTER category:channel key:sample_topic subkey:sample_topicnsqd_1 | [nsqd] 2017/11/23 09:36:45.628817 ERROR: no available nsqlookupd to query for channels to pre-create for topic sample_topicnsqd_1 | [nsqd] 2017/11/23 09:36:45.629965 TOPIC(sample_topic): new channel(sample_topic)nsqd_1 | [nsqd] 2017/11/23 09:36:45.630113 NSQ: persisting topic/channel metadata to nsqd.datnsqd_1 | [nsqd] 2017/11/23 09:36:45.634888 TCP: listening on [::]:4150nsqd_1 | [nsqd] 2017/11/23 09:36:45.635230 HTTP: listening on [::]:4151nsqd_1 | [nsqd] 2017/11/23 09:36:45.636008 LOOKUP(nsqlookupd:4160): adding peernsqd_1 | [nsqd] 2017/11/23 09:36:45.636032 LOOKUP connecting to nsqlookupd:4160nsqd_1 | [nsqd] 2017/11/23 09:36:45.639595 LOOKUPD(nsqlookupd:4160): peer info {TCPPort:4160 HTTPPort:4161 Version:1.0.0-compat BroadcastAddress:b10017538479}nsqd_1 | [nsqd] 2017/11/23 09:36:45.642355 LOOKUPD(nsqlookupd:4160): topic REGISTER sample_topicnsqd_1 | [nsqd] 2017/11/23 09:36:45.645742 LOOKUPD(nsqlookupd:4160): channel REGISTER sample_topic sample_topicnsqd_1 | [nsqd] 2017/11/23 09:36:45.648649 LOOKUPD(nsqlookupd:4160): REGISTER sample_topic sample_topicclient_1 | npm info it worked if it ends with okclient_1 | npm info using npm@5.3.0client_1 | npm info using node@v8.6.0client_1 | npm info lifecycle nsq-client@1.0.0~prestart: nsq-client@1.0.0client_1 | npm info lifecycle nsq-client@1.0.0~start: nsq-client@1.0.0client_1 |client_1 | > nsq-client@1.0.0 start /usr/src/appclient_1 | > node index.jsclient_1 |nsqd_1 | [nsqd] 2017/11/23 09:36:47.295258 TCP: new client(192.168.0.5:47104)nsqd_1 | [nsqd] 2017/11/23 09:36:47.312326 CLIENT(192.168.0.5:47104): desired protocol magic ' V2'nsqd_1 | [nsqd] 2017/11/23 09:36:47.318512 [192.168.0.5:47104] IDENTIFY: {ClientID:0b1264dbb8b4 Hostname:0b1264dbb8b4 HeartbeatInterval:30000 OutputBufferSize:0 OutputBufferTimeout:0 FeatureNegotiation:true TLSv1:false Deflate:false DeflateLevel:6 Snappy:false SampleRate:0 UserAgent:nsqjs/0.10.0 MsgTimeout:0}nsqlookupd_1 | [nsqlookupd] 2017/11/23 09:36:47.327151 200 GET /lookup?topic=sample_topic (192.168.0.5:51278) 28.186µsclient_1 | Message sent successfullynsqd_1 | [nsqd] 2017/11/23 09:36:47.333635 ERROR: [192.168.0.5:47104] - E_INVALID cannot CLS in current statensqd_1 | [nsqd] 2017/11/23 09:36:47.334235 PROTOCOL(V2): [192.168.0.5:47104] exiting ioloopnsqd_1 | [nsqd] 2017/11/23 09:36:47.334762 ERROR: client(192.168.0.5:47104) - E_INVALID cannot CLS in current statensqd_1 | [nsqd] 2017/11/23 09:36:47.334901 PROTOCOL(V2): [192.168.0.5:47104] exiting messagePumpclient_1 | Writer closednsqd_1 | [nsqd] 2017/11/23 09:36:47.353106 TCP: new client(192.168.0.5:47108)nsqd_1 | [nsqd] 2017/11/23 09:36:47.353424 CLIENT(192.168.0.5:47108): desired protocol magic ' V2'nsqd_1 | [nsqd] 2017/11/23 09:36:47.354359 [192.168.0.5:47108] IDENTIFY: {ClientID:0b1264dbb8b4 Hostname:0b1264dbb8b4 HeartbeatInterval:30000 OutputBufferSize:0 OutputBufferTimeout:0 FeatureNegotiation:true TLSv1:false Deflate:false DeflateLevel:6 Snappy:false SampleRate:0 UserAgent:nsqjs/0.10.0 MsgTimeout:0}client_1 | Received message [08e1925aee1e1000]: it really tied the room togetherclient_1 | Received message [08e1925aee1e1001]: This message gonna arrive 1 sec later.client_1 | Received message [08e1925aee5e1000]: Uh, excuse me. Mark it zero. Next frame.client_1 | Received message [08e1925aee5e1001]: Smokey, this is not 'Nam. This is bowling. There are rules.client_1 | Received message [08e1925aee5e1002]: Wu?
管理界面:
三. 参考
- A realtime distributed messaging platform
- demo 代码: thonatos/nsq-demo
- node client: dudleycarr/nsqjs