[聚合文章] NSQ分布式消息队列

消息系统 2017-11-23 23 阅读
NSQ分布式消息队列

NSQ分布式消息队列

苏依苏依
A realtime distributed messaging platform

一. 简介

开源社区可选的队列一搜一大把,断断续续 RabbitMQ/Celery/Kue 都有用过,但或多或少,因为语言以及维护、可扩展性方面,都不是很令人满意。

举例来说 RabbitMQ 语言不通,维护困难;Celery 总有奇奇怪怪的问题(我姿势不对?);Kue 依赖 Redis 就目前的使用来说,还是挺方便的,至少我的测试环境还没出个太大问题;唔,好像忘了 kafka ... (Java 万年黑表示不想多说)

想用 NSQ 的原因大概是:

  • golang 开发
  • 分布式设计
  • 部署简单
  • 客户端丰富
  • http 接口简单易用
  • 稳定性高?(官网文档写的,有待验证!)

二. 使用

据说部署简单,怎么个简单法呢?看下面:

# docker-compose.ymlversion: '3'services:  nsqlookupd:    image: nsqio/nsq    command: /nsqlookupd    # ports:    #   - 4160:4160    #   - 4161:4161  nsqd:    image: nsqio/nsq    command: /nsqd --lookupd-tcp-address=nsqlookupd:4160    depends_on:      - nsqlookupd    # ports:    #   - 4150:4150    #   - 4151:4151  # 这里是demo,不想要可以去掉  client:     build: ./nsq-client        command: npm run start    depends_on:      - nsqlookupd      - nsqd    nsqadmin:    image: nsqio/nsq    command: /nsqadmin --lookupd-http-address=nsqlookupd:4161    depends_on:      - nsqlookupd    ports:      - 4171:4171

运行:

docker-compose up 

输出:

➔ docker-compose upStarting nsqdemo_nsqlookupd_1 ...Starting nsqdemo_nsqlookupd_1 ... doneStarting nsqdemo_nsqadmin_1 ...Starting nsqdemo_nsqd_1 ...Starting nsqdemo_nsqd_1Starting nsqdemo_nsqd_1 ... doneStarting nsqdemo_client_1 ...Starting nsqdemo_client_1 ... doneAttaching to nsqdemo_nsqlookupd_1, nsqdemo_nsqadmin_1, nsqdemo_nsqd_1, nsqdemo_client_1nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:44.688524 nsqlookupd v1.0.0-compat (built w/go1.8)nsqadmin_1    | [nsqadmin] 2017/11/23 09:36:45.519141 nsqadmin v1.0.0-compat (built w/go1.8)nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:44.690764 TCP: listening on [::]:4160nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:44.690825 HTTP: listening on [::]:4161nsqadmin_1    | [nsqadmin] 2017/11/23 09:36:45.519747 HTTP: listening on [::]:4171nsqd_1        | [nsqd] 2017/11/23 09:36:45.626878 nsqd v1.0.0-compat (built w/go1.8)nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639070 TCP: new client(192.168.0.4:34714)nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639093 CLIENT(192.168.0.4:34714): desired protocol magic '  V1'nsqd_1        | [nsqd] 2017/11/23 09:36:45.627841 ID: 481nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639255 CLIENT(192.168.0.4:34714): IDENTIFY Address:56f210cb639a TCP:4150 HTTP:4151 Version:1.0.0-compatnsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639268 DB: client(192.168.0.4:34714) REGISTER category:client key: subkey:nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.645268 DB: client(192.168.0.4:34714) REGISTER category:topic key:sample_topic subkey:nsqd_1        | [nsqd] 2017/11/23 09:36:45.628718 TOPIC(sample_topic): creatednsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.648106 DB: client(192.168.0.4:34714) REGISTER category:channel key:sample_topic subkey:sample_topicnsqd_1        | [nsqd] 2017/11/23 09:36:45.628817 ERROR: no available nsqlookupd to query for channels to pre-create for topic sample_topicnsqd_1        | [nsqd] 2017/11/23 09:36:45.629965 TOPIC(sample_topic): new channel(sample_topic)nsqd_1        | [nsqd] 2017/11/23 09:36:45.630113 NSQ: persisting topic/channel metadata to nsqd.datnsqd_1        | [nsqd] 2017/11/23 09:36:45.634888 TCP: listening on [::]:4150nsqd_1        | [nsqd] 2017/11/23 09:36:45.635230 HTTP: listening on [::]:4151nsqd_1        | [nsqd] 2017/11/23 09:36:45.636008 LOOKUP(nsqlookupd:4160): adding peernsqd_1        | [nsqd] 2017/11/23 09:36:45.636032 LOOKUP connecting to nsqlookupd:4160nsqd_1        | [nsqd] 2017/11/23 09:36:45.639595 LOOKUPD(nsqlookupd:4160): peer info {TCPPort:4160 HTTPPort:4161 Version:1.0.0-compat BroadcastAddress:b10017538479}nsqd_1        | [nsqd] 2017/11/23 09:36:45.642355 LOOKUPD(nsqlookupd:4160): topic REGISTER sample_topicnsqd_1        | [nsqd] 2017/11/23 09:36:45.645742 LOOKUPD(nsqlookupd:4160): channel REGISTER sample_topic sample_topicnsqd_1        | [nsqd] 2017/11/23 09:36:45.648649 LOOKUPD(nsqlookupd:4160): REGISTER sample_topic sample_topicclient_1      | npm info it worked if it ends with okclient_1      | npm info using npm@5.3.0client_1      | npm info using node@v8.6.0client_1      | npm info lifecycle nsq-client@1.0.0~prestart: nsq-client@1.0.0client_1      | npm info lifecycle nsq-client@1.0.0~start: nsq-client@1.0.0client_1      |client_1      | > nsq-client@1.0.0 start /usr/src/appclient_1      | > node index.jsclient_1      |nsqd_1        | [nsqd] 2017/11/23 09:36:47.295258 TCP: new client(192.168.0.5:47104)nsqd_1        | [nsqd] 2017/11/23 09:36:47.312326 CLIENT(192.168.0.5:47104): desired protocol magic '  V2'nsqd_1        | [nsqd] 2017/11/23 09:36:47.318512 [192.168.0.5:47104] IDENTIFY: {ClientID:0b1264dbb8b4 Hostname:0b1264dbb8b4 HeartbeatInterval:30000 OutputBufferSize:0 OutputBufferTimeout:0 FeatureNegotiation:true TLSv1:false Deflate:false DeflateLevel:6 Snappy:false SampleRate:0 UserAgent:nsqjs/0.10.0 MsgTimeout:0}nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:47.327151 200 GET /lookup?topic=sample_topic (192.168.0.5:51278) 28.186µsclient_1      | Message sent successfullynsqd_1        | [nsqd] 2017/11/23 09:36:47.333635 ERROR: [192.168.0.5:47104] - E_INVALID cannot CLS in current statensqd_1        | [nsqd] 2017/11/23 09:36:47.334235 PROTOCOL(V2): [192.168.0.5:47104] exiting ioloopnsqd_1        | [nsqd] 2017/11/23 09:36:47.334762 ERROR: client(192.168.0.5:47104) - E_INVALID cannot CLS in current statensqd_1        | [nsqd] 2017/11/23 09:36:47.334901 PROTOCOL(V2): [192.168.0.5:47104] exiting messagePumpclient_1      | Writer closednsqd_1        | [nsqd] 2017/11/23 09:36:47.353106 TCP: new client(192.168.0.5:47108)nsqd_1        | [nsqd] 2017/11/23 09:36:47.353424 CLIENT(192.168.0.5:47108): desired protocol magic '  V2'nsqd_1        | [nsqd] 2017/11/23 09:36:47.354359 [192.168.0.5:47108] IDENTIFY: {ClientID:0b1264dbb8b4 Hostname:0b1264dbb8b4 HeartbeatInterval:30000 OutputBufferSize:0 OutputBufferTimeout:0 FeatureNegotiation:true TLSv1:false Deflate:false DeflateLevel:6 Snappy:false SampleRate:0 UserAgent:nsqjs/0.10.0 MsgTimeout:0}client_1      | Received message [08e1925aee1e1000]: it really tied the room togetherclient_1      | Received message [08e1925aee1e1001]: This message gonna arrive 1 sec later.client_1      | Received message [08e1925aee5e1000]: Uh, excuse me. Mark it zero. Next frame.client_1      | Received message [08e1925aee5e1001]: Smokey, this is not 'Nam. This is bowling. There are rules.client_1      | Received message [08e1925aee5e1002]: Wu?

管理界面:

三. 参考