Waterstream Bridge Mode

Starting from Waterstream version 1.1.0 Bridge mode may be used to connect Waterstream to another MQTT broker and exchange messages between them so that they would appear as a single distributed broker. It connects to the remote broker as a regular MQTT client and therefore doesn’t need explicit support from it. Local connection bypasses the network layer and directly talks with Waterstream core.

Configuration

Environment variable MQTT_BRIDGES_CONFIG_FILE specifies location of the bridge config file. If running Waterstream in the Docker container - be sure to map the volume with the configuration file properly - e.g.

docker run ... \
  -e MQTT_BRIDGES_CONFIG_FILE=/etc/bridge.conf \
  -v bridge.conf:/etc/bridge.conf:ro ...

Here is an example configuration file:

connectionBackOffInitial: 5000
connectionSuccessMinDuration: 60000

bridges: [
  {
    name: "First Bridge"
    host: "broker1"
    port: 1883
    cleanSession: true
    localClientId: "s1"
    remoteClientId: "bridge1"
    topics: [
      {
        pattern: "area1/#"
        localPrefix: "a/"
        remotePrefix: "b/"
        direction: "OUT"
        qos: 2
      },
      {
        pattern: "#"
        localPrefix: "allIncoming/"
        direction: "IN"
      }
    ]
  },
  {
    name: "Second Bridge"
    host: "broker2"
    localClientId: "s2"
    remoteClientId: "b1"
    topics: [],
    ssl: {
      enabled: true,
      keyPath: "bridge2.key",
      certPath: "bridge2.crt"
    }
  }
]

It uses HOCON file format, which is quite similar to YAML.

Here are the top-level parameters:

  • connectionBackOffInitial - initial back-off time (milliseconds) between attempts to connect to the remote MQTT broker. Unsuccessful attempt will double this interval for the current bridge. Default: 1000.

  • connectionSuccessMinDuration - long (in milliseconds) the connection to the remote MQTT broker must survive to be considered successful and reset a back-off interval to its initial value. Default: 10000.

  • bridges - an array of bridge definitions. Each bridge may connect to a different broker.

Bridge-level parameters:

  • name - human-readable identifier of the bridge for logs and error reporting. Must be unique. If not specified - “host:port” is used whenever there’s a need to identify the bridge.

  • host - host of the MQTT broker. Mandatory parameter.

  • port - port of the MQTT broker. Default is 1883

  • cleanSession - whether bridge should start with a clean session when connecting to the remote broker (and thus have a non-persistent session on that broker). Default is false.

  • sessionExpiryInterval - the session expiry interval in seconds for the bridge client. When the session expiry interval is set to 0 or not specified, the session will end when the network connection is closed. This setting applies only to MQTT 5.0 bridge clients.

  • localClientId - MQTT client ID of the bridge for the Waterstream. All though bridge doesn’t go through the network layer when talking to Waterstream, it needs a client ID to keep track of QoS 1 and QoS 2 messages state. As any MQTT client ID, must be unique across all the MQTT clients connected to the Waterstream. Mandatory parameter.

  • remoteClientId - MQTT client ID to use with the remote MQTT broker. Can be same or different with localClientId, just has to be unique across all clients connected to the remote broker. Mandatory parameter.

  • topics - an array of the topic mappings. If it is empty - no messages will be bridged. Mandatory parameter.

  • username, password - username and password for authentication with the remote broker. Default - not authenticate.

  • ssl - SSL configuration object. If not present - SSL is disabled.

  • protocolName, protocolLevel - specify MQTT protocol version to use with the remote broker. Default is MQTT, 4 - corresponds to MQTT 3.1.1. To use MQTT 5.0 specify protocolLevel: 5.

Topic-level parameters:

  • direction - IN or OUT. Direction of messages replication. Mandatory parameter.

  • pattern - MQTT pattern, can use the wildcards. On the remote MQTT broker will be prefixed with remotePrefix, on the Waterstream - with localPrefix.

  • localPrefix - prefix for the pattern on the Waterstream side. Default is empty.

  • remotePrefix - prefix for the pattern on the remote broker side. Default is empty.

SSL configuration object parameters:

  • enabled - if SSL connection to the remote broker is enabled. Default is false - no SSL.

  • keyPath, certPath - path to the PKCS8 key and X509 certificate which are used for client SSL authentication on the remote broker (remember that Waterstream Bridge works as MQTT client for the remote broker). If either of them is skipped - no client certificate is provided.

  • additionalCaCertsPath - comma-separated list of path to certificates of the authorities Waterstream Bridge trusts to when checking remote broker SSL certificate. Additional - because it doesn’t replace OS default trusted certificates, but adds some custom ones. Default - none, just trust OS default authorities.

  • crlsPath - comma-separated list of Certificate Revocation List PEM files - in case you’d like to invalidate remote broker certificates. Default - none.

Topic mapping is performed with localPrefix and removePrefix. For example - if pattern is #, localPrefix is a/ and remotePrefix is b/, then having IN mapping with this parameters the message on the remote broker in the b/sensor1 topic will be published into a/sensor1 topic in Waterstream. If there’s OUT mapping with the same config - a message from a/sensor2 topic in Waterstream will be published into b/sensor2 topic of the remote broker.

Be careful not to create infinite loops with the topic mapping if you’re using MQTT version 3.X - there’s no automatic detection of such situations. If you use MQTT v5.0 or newer, however, noLocal option is used which means that the remote broker wouldn’t send us back the messages we’ve sent there - so no worries about the loops in this case.

Retained messages handling

MQTT protocol requires servers to only set RETAIN flag in the messages sent to client if the message was sent as a result of new subscription. Therefore, if a message is published to the remote MQTT broker with RETAIN=true, Waterstream will know about this flag only if the bridge is started after that publish. Otherwise, Waterstream will see just a regular message. In the opposite direction, the bridge isn’t a regular client for the Waterstream, therefore it can detect if the message had RETAIN flag. So, if a client publishes a message to Waterstream with RETAIN=true, a bridge will also publish a message with RETAIN=true to the remote MQTT broker.