Flo.w

# MQTT Ingest Endpoint

The endpoint for Flo.w Real-Time ingest is an MQTT broker supporting the MQTT v3.1.1 protocol (opens new window).

# MQTT Ports

By default, the Flo.w Real-Time MQTT broker uses the same domain name as Flo.w engine and exposes the following ports:

Port Description Example URL
1883 Standard MQTT over TCP mqtt://flow.emu-analytics.net:1883
8883 Secure MQTT over TCP with TLS mqtts://flow.emu-analytics.net:8883
8888 MQTT over WebSockets ws://flow.emu-analytics.net:8888
8889 Secure MQTT over WebSockets with TLS wss://flow.emu-analytics.net:8889

# Client Libraries

MQTT client libraries are available for many programming environments and languages.

For Node.js and browser environments, we recommend:

# Authentication

Connection to MQTT brokers is authenticated by a username and password as described in the MQTT v3.1.1 specification (opens new window).

To connect to the Flo.w Real-Time MQTT broker use:

Field Description
Application API Keys
Username Flo.w Application ID
Password Flo.w application API key
Master Keys * see below
Username master
Password Flo.w master API key

The supplied API key must be associated with the specified Application ID and must grant permission to publish and/or subscribe to MQTT topics. See API Keys for more details of API keys and access control lists (ACLs).

The MQTT wildcards, '#' (multi-level) and '+' (single-level), can be used in ACL resource identifiers.

Example API Key permissions are shown below:

// Can publish and subscribe to any application MQTT topic.
{ "mqtt": {"*": "*"}}

// Can publish to any MQTT topic.
{ "mqtt": {"write": "*"}}

// Can subscribe to any MQTT topic.
{ "mqtt": {"read": "*"}}

// Can subscribe to any MQTT topic and publish to dataset 'ingest1' change topic.
{ "mqtt": {"read": "*", "write": "dataset/ingest1/changes"}}

// Can subscribe to any MQTT topic and publish to the 'change' topics for datasets 'ingest1' and 'ingest2'.
{ "mqtt": {"read": "*", "write": ["dataset/ingest1/changes", "dataset/ingest2/changes"]}}

// Can subscribe to any MQTT topic and publish to **any** dataset change topic.
{ "mqtt": {"read": "*", "write": "dataset/+/changes"}}

Application Namespaces

Note that MQTT topic names specified in ACLs should not include the Flo.w Application namespace prefix <applicationID>/. See below:

# Message Topics

# Topic Namespaces

Flo.w MQTT topics are split into application-specific namespaces using a topic name prefix: <applicationID>/. This ensures that Flo.w Applications are only permitted to publish and subscribe to their own topics.

MQTT connections authenticated with an application API Key are restricted to the corresponding application-specific namespace.

MQTT connections authenticated with username: master and a master API Key can subscribe or publish to topics in any namespace (subject to API Key ACL restrictions).

# Ingest topics

To ingest data into a dataset, publish an ingest message to the changes topic associated with the dataset.

Ingest topic: <applicationID>/dataset/<datasetID>/changes

# Error topics

To receive ingest errors, subscribe to the errors topic associated with the dataset. Each ingest error will be emitted as a separate message to the errors topic.

Ingest error messages have a JSON-formatted payload containing a message property and additional error-specific details.

Errors topic: <applicationID>/dataset/<datasetID>/errors

# Custom Topics

A Flo.w Application is free to subscribe or publish to any topic under its namespace (subject to API Key ACL restrictions). Application developers may choose to use the Flo.w MQTT broker to implement real-time messaging between back-end servers and front-end web applications.

← MQTT Messages →