Flo.w

# Rolling Stats Library

flow-rolling-stats (opens new window) is a JavaScript library of rolling time series operators for unevenly spaced data. It is designed to be used in Node.js back-end stream processing services.

The library provides simple moving averages (SMAs), exponential moving averages (EMAs), and various rolling functions. It is based on the utsAlgorithms C library (opens new window).

The implementation details are described in "Algorithms for Unevenly Spaced Time Series: Moving Averages and Other Rolling Operators", Eckner (2017) (opens new window).

# Installation

To install the library from NPM, issue the following command in your project directory:

npm install flow-rolling-stats

# API

Operators are implemented as JavaScript classes with a standard interface. They are designed to be used in real-time processing pipeline where observations are received periodically at uneven time intervals.

Typical usage is shown below (where RollingMean can be replaced with any of the supported operator classes):

const windowedAvg1 = new RollingMean({
  windowSize: { duration: 15, unit: 'minutes' }
});

const windowedAvg2 = new RollingMean({
  windowSize: { duration: 15, unit: 'minutes' }
});

// Process an event as it arrives
on('event', event => {

  // Construct metric
  const metric = {
    timestamp: event.timestamp;
    avgValue1: windowedAvg1.update(event.value1, event.timestamp);
    avgValue2: windowedAvg2.update(event.value2, event.timestamp);
  };

  // Send metric to next stage of pipeline
  emit(metric);
});

# Operators

Operator Description
Rolling Operators
RollingCount Count of observations in window.
RolingMin Minimum observation in window.
RolingMax Maximum observation in window.
RolingSum Sum of observations in window.
RolingMean Mean of observations in window.
Simple Moving Averages
SmaLast Simple moving average using last value.
SmaNext Simple moving average using next value.
SmaLinear Simple moving average using linear interpolation.
Exponential Weighted Moving Averages
EmaLast EWMA using last value.
EmaNext EWMA using next value.
EmaLinear EWMA using linear interpolation.

# Operator Properties & Methods

Property/Method Description
length Number of observations in window.
value Current operator value.
update(value: T, time?: number) Record an observation value at specified time (in Epoch milliseconds). If time is not specified the current time is used.

# Window Size

Window size is specified when constructing an operator class. Supported time units are:

  • ms/milliseconds
  • second/seconds
  • minute/minutes
  • hour/hours
  • day/days

# Recording Observations

Call the update method of an operation to record a new observation. Pass the observation value and timestamp (in epoch milliseconds) as parameters. The update method returns the calculated statistic after the window has been updated.

Note that value property returns the calculated statistic and can be called at any time without performing a window update.

# Numerical Stability

Many of the operators maintain a sum that is updated over the lifetime of the operator. To maintain numerical stability and negate the effects of limited floating point precision the method of Kahan (1965)[1] is used to perform compensated addition.

[1] Kahan, W. (1965). Further remarks on reducing truncation errors Communications of the ACM 8 (1), 40.

← Overview →