Michael Schurter

RFC: statemachine.State as an interface instead of a struct

1

## Pros to an interface - Users can tuck extra metadata into their implementation - Manually creating State structs smells funny ## Cons to an interface - Allows "clever" implementations...

RFC: Metafora v2 - Leader based task assignment; task dependency metadata

3

## Impetus Metafora's current peer-based work stealing approach has limitations: - In a cluster with nodes A and B, each with 10 tasks, if you start node C, any tasks...

## Problem There are a number of circumstances where a task cannot be executed on a node that has claimed it: - A new task type is being added via...

Metafora consumers may want to accept commands from sources other than their coordinator such as: - Signals - Builtin HTTP handler - RPC for cluster management There's really no reason...

As of 8f870a5a taskmgr should **never** have to release a task during `stop()`. This code should probably log or panic or something: https://github.com/lytics/metafora/blob/8f870a5a58f77c9cdb38e32e8356a8991b6250fc/m_etcd/taskmgr.go#L179-L180 Or do nothing except block on the...

From an internal metafora app: ``` 2015/01/01 05:32:01.548419 :50: [WARN] Unexpected error updating node key: 100: Key not found (/workflows/nodes/XYZ) [1682348] 2015/01/01 05:32:01.549067 :50: [ERROR] Error deleting handled command /workflows/nodes/XYZ:...

Tick based balancing instead of time

2

Currently Metafora rebalances every [15 minutes](https://github.com/lytics/metafora/blob/master/metafora.go#L73) [±10 seconds](https://github.com/lytics/metafora/blob/master/metafora.go#L15). The main loop serializes all events via ticks, so we could switch to rebalancing every `N` ticks. The theoretical advantage of tick-based...

@epsniff brought up a good point: > Could the Consumer have a Node() method? _via https://github.com/lytics/metafora/pull/79/files#r22186794_ Right now the _node name_ is an etcd-Coordinator specific concept. None of the core...

Looks like there's a bug in our etcd interaction that can cause metafora to shutdown (cleanly thankfully!). Full logs are here: https://gist.github.com/schmichael/48a2f521828689af7b40 Root problem appears to be this weird log...

## Thoughts on v2 of etcd coordinator ### Inefficiencies - Because tasks are a directory with a claim marker which needs constant refreshing, `Watch()` receives (and has to filter out)...