Meteor and Redux and Angular 2

Like many JavaScript developers, my progression has roughly paralleled that of the industry. When I learned JavaScript, it was as simple as “on mouse over, change color”. Angular 1 and REST ushered in Single Page Applications, RxJS taught us that your app can be thought of as a collection of streams and React, and now Angular 2, provides a way of structuring your apps as hierarchical components.

Each technology had massive impacts on the way you think about your code. To take advantage of the benefits of the technology, you need to abandon your previous approaches and thinking.

Now Redux provides a robust pattern for organizing all your application events and this is one of the biggest mind shifts yet.

In this post I’ll dive into using Redux with Angular 2 and Meteor.  But first:

I Love Redux, what can Meteor do for Me?

If you’re already a big Redux user, but haven’t used Meteor (lately), here’s your TL;DR:

  • At Meteor’s core, is DDP – Distributed Data Protocol. DDP allows you to specify a subset of data that you’re interested in, and when it changes, it shows up in your client via a web socket. (Mongo only for now.)
  • It is trivial to convert these changes into an Observable. (Code is discussed below and you grab it here.)
  • You then use the Observable in your middleware, so changes to the Mongo database just show up as Redux actions for you to process as you please.
  • That’s it! (Well, there’s lots more, but that is my take on a Redux centric way of looking at it and the approach I’ll be exploring in this post.)

I Love Meteor, what’s this Redux Thing?

Redux provides a central nervous system for events and actions to derive a clean, well-defined application state.

If your app is getting hopelessly intertwined leading to the inability to make feature changes, or if you’re struggling with asynchronous “what-happens-when” problems, then consider refactoring it in.  When I did so, it took out the bulk of the spaghetti code that had crept in.

If you’re starting a new app, and have time to learn some cool new stuff, give it a go.  I suspect it will be a dominant approach for some time.

A New Way of Thinking

The basics of Redux are simple:

  • A reducer changes state with pure functions,
  • actions (that have an action type and payload) are fired to trigger calculation of a new state using pure functions, and
  • middleware handle anything asynchronous (actions in, actions out)

Easy, right?

The basic principles are easy to grasp.  But this is such a radically different way of writing application logic, it is challenging (it was for me, at least), to put it into practice in the context of a typically complex single page application. The pay-off is there, and well worth it, but be prepared for it to be slow going at first.

Redux Starter Resources

Ready to learn more about Redux?  Here are some resources I found helpful in my journey that you might not bump into right away when you search.

First for some background thinking on simplicity, I recommend going back to 2012 and watching Rich Hickey’s talk Simplicity Matters. (https://youtu.be/rI8tNMsozo0?t=48). This drives home the value of simplicity and some approaches to achieving it.

One of the big changes you will need to get used to is the concept of immutability. Immutable data increases simplicity as described in this excellent talk by Lee Byron of Facebook (https://youtu.be/I7IdS-PbEgI?t=952.) The whole video is worth viewing, the link is to where Lee explains why immutability is a good idea.

And here is a great talk by Seth Davenport of Rangle on using Angular 2 with Redux. https://www.youtube.com/watch?v=s4xr2avwv3s .

And for a general introduction to reactive programming(in Angular 2, have a look at http://blog.angular-university.io/functional-reactive-programming-for-angular-2-developers-rxjs-and-observables/.

Rolling Up the Sleeves with Meteor, Angular 2 and Redux

The work-in-progress app where I have been experimenting with Redux and Meteor is here. It is currently pinned to Angular 2 RC4 as I await various dependencies to catch up. At the time of this writing, I was part way through moving the app from a RxJS centric approach to a full Redux approach. (I.e. there are tons of bugs and it’s not always the way I would have implemented things if I was starting the app with Redux.)

Grouping Redux Modules and Redux Modules for Meteor

I found it helpful to formalize a group of reducer/middleware/action definitions as in:

import { Reducer } from "redux";
import { Epic } from 'redux-observable';

export abstract class ReduxModule<T> {
  reducer:Reducer<T>;
  epics:Epic[]=[];        // Stream based middleware
  middlewares:any[]=[];   // Normal redux middleware
  enhancers:any[]=[];
  actions:Object;
  abstract initialize():void;
}

This allows you to a have “module” for specific functional areas like Connecting, Login, Uploading, etc. (Perhaps there is a more established library for group, but I didn’t find one. Pointers are welcome.)

Here are some starter modules for Meteor.  It could be the start of a Redux-Meteor library. Let me know by commenting on this page or fill out the contact page if there is interest in collaborating to take this forward.  For now, it is far too immature to be a useful open source project.  It also was worth noting that some parts are mostly Angular free (with the exception of dependency injection), which open’s the possibility of client agnostic implementations.

Observing Changes

It is trivial to convert a reactive Meteor cursor to an Observable:

  static createCursorObserver<T>(cursor: Mongo.Cursor<any>): Observable<IDocumentChange<T>> {
    return Observable.create((observer: Subscriber<IDocumentChange<T>>) => {
      let handle: Meteor.LiveQueryHandle = cursor.observe({
        added: (doc: T) => {
          observer.next({
            changeType: EDocumentChangeType.NEW,
            newDocument: doc
          });
        },

        changed: (nDoc: T, oDoc: T) => {
          observer.next({
            changeType: EDocumentChangeType.CHANGED,
            newDocument: nDoc,
            oldDocument: oDoc
          });
        },

        removed: (doc: T) => {
          observer.next({
            changeType: EDocumentChangeType.REMOVED,
            oldDocument: doc
          });
        }
      });

      return function unsubscribe() {
        handle.stop();
      };
    });

You can then observe the cursor.  For example:

static createUserObserver(userId:string):Observable<IDocumentChange<User>>
{
  return MeteorCursorObservers.createCursorObserver<User>(
	Meteor.users.find({_id: userId})
  );
}

You will be notified of any changes to the portion of the collection you subscribed to.  This allowed me to do things like automatically updating a player’s avatar on all player’s views in a multi-player game as soon as one of the users uploaded a new photo.  (I.e. Meteor magic!)

Write and Respond Pattern

Common approach that emerged was to write to the database and link database changes to Redux events.

For example, you can make a database change locally to mini-mongo (which, of course, will get sync’d up to the server) or a meteor method call (like an RPC), and then an observer, like the one above, fires off a Redux action. For example, an Epic style Redux middleware (Observables in Observable’s out), calls a service to get and Observable of User changes and fires off a user change notification:

watchUsers = (action$: Observable<IPayloadAction>) => {
  return action$.filter(({ type }) => type === UsersActions.WATCH)
    .flatMap(({ payload }) => {
      return UsersService.createUsersObserver().map( (change:IDocumentChange<User>)=>{
        return UsersActions.changeFactory(change);
      });
    });
}

export class UsersActions {

  static changeFactory(documentChange:IDocumentChange<User>):IPayloadAction {
    return {type: UsersActions.CHANGE, payload: {documentChange:documentChange}};
  }

}

If you’re not used to reading RxJS, what this is saying is ‘watchUsers’ is a function that takes an Observable of Redux actions and only let’s through (.filter’s) WATCH request actions.  To the WATCH request, it adds (.flatMap’s) the User change events that have been converted (.map’d) to the Change Redux action.  (This requires someone firing the original WATCH to kick things off.)

Updating State

A Redux reducer is responsible for updating the application’s state.  It is given the current state and an action.  It is up to you to generate a new state.  And I really mean a new state.  You don’t modify the current state. You generate an entirely new state.  If you have a very small state, I find it best to do this like so:

export function loginReducer(
  state: ILoginState = INITIAL_STATE,
  action: IPayloadAction): ILoginState {

  let payload:ILoginAction = action.payload;
  switch (action.type) {
    case LoginActions.LOGIN_REQUEST:
      return Object.assign({}, state, {loggingIn: true});
    case LoginActions.LOGGED_IN:

(Object.assign creates a new copy of the object.)

But WAIT!  Won’t performance suck?  You have to make a complete copy of the application state just to change a single Boolean variable?

Well, the reality is that it only needs to be a logical copy, not a physical copy.  The essential tool for making this pattern work is the immutable.js library from Lee Byron at Facebook: https://facebook.github.io/immutable-js/. It provides an alternative implementation of Map (JavaScript objects) and Arrays (and others) that cleverly only copies the bare minimum while delivering immutability.  (Back to https://www.youtube.com/watch?v=I7IdS-PbEgI.)

Connecting to the Components

Ooooh Kay…

So how to do your Angular 2 components get at the state?  The indispensable tools there is ng-redux (https://github.com/angular-redux/ng2-redux) and typed-immutable-records (https://github.com/rangle/typed-immutable-record), which are being developed and maintained by Rangle.

Basically, you use the @select() decorator and it does all the work to provide you with an Observable of the state:

export class Avatar {
  @select() usersReducer;
  private imageURL:string;

  ngOnInit() {
    this.usersReducer.subscribe( (usersState:IUsersState)=>{
      this.imageURL = User.getAvatarURL( usersState.users.get(this.userId) );
    } );

  }


What’s happening here is that the @select decorator is using the name (‘usersReducer’) to find the part of the application state you’re interested in and creating an attribute that is an Observable of it.  Because it is an Observable, you can subscribe to it to get the most recent values.  There are ways of tying the observable directly to the form using Angular’s 2 built in RxJS support, but I have not explored this yet.

Sourcing Actions from the User

To complete the cook’s tour, let’s look at firing new actions based on user’s activity.  This will boil down to something like:

login(credentials:Credentials):void {
  this.ngRedux.dispatch(
   { 
      type: LoginActions.LOGIN_REQUEST, payload: {credentials: credentials}
   });
}

In the module pattern I’ve used, all the actions dispatchers and factories (makes an action for someone else to dispatch) are collected in a single class.

Request, Manage, Respond, Listen

In the example above, a common pattern can be seen. (Request, manage request, push out a response or exception, listen to the music).

Generally, a sequence is kicked off by a request action.  (Which of course returns immediately without blocking.)

A middleware function, reacts to the request and kicks off the real work, like a call to a server or setting up a watch.  This is where you are back to using Promises and RxJS to manage the asynchronous events.

When the server gives you some data (either as the result of a single request or a watch), you fire a response action. Your reducer updates the application state with the new data from the server.

Error handling is greatly simplified as error information can get stored in the state for notifying the user or handled by the middleware.

Your presentation tier components, sit back and wait for the information they need to arrive at their door step.

A Day in Trenches

I gained my experience by converting an experimental app. Previously, I had directly used RxJS to manage streams. Using streams was far better than not using streams, but Redux provided a much better way of organizing the functionality and addressed some problems that were starting to creep into the code base.

Here are my observations of the benefits I observed:

Lots of “boiler plate” when away

Managing streams directly required a lot of typing each time a new component wanted to tap into it.  With ng-redux, there is just a couple of lines of code to get notified of what you’re interested in. The tooling took care of unsubscribing and other housekeeping tasks.

Consolidated Business Logic

With non-Redux approaches (streamed or otherwise), bits and pieces of logic were smeared throughout the application.  It quickly became difficult to understand where events are fired and who was effected by them.  Managing order of events was still a wrestling match.

With Redux, it was very easy to stay organized in one or two files (the reducer and middleware), and it became a much easier to get a sense of the flow. Of course, I can be critical of my original implementation, but Redux led me down a path that made it easy to stay organized.

Way, Way Easier to Debug

There are advanced debugging tools that can tap into your event stream and even give you time travel debugging capabilities.  However, a 4-line middleware implementation can just console.log out your events and application state.  I found this more than sufficient and an immense improvement.  When things don’t work as expected, almost everything you need to know is in the console.  Even race conditions (like when I got my request/response verbs mixed up) where trivial to detect and fix

Easier to Develop Rich Functionality

It was suddenly obvious and trivial to implement things like file upload progress and cancel buttons.

Furthermore, the messy-ping-pong event handling of the past tended to cause a form of paralysis – your developer pain avoidance instincts, would cause you to not bother.  I found that with Redux, it was just a lot easier to reason about things.

For example, a problem occurred where I received events from a user before I was notified that the user was part of a game.  It was easy for me to buffer and release within the middleware components to provide a solution.  (If I was handier with RxJS, I probably could have handled it declaratively, but my head was spinning enough as it was.)

State is Rock Solidly Understood

How many times have been perplexed by your app’s behavior? Why is it doing that? What value is in this object? Why did it behave that way?  While, I’m still in my early days with Redux, this problem seems to be squarely addressed.  The state at the time of every action event is transparent, in a single, accessible location.

Less Deep Nesting

Promises were supposed to end callback hell.  They are useful, but I still found three or four deep chains of promises were not uncommon. Better than callbacks, but still unpleasant.  While you will still need some of this style of coding in your asynchronous middleware, this problem seems to be diminished.  You just blurt out what is going on when it happens and handle the issues by displaying to the user or reacting with the middleware logic. This reduced the need for micromanaging exceptions.

What are the Problems?

Yuri Takhteyev, CTO of Rangle, in another great intro to Redux (https://www.youtube.com/watch?v=s9AC8KTcce8) , identifies these Caveats about Redux:

  • It’s addictive
  • You won’t be happy using anything else
  • Your friends won’t understand your obsession

I tend to agree.

From a more newbie perspective, I would call out that it is hard-work to learn.  Some challenges I encounter were:

  • It is a radically different way of writing application logic. Don’t expect to be productive out of the blocks. The pay-off is there, it won’t take long, and it is well worth it.
  • It’s not just Redux you have to learn. I had to take frequent detours to learn other leading edge JavaScript technologies. For example:
    • TypeScript generics.
      • They’re great, but it took time and practice before I could easily grok something like:
static createCursorObserver<T>(cursor: Mongo.Cursor<any>): Observable<IDocumentChange<T>> {

Definitely not your father’s JavaScript

  • New ES6 syntax. It’s great too, but until you’re used to it, you’ll need to hit the books to understand what is happening.  For example, here’s how you define a middleware:

gamePlayMiddleware = (gameState: IGamePlayState) => next => (action: IPayloadAction) => {
		

Got it?

  • RxJS: vastly powerful. Expect to dedicate some time learning the various patterns and operators.  I’m just beginning my journey and it seems like it will take some time to speak it fluently.
  • Doing things the Immutable way. Simple objects are trivial, but when you have deeply nested structures and need to chain changes, it’s very easy to make mistakes when you are not used to it. Some of the tooling is pretty good and provides many useful warnings, but old habits die hard, and you will code mistakes.
  • Tooling is catching up. From a Meteor/Angular 2 perspective, using Redux is not something you’ll find a lot of templates and examples for.  Of course, React is stable and Angular 2 seems to embracing Redux with an almost identical library called NgRx.
  • A drop dead easy Undo function. 4o lines of squirely and imperfect state recreation logic became a simple matter of storing state at each step and restoring the parts that were necessary.

Conclusion

I’m sold on Redux.  It’s profoundly different and way better. But, simplicity is hard work, especially when you are using it for the first time and advancing your skills with related technologies. The hard work will pay off.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s