The public release of the Facebook embed is finally here. You can now embed any public post from your friends, neighbors, and favorite celebs. We worked diligently overnight to make this embed available for your site. Now get going and try out some yourself.

image

Find all the technical details in our changelog. Enjoy and play safe!

Posted 1 year ago by artgibby
Embed This
In this three part series, we examine DataSift, and how using Embedly can enhance the data provided by DataSift. In the first post we examined DataSift data and how Embedly can augment URL meta-data. In this post, we’ll start writing some code to supplement DataSift data with Embedly data in Node.js, pushing the stream to clients with Pusher. In the third and final post, we’ll create a client side application that leverages DataSift, Embedly and Pusher to show a live stream of the most popular URLs being tweeted about any subject.
http://i.imgur.com/S9VlRxE.jpg

As we all know, The Internets are a Series of Tubes. In this post we’ll create some glue to connect some of those tubes together, pulling data from DataSift and Embedly Extract, and delivering it to our happy customers via Pusher.

If you didn’t read the first post, DataSift is the simplest way to get a live stream of data from The Social Internets. It’s as simple as defining a query, and performing an HTTP GET.

Pusher, in case you haven’t heard, is the simplest way to get one stream of data into the hands of a bunch of consumers as quickly as possible. Think of it as a giant splitter for the Internet.

We’ll use Nodejs for our glue. Nodejs is one of the most excellent Internet glues available. It speaks tons of protocols, and it’s especially good at HTTP. It’s great for demos like this one, because everyone knows Javascript.

We’ll also get started on the basics of what will eventually be the incredibly beautiful UI for our twitter link stream. We’ll need something to test our tubes, anyway. We won’t do the beautifying until the third and final post, that is where Embedly will really shine.

Of course, the source for the entire app is publicly available on Github. We’ll focus on the implementation of the backend in this post, which can be found in the server directory. This code is not production ready, it’s merely a demonstration of the basic structure of such an app.

Overview

http://i.imgur.com/OIdaD2b.jpg

The glue we’ll make is a Nodejs server that is composed of three main parts. A webserver, a stream collector and a stream worker. The web server simply exposes an API for the frontend to manage the subjects. We use DataSift’s REST API to compile DataSift’s Curated Stream Definition Language for each of our subjects. The collector consumes streams from DataSift. It also listens for events from the web server so it knows when to start and stop listening to particular subjects. The worker consumes the DataSift events, adds Embedly extract data, and pushes them to Pusher. We’ll use Redis to communicate between the components.

HTTP API

As mentioned, we’ll use DataSift’s REST API to compile CSDLs for each subject. DataSift’s API returns a hash for successfully compiled CSDLs that is used with their Streaming API to consume the stream. Each subject will have an associated Pusher channel, which is named according to the hash of the DataSift CSDL.

The subjects API has the following pattern:

/1/subject(/:subject)

Performing a GET on /1/subject will return a JSON object mapping known subjects to DataSift hashes. Performing a GET on a particular subject will return on object mapping just that subject to it’s hash. POSTing to a subject will create it if it doesn’t exist, and return an object containing a hash field. DELETEing will simply remove the subject from our store. Both the POST and DELETE methods publish an event over a Redis channel so that the collector can take the appropriate action, whether it is subscribing or unsubscribing from the given subject.

The csdl.js module encapsulates the work of validating and compiling the CSDL. It reads as follows:

var _ = require('underscore'),
    settings = require('./settings'),
    request = require('superagent'),
    sprintf = require('sprintf').sprintf;

var CSDL = function(opts) {
  if (!(this instanceof CSDL)) {
    return new CSDL(opts);
  }

  _.defaults(this, opts, {
    username: settings.datasiftUsername,
    apiKey: settings.datasiftKey,
    host: 'http://api.datasift.com',
    validateEndpoint: '/v1/validate',
    compileEndpoint: '/v1/compile'
  });
}

/**
 * compile csdl. See http://dev.datasift.com/docs/rest-api/compile
 * for details. Callback takes (error, res). See
 * https://github.com/visionmedia/superagent for details.
 */
CSDL.prototype.compile = function(src, fn) {
  request
    .post(sprintf("%s%s", this.host, this.compileEndpoint))
    .set('Accept', 'application/json')
    .send({csdl: src, username: this.username, api_key: this.apiKey})
    .timeout(3000)
    .end(fn);
}

/**
 * validate csdl. See http://dev.datasift.com/docs/rest-api/validate
 * for details. Callback takes (error, res). See
 * https://github.com/visionmedia/superagent for details.
 */
CSDL.prototype.validate = function(src, fn) {
  request
    .post(sprintf("%s%s", this.host, this.validateEndpoint))
    .set('Accept', 'application/json')
    .send({csdl: src, username: this.username, api_key: this.apiKey})
    .timeout(3000)
    .end(fn);
}

exports = module.exports = CSDL;

subject.js manages our subjects:

var _ = require('underscore'),
    settings = require('./settings'),
    sprintf = require('sprintf').sprintf,
    step = require('step'),
    util = require('util'),
    events = require('events'),
    redis = require('redis'),
    CSDL = require('./csdl');

Subject = function(opts) {
  if (!(this instanceof Subject)) {
    return new Subject(opts);
  }

  _.defaults(this, opts, {
    redis: redis.createClient(),
    key: 'DS_CSDLS',
    channel: 'DS_CSDL_EVENT',
    api: CSDL(),
    pubsub: redis.createClient()
  });

  if (!this.redis) {
    throw new Error('redis not specified');
  }

  var self = this;

  self.pubsub.on('message', function(key, evraw) {
    var ev = JSON.parse(evraw);
    self.emit(ev.type, ev.hash);
  });
  self.pubsub.subscribe(self.channel);
}

util.inherits(Subject, events.EventEmitter);

Subject.prototype.find = function(query, fn) {
  if (!fn && query instanceof Function) {
    fn = query;
    query = null;
  }
  if (query) {
    this.redis.hget(this.key, query, fn);
  } else {
    this.redis.hgetall(this.key, fn);
  }
}

Subject.prototype.delete = function(subject, fn) {
  var self = this;
  self.redis.hget(self.key, subject, function(err, res) {
    if (res) {
      self.redis.hdel(self.key, subject, function(err) {
        if (!err) {
          self.redis.publish(self.channel,
            JSON.stringify({type: 'unsubscribe', hash: res}));
        }
        fn(err);
      });
    } else {
      fn(err);
    }
  });
}

Subject.prototype.add = function(subject, fn) {
  var csdl = _.template(settings.csdlTemplate, {subject: subject}),
      self = this;

  step(
    function() {
      self.find(subject, this);
    },
    function(err, res) {
      if (res) {
        fn(null, {hash: res});
      } else {
        self.api.validate(csdl, this);
      }
    },
    function(res) {
      if (res) {
        if (res && res.status == 200) {
          self.api.compile(csdl, this);
        } else {
          console.error(res);
          fn(new Error(sprintf('failed to verify CSDL for %s', subject)));
        }
      }
    },
    function(res) {
      if (res) {
        if (res && res.status == 200) {
          self.redis.hset(self.key, subject, res.body.hash, function(err) {
            if (err) {
              console.error(err && err.stack);
              fn(new Error(sprintf('failed to save hash for %s', subject)));
            } else {
              self.redis.publish(self.channel,
                JSON.stringify({type: 'subscribe', hash: res.body.hash}));
              fn(null, res.body);
            }
          });
        } else {
          console.error(res);
          fn(new Error(sprintf('failed to compile CSDL for %s', subject)));
        }
      }
    }
  );
}

exports = module.exports = Subject;

Key points are that it uses a template CSDL defined in settings.js and that it publishes POSTs and DELETEs over the Redis channel. It also keeps track of what subjects we are subscribed to in the event of a restart and to respond to the /1/subject query. Here is the CSDL template that we used:

twitter.links exists
AND
twitter.text contains_any "<%= subject %>"
AND
twitter.user.lang in "en"

The Subject class will inject subject into the template and use the CSDL module to compile the CSDL.

The Collector

The collector is an simple module that subscribes and unsubscribes to/from CSDLs in response to events on our web frontend. It also subscribes to any existing channels persisted in Redis on start. Any events consumed will simply be pushed to a list in Redis for consumption by the worker. Here is the listing:

var DataSift = require('datasift'),
    step = require('step'),
    utils = require('./utils'),
    settings = require('./settings'),
    consumer = new DataSift(settings.datasiftUsername, settings.datasiftKey),
    connected = false,
    Subject = require('./subject'),
    subject = new Subject(),
    _ = require('underscore'),
    debug = require('debug')('subject-server:collector');

step(
  function() {
    utils.redis(this);
  },
  function(err, redis) {
    subject.on('subscribe', function(hash) {
      if (connected) {
        debug('subscribing %s', hash)
        consumer.subscribe(hash)
      }
    });
    subject.on('unsubscribe', function(hash) {
      if (connected) {
        debug('unsubscribing %s', hash)
        consumer.unsubscribe(hash)
      }
    });
    consumer.on("connect", function(){
      debug('connected');
      subject.find(function(err, res) {
        _.values(res).forEach(function(hash) {
          debug('subscribing %s', hash);
          consumer.subscribe(hash);
        });
      });
      connected = true;
    });
    consumer.on('interaction', function(data) {
      debug('interaction', data.hash);
      redis.lpush(settings.inqueueKey, JSON.stringify(data));
    });
    consumer.on('disconnect', function() {
      debug('disconnected');
      connected = false;
    });
  }
);

exports = module.exports = {
  start: function() {
    consumer.connect();
  },
  stop: function() {
    consumer.disconnect();
  },
  stat: function() {
    return connected ? 'connected' : 'disconnected';
  }
}

Worker

The worker is where the three services intersect. The events that we publish over Pusher will be URL based and not tweet based. That means that if a tweet contains multiple URLs, it will translate to multiple Pusher events. We’ll take the tweet we received from DataSift, scrape it for URLs, call Embedly extract on the URLs, then publish both the DataSift data and the extract data via Pusher. Here is the meat of the worker module:

step(
  // get next event from the queue
  function() {
    debug('fetching from ' + settings.inqueueKey);
    redis.brpoplpush(settings.inqueueKey, settings.outqueueKey,
                     1000000, this);
  },
  // fetch extract info
  function(err, data) {
    if (err) {
      console.error(err.stack);
      return setTimeout(utils.partial(redis, embedlyApi, procNext));
    }

    debug('got data')
    try {
      var ev = JSON.parse(data),
          urls = findUrls(jpath(ev, 'data.interaction.content'));

      embedlyApi.extract({urls: urls}, utils.partial(ev, this));
    } catch(e) {
      console.error(e.stack);
      return setTimeout(utils.partial(redis, embedlyApi, procNext));
    }
  },
  function(err, args) {
    if (err) {
      console.error(err.stack);
      return setTimeout(utils.partial(redis, embedlyApi, procNext));
    }

    var extracts = args[0],
        ev = args[1];

    var self = this,
        procCount = 0;

    if (extracts.length > 0) {
      extracts.forEach(function(extract) {
        try {
          if (extract.url && extract.type != 'error') {
            procCount += 1;
            debug('got extract for %s', ev.hash);

            pusher.trigger(ev.hash, 'tweet', {
              'event': ev,
              'embedly': extract
            });
          }
        } catch(e) {
          console.error(e.stack);
        }
      });
    }
    if (procCount == 0) {
      debug('no extracts processed');
      return setTimeout(utils.partial(redis, embedlyApi, procNext));
    }
    this();
  },
  // next iteration
  function(err) {
    // don't overflow stack
    debug('next');
    return setTimeout(utils.partial(redis, embedlyApi, procNext));
  }
);

Conclusion

http://i.imgur.com/lG6mpLM.jpg

That’s all folks. If you are considering using this as the starting point of a real app, please take a look at Pusher’s Server API. You can do things like authorize users and register web hooks to be notified when channels are occupied or vacated. This could be used to cleanup DataSift channels when they aren’t being used.

If you want to try the app out, feel free to clone the repo and follow the instructions in the README. It isn’t turnkey, since you have to sign up for accounts on the three services, but they all offer free accounts and it shouldn’t take to long to sign up.

In the next post we’ll finish building the frontend of the app to create a beautiful frontend for our twitter URL stream.

Posted 1 year ago by dokipen
Embed This

image

Recently, we had an Embedly hack week where internally we played with ideas to make something cool.  I made a Reddit discussion network visualization, powered by D3 with previews generated by Embedly jQuery, and UI by Foundation

Why I made this

I’ve been using Reddit a lot lately.  My introduction to user-generated-link-site-addiction was Hacker News.  When I moved onto Reddit, I first noticed how deeply nested conversations were. I found that interesting, especially because sometimes these deeply nested threads turned out hilarious. I like playing with social data, and figured I could visualize this nested structure as a network by linking comments to what they are referring to- as a new way to browse Reddit.  At the same time I could see how different conversations are structured. 

How it works

Getting Reddit data is fairly simple.  You can see a previous post where we analyze subreddits.  You just add a “.json” to the URL.  Given a discussion, I recursively crawl the json response and pull all of the comments, noting the username, ID, parent, and body of the comment. 

The D3 force directed layout needs an array of nodes and links, so I add those as the comments are parsed. The size of the node is based on the score.  The original poster (OP) is orange, and all other nodes are black unless they comment more than once, in which case they are given a color.

To improve the user experience, I added link previews with Embedly jQuery and buttons for the Reddit Front Page.  Each link from the front page is run through Embedly to get an embed of the link.  A hover event for each button displays the respective preview.

There are also link previews for the comments. On hovering over a comment, the body is parsed for link text.  The link text is replaced with a URL (just wrap it with an `a` tag), and used to get the embed preview.  You can see an example of this in the Arnold Schwarzenegger AMA below.

Cool things I noticed

One common pattern for deeply nested threads is that a user shows up in alternating responses to their comment, reflecting a dialogue. You can see this in the thread below the original post (large orange node) below. Click the image to open up the network. 

image

You can also see how conversations can get ‘derailed’ and focus on the top comment thread, instead of the original post, as seen in the thread to the right of the first post. 

AskReddit’s have much more comment upvoting than other discussions.  You can see this by how large the nodes are. 

image

AMA’s, as expected, have lots of comments from OP, as well as upvoting.  Here is the top AMA from the last year:

image

and here is the infamous Morgan Freeman one.  It was far less engaging, and you can see the difference:

image

And while we’re on AMAs, here is the Arnold Schwarzenegger one.  I point it out because he used a unique answering method of handwriting the responses and posting them onto imgur.

image

This is a recent TIL I liked. You can see quite a few double (or more) comments by the same user along the threads. This was a particularly popular one, so there is more upvoting on this one than I’ve normally seen. 

image

Here is one of the higher scoring /r/javascript posts.  As well as being much smaller ,it is also focused much more towards dialogue.  More users post more than once, with comments responding back and forth, highlighted by the node colors.

image

Here is the top post over the past year in /r/programming. It is showcasing a project.  It looks like an AMA given the amount of times OP responds. 

image

Try it out

If you haven’t already, you can play around with the network here. Paste a link to the comments section of a reddit submission to see the network. The observations above are from looking at it over the past few days.  I’d love to hear other things you notice from browsing reddit networks, and of course, here is the code, everything is client side. 

Posted 1 year ago by whichlight
Embed This

Last night I had the pleasure of speaking to the Boston JS Meetup on Ember. It was a great crowd and I want to thank everyone for coming out.

I put together about 50 slides on the subject and wrote a ton of notes per slide. I’d thought I’d share them on the blog as well.

(#1) Intro

The goal of this talk is not actually to teach you how to build an application with Ember. It’s to get you into the correct frame of reference. We could go through a todo list, but lets be honest, you don’t care.

I really don’t want to be the guy standing at a desk, building a demo, explaining every line of code. I’d rather share a few stories, why we at Embedly still like it and why you might want to give it a shot.

(#2) Me

I founded a small start up here in Boston.

From the time I started coding, I’ve been told to stay away from actual JavaScript and use jQuery instead. I’m the person that every Javascript purist hates.

I’m not an Ember expert. I wrote a few tutorials here and there, but in general, a lot of you out there know more about the framework.

(#3) What we Built

We rewrote our entire app dashboard in Ember. This allows users to see usage, manage their account and select products.

(#4) Why?

There are really two reasons we chose to work with Ember, one is people based and the other about technology.

Startups thrive on new things. We don’t actually need to use a cutting edge framework that is pre 1.0, but we have to at the same time.

By making work interesting we attract talent and retain the talent that we already have. If no one ever got fired at a big company for choosing Java, no one got fired at a startup for choosing Ember.

The other is that we liked that Ember was built on technology that we were already familiar with in jQuery. It’s sort of baby steps into the deep murky waters of JavaScript MVC frameworks.

(#5) No comparing and contrasting.

I know one of you showed up with the idea that you were going to prove to someone in this audience that your framework of choice is better than Ember.

I’m not going to talk about it.

If anyone is going to ask how Ember compares to Angular, don’t, you should just tweet hateful words at me instead.

Here:

@screeley Angular is faster than Ember, here’s the link to prove it: bit.ly/11mNonE

@screeley Backbone is 1,500 lines of code, Ember has 27,000. You suck.

@screeley neckbeard, neckbeard, neckbeard. I hate you.

For those who love Ember already I’ll take

@screeley you are a gentleman and a scholar, i <3 you #yolo

(#8) Ember Intro

I gave this talk to the guys in the office yesterday and it was made pretty clear that without at least a little primer on Ember, the talk itself wasn’t that interesting. Who knows even with the primer it may not be interesting.

In order to at least catch the people up in the room that have not used Ember, let’s just take a brief look at Ember, how it’s structured and go from there.

Ember is a JavaScript MVC framework that came out of Sprout Core 2.

(#9) Application

Everything in Ember starts with creating an Application, this will be the default namespace for your app for the rest of it’s life.

window.Dolphin = Em.Application.create({
  rootElement: $('#dolphin'),
  ready: function(){}
});

(#10) Router

The Router determines how your application responds to states, in the background Ember is creating a StateManager. States are just URL paths allowing the user to move through the application.

Dolphin.Router.map(function() {
  this.route('about');
  this.resource('company', function(){
    this.route('team');
  });
});

This router creates the following URL paths:

/
/about
/company
/company/team

The router is about declaring hierarchy in your application. The only real difference between declaring a “route“ and a “resource“ is that a resource can have children that inherit from the parent.

(#11) Templates

Handlebars is the templating language for Ember. It’s biggest strength is on data bindings, meaning when data changes it will update the html.

Ember just adds helpers to Handlebars to make building pages easy. By default you should create two templates.

application.handlebars
<div class="row">
  <div class="large-12 columns">
    {{outlet}}
  </div>
</div>
index.handlebars
<h1>Welcome to Dolphin!</h1>
<p>Helping Dolphins become the smartest species since 2013</p>

The contents of index.handlebars will be placed in the “{{outlet}}“ of application.handlebars.

(#12) Models

A model can really be any object, it doesn’t have to be an Ember Data model. While the docs suggest this, it’s not the case.

The easiest example is the following:

Dolphin.CompanyTeamRoute = Em.Route.extend({
  model: function(){
    return {name: 'sean'}
  }
});

I can then use that model in the company.team.handelbars template.

<div>
  <h1>{{name}}</h1>
</div>

If you then started to hire people we can change it to a list of people.

Dolphin.CompanyTeamRoute = Em.Route.extend({
  model: function(){
    return [
      {name: 'sean'},
      {name: 'kawandeep'},
      {name: 'andy'}
  }
});

And the template would then look like this:

<ul>
  {{#each controller}}
    <li>{{name}}</li>
  {{/each}}
</ul>

When you are ready for something like Ember Data you can use the following:

Dolphin.CompanyTeamRoute = Em.Route.extend({
  model: function(){
    return Dolphin.User.find({is_team: true});
  }
});

(#15) Controllers

The Controller conceptually sits between your Model and View, and handles logic on your data. In Ember, the controller is a good place for code that reacts to changes in the Model.

This is a simple example of how to use a controller to modify the data sent to the template.

Dolphin.CompanyTeamController = Em.ArrayController.extend({
  sortAscending: ['years_coding'],
  totalYears: function(){
    return this.reduce(function(total, user) {
      return total + user.get('years_coding');
    }, 0);
  }.property('@each.years_coding')
});

The line between what belongs in a View or Controller can be fuzzy. As a rule of thumb, if your method involves jQuery, it should probably be a View. If your method needs to talk to your backend, it should be a Controller.

(#16) Views

Views handle presentation of your data to the user. In Ember, this means stuff like setting up animations and handling user interactions.

This example fades all the images within a view in with “opacity“ once the view has been rendered (didInsertElement).

Dolphin.CompanyTeamView = Em.View.extend({
  didInsertElement: function () {
    // fade images in.
    this.$().find('img').animate({
      opacity: 1
    }, 2000)
  }
});

(#17) Arrays

Ember’s standard array gives you a lot of utility functions like map, reduce and filter. You should use these early and often.

(#18) Builtins

Ember also comes with a number of functions that can be useful everywhere. We use isNone, isEmpty and keys often in our application.

(#20) Law #1: Do things the Ember way.

We are going to come back to this time and time again, but don’t fight the framework. Don’t try to get clever, just read the documentation and ask questions.

A lot of people talk about Ember’s learning curve as opposed to other frameworks. It takes awhile to understand it, but when you do you will be amazed of how productive you can be.

(#21) Law #2: Naming Conventions are Everything.

This is part of the magic that turns people off. When you name a route, or even a URL you are condemning it the that name for the rest of it’s live.

/#/about

Creating this path spawns 3 different objects and a template that you now have to keep track of.

AboutRoute
AboutView
AboutController
about.handlebars

I tell you this, because getting clever is only going to cause you pain.

(#23) Law #3:Think like Russian Dolls

The general layout is all done via outlets, which work similar to Russian dolls. Know what information you will need to have in order to render the template. You can do clever things when getting models, but think of it this way.

A lot of Ember is designing URLS.

To build the classic blog comment example, including a comment detail page, we will need to know the Post that the comment is attached to, and the Blog that it is on.

/blog/:id/post/:id/comment/:id

The router looks like so:

this.resource('blog', {path:'/blogs/:blog_id'}, function(){
  this.resource('post', {path:'/posts/:post_id'}, function(){
    this.route('comment', {path: '/comment/:id'});
  });
});

You then need 4 templates:

application.handlebars
blog.handlebars
post.handlebars
comment.handlebars

(#27) Law #4: You will never use “text/x-handlebars“

You may have noticed that there are separate “.handlebars“ files for each template. A lot of the intro stuff has something like this:

<script type="text/x-handlebars" data-template-name="blog">
  <div>
    {{outlet}}
  </div>
</script>

It’s useless, no one is going to add 50 templates to one index.html file. You should use something like “ember_templates“ instead. It precompiles all your templates.

Another way is pre compile them yourself into Ember.TEMPLATES dictionary:

Ember.TEMPLATES['blog'] = Em.Handlebars.compile('<div>{{outlet}}</div>');

(#29) Law #5: You needs to use “needs“

With the Russian Dolls you will find yourself needing the parent controller often (i.e. post needs blog).

var CommentController = Em.ObjectController.extend({
  needs:['blog', 'post],
  postBinding: "controllers.post",
  blogBinding: "controllers.blog",
});

The way to do this is with “needs“. You can use setUpController on the route but that’s a pain.

(#31) Law #6: jQuery is not the enemy.

“this.$()“ is available on every View. Use is, sometimes it’s much easier that writing it the Ember way.

For example if we wanted to fade in the actual text of a comment, we might have the following.

var CommentView = Em.View.extend({
  template: Em.Handlebars.compile('<b>{{name}}</b><p>{{{body}}}</p>'),
  didInsertElement: function(){
    this.$().find('p').animate({
      opacity: 1
    }, 1000);
  }
});

You could write another view that just handled the p tag, but you are just adding bloat.

(#33) Law #7: Ember Data is great at getting data.

I’ve witnessed many a post on how awful Ember Data is. These idiots didn’t read the giant disclaimer that said “not production ready”.

Ember Data is really good at at getting data, it’s not great at saving data. So, don’t be silly, just write the saving data part yourself.

Our library is about 100 lines long, and just uses jQuery to handle the form submission.

$('form').submit(function(){
  //do what the world taught you.
})

(#34) Law #8: Understand Ember Data’s protocol and protect yourself.

Ember Data expects a certain data structure when digesting data. i.e.

/api/blogs

{
  "blogs": [
    {
      "id": 1,
      "name": "Embedly Blog"
    }, {
      "id": 2,
      "name": "Dolphin Blog"
    },
    ....
  ]
}

This is all well and good, you will build a generic solution to handle this for all your objects. And then, one day you will discover that this works.

/api/users

{
  "users": [
    {
      "id": 1,
      "name": "Sean",
      "email": "sean@embed.ly"
    },
    {
      "id": 2,
      "name": "Kawandeep",
      "email": "kawandeep@embed.ly"
    },
    ....
  ]
}

Ember isn’t going to handle this for you, so get used to protecting your users and handle permissions yourself.

(#37) Law #9: deferReadiness is your friend.

A lot of times you are going to want to grab some data before you start up the application. In our case we needed the user and organization before we started everything up.

In order to make sure everything was present, we wanted to wait, ever so patiently for the server to return everything first.

It looks something like this:

App.deferReadiness();
// Wait for all the javascript files to load.
$(document).ready(function(){
  App.User.current(function(user, profile, organizations){

    // Set everything else up.
    App.set('user', user);

    // Will start everything up.
    App.advanceReadiness();
  });
});

It will save you some time.

(#39) Law #10: Save high level objects on the Application.

When you start looking back at the Russian Dolls you’ll want to make everything dynamic. For example everything in our dashboard is under organization.

/organization/screeley

Everything is under that, from tickets, to keys to admin is all dependent on that organization.

We can do “needs“ for everything all the way down, but you will find yourself in a handlebars template doing something like.

var Clip = Em.Namespace.create();

Clip.Copy = Em.View.extend({
  tagName: 'a',
  didInsertElement: function(){
    var self = this;
    this.$().attr('data-clipboard-text', App.get('org.api_key'));
    var clip = new ZeroClipboard( this.$().get(0), {
      moviePath: "/scripts/vendor/zero/zero-clipboard.swf"
    });
  }
});

We use this view all over the place, to make sure we had the org passed around correctly all the time would be super annoying.

It will make your life better if you just save high level objects on the App.

(#41) Law #11: Objects, objects everywhere.

Everything in Ember is an object, you better like objects. For those of you that are “javascript is for functions” people, I’ll give you an example of some goodness that objects bring to the table:

App.AuthedRoute = Em.Route.extend({
  redirect: function(){
     if (Em.isNone(Shine.get('user'))){
       this.transitionTo('login');
       return false;
     }
     return true;
  }
});

This allows us to define what Routes we need to be authenticated for, so the browser can take care of it before the server has to get involved.

(#43) Law #12: Use promises.

This is probably my favorite snippet in the code base:

App.OrganizationRoute = App.AuthedRoute.extend({
  model: function(){
    var promise = Em.Deferred.create();
    App.Organization.find({slug:params.slug}).then(function(results){
      if (Em.empty(results)){
        promise.resolve();
      } else {
        App.set('org', results.objectAt(0));
        promise.resolve(results.objectAt(0));
      }
    });
    return promise;
  }
});

By default, if you pass an object to find it will return a list, but we wanted a single object. Instead of hacking the api to take a slug, we used a Deferred obj and then resolve the promise when we have it.

You will notice that Ember Data passes back a Deferred Obj, use it.

(#44) Law #13: States move the world.

A lot of the Router and Transitions are all based on State Machines. I don’t know a ton about it to be honest, but when you are debugging and going into the source, you will see a lot of it.

Here are the silly basics.

$('body').append(
  ['<a class="view-terms">View</a>',
    '<div id="terms" style="display:none">',
    '<p>These are the terms</p>',
    '<a id="accept">Accept</a>',
    '<a id="decline">Decline</a>',
  '</div>'].join(''));

var terms = Em.StateManager.create({
  initialState: 'closed',
  open: Em.State.create({
    enter: function(manager){
      $('#terms').show();
      $('#accept').on('click', function(){
          manager.transitionTo('welcome');
      });
      $('#decline').on('click', function(){
        manager.transitionTo('decline');
      });
    },
    exit: function(){
      $('#accept').off('click');
      $('#decline').off('click');
      $('#terms').hide();
    }
  }),
  closed: Em.State.create({
    enter: function(){
      console.log('closed')
    },
  }),
  welcome: Em.State.create({
    enter: function(){
      alert('Hello !');
    }
  }),
  decline: Em.State.create({
    enter: function(){
      alert('Goodbye !');
    }
  }),
});

$('a.view-terms').on('click', function(){
  terms.transitionTo('open');
});

This is obviously a lot of code for a little bit of work, but the idea is that you will notice enter, exit and transitionTo used a lot in common objects.

(#46) Law #14: Ember gives you debugging tools, use them.

Four Sub Points here:

Use the un-minified version of Ember for development, it by default prints out some useful stuff.

If you need more logging you can use some constants when creating the application.

App = Ember.Application.create({
    LOG_STACKTRACE_ON_DEPRECATION : true,
    LOG_BINDINGS                  : true,
    LOG_TRANSITIONS               : true,
    LOG_TRANSITIONS_INTERNAL      : true,
    LOG_VIEW_LOOKUPS              : true,
    LOG_ACTIVE_GENERATION         : true
});

Inside the templates you can throw in

{{debugger}}

or if you want to see what a specific value is, to

{{log controller}}

(#48) Law #15: Let Handlebars helper you.

Handlebars allows you to extend the template language. It can be very helpful, here’s a quick example. 

Em.Handlebars.registerBoundHelper('prettyCents', function (as_cents) {
  var as_dollars = as_cents / 100;
  return '$' + as_dollars;
});

(#50) Law #16: Use Ember’s standard library.

This goes back to the beginning, but remember that Ember gives you a lot of awesome builtins, use them.

(#51) Law #17: Use Grunt.

In all honestly this will decrease your development time drastically. We wrote a post on this, but with the right Gruntfile it takes about 3 minutes to set up. You get live reload, pre compiling of templates and js hinting out of the box for nothing.

This will save you from a hell you didn’t know existed.

(#52) Law #18: It’s probably not Ember’s fault, you just think it is.

Embedly spent 2 weeks debugging an issue that we thought was an Ember issue. It turns out that it wasn’t. Make sure that you give Ember the benefit of the doubt. 

Posted 1 year ago by screeley
Embed This
Recap of the Embedly Startup Office Hours at Storify
Last week Sean and I held a mixer in the Storify offices, the Embedly Startup Office Hours.  Folks in attendance included Code for America fellows, data scientists, freelance developers, and old friends.  We learned about what people were working on, and talked a bit about the API.  Most importantly, we got to know one another a bit better. Thanks all who attended!
If you’re in Boston next week, check out Sean’s talk at the Boston JS Meetup.
Recap of the Embedly Startup Office Hours at Storify
Last week Sean and I held a mixer in the Storify offices, the Embedly Startup Office Hours.  Folks in attendance included Code for America fellows, data scientists, freelance developers, and old friends.  We learned about what people were working on, and talked a bit about the API.  Most importantly, we got to know one another a bit better. Thanks all who attended!
If you’re in Boston next week, check out Sean’s talk at the Boston JS Meetup.
Recap of the Embedly Startup Office Hours at Storify
Last week Sean and I held a mixer in the Storify offices, the Embedly Startup Office Hours.  Folks in attendance included Code for America fellows, data scientists, freelance developers, and old friends.  We learned about what people were working on, and talked a bit about the API.  Most importantly, we got to know one another a bit better. Thanks all who attended!
If you’re in Boston next week, check out Sean’s talk at the Boston JS Meetup.
Recap of the Embedly Startup Office Hours at Storify
Last week Sean and I held a mixer in the Storify offices, the Embedly Startup Office Hours.  Folks in attendance included Code for America fellows, data scientists, freelance developers, and old friends.  We learned about what people were working on, and talked a bit about the API.  Most importantly, we got to know one another a bit better. Thanks all who attended!
If you’re in Boston next week, check out Sean’s talk at the Boston JS Meetup.
Recap of the Embedly Startup Office Hours at Storify
Last week Sean and I held a mixer in the Storify offices, the Embedly Startup Office Hours.  Folks in attendance included Code for America fellows, data scientists, freelance developers, and old friends.  We learned about what people were working on, and talked a bit about the API.  Most importantly, we got to know one another a bit better. Thanks all who attended!
If you’re in Boston next week, check out Sean’s talk at the Boston JS Meetup.
Recap of the Embedly Startup Office Hours at Storify
Last week Sean and I held a mixer in the Storify offices, the Embedly Startup Office Hours.  Folks in attendance included Code for America fellows, data scientists, freelance developers, and old friends.  We learned about what people were working on, and talked a bit about the API.  Most importantly, we got to know one another a bit better. Thanks all who attended!
If you’re in Boston next week, check out Sean’s talk at the Boston JS Meetup.

Recap of the Embedly Startup Office Hours at Storify

Last week Sean and I held a mixer in the Storify offices, the Embedly Startup Office Hours.  Folks in attendance included Code for America fellows, data scientists, freelance developers, and old friends.  We learned about what people were working on, and talked a bit about the API.  Most importantly, we got to know one another a bit better. Thanks all who attended!

If you’re in Boston next week, check out Sean’s talk at the Boston JS Meetup.

Posted 1 year ago by whichlight

Sean and I are going to be in San Francisco next week. If you’re in the bay area, come join us for a startup office hours session with Embedly. The event will be on Thursday, July 11th, from 6:30 to 8:30pm.  The kind folks at Storify have allowed us to host the event in their space.

Basically, we want to have an opportunity to hang out with developers in San Francisco  have some pizza and drinks, and talk about startups.  

Sean and I are eager to hear your ideas, or offer any tips and anecdotes we’ve picked up along the way at Embedly.  Make sure to RSVP for the event here

Looking forward to meeting you! Feel free to ping me beforehand at @whichlight

 

Posted 1 year ago by whichlight
Embed This
In this three part series, we’ll examine DataSift, and how using Embedly can enhance the data provided by DataSift. In this post we’ll examine Datasift data and how Embedly can augment URL metadata. In the second part, we’ll start writing some code to supplement DataSift data with Embedly data in Node.js, pushing the stream to clients with Pusher. In the third and final post, we’ll create a client side application that leverages DataSift, Embedly and Pusher to show a live stream of the most popular URLs being tweeted about any subject.

Social media monitoring companies love Embedly: it makes sense that if 1 in 4 tweets contains a URL, understanding the content behind that link is important. We set out to understand what we could offer these companies and what the major benefits would be. To do so we go straight to the source to understand the data and also elicit the help of Datasift.

Overview

Being a Haverhill native, I wanted to feature my home city in this post. After spending a week filtering for tweets about Haverhill staples, like Rob Zombie, The First Macy’s and Archie Comics and only collecting 12 tweets, I decided that the Bruins would be the next best thing. Our hometown hockey team was in the finals and everything seemed to be going right. In my eternal optimism, it seemed like a good idea. In retrospect, knowing Boston sports history, I could have gone with a safer bet.

http://imgur.com/69syu8a

Data Summary

Here is a comparison of image extraction for the 1796 URLs that we were able to find during our time period. Embedly increased our total images by more than 5 times, and almost doubled the amount of URLs with associated images.

http://imgur.com/a/mvjxC

You’ll also notice that DataSift didn’t pull out any keywords for the article, but Embedly got a bunch. Here is a comparison of entity and keyword extraction for the 1796 URLs. Embedly was able to increase our keyword extraction by about 4 times.

http://imgur.com/a/lut5w

As a bonus, Embedly also pulls out the content of any links to articles. About 13% of the URLs were seen as articles by Embedly and had their content extracted. Since most of the tweets for this particular subject were images, the low percentage makes sense. Embedly will also give you a list of related articles.

Details

DataSift is “the most sophisticated data platform used to filter insights from the world’s most popular social & news sources.” After playing with it for a few days, I can attest that it is an awesome tool. The admin interface is sweet, and creating a filtered stream of tweets using the the web editor was a breeze.

I filtered for all tweets with ‘#bruins’ and ‘http’ in their content to get all tweets containing URLs about the Bruins. Here’s an image of the DataSift stream editor demonstrating how simple it was to setup:

http://i.imgur.com/9M9ZBpQ.png

I took all the data, scraped out and normalized the URLs, and then ranked the URLs according to how many unique users tweeted them. The top tweeted URL was a snapshot of Zdeno Chára headed back home after a warm up game of FIFA at the Embedly office. Here is an annotated version of the photo:

http://imgur.com/ziAV9oL

And here is an example interaction given to us by DataSift:

{
  "demographic": {
    "gender": "male"
  },
  "interaction": {
    "schema": {
      "version": 3
    },
    "source": "web",
    "author": {
      "username": "TheSargeaki",
      "name": "John Sartzetakis",
      "id": 195955539,
      "avatar": "http://a0.twimg.com/profile_images/1141532101/John_normal.jpg",
      "link": "http://twitter.com/TheSargeaki"
    },
    "type": "twitter",
    "created_at": "Wed, 19 Jun 2013 19:34:46 +0000",
    "content": "RT @NHLQubes: I caught Zdeno Chara riding his bike outside of TD Garden yesterday afternoon before Game 3. Pretty awesome. #Bruins http://t.co/GeZ9golwW2",
    "id": "1e2d9174c2c8af00e0747c671f1a0fd4",
    "link": "http://twitter.com/TheSargeaki/statuses/347437320342999040"
  },
  "klout": {
    "score": 14
  },
  "language": {
    "tag": "en",
    "confidence": 100
  },
  "salience": {
    "content": {
      "sentiment": 5
    }
  },
  "twitter": {
    "id": "347437320342999040",
    "retweet": {
      "text": "I caught Zdeno Chara riding his bike outside of TD Garden yesterday afternoon before Game 3. Pretty awesome. #Bruins http://t.co/GeZ9golwW2",
      "id": "347437320342999040",
      "user": {
        "name": "John Sartzetakis",
        "statuses_count": 126,
        "followers_count": 6,
        "friends_count": 25,
        "screen_name": "TheSargeaki",
        "profile_image_url": "http://a0.twimg.com/profile_images/1141532101/John_normal.jpg",
        "lang": "en",
        "id": 195955539,
        "id_str": "195955539",
        "favourites_count": 5,
        "created_at": "Tue, 28 Sep 2010 00:02:10 +0000"
      },
      "source": "web",
      "count": 687,
      "created_at": "Wed, 19 Jun 2013 19:34:46 +0000",
      "hashtags": ["Bruins"],
      "media": [{
        "id": 347091365844373500,
        "id_str": "347091365844373504",
        "media_url": "http://pbs.twimg.com/media/BNEdwh3CUAAZa9m.jpg",
        "media_url_https": "https://pbs.twimg.com/media/BNEdwh3CUAAZa9m.jpg",
        "url": "http://t.co/GeZ9golwW2",
        "display_url": "pic.twitter.com/GeZ9golwW2",
        "expanded_url": "http://twitter.com/NHLQubes/status/347091365835984897/photo/1",
        "type": "photo",
        "sizes": {
          "medium": {
            "w": 600,
            "h": 800,
            "resize": "fit"
          },
          "small": {
            "w": 340,
            "h": 453,
            "resize": "fit"
          },
          "thumb": {
            "w": 150,
            "h": 150,
            "resize": "crop"
          },
          "large": {
            "w": 1024,
            "h": 1365,
            "resize": "fit"
          }
        }
      }],
      "lang": "en"
    },
    "retweeted": {
      "id": "347091365835984897",
      "user": {
        "name": "Matt Cubeta",
        "url": "http://www.nhl.com/ice/newsindex.htm?author=2729&view=headline",
        "description": "http://NHL.com Fantasy Hockey analyst. I probably root too hard for Brandon Dubinsky, Hiroki Kuroda and Ray Felton. Opinions are mine, not the NHL's.",
        "location": "New York, NY",
        "statuses_count": 1896,
        "followers_count": 1920,
        "friends_count": 824,
        "screen_name": "NHLQubes",
        "profile_image_url": "http://a0.twimg.com/profile_images/3108149042/1a68dc585c8471f4225a5d084749561c_normal.jpeg",
        "lang": "en",
        "time_zone": "Eastern Time (US & Canada)",
        "utc_offset": -18000,
        "listed_count": 51,
        "id": 164404466,
        "id_str": "164404466",
        "created_at": "Thu, 08 Jul 2010 20:09:08 +0000"
      },
      "source": "web",
      "created_at": "Tue, 18 Jun 2013 20:40:05 +0000"
    }
  }
}

If we run the URL through Embedly’s Extract endpoint, we get the much more information about the URL. Here are some interesting parts, the image extraction and keyword/entity extraction:

..snip..

"images": [{
  "caption": null,
  "url": "https://pbs.twimg.com/media/BNEdwh3CUAAZa9m.jpg:large",
  "height": 1365,
  "width": 1024,
  "colors": [{
    "color": [181, 184, 180],
    "weight": 0.41845703125
  }, {
    "color": [21, 21, 24],
    "weight": 0.320068359375
  }, {
    "color": [70, 66, 64],
    "weight": 0.18701171875
  }, {
    "color": [131, 121, 59],
    "weight": 0.053466796875
  }, {
    "color": [123, 126, 130],
    "weight": 0.02099609375
  }],
  "entropy": 5.77213454954,
  "size": 203452
}, {
  "caption": null,
  "url": "https://pbs.twimg.com/media/BNEdwh3CUAAZa9m.jpg",
  "height": 800,
  "width": 600,
  "colors": [{
    "color": [178, 180, 177],
    "weight": 0.454833984375
  }, {
    "color": [20, 21, 25],
    "weight": 0.294189453125
  }, {
    "color": [66, 64, 66],
    "weight": 0.194091796875
  }, {
    "color": [132, 123, 74],
    "weight": 0.056884765625
  }],
  "entropy": 5.209212398489663,
  "size": 73595
}, {
  "caption": null,
  "url": "https://si0.twimg.com/profile_images/3728540001/87a5ed2f38245f00839d146438525fc4_normal.jpeg",
  "height": 48,
  "width": 48,
  "colors": [{
    "color": [219, 167, 135],
    "weight": 0.15576171875
  }, {
    "color": [163, 123, 97],
    "weight": 0.148193359375
  }, {
    "color": [36, 25, 22],
    "weight": 0.096435546875
  }, {
    "color": [74, 66, 65],
    "weight": 0.083740234375
  }, {
    "color": [110, 83, 62],
    "weight": 0.078369140625
  }],
  "entropy": 6.559165189070405,
  "size": 982
}, {
  "caption": null,
  "url": "https://si0.twimg.com/profile_images/378800000025528513/e21842ee327d8120b116edc5f0d8c9f4_normal.jpeg",
  "height": 48,
  "width": 48,
  "colors": [{
    "color": [217, 202, 162],
    "weight": 0.373046875
  }, {
    "color": [152, 77, 53],
    "weight": 0.062744140625
  }, {
    "color": [168, 113, 93],
    "weight": 0.054931640625
  }, {
    "color": [113, 43, 40],
    "weight": 0.038330078125
  }, {
    "color": [67, 9, 21],
    "weight": 0.033447265625
  }],
  "entropy": 5.829762603947044,
  "size": 1289
}, {
  "caption": null,
  "url": "https://si0.twimg.com/profile_images/2539374850/fqv4h6wxlkhq1a5bx897_normal.jpeg",
  "height": 48,
  "width": 48,
  "colors": [{
    "color": [188, 193, 202],
    "weight": 0.20361328125
  }, {
    "color": [143, 147, 154],
    "weight": 0.185546875
  }, {
    "color": [90, 88, 89],
    "weight": 0.109375
  }, {
    "color": [33, 33, 35],
    "weight": 0.06396484375
  }],
  "entropy": 5.115549382910704,
  "size": 4538
}],

..snip..

"entities": [{
  "count": 3,
  "name": "Zdeno Chara"
}, {
  "count": 1,
  "name": "Blackhawks"
}, {
  "count": 1,
  "name": "Dubie"
}, {
  "count": 1,
  "name": "Hawks"
}, {
  "count": 1,
  "name": "Kadri"
}, {
  "count": 1,
  "name": "Boston Bruins"
}, {
  "count": 1,
  "name": "Chara"
}],
"keywords": [{
  "score": 90,
  "name": "nhlqubes"
}, {
  "score": 35,
  "name": "chara"
}, {
  "score": 26,
  "name": "zdeno"
}, {
  "score": 23,
  "name": "bike"
}, {
  "score": 23,
  "name": "awesome"
}, {
  "score": 22,
  "name": "pretty"
}, {
  "score": 21,
  "name": "bruins"
}, {
  "score": 20,
  "name": "gez9golww2"
}, {
  "score": 20,
  "name": "devenp"
}, {
  "score": 18,
  "name": "riding"
}],

..snip..

This particular URL is pretty simple, but even for something as simple as a tweet, we can see that Embedly is giving us more information about the images, including Colors, and even pulling out entities and keywords. But to really see what Embedly’s Extract endpoint is capable of, we’ll have to look at URLs from sites less popular than twitter.

Where Embedly really shines is with articles. The eleventh highest URL in our results was an article from Boston Magazine. Here are the images Extract was able to pull out, in order of importance. DataSift got the first one.

http://imgur.com/a/ax2Yl

That’s it for the first part in this series on Embedly and Datasift. Hopefully you’ve seen the value of using Embedly to mine URL metadata from your Datasift stream and got inspired to create a really engaging experience for your users. Next time we’ll get to work writing some actual backend code to mash up Embedly and DataSift and push the data to clients with Pusher. See you next time!

Posted 1 year ago by dokipen
Embed This

The world’s largest supplier and manufacturer of athletic apparel and equipment, Nike boasts a whopping global revenue of about $24B USD without breaking a sweat. What else? Nike spent close to $900M USD on advertising in 2011. Sound like a big number? That was in the U.S. alone. Unreal.

Why spend so much on advertising? Because it works. Why does it work? Content Marketing. It’s not the action of pushing ads that encourage you to buy something, it’s the goal to inspire and communicate through the creation of sharable content that your brand is the best on every level.

As I spent the better part of the last week perusing through Nike’s Facebook page and countless Instragram, Twitter, and Youtube accounts, it’s clear that the majority of any material shared includes an image, video or other embedded content. So, what can you do to up the ante on your content marketing game? Here’s four tips to consider:

  1. Cover all your bases. Nike has hundreds of social accounts that target specific sports, products, and locations. On Twitter there’s at least 100 verified accounts, Facebook boasts roughly 20 verified profiles, and countless Instagram accounts the deeper you search.image
  2. Create beautiful and engaging content. You don’t need an endless marketing budget to make a promotional video or a shareable image. Through all of their channels and accounts Nike publishes content on a daily basis. Looking at their main accounts: Twitter is updated 3 - 5 times a day, Facebook posts are published 5 - 10 times per month, Youtube has 2-3 videos uploaded a month, and Instagram a new image is added daily.image
  3. Realize who your customers are and cater your content to them. They make your brand and today, more than ever before, what people say on the web and behind closed doors actually matters. Nike’s followers are all over the world and in large quantities across social networks.imageimage
  4. Put your brand services/products alongside your community. Nike’s products are front and center on their homepage along with their Twitter, Instagram, Facebook, and Youtube channels for customers to engage with. Nike Community is a curated display of their social accounts with an interaction factor that allows customers to view images and videos right on Nike.com.image

    image

    image

    image

These steps should feel natural and scale with your brand. Content marketing is about sharing relevant content and engaging with your customers.

Sources:

  1. http://nike.com
  2. http://twitter.com/nike
  3. http://facebook.com/nike
  4. http://instagram.com/nike
  5. http://www.youtube.com/user/nike
  6. http://www.statisticbrain.com/nike-company-statistics/
  7. http://www.statista.com/topics/1243/nike/
Posted 1 year ago by ninerr
Embed This

Posted 1 year ago by thejohnnest
Embed This

Instagram has rolled out their newest feature and the biggest one since they launched just two years ago. Say hello (literally) to Instagram Video. You can now record, edit and filter a 15 second video to your heart’s desire. 

Oh yeah and we’re supporting them now. #FilterOn

You can use this now from our Embed product. 

Posted 1 year ago by ninerr
Embed This
Tagged: