kalv.co.uk

Engaging with early users to learn Habits

2012-04-12T00:00:00Z

As I’ve been working to get our startup Goodbits up and running I’ve been using many different tools to get as much insight on our customers activity. The one tool that I’ve been using the most has been Kissmetrics which has allowed me to look at specific events or actions that a user carried out on the site. Making it far more useful than the popular choice of Google Analytics or similar.

It has been very important to learn what daily habits our users are carrying out, so the better information I can compile the better I can deduce common usage patterns that I have to improve and refine. Check out this post from Nir which is an interesting read on Habit forming.

Now there is a new kid on the block and I’m loving it. Intercom, a web app that is a CRM (Customer Relationship Management) Tool on my customers with some serious powerful features.

How it works

Like with most of these services, you drop in a javascript snippet into your application and then set a bunch of settings, the email address of your signed in user, etc. By default Intercom then records information like ‘signed up’, ‘last seen at’, the number of ‘sessions’ & then looks up more information on that customer.

var intercomSettings = {
  app_id: 'app_id',
  email: 'john.doe@example.com', // TODO: User's e-mail address
  created_at: 1234567890, // TODO: User's sign-up date, Unix timestamp
  name: 'John Doe'
};

The added lookup on information is that it uses FullContact to link to their social media profiles, this gives you better insight of the type of user they are. It’s great, with a couple of clicks I can see what field this customer works in, how interactive they are with certain social media tools and better understand the types of customers we have using our product.

“Whoa, this feels like getting a message from God”

It doesn’t stop there, the interface is very well designed allowing me to filter by data stored for a user and then message them, yep message them. Either directly so that when they next use my site a popup would appear or through email.

I’ve always hated spam messages from an app I use, this filtering allows me to target specific behavior and I have seen that users engage back with us more than they would with a generic mail shot from MailChimp.

I have more confidence that they will engage back with me. Intercom have a column in their app with a heart in it for relationship.

Getting more from Intercom

You can also add custom information to intercom, which then allows you to further filter your users.

var intercomSettings = {
  // .. add this
  custom_data: {
    'invites sent': 4,
    'has added twitter service': true,
    'number of sections created': 5
  }
};

This can be quite intensive on your to render this data out on every request, so instead of caching it, I am posting this data in a scheduled task hourly using their API and Rubygem

Auto Messaging

You can now set up filters and then create an auto message which be sent out to customers that fall into that filter. I am using this for some great scenarios, ones where I’ve before written manual code to fire emails.

Ask users why they haven’t come back to our application after a period of time to better understand what it is we’re not doing right
Send targeted messages describing what they can do with our tool that they haven’t already enabled and if they didn’t use those features, why.

Constantly improving

They are adding improvements all the time, I wanted auto messages and it was something they released. I wanted to use my custom data in generated messages, deployed and notified. I wanted to easily see the number of emails being sent from their auto messages features, they updated the interface and deployed it. This reaction to customer needs is what I try to do with products I work on, so using a tool that do this for me, is satisfying.

I decided to write this post because Intercom is starting to become a must have for me in building web products. It’s starting point in reviewing analytics is great, the customer.

How a Ruby guy decided to add node.js to the mix

2012-01-17T00:00:00Z

I’ve been a Ruby guy for over 4 years now and have always found the language fun and the frameworks suitable for building web based applications, I love it.

Since the release of Node.js in 2009, I and my colleagues at GoFreeRange were very excited about the event loop, non-blocking IO goodness for scalability and speed on the web. We had even played on something to help testing (was only for a day, just to learn more about the tech).

So I’ve been looking for the right time to start developing a real application that benefits from what Node.js has to offer over what I can do quickly and efficiently in Ruby.

I have yet to see an API or website really need the scalability or speed offered by node.js over what I know with ruby, so I’ve always chosen to use what I know, has good conventions and good maintainability.

But last week the time finally arrived where Node.js might be a better fit.

This won’t be a post outlining benchmark metrics but just an overview of my findings and how I settled on Node.js for this specific need.

The problem

Our feed checking mechanism at RSS Hero was built using Resque so that we could fetch multiple feeds and check them (making sure they are a feed). Then to subscribe with Superfeedr if successful.

This process was not fast enough for fetching lots of feeds, I found that activating a couple of users that each had 1000 subscriptions or so, it would stack up and take quite a while to churn through.

Even though I could increase the number of resque workers running in parallel I would end up using more memory on our limited production servers.

HTTP feed fetching is going to increase, one of our upcoming milestones is to take load away from superfeedr and fetch some feeds ourselves.

Proposed Solutions

To make this faster I needed to use less memory and be able to fetch multiple HTTP requests in parallel. The shortlist that I came up with was:

Eventmachine & em-http-request – A ruby event loop allowing async processing
Typhoeus – Http fetching using curb (curl)
Node.js – event driven, non-blocking IO

So to figure out which was best I set a time box of ½ day to spike on each method and learn about them.

With each test setup I used Redis pubsub to push urls to be fetched onto a channel and have the fetching code consume that channel. This would allow easy integration from the existing rails app.

Also to remove bandwidth fluctuations from my ISP I ran the tests on a Linode server.

Eventmachine

This was my first choice and I believed it would be good enough for our needs. I would be able to use my existing tested ruby code for the feed checking and connect it to redis.

The first issue and main one was eventmachine kept on stalling. When the sample data was loaded, the compatible redis library (em-hiredis) would read off all the messages off the pubsub channel and kick off http requests, responses then stopped after the first 100 requests.

I thought it might be an issue with trying to send too many requests at one time. So I tried using em-synchrony which is a lovely library to allow synchronous control in the event loop, but it stopped the async nature of the http requesting so slowed it back down.

To better manage the concurrency I then tried to store the received urls from redis into a shared variable and then poll that collection (EM::PeriodicTimer) and only fire off a limited amount of http requests. This worked well until 300-400 requests were sent, again eventmachine stalled for a 1-5 secs and then came back again.

After spending a bit of time debugging and digging into my limited knowledge of Eventmachine and em-http I couldn’t see what was happening. All I could see was that the event loop was not ‘ticking’, the timer was not firing so maybe something had blocked the IO loop. Using 1000 requests to a localhost web address worked fine, so maybe it was something in the network layer, maybe an address not returning, DNS hanging. I did find evidence to show that the HTTP Connection was not fully asynchronous so perhaps one or a couple of network connections were blocking.

I fiddled with adding timeout settings to the http request but nothing stopped hitting this stall.

With my limited time for this spike and this stability issue I moved on.

Typhoeus

I have used curb before for fast feed fetching at Twitterfeed, where I used a modified feedzirra that allowed multiple http feed fetching, this worked very well at the time.

Comparing this though to the speed I was getting with Eventmachine it was slow, a fire off of 10 http requests would still wait for them all to return, which was too slow.

Again I had to manage my own concurrency when reading urls to be fetched from redis.

This being too slow, I moved on to node.js.

Node.js

Already knowing javascript, I got it running pretty quickly and the responses came through fast. I can’t say exactly but it responses for for the first hundre came through a lot quicker and it stayed consistently fast.

I didn’t have to worry about concurrency control, internally node.js seemed to manage and handle the requests fine.

I had to write my own code to handle redirects and to put a cap on how many to follow before throwing an exception as I was using the http request library included in Node.js.

The results from the little testing on Node.js was amazing. It felt solid and fast. This was enough for me to take the plunge and deploy some node.js.

To Summarize

I chose to use node.js because it was faster, stable and used a lower memory footprint on my server.

RSS Hero is now using node.js for it’s http fetching and memory usage has reduced from 400mb of RAM to 20mb and it’s a lot faster.

Turn off annoying Skype auto gain

2011-11-24T00:00:00Z

If you like me use Skype quite bit for calls or coding together with other developers, you might find that when you’re quiet Skype tries to be clever and automatically increase the microphone gain. This is crap as it starts to pick up the background noise.

You can’t easily turn off automatic gain control in Skype, it’s a setting you can apply in a xml file. Hunting around I found the solution again and this time I’ve stuck in a gist in case you want to try it too. Try it, even if you have a headset with a microphone built in, this makes a difference.

Learning Ruby / Rails

2011-11-23T00:00:00Z

I’m often asked about where to go and what to read to learn about Rails. In fact just the other day, so I’m going to outline some details here so to not repeat myself in the future or even forget! This is not a definitive guide but rather some links to things you should read and think about.

Learn Ruby

Sure you can go on right ahead and start with Rails and start building a web application but if you’re unfamilar with Ruby then please take the time to learn the basics, it will help considerably.

There are quite a few places you could learn about Ruby, books being one of them, here are some of my favourites.

why’s (poignant) Guide to Ruby_Guide_to_Ruby) This is a different kind of programming book, it’s distributed under Creative Commons and has some interestig cartoon graphics to go through Ruby.
Ruby Koans Created by Jim Weirich and Joe O'Brien this is a hands on way to learn more about Ruby it’s a test driven approach for you to learn Ruby’s internals. I myself still use these tests to remind me or even learn about things I didn’t know about.

Test Ruby

Most guides on the web don’t include tests. And what I mean about tests is using rspec, test unit, cucumber to progamatically test your application. Not having someone click through the site or pages. The importance of testing isn’t directly required to learning about ruby or rails BUT will help you in understanding what’s going on (like the Ruby Koans) and get you into a good practise when developing your application.

Rails

To learn about the rails framework and it’s make up check out these sources, there are some great screencasts you can now watch.

Peepcode – Is paid but worth it. Even watching some of the play by play is a good watch to see how others work.
Railscasts – Very good free and now has pro content on how to do common things in rails. Has good screencasts on commonly used gems that enhances rails.
Rails Guides – is a good source of information too on the rails framework.

News and updates

I found it helpful to follow some sources for updates on what gems are out there or new gems that do something clever. Here are some that I followed for this and still do.

RubyFlow – a good source for new information on Ruby/Rails
Ruby5 Podcast – I used to listen to listen to podcasts quite a bit when I had time, but have stopped now. Give it a go, sometimes it quite good to hear discussion on implementation decisions and justificiation on gem use.
Pivotal Blabs – This is a blog ran by Pivotal Labs where they share great little tips from their standups.

Edit:

@taylorbuley comments on twitter to see also Codeacademy.com which seems to be for learning javascript – might eventually have some Ruby, still worth a look if you want to understand javascript.

Making Textmate play nice with RVM

2011-07-27T00:00:00Z

I finally sat down for a while this morning to get RVM working well with Textmate ruby running commands. Whilst pairing with the team at GoFreeRange we tend to stay in Textmate, write tests and then run them from there. But recently using RVM and .rvmrc config files within 1.9.2 projects we haven’t been able to get the tests to run.

To make it easier than creating a textmate rvm wrapper, I run rvm so that it sets up the right ruby and paths. Then check if there is a .rvmrc and take that into account.

I add these two lines to any Textmate commands (Bundles –> Bundle Editor –> Edit Commands…) for example the Ruby -> Run command.

[[ -f "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm"
[[ -f "$TM_PROJECT_DIRECTORY/.rvmrc" ]] && . "$TM_PROJECT_DIRECTORY/.rvmrc"

Small things I like about HTML 5

2011-06-14T00:00:00Z

Most people are talking about the big things that HTML 5 is bringing to the table, video, audio, Websockets APIs, etc. I thought I’d share a small selection of the small attributes that I’ve used so far.

the placeholder attribute

You can now add the placeholder attribute on form input elements. This will then display the text until the user provides focus to the input element.

<input type="text" placeholder="Enter your email address"/>

Wow this has made things so much easier. No more extra javascript to remove the value when clicked and then put it back when the user clicks away and it’s empty, blah di blah.

email input type

To better improve what I wrote above, you can now define an email type on input elements. Allowing mobile browsers to show a different keyboard for those elements.

<input type="email" placeholder="Enter your email address"/>

There are more wonderful attributes to provide browsers with better information on what needs to be entered like ‘min’ and ‘step’ for ‘number’ type input elements. All good stuff, it’s great to see browsers implementing more behaviour on the structure of webpages.

the cache manifest

<!DOCTYPE HTML>
<html manifest="/cache.manifest">
  <head>
    <title>Test</title>
    <script src="application.js"></script>
    ...

Offline web pages made simple. Simply add the manifest attribute to html pointing to a web served file. This will setup which files will be available offline. So for this example a file ‘cache.manifest’ would be created.

CACHE MANIFEST 
/application.js

You can even define what is required for network access or even just point to an offline page. Nice.

CACHE MANIFEST
FALLBACK:
/ /offline.html
NETWORK:
*

This file has to be served as type ‘text/cache-manifest’. Read more on cache manifest at diveintohtml5.org. If you use Ruby checkout Rack::Offline to help manage your offline files.

Fastest Ruby RSS Parser?

2010-12-10T00:00:00Z

I was working on a project the other day that needed to parse a lot of RSS feeds and would need to poll & parse them frequently. So once again I had the question ‘which ruby rss parser should i use?’ in my mind.

Back when I was working for twitterfeed we performed some testing to find the optimal library that could fetch and parse the feeds fast. It was found then that Feedzirra performed the best as you could use a multiple fetch using curb.

For now I only wanted to know how fast these libraries would just parse the feeds, I was not worried about the http fetching.

I had some time so I returned to a nice performance test suite created by Julien Genestoux of Superfeedr.

I forked the suite, updated it to use bundler, order the results and to use the latest gems and then re-ran the performance test. You can find my fork here if you are interested.

The results were (ran on my macbook pro 2.53ghz, 4gb ram, SSD):

ruby feed-parser-bench.rb 1000

feedzirra => Average: 0.0278322427572428 (0 errors) RSS: 0.0260872052845528 Atom: 0.0295189980353635
mrss => Average: 0.042981384305835 (7 errors) RSS: 0.039129512244898 Atom: 0.0467262599206349
syndication => Average: 0.0495069849548646 (4 errors) RSS: 0.0398204020408163 Atom: 0.0588687712031558
rfeedparser => Average: 0.452556951048951 (0 errors) RSS: 0.389084115853658 Atom: 0.513909868369352
simplerss => Average: 1.29865328871129 (0 errors) RSS: 0.20822887195122 Atom: 2.35265881532417

The result was simple – Feedzirra was still the fastest.

Playing with Macvim

2010-11-29T00:00:00Z

I have been spending some time over the last week of so with macvim. Thought I’d share my experiences so far.

Why

I wanted to play around with VIM again really, one day a team member reminded me of the ‘c-w’ vim command when working on a server (it clears the word from the start of it). At this point I remembered the power that vim has and I wanted to get back to that and see if it would make it easier for me to edit and write code.

I also felt a frustration when moving from keyboard to mouse. Could I be as productive and not have to touch the mouse (perhaps still use to explore around a project directories).

Some people could argue these pains might not be enough to justify switching or spending the time to learn a new tool. I just thought I’d give it a try, I can always switch back to my long time home textmate.

Setting it up

I simply installed macvim through homebrew brew install macvim. If you don’t use homebrew to manage your packages on your mac, i seriously advise you to look into it.

Once that was installed I installed the color IR_Black_theme to make it look a bit prettier than the default (I prefer dark backgrounds).

I pulled parts of the .vimrc and .gvimrc from this repo from railsjedi (found by just searching github for macvim).

The main plugins I installed were:

ack.vim: Allows me to search the project using ack, similar to AckMate. To use it in vim I would :Ack search_for_this, i have now setup a mapping for my lead key and character (a shortcut really to the command)
bufexplorer.vim: This is a neat way to browse what files you have open in the buffer. I use this with a lead key shortcut too. Allowing quick switching files.
rails.vim: A rails plugin that really I find the syntax highlighting helpful, it does provide other things but i haven’t used them much.

A note on NERDTree, I started using this plugin which helps with directory navigating. But dropped it to use the included netrw in vim, it seems to do everything I need for now and thought it best to learn as much of the real tool as possible.

So on to the findings.

:( Go to File / Peepopen

The first thing that became apparent was not having a Go to File, the Command-T option that would list the files in your directories and allow you to quickly go to a file.

I have been using peepopen in textmate for this and it works with macvim! You simply switch the editor and reload, works really easily.

:) Love split screen

Splitting the code on a 23" screen helps for both showing test with code and view with controllers/models. I think this is a preference thing, I myself like it. Even when just learning new code from a project having the ability to have multiple windows makes it easier.

:) Shortcuts for text and visual mode

To remove and move text with commands on the keyboard is probably the most thing i’ve used. Having the ease to delete to the end of line and start writing, delete just the word or a couple and then having the visual mode helps too. Sure there is shift select in textmate for visual mode, but holding down the shift sometimes (not all) is annoying.

:( Running tests

When in a rails project using textmate I would always run a focussed test and window popup would open. With the vim setup I haven’t found an easy way to do this. I have to switch to a terminal and run the test: ruby -Itest test/blah -n '/the test to focus on/'. This slows things down a little by only the fact that running a new focussed test means writing out the matcher, obviously repeating a test is quick, just means alt-tab, up and bang to run the command again.

:( No markdown preview

Recently I’ve found myself writing more markdown and liked the ability to preview the page in textmate. Haven’t really looked if there is anything nice for vim on this.

:( Esc to complete

I barely used this in textmate, but have found myself wanting it now and then. It autocompleted the word you were writing when you pressed esc.

:/ To tab or not tab.

I’m finding it hard to decide on whether to open files in multiple tabs or to just switch between the buffers, this just get’s messy that’s all, i will eventually figure it out.

:) Even more fun to code

Learning something new with vim is fun and adds to the fun of doing the work I do. So for me this is a plus.

:) Remote developing.

I love the potential of using vim not just the macvim version on remote machines for pair development or even lightweight client’s being able to develop by using a remove machine having components installed (ipad for example).

Conclusion

I’m going to continue using Macvim, as I’m loving what you can do to the text in visual mode and the ability to delete and move lines/words around on the keyboard.

What do you think? Would you try or already use Macvim? Have any tipes?

LRUG talk on websockets

2010-11-10T00:00:00Z

I had the pleasure of giving a talk at LRUG this month on Websockets and Cramp.

I had some great fun using Showoff creating the slides. It allows you to write the slides in markdown and serve them using Sinatra and some awesome javascript to make things look pretty. Making it full screen with Chrome worked a charm.

You can find the slides over at http://github.com/kalv/lrug-websockets and the example chat app that I built over at http://github.com/kalv/yact.

Stop being clever Capybara

2010-10-27T00:00:00Z

Today we realised that there was a bunch of functionality being tested that didn’t work when javascript was disabled and had to spend a while figuring it out.

We found that it was because of the Rails 3 ability to make links become either HTTP DELETE or POST requests. So when creating a link in rails like this:

link_to "Delete blog post", blog_post(@blog), :method => :delete

It would output into a html link:

<a href="/blog/1" data-method="delete">Delete blog post</a>

The data-method is a HTML5 custom attribute by the looks of things. When the page is loaded the default rails.js javascript adds an event handler to those link elements to turn it into a form that will actually send a DELETE http request when it is clicked.

When testing these links in Capybara (an integration testing framework) for some reason it performs this javascript behaviour within it’s rack-test driver. Isn’t this wrong? Shouldn’t integration testing be just clicking through the site when javascript is disabled. And when you want to test javascript you use a javascript driver explicitly, like you can in cucumber and capybara:

@javascript
    Scenario: do something AJAXy
      When I click the AJAX link
      ...

This magic that capybara performs would lead a developer into believing that those links work without javascript but they don’t. Sure some people might say why worry about it, most browsers now know about javascript, but what happens if someone chooses to turn it off or it’s disabled for other reasons.

Moving forward I’m going to try and steer clear from using the ‘:method => :delete’ in rails or monkey patch capybara to stop it changing the test request based on the ‘data-method’.