So You Want To Be A Facebook Entrepreneur?

If you haven’t done so yet, check out this post over at TestFacebook. Lots to think about before starting that venture centering around a Facebook application.

Professional Services

Facebook Testing is a consulting service to help you test your Facebook applications. By focusing on the unique aspects of testing Facebook applications, they can provide you with a unique service. No need to explain to them the differences between Canvas and Connect applications, or what FBML and FQL are – they already know — and have experience testing them.

Facebook Testing is able to help you automate the testing of your Facebook application, and can do this for both functional and load testing. For more information,  please visit Facebook Testing.

Partnership Opportunities

For those of you who follow this blog, it may seem like we haven’t been too active for a while. Let me assure you that this is very far from the truth – in fact we’ve been very busy lately. We’re now ready to give you a small peek behind the curtain to let you know what we’ve been up to.

When we originally conceived of FriendRunner we imagined that it would be an all-in-one tool that customers could use to load test their Facebook applications. That’s what we built, and so far it’s worked out fairly well. However, we’re beginning to realize that there are some things we could be doing better:

• FriendRunner duplicates functionality that exists in other tools

FriendRunner needs to do things like record and edit test scripts, graph timing results, and lots of other things that standard load testing tools already do (and frankly do very well). Duplicating this effort in FriendRunner takes our attention away from providing more features on the Facebook-centric aspects that really add value.

• FriendRunner customers need to learn yet another tool

People who want to load test Facebook applications don’t spontaneously appear – they typically have experience load testing other web applications and have experience with other load testing tools. Forcing these people to read new documentation and learn how to use a new user interface seems like a waste of their time. Since they’re already experts in other tools, it would be better if we allowed them to use FriendRunner through them.

• People are very clever and come up with usage scenarios for FriendRunner that we never imagined

We occasionally come across potential customers who want to do things with FriendRunner that it was never really designed to do. Instead of saying “No, No, No, you’re using it wrong!!” we’d much rather embrace those people’s resourcefulness and provide them with a great solution.

All of this points to the fact that we’d be much better off by modularizing FriendRunner, and allowing other tools to do part of the work where appropriate. And that’s what we’ve been working on,  so now FriendRunner can be easily integrated with other load testing tools to allow them to test Facebook applications. This opens up a world of possibilities for others to partner with us to provide load testing services for Facebook applications.

Load Testing Tool Vendors

This one’s pretty obvious. Standard load testing tools can’t really deal with Facebook applications, but they can now thanks to FriendRunner. There are several ways of integrating FriendRunner with another tool ranging from the really simple to the highly integrated (which is still pretty easy to do). Vendors of these tools that would like to learn more about how to do this should look here.

Load Testing QA Consultants

Consultants that perform QA services for other companies can expand their offerings by using FriendRunner. Since FriendRunner can work with existing load testing tools, the learning curve is very small and they can be offering services in no time. QA consultants that would like to learn more about this should look here.

Digital Media Agencies

Agencies and contractors who build Facebook applications for brands and companies can use Friendrunner as an additional premium service to offer to their customers. To learn more about how you can benefit while offering your customers more services, have a look here.

Facebook’s new location service: Places

Back in March, we predicted the imminent announcement of the Facebook location-based service. Well, we were a little early on that call as Places was announced by Facebook yesterday.

Places is an exciting new turn for Facebook as many cool features and applications will be built using it, and Yes Virginia, FriendRunner will fully support it so that you may load test your location-aware Facebook application. Places is built around the idea of users “Checking in” at places from their mobile device so that their friends and applications can know where they are and where they have been. From a functional point of view, this looks a lot like foursquare.

Facebook has created an API so that applications may interact with Places. Currently, this API only supports Graph API apps, so those of you who are building with the classic RESTful API are SOL. I don’t think this will ever change, as trying to represent something as rich as location using something as complex as the RESTful API will simply end up as a fiasco. Facebook currently allows developers to read users’ checkin information, and some time in the future (apparently currently in private beta) applications will be able to make checkins on their own.

Alright, so where does FriendRunner fit into all of this. Well first off if you weren’t aware, FriendRunner does support, and we absolutely love the Graph API. Without that support there would be no way for us to implement Places. Now the reason we love the Graph API so much is that it’s so well thought out and regimented that querying a list of user checkins looks exactly like querying a list of user friends which looks like querying a list of a user’s favorite books. This means that we should have this support coded up fairly soon so that when you’re ready to load test your location-aware apps, FriendRunner will have you covered.

So go at it and start using location-awareness in your Facebook applications. When the time comes to test these features, let us know how FriendRunner can help you.

Contest

Since we’re talking about places and locations, see if you can identify the beautiful Georgian mansion featured here. This National Historical Landmark was built by a signer of the Declaration of Independence and was once (a long time ago) the workplace of Sericon Technology’s president. When you think you have the answer, let us know at contest@friendrunner.com. The first one to correctly answer will be featured on this page, and will enjoy worldwide fame from people who love testing Facebook apps. Extra credit if you can also provide the house’s coordinates.

Winner

 Dr. William M. Spears of Swarmotics has correctly identified Homewood House on the campus of The Johns Hopkins University as our mystery location. He’s also noted its coordinates as 39°19′46″N 76°37′9″W, as well as adding his personal comment that it is a “Good frisbee and tanning location (i.e., the ‘beach’).”

Congratulations and good work William!

Graph API?? Graph API!!

A few months ago, Facebook announced the new Graph API at their f8 conference. At the time, we were so excited about it we immediately posted something.

If you haven’t looked at it yet, the Graph API is a much simpler way to write Facebook applications when compared to Facebook’s “Classic” RESTful API. To understand the Graph API you only need to understand a handful of logical and internally consistent concepts, and then put those concepts together to build a powerful application. This is in sharp contrast to the classic way to build Facebook apps where nothing is consistent, and new features from Facebook simply look like patches on top of other patches. Since the new API is so simple and language agnostic, we’re even starting to see many libraries released in every conceivable language to make development that much easier.

For us here at FriendRunner, the Graph API brings yet another challange. We not only need to understand how the Graph API works from the application side, but we also need to support it from the Facebook side. Since FriendRunner effectively replaces the Facebook server, we need to support everything that it does. We’ve even had a few customers that we’ve had to turn away because they’re some of the cool kids who’ve started using the Graph API. We have to apologize and tell them that “FriendRunner doesn’t support the Graph API.”.

Until now.

We’re happy to report that FriendRunner can now be used to test your Facebook apps developed with the Graph API. Building this was not easy, as the API is simple yet vast. There’s a lot of ground that it covers, and to really be compliant, we need to support it all. And not just support it – we need to support it scalably so that we can run load tests off of it. Not only is our Graph API scalable, but we’ve hooked it into our Advanced Monitoring system so that it’s provably scalable.

We’re really proud of this work as it’s a major feature that would put FriendRunner light years ahead of the other Facebook app testing platforms if there were other Facebook app testing platforms. So if you’ve been building your Graph API app and were worried about how you can properly load test it, FriendRunner has you covered.

Hyper-viral Facebook Applications

Are you planning on writing a Facebook application that will go really viral and have millions of users? Do you avoid discussing the number of users you think your app will attract for fear that people will think you’re crazy? Then this is for you.

Facebook recently updated their Developer Principles & Polices document to deal with applications that have huge numbers of users or make lots of API calls to the Facebook server. This document, which your application needs to meet the requirements of, contains a short section (II.11) about very popular applications. It now states:

If you exceed, or plan to exceed, any of the following thresholds please contact us as you may be subject to additional terms: (>5M MAU) or (>100M API calls per day) or (>50M impressions per day).

Clearly, they’re concerned about large and popular applications overwhelming their infrastructure, and want to deal with them proactively. It isn’t commonly known what Facebook’s additional terms are that large applications need to meet, but it’s certainly reasonable for them to exist.

So what’s the FriendRunner connection? Well, if you’ve got more than five million active users, congratulations, but you’re going to have to abide by the additional Facebook rules. However, if you don’t have that number of users but you’re pushing against the hundred million API call limit, FriendRunner can possibly help. One of the things that FriendRunner does well is allow you to profile and debug the API calls that your application makes to the Facebook server. If unnecessary or redundant calls are being made, FriendRunner can help you find them to decrease your use of the Facebook server and possibly avoid any additional terms that you must meet.

FriendRunner adds Advanced Monitoring

One of the first things taught to medical students is Primum non nocere, which roughly translates into First, do no harm. This is understood to mean that it’s better to do nothing than to treat a patient in a way that you do more damage than good, or put another way: Don’t let the cure be worse than the illness.

So what does this have to do with FriendRunner? Lots – in fact it was the inspiration for our latest feature that you don’t even get to see since it works behind the scenes, but will be thankful it’s there. To understand why, you need to think about the two jobs that FriendRunner performs:

1. Calls the test application in the same way that Facebook would so the application can experience the load. While doing this, FriendRunner monitors the time it takes the application to respond to determine its health. (Steps 2 and 5 in the diagram below)
2. Allows the application to call the FriendRunner ”proxy” Facebook server to service any Facebook API calls that the application needs to make. (Steps 3 and 4 in the diagram below)

That’s really all that FriendRunner does. These two functions act together to form a system that can put enough load on the test application so we can learn something meaningful, and hopefully make the application more scalable and better.

But what about FriendRunner? What happens when it gets large loads put on it? Once we start running with a few thousand virtual users, will the proxy Facebook server be able to satisfy API requests fast enough? And what if it can’t? Well, that’s pretty easy to answer: a poorly performing API will result in poor performance at the test application level. In other words, FriendRunner (the tool used to measure an app’s performance) will be the reason that the app being tested exhibits poor performance.

This is the stuff that keeps us up at night. However, in reality, things really aren’t so bad. FriendRunner’s server that replies to API calls is not a single server, but a collection of state-less scalable servers deployed onto a computing infrastructure with almost infinite expandability. It’s very unlikely that it would ever be the cause of a problem and make an application under test appear to be slowing down. Yet, we still think about these things to ensure that our customers get our best service.

Our initial attempts to allay these concerns were to simply monitor the servers that the FriendRunner software ran on. We’d monitor the health of the computers, and watch the memory and CPU usage. This works pretty well at the macro level, but still doesn’t really say what happens at the individual call level. The server’s memory may be doing just fine, but we still may be affecting the performance of the application under test. Truthfully, we’ve never seen any evidence of this, but it still weighs heavy on us. Remember: First, do no harm. We take our customers very seriously, and don’t want to be the cause of misinformation.

Which brings us to our latest feature, Advanced Monitoring. We’ve now instrumented our Facebook API server so that we can measure the performance of the FriendRunner server at the API call level to prove that the system acts consistently. And that’s a very important point to this whole process – we aren’t interested in measuring our performance because we want to make it faster. No, we measure to prove consistency – something that’s very important when measuring the performance of another system. Our calls should return in about the same amount of time regardless of whether there are ten or ten thousand users running in the system. That’s a difficult standard to live up to, and not only do we do it, but Advanced Monitoring can now prove that this is what happens.

Unfortunately, despite this very sophisticated system working very well, there’s really nothing that you, the user, can see. But we’ll sleep much better at night knowing that FriendRunner is not doing any harm. And you … well, you should be happy that FriendRunner will always provide you with a large and consistent load so that testing and tuning your application is as simple as possible.

Hobsons chooses FriendRunner

We are pleased to announce that one of our first customers has gone live with its application! mySchools by Hobsons is a new Facebook application that was released a few weeks ago. This innovative tool is designed to help students find the most suitable school to attend.
 
It was very exciting for us to start using FriendRunner on an app that was mission critical to the company that developed it. We validated a number of our assumptions, and learned a bit more about the information that we can generate to be of maximum use to our customers.

To learn more about how FriendRunner helped Hobsons and can also help your application, please click here.
 
Stay tuned, Hobsons is only the first customer story we can let you know about. We have more successes that we’ll be announcing soon.

How to write a load test script – Part 2

In the last post, we discussed some strategies for deciding which user actions should go into your load test. It’s all pretty simple, include those actions that a typical user will take when using your app. But how easy is this to do in a very realistic manner. Let’s say users will visit one page twice as often as another, or the application is a game and the user will take different actions based on whether he won or lost. How can we do this in FriendRunner?

To get to the answer to these questions, you first need to understand a small secret about FriendRunner. Load test scripts are created in Selenium. However, that isn’t where the process of test script generation ends. The Selenium script is simply an easy way to communicate intentions about what the test should do, but there is no way to define all of the richness that a test should be able to hold. But that’s okay, the Selenium script isn’t even the thing that’s executed during a Friendrunner test. When you submit your Selenium script, FriendRunner will load it back into Selenium, and convert it into Java code. When you run your test through FriendRunner, it is this code that is actually controlling what your virtual users are doing.

This is great because it means that the virtual users are being controlled programmatically. If we need them to do some fancy things, all we’ll need to do is hand tweak the program. Let’s look at different things we can do:

Going to a page only some of the time

If there’s a page that should be a part of a load test, but users will only go to it on occasion, then we’ll need to take advantage of FriendRunner’s ability to use randomness in load test scripts. At any point in the script, instead of saying that the virtual user should go to some page, we can say that the user goes to some page X% of the time. Then, when we get to that point in the script we generate a random number between 1 and 100. If this number is less than X, then we’ll go to that page. This behavior can be used in any situation when some amount of randomness will help to make the test script more realistic.

User action determined by what’s on the page

Often, it’s a bit tricky to write a static serial test script because the application may respond to user requests in an unpredictable way.  For instance, if the application is a game, the response to a user’s move may depend on whether that user has won or lost the game. The action that the user takes next on a winning page may not even make sense if the page is a losing page. This means that the page must be “read” in order to determine the next action to take. FriendRunner is able to search a page for a piece of text, and control the next user action beased on whether that text is or isn’t there.

Two users who perform different actions

Each virtual user in FriendRunner has a user id, just like in Facebook. If we’d like 1% of the users to perform advanced functionality, and the rest of the users to perform normal functions, we can do that easily be taking advantage of the user ids.

Warning - Don’t get carried away

When writing your load test script, it’s often useful to take a break every half hour , take a deep breath, and say to yourself “I’m writing a load test script.” It’s often very easy to fall into the trap of trying to throw every user action and function into your test script, but this must be avoided. You aren’t writing a functional test, it’s a load test, and you should treat it that way. In our experience, even very large Facebook applications can be load tested with a handful of user actions.

The reason that you want to keep your load tests simple is because timing is so important to interpreting the results of the test. If you’d like to know the point at which your test, and your application began to go south, the best place to look is the transaction response graph. This graph shows you the average time it took to run through one cycle of the test script. If the test script is very long or complex, this graph will be less predictable and harder to interpret.

How to write a load test script – Part 1

One of the hardest parts of running a load test for your Facebook application is simply figuring out what to test. Since we’ve been through this a few times, we’ve come up with some ideas and tips for  how to do this properly and efficiently so that you can build a load test script that effectively tests your app.

An interesting observation we made was how closely the process of creating a test script is to Richard Feynman’s famous algorithm for solving every problem that you could possibly imagine. So let’s look at the steps you’ll need to take:

Building your FriendRunner test script

Step 1: Write down all of the things your user can do in the app

This is the tedius part of the exercise. Make a list of all of the features available to a user while he uses your application. Sometimes it’s useful to clump several discrete user actions into a single logical action. For instance, a user may browse to an account information page in your app, modify his information, and then save that change. When deciding what gets into a load test, it’s often better to think of user actions at a more macro level.

Step 2: Think really hard about what the user typically does when using your app

Does you application have a  page where you discuss your Privacy Policy? If not, you probably should, but that isn’t the issue. The issue is that very few of your users are ever going to this page. Ones that do will go once, and never return. Serving this page will probably incur very little load on your server that fulfills the request. Ergo, looking at the Privacy Policy should not be a part of a load test. Seems kind of obvious when you see it written down, but we’ve seen many things like this in proposed scripts.

What you should be putting into your script are the “normal” things that your user does when he interacts with your application. You can figure out what this is by observing how users interact with similar services (e.g. if you’re already running a non-Facebook version of your web application) or with the liberal use of some common sense.

Step 3: Write down the list of things to do for your load test script

Here’s the easy part. Once you decide on what a normal user does when he interacts with your app, put the steps together in a reasonable order. If several steps have some type of dependency on each other, take this into account. It’s also important to consider whether one user’s actions will have consequences for another user.

Special handling of FriendRunner load test scripts

In our next post, we’ll discuss some cool FriendRunner features that will allow your test scripts to more closely approximate the real world. Things such as going to some pages for some users but not others, or changing the user behavior based on information that’s displayed on the pages shown.

Contest

Identify the company that’s represented on the t-shirt that Feynman wears in the above picture, and name one thing that he did for that company. First person to answer correctly will be named Alpha Geek, and have their name posted on this page. Send your responses to contest@friendrunner.com.

Winner

Dr. Saul Kravitz was the first one to identify Thinking Machines as the company represented on Richard Feynman’s shirt. Feynman worked there for several years doing everything from painting the walls to developing deep insights into parallel computation.

For more info about Feynman’s relationship with Thinking Machines, see http://www.longnow.org/essays/richard-feynman-connection-machine/