Facebook’s new location service: Places

Back in March, we predicted the imminent announcement of the Facebook location-based service. Well, we were a little early on that call as Places was announced by Facebook yesterday.

Places is an exciting new turn for Facebook as many cool features and applications will be built using it, and Yes Virginia, FriendRunner will fully support it so that you may load test your location-aware Facebook application. Places is built around the idea of users “Checking in” at places from their mobile device so that their friends and applications can know where they are and where they have been. From a functional point of view, this looks a lot like foursquare.

Facebook has created an API so that applications may interact with Places. Currently, this API only supports Graph API apps, so those of you who are building with the classic RESTful API are SOL. I don’t think this will ever change, as trying to represent something as rich as location using something as complex as the RESTful API will simply end up as a fiasco. Facebook currently allows developers to read users’ checkin information, and some time in the future (apparently currently in private beta) applications will be able to make checkins on their own.

Alright, so where does FriendRunner fit into all of this. Well first off if you weren’t aware, FriendRunner does support, and we absolutely love the Graph API. Without that support there would be no way for us to implement Places. Now the reason we love the Graph API so much is that it’s so well thought out and regimented that querying a list of user checkins looks exactly like querying a list of user friends which looks like querying a list of a user’s favorite books. This means that we should have this support coded up fairly soon so that when you’re ready to load test your location-aware apps, FriendRunner will have you covered.

So go at it and start using location-awareness in your Facebook applications. When the time comes to test these features, let us know how FriendRunner can help you.

Contest

Since we’re talking about places and locations, see if you can identify the beautiful Georgian mansion featured here. This National Historical Landmark was built by a signer of the Declaration of Independence and was once (a long time ago) the workplace of Sericon Technology’s president. When you think you have the answer, let us know at contest@friendrunner.com. The first one to correctly answer will be featured on this page, and will enjoy worldwide fame from people who love testing Facebook apps. Extra credit if you can also provide the house’s coordinates.

Winner

 Dr. William M. Spears of Swarmotics has correctly identified Homewood House on the campus of The Johns Hopkins University as our mystery location. He’s also noted its coordinates as 39°19′46″N 76°37′9″W, as well as adding his personal comment that it is a “Good frisbee and tanning location (i.e., the ‘beach’).”

Congratulations and good work William!

Graph API?? Graph API!!

A few months ago, Facebook announced the new Graph API at their f8 conference. At the time, we were so excited about it we immediately posted something.

If you haven’t looked at it yet, the Graph API is a much simpler way to write Facebook applications when compared to Facebook’s “Classic” RESTful API. To understand the Graph API you only need to understand a handful of logical and internally consistent concepts, and then put those concepts together to build a powerful application. This is in sharp contrast to the classic way to build Facebook apps where nothing is consistent, and new features from Facebook simply look like patches on top of other patches. Since the new API is so simple and language agnostic, we’re even starting to see many libraries released in every conceivable language to make development that much easier.

For us here at FriendRunner, the Graph API brings yet another challange. We not only need to understand how the Graph API works from the application side, but we also need to support it from the Facebook side. Since FriendRunner effectively replaces the Facebook server, we need to support everything that it does. We’ve even had a few customers that we’ve had to turn away because they’re some of the cool kids who’ve started using the Graph API. We have to apologize and tell them that “FriendRunner doesn’t support the Graph API.”.

Until now.

We’re happy to report that FriendRunner can now be used to test your Facebook apps developed with the Graph API. Building this was not easy, as the API is simple yet vast. There’s a lot of ground that it covers, and to really be compliant, we need to support it all. And not just support it – we need to support it scalably so that we can run load tests off of it. Not only is our Graph API scalable, but we’ve hooked it into our Advanced Monitoring system so that it’s provably scalable.

We’re really proud of this work as it’s a major feature that would put FriendRunner light years ahead of the other Facebook app testing platforms if there were other Facebook app testing platforms. So if you’ve been building your Graph API app and were worried about how you can properly load test it, FriendRunner has you covered.

Hyper-viral Facebook Applications

Are you planning on writing a Facebook application that will go really viral and have millions of users? Do you avoid discussing the number of users you think your app will attract for fear that people will think you’re crazy? Then this is for you.

Facebook recently updated their Developer Principles & Polices document to deal with applications that have huge numbers of users or make lots of API calls to the Facebook server. This document, which your application needs to meet the requirements of, contains a short section (II.11) about very popular applications. It now states:

If you exceed, or plan to exceed, any of the following thresholds please contact us as you may be subject to additional terms: (>5M MAU) or (>100M API calls per day) or (>50M impressions per day).

Clearly, they’re concerned about large and popular applications overwhelming their infrastructure, and want to deal with them proactively. It isn’t commonly known what Facebook’s additional terms are that large applications need to meet, but it’s certainly reasonable for them to exist.

So what’s the FriendRunner connection? Well, if you’ve got more than five million active users, congratulations, but you’re going to have to abide by the additional Facebook rules. However, if you don’t have that number of users but you’re pushing against the hundred million API call limit, FriendRunner can possibly help. One of the things that FriendRunner does well is allow you to profile and debug the API calls that your application makes to the Facebook server. If unnecessary or redundant calls are being made, FriendRunner can help you find them to decrease your use of the Facebook server and possibly avoid any additional terms that you must meet.

FriendRunner adds Advanced Monitoring

One of the first things taught to medical students is Primum non nocere, which roughly translates into First, do no harm. This is understood to mean that it’s better to do nothing than to treat a patient in a way that you do more damage than good, or put another way: Don’t let the cure be worse than the illness.

So what does this have to do with FriendRunner? Lots – in fact it was the inspiration for our latest feature that you don’t even get to see since it works behind the scenes, but will be thankful it’s there. To understand why, you need to think about the two jobs that FriendRunner performs:

1. Calls the test application in the same way that Facebook would so the application can experience the load. While doing this, FriendRunner monitors the time it takes the application to respond to determine its health. (Steps 2 and 5 in the diagram below)
2. Allows the application to call the FriendRunner ”proxy” Facebook server to service any Facebook API calls that the application needs to make. (Steps 3 and 4 in the diagram below)

That’s really all that FriendRunner does. These two functions act together to form a system that can put enough load on the test application so we can learn something meaningful, and hopefully make the application more scalable and better.

But what about FriendRunner? What happens when it gets large loads put on it? Once we start running with a few thousand virtual users, will the proxy Facebook server be able to satisfy API requests fast enough? And what if it can’t? Well, that’s pretty easy to answer: a poorly performing API will result in poor performance at the test application level. In other words, FriendRunner (the tool used to measure an app’s performance) will be the reason that the app being tested exhibits poor performance.

This is the stuff that keeps us up at night. However, in reality, things really aren’t so bad. FriendRunner’s server that replies to API calls is not a single server, but a collection of state-less scalable servers deployed onto a computing infrastructure with almost infinite expandability. It’s very unlikely that it would ever be the cause of a problem and make an application under test appear to be slowing down. Yet, we still think about these things to ensure that our customers get our best service.

Our initial attempts to allay these concerns were to simply monitor the servers that the FriendRunner software ran on. We’d monitor the health of the computers, and watch the memory and CPU usage. This works pretty well at the macro level, but still doesn’t really say what happens at the individual call level. The server’s memory may be doing just fine, but we still may be affecting the performance of the application under test. Truthfully, we’ve never seen any evidence of this, but it still weighs heavy on us. Remember: First, do no harm. We take our customers very seriously, and don’t want to be the cause of misinformation.

Which brings us to our latest feature, Advanced Monitoring. We’ve now instrumented our Facebook API server so that we can measure the performance of the FriendRunner server at the API call level to prove that the system acts consistently. And that’s a very important point to this whole process – we aren’t interested in measuring our performance because we want to make it faster. No, we measure to prove consistency – something that’s very important when measuring the performance of another system. Our calls should return in about the same amount of time regardless of whether there are ten or ten thousand users running in the system. That’s a difficult standard to live up to, and not only do we do it, but Advanced Monitoring can now prove that this is what happens.

Unfortunately, despite this very sophisticated system working very well, there’s really nothing that you, the user, can see. But we’ll sleep much better at night knowing that FriendRunner is not doing any harm. And you … well, you should be happy that FriendRunner will always provide you with a large and consistent load so that testing and tuning your application is as simple as possible.

Hobsons chooses FriendRunner

We are pleased to announce that one of our first customers has gone live with its application! mySchools by Hobsons is a new Facebook application that was released a few weeks ago. This innovative tool is designed to help students find the most suitable school to attend.
 
It was very exciting for us to start using FriendRunner on an app that was mission critical to the company that developed it. We validated a number of our assumptions, and learned a bit more about the information that we can generate to be of maximum use to our customers.

To learn more about how FriendRunner helped Hobsons and can also help your application, please click here.
 
Stay tuned, Hobsons is only the first customer story we can let you know about. We have more successes that we’ll be announcing soon.

How to write a load test script – Part 2

In the last post, we discussed some strategies for deciding which user actions should go into your load test. It’s all pretty simple, include those actions that a typical user will take when using your app. But how easy is this to do in a very realistic manner. Let’s say users will visit one page twice as often as another, or the application is a game and the user will take different actions based on whether he won or lost. How can we do this in FriendRunner?

To get to the answer to these questions, you first need to understand a small secret about FriendRunner. Load test scripts are created in Selenium. However, that isn’t where the process of test script generation ends. The Selenium script is simply an easy way to communicate intentions about what the test should do, but there is no way to define all of the richness that a test should be able to hold. But that’s okay, the Selenium script isn’t even the thing that’s executed during a Friendrunner test. When you submit your Selenium script, FriendRunner will load it back into Selenium, and convert it into Java code. When you run your test through FriendRunner, it is this code that is actually controlling what your virtual users are doing.

This is great because it means that the virtual users are being controlled programmatically. If we need them to do some fancy things, all we’ll need to do is hand tweak the program. Let’s look at different things we can do:

Going to a page only some of the time

If there’s a page that should be a part of a load test, but users will only go to it on occasion, then we’ll need to take advantage of FriendRunner’s ability to use randomness in load test scripts. At any point in the script, instead of saying that the virtual user should go to some page, we can say that the user goes to some page X% of the time. Then, when we get to that point in the script we generate a random number between 1 and 100. If this number is less than X, then we’ll go to that page. This behavior can be used in any situation when some amount of randomness will help to make the test script more realistic.

User action determined by what’s on the page

Often, it’s a bit tricky to write a static serial test script because the application may respond to user requests in an unpredictable way.  For instance, if the application is a game, the response to a user’s move may depend on whether that user has won or lost the game. The action that the user takes next on a winning page may not even make sense if the page is a losing page. This means that the page must be “read” in order to determine the next action to take. FriendRunner is able to search a page for a piece of text, and control the next user action beased on whether that text is or isn’t there.

Two users who perform different actions

Each virtual user in FriendRunner has a user id, just like in Facebook. If we’d like 1% of the users to perform advanced functionality, and the rest of the users to perform normal functions, we can do that easily be taking advantage of the user ids.

Warning - Don’t get carried away

When writing your load test script, it’s often useful to take a break every half hour , take a deep breath, and say to yourself “I’m writing a load test script.” It’s often very easy to fall into the trap of trying to throw every user action and function into your test script, but this must be avoided. You aren’t writing a functional test, it’s a load test, and you should treat it that way. In our experience, even very large Facebook applications can be load tested with a handful of user actions.

The reason that you want to keep your load tests simple is because timing is so important to interpreting the results of the test. If you’d like to know the point at which your test, and your application began to go south, the best place to look is the transaction response graph. This graph shows you the average time it took to run through one cycle of the test script. If the test script is very long or complex, this graph will be less predictable and harder to interpret.

How to write a load test script – Part 1

One of the hardest parts of running a load test for your Facebook application is simply figuring out what to test. Since we’ve been through this a few times, we’ve come up with some ideas and tips for  how to do this properly and efficiently so that you can build a load test script that effectively tests your app.

An interesting observation we made was how closely the process of creating a test script is to Richard Feynman’s famous algorithm for solving every problem that you could possibly imagine. So let’s look at the steps you’ll need to take:

Building your FriendRunner test script

Step 1: Write down all of the things your user can do in the app

This is the tedius part of the exercise. Make a list of all of the features available to a user while he uses your application. Sometimes it’s useful to clump several discrete user actions into a single logical action. For instance, a user may browse to an account information page in your app, modify his information, and then save that change. When deciding what gets into a load test, it’s often better to think of user actions at a more macro level.

Step 2: Think really hard about what the user typically does when using your app

Does you application have a  page where you discuss your Privacy Policy? If not, you probably should, but that isn’t the issue. The issue is that very few of your users are ever going to this page. Ones that do will go once, and never return. Serving this page will probably incur very little load on your server that fulfills the request. Ergo, looking at the Privacy Policy should not be a part of a load test. Seems kind of obvious when you see it written down, but we’ve seen many things like this in proposed scripts.

What you should be putting into your script are the “normal” things that your user does when he interacts with your application. You can figure out what this is by observing how users interact with similar services (e.g. if you’re already running a non-Facebook version of your web application) or with the liberal use of some common sense.

Step 3: Write down the list of things to do for your load test script

Here’s the easy part. Once you decide on what a normal user does when he interacts with your app, put the steps together in a reasonable order. If several steps have some type of dependency on each other, take this into account. It’s also important to consider whether one user’s actions will have consequences for another user.

Special handling of FriendRunner load test scripts

In our next post, we’ll discuss some cool FriendRunner features that will allow your test scripts to more closely approximate the real world. Things such as going to some pages for some users but not others, or changing the user behavior based on information that’s displayed on the pages shown.

Contest

Identify the company that’s represented on the t-shirt that Feynman wears in the above picture, and name one thing that he did for that company. First person to answer correctly will be named Alpha Geek, and have their name posted on this page. Send your responses to contest@friendrunner.com.

Winner

Dr. Saul Kravitz was the first one to identify Thinking Machines as the company represented on Richard Feynman’s shirt. Feynman worked there for several years doing everything from painting the walls to developing deep insights into parallel computation.

For more info about Feynman’s relationship with Thinking Machines, see http://www.longnow.org/essays/richard-feynman-connection-machine/ 

Facebook Graph API on Android

Last week at Google I/O, Google’s developer conference, an important announcement about Facebook was made that’s gone fairly unnoticed (well, All Facebook did notice it). Android now has its very own API to take advantage of the Facebook Graph API, and is callable directly from any Android application. This is very similar to what’s happening with Facebook Connect for iPhone, but technically more sophisticated. The Android library supports the new Graph API, as well as support for authentication through OAuth 2.0. All the things the cool kids are using to build their Facebook apps.

Of course you’re wondering “How does FriendRunner fit into all of this?”. Is there really a need to load test an application that’s running on a mobile device? The answer to that is no! It’s a mobile device – that doesn’t even make any sense. Well…., unless there is also a backend server you run that the Android app talks to. If that’s the case, and you want to test the scalability of your backend server, then using FriendRunner makes loads of sense.

The way we’d imagine this to work is the following: We’d take your Facebook-enabled Android app and instead of loading it onto an Android device, we’d run it with an Android simulator. In fact, we’d run a whole lot of simulators in parallel. They’d all interact with your with your backend server giving it quite a workout, and allowing you to determine whether it would survive in the wild. The FriendRunner goodness comes into play because the Android app also needs to interact with a Facebook server, but typical Facebook load testing rules will apply, which are Don’t Do This. FriendRunner alleviates this problem by allowing the Android app to interact with it instead of a real Facebook server, and so allows a full load test to be run.

So do you think a tool like this would be of value? Does this address issues that mobile developers sometimes find themselves faced with, and would it solve their problems. Leave us a note and let us know what you think.

No more 24 hour rule!

One of the more interesting announcements coming out of f8 was the relaxation on the prohibition of cacheing data for more than 24 hours. Now I’ve always thought that this prohibition came from the privacy world – don’t save people’s information because it isn’t yours. Turns out, it isn’t a privacy thing at all – it’s really a user experience issue. The problem with holding on to people’s data is what happens if it changes? If you store the data and rely on it being current, things will fall apart when the user changes it several weeks from when you stored it. However, if you can only hold onto it for 24 hours, the chances of something going wrong go down dramatically.

These concerns go away now because of the new service that allows developers to register with Facebook to be notified when a user’s information changes. This will allow any cached data to be updated without any concerns of getting out of whack. This can dramatically simplify writing applications, and it was one new feature that seems to be universally embraced by developers.

And how does this affect FriendRunner? The FriendRunner profiler features that allow developers to see what APIs get called during the execution of their application become more important. There is now no excuse at all to call the same API with the same data, since the result may be cached, FriendRunner can help you find these types of problems, and identify places where you can optimize your code.

The Facebook Graph API

The new Facebook Graph API announced at f8 represents a revolution in the way that Facebook applications will be developed from here on in. You want to get a list of a users’ friends? No longer will you need to weed through the PHP documentation to try to figure out which API to call, and which parameters to pass. Now, all you need to do is make a simple RESTful HTTP call, and get a nicely formatted JSON response in return. How do you format the REST call? Pretty simple, I’m guessing that after you do 2 or 3 of these it will become second nature, and you won’t even need to look at the documentation.

So how will this affect FriendRunner? My initial thoughts is that it will affect FriendRunner in a positive way. All of the original API will still be in force, so nothing there will change. What will change it that FriendRunner will need to support all of the REST API calls at https://graph.facebook.com. But that won’t be so hard because it’s simply getting at the same data that we already provide, just in a different way. But even if it is hard, that’s okay – we subscribe to the “Why does Rice play Texas” philosophy.

Since the RESTful API is language agnostic, the FriendRunner support can be written any way we want, and not necessarily in PHP. This makes us very happy.

Stay tuned to get more information about when we’ll have the new API coded up and ready. In the meantime, all of the investment you’ve made in wriing Facebook apps the old way is secure, and your apps will continue to work. And FriendRunner will continue to be available for testing them.