Rapportive

Raplets API

Applications for the Rapportive Inbox Platform

This is version 0.5 (March 2011) of the Raplet documentation. For changes since previous versions, see the changelog. For updates, please subscribe to the raplet-announce mailing list. For help, questions or feedback, please email the raplet-dev mailing list.

Overview

The aim of Rapportive is to give email users access to relevant contextual information as part of their normal email workflow. In its current form, Rapportive displays a sidebar next to an email thread in Gmail, showing information about one particular person. The information is collated from social networking sites and other sources. Rapportive chooses one sender or recipient of an email thread to show by default; if the user wishes to see information about another person, they can hover their mouse over that person’s email address.

Soon after Rapportive was released, users started asking us whether we could add various other pieces of information to the sidebar. In response, we introduced Raplets, which are a flexible way for any web developer to insert content into Rapportive, and to provide rich integrations with external services.

A Raplet is a piece of code that runs on your server and is accessible via a URL. Any user can add Raplets to their sidebar by visiting raplets.com and entering the Raplet URL. We showcase a number of existing Raplets at raplets.com and you can integrate with that page to provide a seamless installation experience.

Rapportive is currently implemented as a browser extension that dynamically modifies the Gmail page to insert the sidebar. However, Raplets are designed with future portability in mind. As Rapportive expands into other email clients and communication platforms, we will keep Raplets compatible with this specification, provided that it is technically feasible to do so.

Use Cases

Who would benefit from developing a Raplet? There are many potential use cases:

How Raplets Work

Let’s run through an example to get you started quickly.

Your server and URL

Your Raplet runs on your own server. You may use any programming language, and any hosting provider (or even a server on your local network or your local machine, provided that your browser can access it by HTTP). If you need to identify individual users, you can use cookies or the configuration protocol. If your data is publicly accessible, you don’t need authentication.

In this example, let’s say you are developing the Raplet on your local machine, and you are serving it at the following URL: http://localhost:8080/raplet.php

Any valid URL is fine, with the following restrictions:

Request Format

Raplets are implemented using JSONP — but don’t worry if you’ve not heard about it before; it’s easy.

Say you get an email from Rahul Vohra, whose email address is rahul@rapportive.com and whose Twitter username is @rahulvohra. If you have Rapportive and your Raplet installed, Rapportive will ask your browser to make a GET request to your Raplet URL, and include some information about Rahul in the query parameters. At the moment, the following parameters are included (URL-encoded as required for URLs):

Some values may be unknown for some people. If a parameter is missing or its value is the empty string, your Raplet should simply treat it as unknown. The parameters may appear in any order, and parameters may be added or removed in future. If your Raplet doesn’t recognize a parameter, it should ignore it.

For example, if your Raplet is at http://localhost:8080/raplet.php, your web browser will request something like the following:

http://localhost:8080/raplet.php?email=rahul@rapportive.com&name=Rahul%20Vohra&twitter_username=rahulvohra&callback=jsonp123456789

Response Format

Ok, your server has received a Raplet request; you have read out the parameters, queried your database and maybe found something relevant about that person. Now you build your response to be sent back to the browser.

The response must be formatted as a JSON object. The following properties are currently supported in the response object:

All of this must be encoded as a JSON object (see the specification at http://json.org for details). Your JSON must be valid (invalid JSON causes errors which are hard to diagnose), so we strongly recommend that you use a standard JSON library that escapes everything correctly. JSON libraries are available for all mainstream programming languages.

Finally, when you have encoded your JSON, there’s one last thing you need to do. Before the JSON object, you need to prepend the value of the callback query parameter (e.g. jsonp123456789) and an opening parenthesis ’(’. After the JSON object, you need to append a closing parenthesis ’)’. And that’s your entire response. It should look something like this:

jsonp123456789({
    "html": "<h2 class='name'>Rahul Vohra</h2><p>Lifetime value: $1,000,000!</p>",
    "css": "h2.name { color: grey; }",
    "js": "alert('Hello world!');",
    "status": 200
})

Make sure there is no spurious HTML anywhere in the response, otherwise your Raplet will not work. Some programming environments (e.g. PHP) print warning messages directly to the page, which will cause your response to be invalid, so please ensure that all errors and warnings are handled properly. If you can, you should send your response with Content-type text/javascript, although most browsers will accept your response even if it’s a different content type.

Add The Raplet to Rapportive

Open Gmail in a browser with Rapportive installed. Click “Rapportive” at the top of the Gmail page, then click “Add or Remove Raplets”. You will be logged in to Rapportive if necessary. (If you are already logged in, you can take a short cut by going directly to http://raplets.com.)

On the Raplets page, click “Add” in the “Custom Raplet” section, and paste your Raplet URL into the field provided. In this example it would be http://localhost:8080/raplet.php.

Go back to Gmail and look up a person in Rapportive, e.g. by hovering over an email address. The sidebar should load, including your Raplet. It’s usually not necessary to reload your Gmail tab, but if it’s not working as expected, try reloading Gmail as your first measure.

If it doesn’t work, please see the troubleshooting section. If it does, congratulations on your first Raplet! If you want it to be added to our directory of featured Raplets on the Rapportive website, please contact supportive@rapportive.com.

Using JavaScript

The best Raplets provide rich interactive behavior that allows users to accomplish tasks without ever having to leave their inbox. You can make your Raplets interactive using JavaScript.

JavaScript and jQuery

You can include JavaScript in your Raplet as a string in the js property of the JSON response object. As with all other content, please make sure that it is correctly escaped for JSON. The Raplet’s JavaScript code is executed after the Raplet HTML has been inserted into the page, and every time the sidebar is refreshed.

jQuery is automatically provided for you (currently version 1.4.2), and we strongly recommend that you use it. You can use jQuery’s functions jQuery and $. They are automatically scoped to your Raplet: for example, $('a') selects all links within your Raplet, not all links in the entire Gmail page. Please see the section on security for further detail.

For example, a Raplet response with JavaScript may look a bit like this:

jsonp123456789({
    "html": "<h2 class='name'>Rahul Vohra</h2><p>Lifetime value: $1,000,000!</p>",
    "css": "h2.name { color: grey; }",
    "js": "$('h2.name').click(function(){ $(this).text('You clicked me!'); });",
    "status": 200
})

Sending Data to Your Server

To provide interactive features, your Raplet may need to send information back to your server (e.g. if you want to provide a button which adds a new contact to your CRM). You would normally do this using Ajax, but the standard techniques and libraries that rely on XMLHttpRequest do not work in Raplets. This is because the browser page was loaded from mail.google.com, and for security reasons, browsers block cross-domain requests to any server other than mail.google.com.

We currently have two methods which you can use to send data back to your server: JSONP and IFrames.

Metadata and Configuration

There is more information surrounding Raplets than just looking up people. Unless you are the only person using your Raplet, you probably want to give it a name, description, icon and other information that helps users make the most of your Raplet. This is achieved by the Metadata protocol.

In addition, some Raplets need to be configured by the user. For example, if you are building on a third-party API which requires that users provide their API key or password, you need some way of asking the user for that information when they install your Raplet. This is achieved by the configuration protocol.

If you submit your Raplet to Rapportive to be included in our list of featured Raplets, the information about your Raplet will also be taken from this metadata.

The Metadata Protocol

When your Raplet is configured, Rapportive may ask your server for additional information about your Raplet. This is done with a request to your Raplet URL, but instead of the email, name and other person-specific parameters, the following query parameters are given:

For example, a metadata request may look like this: http://localhost:8080/raplet.php?show=metadata&callback=jsonp123456789

If your Raplet receives a metadata request, instead of the usual Raplet response, it should return a JSONP object with the following properties (all required unless marked “optional”):

For example, your response may look something like the following:

jsonp123456789({
    "name": "CrunchBase",
    "description": "Essential intelligence for every entrepreneur and investor. ",
    "welcome_text": "<p><a href=\"http://www.crunchbase.com\">CrunchBase</a> is an essential resource for tech entrepreneurs and investors.</p>\n<p>...</p>",
    "icon_url": "http://rapportive.com/images/raplets/crunchbase-icon.png",
    "preview_url": "http://rapportive.com/images/raplets/crunchbase-preview.png",
    "provider_name": "Rapportive",
    "provider_url": "http://rapportive.com/",
    "data_provider_name": "the CrunchBase API",
    "data_provider_url": "http://www.crunchbase.com/help/api"
})

The Configuration Protocol

If your Raplet shows the same information to any user, there is probably no need to configure it. However, if distinguishing users is important, you can use this protocol to configure your Raplet when a user installs it.

The configuration protocol is based on the draft OAuth2 specification, and serves several purposes:

The basic workflow of the configuration protocol is:

  1. As part of the metadata for your Raplet you specify a config_url, which can be any URL on your servers.
  2. When a user wants to install your Raplet, Rapportive opens a popup window (600 pixels wide, 500 pixels high), loading a page on your server (derived from config_url as explained below) in that window.
  3. Your page can do whatever it needs to do: ask the user for their username and password, present a configuration form, and/or ask the user for permission to pass data to Rapportive. This might be a single button, or it might be a multi-step process.
  4. When you are finished, you redirect the browser back to a special URL (including a token, as explained below), we securely store the token on Rapportive’s servers, and the popup window is closed.
  5. The Raplet is now installed. Whenever the user looks up a contact in Rapportive, the token is included as one of your Raplet’s query parameters.

The crucial part of this protocol is correctly constructing the various URLs.

For example, say that you specified "config_url": "https://example.com/config" in your Raplet metadata. When the user chooses to add your Raplet and we open the popup, it initially loads a URL which we construct by adding three query parameters to your config_url:

An example request might look like:

https://example.com/config?client_id=rapportive&response_type=token&
    redirect_uri=http%3A%2F%2Frapportive.com%2Fraplets%2Foauth_callback%2FBAh7BzoPc--mFwbGV0X3Vyb

You now need to ask the user to sign in and specify their preferences. Once that is done, you need to construct a token containing all the information you need in the Raplet: for example, their user ID, their account name, and/or a secret key or signature to verify that the information is true. (If you use a secret key, make sure that you use a different key for every user!)

Your token can be any string, and can be structured in whatever way you like. It could be a random string which you look up in your database, or it could be a string of properties, signed with a HMAC to protect it from tampering. For example, if you want to remember that the user who installed the Raplet had a user_id of 100, that they selected to use the account called test in the Raplet preferences, your token could be something like user_id$100~account$test~key$98765 (where the last part is your HMAC). In this example, we used $ and ~ as separator characters because they don’t need to be URL-encoded, but you can choose others.

After constructing the token, you should take the redirect_uri that we passed to you previously, append #access_token= and then append your token (encoded as though it were a HTTP query string; for example, ?, = and & should be replaced by %3F %3D and %2F). Then you redirect the user’s popup window to that URL you constructed.

An example redirect might look like:

https://rapportive.com/raplets/oauth_callback/BAh7BzoPc--mFwbGV0X3Vyb#access_token=user_id$100~account$test~key$98765

Rapportive will remember the access_token you included in the redirect, and store it on the user’s account. Now that the Raplet has been installed, whenever the user looks up a contact, Rapportive will include that token whenever it makes a request to your Raplet, using a parameter called oauth_token.

For example, looking up rahul@rapportive.com now makes requests like the following to your Raplet:

http://localhost:8080/raplet.php?email=rahul@rapportive.com&name=Rahul%20Vohra&twitter_username=rahulvohra
    &oauth_token=user_id$100~account$test~key$98765&callback=jsonp123456789

With the oauth_token you can now identify the user and present the Raplet appropriately.

Configuration Parameters (Deprecated)

Before introducing the configuration protocol, as described above, Rapportive used a different mechanism for configuring Raplets. That mechanism is documented here. It will continue to be supported until at least February 2012, but it should not be used for any new developments.

Security

Openness of JSONP

Rapportive uses JSONP to load Raplets because it enables cross-domain requests (so that a page from mail.google.com can make a request to your Raplet server, and receive the response). This feature is also JSONP’s greatest weakness: not only Rapportive and pages on mail.google.com, but pages on any domain can make requests to your Raplet URL.

This creates a security risk if your Raplet provides access to sensitive data: if an attacker knows your Raplet URL and they can trick you into visiting a specially prepared web page (e.g. by sending you an email like “hey, look at my funny pictures of cats on this site!”), they can use your browser to send requests to your Raplet, leak the data back to their own servers and thus potentially gain access to your sensitive information.

You can prevent this kind of attack using the configuration protocol to generate a unique token for each user, as follows:

  1. For each user of your application who may use a Raplet, generate a “token”, which is a string that is sufficiently random that it cannot be found by guessing or brute force (typically either a per-user random string which you store in your database, or a HMAC).

  2. Use the configuration protocol to provide this token to Rapportive when the user adds your Raplet.

  3. We store this token for each Rapportive user on Rapportive servers, and include it as a query parameter on all requests to your Raplet by that particular user.

  4. When your Raplet receives a request for data, you check that a token has been included as a query parameter, and that it matches the token you generated for that user. If the token is missing or does not match, you reject the request and provide a helpful error message.

Your Raplet is now secure because the attacker cannot find out your users’ secret tokens. (Metadata requests are not protected by tokens, but they shouldn’t contain any substantial sensitive information anyway.)

JavaScript sandboxing

In the current version of Rapportive, JavaScript inside Raplets is run with only minimal sandboxing: exceptions are caught, and jQuery selectors are by default scoped to the DOM element containing the Raplet HTML.

A malicious Raplet author could easily circumvent these limitations and access any part of the Gmail page, which may contain large amounts of sensitive information. We are therefore warning users that they should only install Raplets from sources they trust.

We recognize that trust alone is not enough, so we are planning to fully sandbox Raplets in future. By sandboxing, we will eliminate all known ways of accessing or modifying data to without the user’s explicit permission. We are also planning to introduce additional APIs through which Raplets can request permission to access information other than the parameters we pass to the Raplet request.

To avoid breaking your Raplet when we introduce sandboxing in future, we ask you to write your JavaScript very conservatively:

This is not an exhaustive list of limitations. In general, if you write your JavaScript according to best practices, it should continue working when we introduce sandboxing of Raplets.

Privacy

By observing the email addresses and other information sent to your Raplet, you could learn a large amount of potentially sensitive information about a user. We expect you to be absolutely transparent and honest about how you will use and safeguard this information, and it is up to you to make sure that users of your Raplet understand your policies.

If you request that your Raplet is added to the list of suggested Raplets on the Rapportive website, we will need to periodically verify that your handling of data meets Rapportive’s legal and ethical standards.

Advanced Usage

Collapsible Sections

The Rapportive sidebar consists of a number of vertically stacked sections which may appear in either a collapsed or an expanded state. The collapsed state is typically very compact (less than 30 pixels high, if possible) to display only essential information, whereas the expanded version may contain substantial detail. If the expanded content is very tall it may be scrolled.

Sections are expanded and collapsed based on the available screen space and the user’s choices. If a section is collapsed, the user can hover their mouse over it to see the expanded contents, or they can click a button to expand that section in the sidebar. If space is limited, other sections may be collapsed when a section is expanded, in order to free up space.

By default, your Raplet is given its own section, and your content is rendered to the expanded version. The collapsed version of the section is automatically generated from the icon and Raplet name specified in the metadata. The collapsed version is only shown if your expanded raplet has content; if the expanded version would appear empty, the section is invisible in both states.

You can customize the collapsed version of your Raplet with a response like this:

jsonp123456789({
    "status": 200,
    "css": "h2.name { color: grey; }\n .collapsed { white-space: nowrap; overflow: hidden; }",
    "js": "$('h2.name').click(function(){ $(this).text('You clicked me!'); });",
    "sections": [
        {
            "expanded_html": "<h2 class='name'>Rahul Vohra</h2><p>Lifetime value: $1,000,000!</p>",
            "collapsed_html": "<div class='collapsed'>$1,000,000</div>"
        }
    ]
})

Your Raplet response is interpreted as follows:

Feedback on this new user interface is particularly welcome. Please email the raplet-dev mailing list or supportive@rapportive.com.

Raplet Parameters from Message Content

If you operate an application which sends automated notification emails, Rapportive can tell your raplet when the user is looking at one of your notification emails.

For example, if you have a helpdesk system, you can pass the ticket number from an email to your raplet. This is particularly useful if you want to offer contextual actions based on a message: your Raplet can execute JavaScript, so you can let the user update the status of the ticket or assign it to a colleague, without needing to leave their inbox.

To pass a special parameter to your Raplet, you need to include a special link in the body of your notification emails. In plaintext emails, you can just include a URL; in HTML emails, we look at the href attribute of links in the message body, and you can hide the link by making the <a> tag empty.

Let’s say your raplet URL is http://localhost:8080/raplet.php and you want to pass two additional parameters from a notification email: ticket_id=123 and user_id=456. In that case, you need to include the following link in your email:

<a href="https://rapportive.com/raplets/raplet_params?raplet=http%3A%2F%2Flocalhost%3A8080%2Fraplet.php&amp;ticket_id=123&amp;user_id=456"></a>

The link is constructed as follows:

When Rapportive is loaded on a message with such a link set, and your Raplet is installed, it will send a request with a URL like the following:

http://localhost:8080/raplet.php?email=noreply@myservice.com&ticket_id=123&user_id=456&callback=jsonp123456789

Your Raplet can then respond as usual, and provide additional context because it knows that the user is looking at an email about ticket 123, from user 456.

Please note that anyone could send you an email containing a link of this pattern; you should not trust the Raplet parameters, but validate them just like any other user input.

Sidebar Override from Message Content

Using the same technique as passing Raplet parameters based on message content, you can choose to override the default lookup for the entire Rapportive sidebar.

This is particularly useful if you send notification emails relating to actions taken by a particular person, but the notification email sender is a non-personal email address such as noreply@example.com. Now, noreply@example.com is unlikely to have an interesting Rapportive profile, but the person behind the notification may well have an interesting profile.

To override the email address for which the Rapportive sidebar is shown, include a link like the following in your email:

<a href="https://rapportive.com/raplets/lookup?email=rahul%40rapportive.com"></a>

Alternatively, to show the Rapportive sidebar for the person with a particular Twitter username, include a link like the following:

<a href="https://rapportive.com/raplets/lookup?twitter_username=rahulvohra"></a>

The link is constructed as follows:

Including this link causes the entire Rapportive sidebar (not just your Raplet) to show the profile for rahul@rapportive.com or @rahulvohra instead of noreply@example.com. It only affects the profile that is shown by default when the conversation is loaded in Gmail; when a user hovers over an email address the Rapportive sidebar will update in the usual manner.

Note that this makes the email addresses of your users accessible to any recipient of your notification emails. Only use this technique in cases where the recipient already has access to the email address via some other channel (e.g. they can see the email address in your web app).

Troubleshooting

Debugging Your Raplet

If your Raplet does not work, diagnosing what is wrong is not straightforward. Here are some suggestions:

Third-Party Cookies

Some of your users may have third-party cookies disabled. If your Raplet sets cookies — for example, after authenticating a user — then these users will either have to enable third-party cookies, or they will have to add an exception for your server’s hostname.

Support

We would like to invite you to share your experiences and thoughts on our raplet-dev mailing list. This is also a good place to ask any questions you have or get help if you get stuck.

Please also make sure that you subscribe to the raplet-announce list. This is a low-traffic list on which we post updates that affect all Raplet developers.

Appendix

When you want to release your raplet, you can provide users with a link to install it using the Raplets gallery. To do so, link the user to https://rapportive.com/raplets and include a preset= query parameter set to the URL of your raplet.

For example, the Rapportive Crunchbase raplet is available at https://rapportive.com/raplets/crunchbase, so we could link users to:

https://rapportive.com/raplets?preset=https://rapportive.com/raplets/crunchbase

To test this experience, first log into Rapportive by opening Gmail, and clicking “Log In” in the Rapportive menu at the top of the page, and then visit https://rapportive.com/raplets?preset=https://rapportive.com/raplets/crunchbase.

We recommend warning your users that they will have to be logged in to Rapportive before clicking on the link, as they will otherwise see an error page telling them to log in.

Changelog

This document is maintained in a public repository on GitHub, where you can see a detailed log changes.

Version 0.5 (March 2011)

Version 0.4 (February 2011)

Version 0.3 (May 2010)

Version 0.2 (April 2010) — first public release

Version 0.1 (March 2010)

© Rapportive, All Rights Reserved

Follow @rapportive on Twitter