API documentation

Welcome to the Haystack API. All the data available through this service was collected using Lumen, an Android application that analyzes mobile traffic and helps identify privacy leaks and the organizations collecting this information.

Before starting using our service, you might be interested in reading our privacy policy.
How to get access to the API
To gain access to the API, you first need to register an account. Registrations must be done through this form . We ask you to explain why you want access the data. This is mandatory. Upon validation of your account, you will be able to access the API here . You then have two ways to search through the data: either using the web interface or using the command line version.
Searching using the web interface
You can look for a specific application, or a specific domain name to get the flows information that Lumen has detected. The search must be done using one of the following criteria: The FQDN does not need to be complete: for instance, searching for google will match all domains names that contain google.

Upon running a successful query, you will get the first results page. Each results page contains at most 20 entries. You will also know how many entries were matched by your query and the total number of results pages.

Keep in mind that we limit the number of queries you can do. The current limits are 2,000 queries per day and 5 per minute. Each time you want to get a new page of results, a query will be deducted from your total.
Searching using the command line interface
To be able to use the API using a command line tool, you need to perform a HTTP POST request at the following URL:

https://androidobservatory.com/lumen-api/curl/

You need to pass your access token as a parameter to every POST request. To retrieve your token, login to the web interface. You will then see the API token button, as illustrated below.
Example of API token
The search criteria are the same as for the web API, namely: To look for a specific version of an application, append version=<version> to your request. For instance, to look for contacted package names containing vpn: 

$ curl https://androidobservatory.com/lumen-api/curl/?token=<token>&app_package=vpn

If your query returns some results, you will get a JSON of the following format: 

{
    "paging": {
        "first": "https://androidobservatory.com/lumen-api/curl/?app_package=vpn&token=&page=0",
        "last": "https://androidobservatory.com/lumen-api/curl/?app_package=vpn&token=&page=5",
        "nb_matches": "113",
        "nb_pages": "5",
        "pages": [
            "https://androidobservatory.com/lumen-api/curl/?app_package=vpn&token=&page=0",
            "https://androidobservatory.com/lumen-api/curl/?app_package=vpn&token=&page=1",
            "https://androidobservatory.com/lumen-api/curl/?app_package=vpn&token=&page=2"
        ]
    },
    "applications": [
        {
            "app_name": "VPN Unlimited Proxy AppVPN",
            "app_package": "appvpn.vpn",
            "app_version": "48",
            "dst_ip": "52.94.232.33",
            "dst_port": 80,
            "fqdn": "aax-us-east.amazon-adsystem.com",
            "is_tls": null
        },
        ...
    ]
}

If your query does not match any entry of our database, you will get a JSON of this format instead: 

{
    "Message": "Sorry, no application matches your search."
}

You will get a JSON of the same format in case of error.


Home