FCM Queued Messages on Android

On Android, a great deal of app data being sent from the cloud may be arriving at each app through a shared conduit: FCM Queued Messages. As the data travels along this path, it can leave traces behind that can give us a great second opportunity to recover said data. Amongst other apps, we’ve seen this “conduit” being used by Facebook, Tumblr and YouTube to name just a few. In this blog I’ll attempt to set out what this artefact is, why it exists in the first place, and how the data is structured.

Background

You can find the artefact in question on the data partition under: /data/data/com.google.android.gms/files/fcm_queued_messages.ldb.

Before we get into the bits and bytes, we should take a moment to expand some acronyms in the path above. First the “gms” in the package name: “GMS” here is Google Mobile Services (https://www.android.com/intl/en_uk/gms/). From a user’s perspective, this is the bundle of Google apps that is included with the official version of Android used by Google and its partners (which includes most major hardware manufacturers); for developers though, GMS also provides a bunch of APIs (Application Programming Interface) to make developing apps easier in a number of areas.

One of those APIs is “fcm”, which stands for Firebase Cloud Messaging (https://firebase.google.com/docs/cloud-messaging for more details). This isn’t messaging as in chat – although it can certainly be used in the development of chat apps – rather it is messaging between a server and an app (possibly running in the background).

The final acronym is “ldb” which, if you have followed along with my recent posts, you may already have guessed, is short for “LevelDB”. This artefact is indeed a LevelDB (https://github.com/google/leveldb) data store, which is contained within this folder.

The Nuts and Bolts

As we have previously discussed, LevelDB is a key-value store where each of the keys and values can be any arbitrary blob of data. In this case, the keys are an ASCII encoded text string (or possibly UTF-8, or an extended ASCII code page, it’s impossible to tell for sure with the characters used) structured along the lines of:

The decimal number to the right of the colon is, rather usefully, a timestamp – in this case a microseconds Unix epoch timestamp which correlates to the time that the message was raised. The meaning of the hex value to the right of the percent sign is not totally understood at this point, but it does seem to be the same for all notifications relating to a particular app on the device (there is a way to identify which app that actually is in the record’s value).

Before we get to the value, we should address a behaviour which is observed in the database:

In LevelDB, records are arranged as a log of changes to keys. As you can see in the screenshot above, when we lay the records out sequentially, a record with a key arrives into the database and then is immediately deleted – as it gets forwarded to the appropriate app. As the name of the database suggests: this database is a queue of messages waiting to be delivered and in most cases the message will be delivered pretty rapidly – emptying the queue as it does. Not that this matters for forensic recovery: if we have a viewer or software library (https://github.com/cclgroupltd/ccl_chrome_indexeddb) which can view the deleted records, we can go back and pick up the deleted keys from back when they were live.

The value of the record will be a protocol buffer (https://developers.google.com/protocol-buffers): an object serialisation format which originated at Google, so it’s no great surprise to see it here. Protocol buffers are generated by custom serialisation code so they aren’t really meant to be read by other applications. But the format has enough context about the data stored inside, so by using an appropriate viewer we are able to reveal the structure of the data.

The main data of interest lives within the first object we encounter with tag 0x12. In that object we should find, amongst other items, a string entry with tag number 0x2A which gives the name of the package that the message is associated with, and one or more objects with tag number 0x3A which will each contain a string key and value – the message being sent from the server to the client application.

Up until this point, the data structures we encounter will be consistent regardless of the application they relate to, but the contents of the keys and values within 0x3A objects will be specific to the particular application that they relate to. A good number of applications that we’ve encountered make use of JSON (Tumblr and Facebook come to mind) which makes processing the records nice and structured. Some applications use simple, flat text fields, whereas some others use base64 to encode other binary structures (YouTube as an example encodes a further protocol buffer in these fields).  The contents of these values vary in their detail, with some applications providing only very basic data plus a URL to a REST API endpoint, others supplement the data with material to display in notifications, such as message subjects, snippets of text, titles of posts and so on. The best results of course come from those which contain the full data associated with an event, such as chat messages with metadata.

‍

Limitations, Benefits

The really nice thing about this artefact is that it operates entirely independently of the application the data is on its way to. Using the example of a chat message: once delivered on to the app, that message could be deleted by the app or by the server, but the recoverable record in the FCM LevelDB store will be completely unaffected by that operation – as far as LevelDB is concerned that record is deleted anyway; indeed, we’ve had situations where we were able to recover thousands of records related to apps that were no longer installed on the device at all, or months more of earlier interactions than were recoverable from the main app database.

There is a limitation to this artefact however: all of the data in this database will be “incoming” – the queue is only for messages coming from the server to the app. What that means in practice varies from app to app; for chat messages this does mean that often you will only see one side of the conversation, but you may still see incoming chats, association evidence, general evidence of interaction with an app and so on.

FCM Queued Messages can provide you with an excellent “second chance” opportunity to recover data related to any app which makes use of Firebase Cloud Messaging, regardless of whether the data has been deleted by the application, or indeed, if the application is even still installed.

‍