How to set up Firestore and Algolia using Cloud Functions.

A while back I attempted to enable full-text search in my app using Algolia and Firestore (at that moment, in beta). There was little documentation on how to achieve this combination and it turned out to be quite cumbersome.

It’s 2019 now, and hopefully things have moved forward. I’m giving Algolia a second chance and it is a great opportunity to update this guide.

Overview

What we want to achieve is the following architecture:

Basic architecture of what we want to set up.

There are basically four components to this scheme. The app (how users interact with whatever we’re building), Firestore (the database where we keep data), Algolia (the “database” optimized for full-text searches) and Cloud Functions (the backend server that will keep Firestore and Algolia always in sync).

So what we have to do is:

1. Bring our existing Firestore documents into Algolia.
2. Build the functions that will keep Algolia always in sync with Firestore.
3. Define Algolia’s search settings.
4. Perform search operations in our app.

Simple enough, in theory. So let’s get to it.

Part 1: Get ready

Step 0: Create your accounts in Firebase and Algolia

Not much to say here. You probably already have a Firebase one, so go ahead and create your account in Algolia, if not already done.

Keep in mind that, in order to use Firebase Cloud Functions to automatically index Firestore collections into Algolia, your Firebase account plan must be a paid one. The free Spark Plan does not allow requests to external APIs.

Step 1: Install Cloud Functions

Firebase Cloud Functions is a great service for building what people call a “serverless” backend (although this name is not technically right). If you are new to it, check their docs, this 2 min introduction video or this nice Google I/O video explaining its possibilities.

If you haven’t yet used Cloud Functions, go ahead and set it up. I won’t cover details in here because the docs are already good, just make sure you are using a Firebase supported LTS version of node (currently Node.js 8).

Go to your project’s root folder and run:

1) nvm install 8.16.0
2) npm install -g firebase-tools
3) firebase login
4) firebase use -add
5) firebase init

1. Installs the right version of Node. If you are reading this after 2019, do check the docs for the current supported version.
2. Installs Firebase CLI so we can build Cloud Functions.
3. Log into our Firebase account so we can access our project.
4. (Optional) In case you are using several Firebase projects for different environments (Development, Staging, Production etc.), create aliases to switch between them.
5. Firebase init will take you through a few questions to help you set up the base files. I’m using Typescript (as it gives us async/await capabilities) with ESLint installed.

Step 2: Install Algolia’s Node.js SDK

Now cd into the functions directory and install algoliasearch, which is Algolia’s Node SDK that we can use to update items in their indices.

cd functions
npm install algoliasearch --save

Step 3: Set up config variables with Algolia keys

In order to use algoliasearch, we will need API keys to securely access Algolia. Firebase Cloud Functions offers a secure way to store this information in its environment configuration:

$ firebase functions:config:set algolia.appid="YOUR_APP_ID" algolia.apikey="YOUR_API_KEY"

Replace YOUR_APP_ID and YOUR_API_KEY with your own keys, which can be found in your Algolia’s dashboard, and run the command.

Please be careful: the API key you will use in this command is the Admin API Key, which has super powers! Do not place this in any repository or in your front-end app or in other unsafe places. Keep it safe! Use it ONLY from your backend: this key is used to create, update and DELETE your indices.

Done and done. Moving forward.

Part 2: Set up your Algolia Indices

Step 1: Create an index in Algolia

Now is the time to figure out which Firestore collections you want to index in Algolia. Each collection will have an Algolia index of its own, so you should create indices only for those collections you will be searching through. Algolia talks about data structuring and it’s a good idea to read it.

Since I work with two different environments (develop and production), I’m creating two indices for my Firestore collection:

MyCollectionName_prod
MyCollectionName_dev

Step 2: Bring existing Firestore documents to the created index

Algolia offers two ways to import existing data to the index: manually uploading a JSON file or using the API. I’m choosing to use the API as the script can be easily reused to index both development and production environments. Also, in case we ever need to re-index Algolia, it will be just a click away.

This will be done only once, as we will also create cloud functions to automatically index all changes in the Firestore collection.

Algolia’s documentation recommends creating a local dotenv environment and a local node function to perform this operation. My previous article explains how to do it for Firestore, if you’re interested.

However, I’m going to use a cloud function to bring my collection data into Algolia. I think it’s cleaner, easier, and I can always enable/disable it whenever I need to re-index Algolia.

Here’s how my cloud function looks like. Please read the comments for explanations.

Needless to say, but the code above is filled with boilerplate text, so please revise carefully and update everything to your needs. Here’s what you need to change:

PRODUCTION-PROJECT-NAME: you can find it your Firebase console.
COLLECTION_prod: the name of your production index.
COLLECTION_dev: the name of your development index.
COLLECTION: the name of your Firestore collection that you're indexing.
relevantProperties: the properties you want to include in the record.

If you want a better understanding on which properties you should include in your record, refer to Algolia’s recommendation on the topic. In a nutshell, each record in your index should contain enough information to be found and to allow a full display of its content in your client app (when the record is retrieved).

With this function ready, it’s time to deploy:

firebase deploy --only functions:sendCollectionToAlgolia

This command will generate a URL (printed in the terminal) that you can use to call the function. Paste the URL in a browser and hit enter. If all went well, you should see the success message “MyCollectionName was indexed to Algolia successfully” and you should also be able to see all your records in your Algolia’s Dashboard.

Step 3: Configure searchable attributes and custom ranking

Now we need to tweak the settings so searches return the most relevant results. We need to decide which attributes to use for searching, and in what order and manner we’ll search them.

We should follow Algolia’s suggestions on how to do that.

This is a really important step, so I recommend going through each of the search settings and reading their guides, so you get a sense of how to best set yours.

Step 4: Listen to changes in Firestore documents

Cool, we’re done with step A from the scheme illustrated in the beginning of the this post. Off we go to step B.

Now that we have all our collection documents indexed in Algolia, we need a way to keep them always up to date.

As you might know, Cloud Functions allows us to create listener functions that are triggered every time a change occurs in a given collection. Here’s how my functions are looking like:

Listener functions to update Algolia. The functions inside each listener are described in the next step.

As you can see, they are very simple and straightforward, and most of the time they will be just that.

However, there might be situations where you only want to index documents that match a certain criteria, say, if the document is complete in terms of information, or if it has been reviewed and accepted. This is the place to check those conditions and only send to Algolia what will indeed be used on searches. Remember they charge for how many objects you have stored (after your quote has been reached), so it may be a good idea to keep a good filter on this.

Moving on!

Step 5: Update Algolia index on document changes

We’re almost done. The next step is to create the functions saveDocumentInAlgolia(), updateDocumentInAlgolia() and deleteDocumentFromAlgolia().

They are also simple:

This is my version, not the simplest one because I do need to filter which documents are indexed to Algolia. If you don’t, simply remove all the conditions.

firebase deploy --only functions

And that’s it! Deploy these functions and make some tests. Create, update and delete some documents from your Firestore collection and they should be almost instantly updated in Algolia.

Our backend is done. What about the front-end?

Part 3: Enable beautiful and fast search on your app

After setting up our Algolia indices and keeping it up to date with Firestore, all there’s left is to make use of it.

Implementation of this part will be very specific to your own UX, the programming language you use and so on, so feel free to skip this part.

Algolia has several production-ready UI libraries that you can use to enable instant search functionality in your app. Go ahead and dig in to see if they are suitable for your needs.

They highly recommend the use of such UI libraries, but if you prefer to build your own, it’s possible as well. If you’re developing for iOS in Swift, this would be the guide to use their UI components.

Step 1: Install Algolia Client SDK

In the following lines I will only summarize the steps I took and add some comments when relevant, so do follow Algolia’s guides for this part.

First, we install InstantSearch. I’m choosing not to use the InstantSearch UI Library, even though Algolia highly recommends using it. It just seems much simpler to just get access to the main REST API calls and handle it from there.

Add pod 'InstantSearchClient', '~> 7.0' to your Podfile and run pod install. I did find a bunch of warnings in their SDK, so I took the liberty of fixing them (I do not like warnings cluttering my code 🤷‍♂️). Most of them were Swift 5 related, so I won’t even mention here as they will probably fix it soon.

Step 2: Config Algolia client

In the view controller (or view model, if you’re using MVVM) where search will be performed, we config Algolia and create the search function:

Please be careful: the API key you will use in the setupAlgoliaSearch() function is NOT the Admin API Key! You should use the Search API Key available in your Algolia Dashboard, as it has rights to search only (and not delete anything). Also, it might be a good idea to retrieve such keys from Firestore, so you don’t ship your app with such information exposed (even though they are not critical).

With that in place, we can move on to actually performing searches.

Final step: Search and handle results (a.k.a Hits) 🎉

The only thing left to do now is search and handle results. There’s no need for me to explain it in details since it’s not really related to Algolia and Firestore. You just have to call myViewModel.searchCollection(forText: String) every time your TextField changes, and update a TableView with the contents of myViewModel.myObjects.

And that’s it!

I hope this guide is helpful to someone out there! I struggled a lot the first time I did it, and I wish I had found some step by step guidance like this one. Please drop a message below if you have any suggestions on how to do it better, or if you found any mistake. Or just say hey! 👋