Countries, cities, professional areas, languages, currencies - all these are the names of directories inside our mobile application. They change very rarely, but are used everywhere, and therefore must be up-to-date and should not waste user and developer time updating them. In this article, we will figure out how to make working with directories simple and convenient.

How it all began

Reference books have been living in our application for a very long time. Until recently, they were plain text files in which we saved responses from the server in JSON format. There were about 20 megabytes of such data, and for a mobile application this is really a lot. Loading this data in the application was very slow, and the wrapper was written in Objective-C. In short, fear and terror are legacy. 

A year ago, we started working on a new application for employers. One of the blockers for us was the creation of a new feature for working with directories. In its current form, we did not want to drag it into a fresh and not desecrated application at all, so we decided to change the directories. 

I walked around the developers, talked with them about the wishes for the new feature and made a list of requirements:

• data should be available as quickly as possible, right at the start of the application;

• it is necessary to provide updates of directories from the server;

• Note that the structure of the information may change. For example, new directories can be added, old links can be changed, which means we need to provide for the possibility of migrations;

• provide a quick search for information in these directories;

• all this should work in a separate feature that will be rummaged between our applications.

Looking at these requirements, I realized that the question “how to store reference information” simply does not exist - we need a database. It will provide us with the storage of directory information, quick search, the possibility of migrations and updates.

In total, we close most of the requirements almost “out of the box”. It remained to understand what kind of base we want to use and how we will work with it. 

As a result, we got several tasks:

• first, we need to learn how to supply data with the application;

• secondly, it is necessary to make working with data inside our application convenient;

• thirdly , we must make sure that the data is updated only when necessary. 

In an ideal world, the developer does not need to update the data in the bundle by hand. In an ideal world, all the necessary updating happens either on a schedule on the CI side, or during the creation of a release candidate for regression. Some scripts are launched and a fresh version of the database is added to the repository. The validity of this base is provided by integration tests. In general, we needed to automate the delivery of data along with the application as much as possible and combine the generation of dictionaries between iOS and Android. 

Databases and their dear comrades

Taking the basic requirements as a basis, I have collected the main possible options for working with a database in iOS. There are three options: CoreData, Realm, SQLite. Additionally, I looked at less popular options like YAP, but quickly abandoned them. Now I will briefly talk about the pros and cons of each of the main options.

coredata. The advantages are obvious: the most native, adding a framework does not affect the size of the application, most of our developers have already worked with it. Plus, it's pretty easy to write migrations in it. The cons are also clear. The database can only be generated on mac or ios. You can't just take and generate a sqlite file and replace it. Be sure to use the version of the models from your project. CoreData is only suitable for iOS - no cross-platform. And it also has a rather cumbersome API. Yes, Apple has improved it, but still, in terms of convenience, CoreData loses a lot to other solutions.

realm. Of the benefits: cross-platform, fast data search, simple and understandable code in the project. There are also disadvantages. For generation, you need a project with a connected Realm - a script is indispensable here. This is a third party library and an additional dependency that also increases the size of the application from 5 to 14 mb. In addition, our Android developers did not want to drag Realm to them at all. Accordingly, we lose a plus from the cross-platform.

SQLite. The main plus is that the database is easy to generate. Just write scripts that create tables and insert data. This is the ultimate cross platform: Android's database engine also uses SQLite. Cons and other advantages depend on which library you choose to work with the database. I considered three options. 

One of them is native , it has a big plus in the form of no dependencies. But there is also a fat minus - the API is generally inconvenient. Apple suggests using the OpaquePointer to work with the base.

GRDB.swift. Pros: simple work with data, fast work speed, excellent support by the author and the community. Cons: it does not have Carthage support, and this was the principled position of the author of the library. For us, this has become a stop factor, because we are actively working with Carthage and all our main libraries are connected through it. 

And finally, SQLite.swift. Pros: simple and convenient, there is community support. We already had this library in our project as a dependency on another library. Now for the cons. At the time of research, the last commit was in the repository about two years ago. But before recording the video version of this article, I went to the library page, and it was updated quite recently. It can also be noted that a separate manager is used for migrations, but this is a common problem with working with sqlite.

For each of the five database options, I created a small test project that demonstrated the possibilities of interaction: how to work with models, how the database itself can be generated and updated. And then the team and I held a small vote to select the absolute favorite. It is worth noting that we were solving a specific task of working with directories, and did not choose a single database for our application.

We evaluated the work with the database according to 4 indicators: the convenience of generation, the convenience of working in code, data migration and quick search in the database. But there were only two really important criteria - the convenience of generation and the convenience of working in code, taking into account the goals that we set. As a result, the first place in terms of generation was deservedly taken by SQLite, the second by CoreData, and the third by Realm. In our conditions, it was very difficult to generate a base on Realm.

But in terms of ease of use, Realm was in first place, of course, taking into account the work scenarios that we developed based on the results of our vote. In second place was SQLite, but not native, but when using any library. And on the third - CoreData. For the rest of the points, any option gives a good search speed, and we generally refused to migrate. But more on that later.

As a result, we chose the easiest option for generation - SQLite and the SQLite.swift library, which was already in the project. The generation of the SQlite database itself is quite simple. We wrote a Python script that downloads data from the server and puts it into tables, with each separate directory entering it into its own table. Or like, for example, in the subway, in two tables: separate stations, separate lines. Our final database weighs about seven megabytes, but before adding it to the bundle, I put the whole thing in a zip archive and compressed it to two and a half.

References, tests, data

I won't get too deep into the manuals, because there's nothing interesting here. In general terms, each table had its own provider. And it had two types of get methods: to work synchronously and asynchronously through Combine.

Synchronous methods were needed to simplify work with legacy code, since everything worked synchronously there. And to make it possible to switch to directories gradually, I added an additional legacy wrapper for them. A synchronous call is sometimes much easier to make, and for simple requests it works out very quickly.

And, perhaps, the most interesting point is the tests. Tests were written for the entire database and for each table, which make sure that the model in the code matches the database tables that are in the application bundle. This is how we check that our Python script has generated exactly what we need and that the code is ready to go.

And now we have to solve only one task - updating the data during the operation of the application. In the course of discussions and selection of a database, in order to simplify life, we decided to abandon the possibility of migrating one version to another. 

Now everything works like this:

• Firstly, if a user downloads a new version of our application, the base from the bundle will replace the one that he has now at launch. This simple mechanism allows us to completely forget about migrations and not be afraid that the data models in the code and the table will not match.

• Secondly, we decided to add a special table with metadata to the database. In it, we store the date of the last update for each directory, as well as a special ETAG value. Seven days after the date of the last update, the application will try to download new data taking into account ETAG and change the date in the table to a new one. Using ETAG means that if the data has not changed on the server, then nothing will change in the database, and there will be no extra traffic or load.

Results 

By switching to SQLite for references, we got rid of JSON files and reduced the size of our application by 15 MB. Updating the version of directories in a bundle has become much easier, and working with them in code is much more comfortable. And my main message for today: “Do not be afraid to use the SQLite database in your projects. In some scenarios, it can give a tangible head start to other solutions.”