If your app stores user data in a database, there are a number of occasions where you may need to make changes to multiple objects at once. The most likely scenario is you are loading the initial database state for a new user as part of the onboarding process, but it may also come at times where a user modifies a bunch of records at once, or an app upgrade requires the app to modify those values somehow.
If you’re using CloudKit as your database, you follow the common advice: per Apple’s recommendation, and for the sake of performance, you want to make as few of these CloudKit operations as possible, so you create or modify records in one batch, using a CKModifyRecordsOperation.
But then if you want to sort these records in order of when they were created, or last modified, using a creationDate or modificationDate NSSortDescriptor, those records are out of order. You inserted them 1-2-3-4-5, but you get 2-4-3-5-1.
Even worse, every time you ask for those records, the order is different. So the next time, it’s 3-1-2-5-4.
I added the records in the order I wanted them; why is CloudKit messing up the order?
There are two things happening that cause this behavior: one of them is specific to CloudKit’s batch creation and modification, and one of them is common behavior across many database systems, including CloudKit. We’ll dig into both issues, and then talk about a couple of ways you can make your records sort by date cleanly.
Why is this happening?
The first problem related to CloudKit is that when you create multiple records in CloudKit using a single CKModifyRecordsOperation, the records are assigned the same creation date. This also applies if you are sorting by the last modified date, and you modify multiple records at the same time.
But if I were to write those out to a file, they would still be in the order my app put them in. Surely the database would do something similar to that, right?
Well, here is where we run into the second problem, which is common across many databases of many varieties, not just CloudKit. Database storage is much more complicated than simple files. When you write those records into a database, the database is highly optimized to store those records in ways that allow them to be retrieved as quickly as possible. In order to optimize that process as much as possible, once it’s in a database, no record has any relation to any other record, other than what is present in its fields.
When you create those 5 records in one batch, once they’re in the database, there is no batch anymore. All the database stores is 5 records with the same creation date, with no relation to each other anymore.
When you perform a fetch and sort by creation date, the database will optimize its retrieval by whatever it can get the most quickly first. Because CloudKit is an enormous database system, shared across tens of thousands of servers, apps, disks, etc., the order in which your app would receive those records is essentially random, even across different fetch requests.
What can you do about this?
I dropped a little hint in the bolded statement in the previous section. To review, I said that no record has any relation to any other record, other than what is present in its fields. So to solve this, you need to make sure that if you want strict ordering for your records, you must make sure the ordering is fully specified by the fields of your records. Let’s see a couple of ways you can do that.
The simplest implementation would be to create and modify records one-by-one, which would guarantee the creation and modification timestamps are unique. If you are creating/modifying records in small batches (less than 5 at a time), the small performance hit of writing the records one-by-one may be worth the simple implementation.
If you want or need to preserve batch operations, then you need to make a small addition to your CloudKit schema in order to keep the ordering of the records within batches. Let’s call this batchOrderId, and the type is a simple Int.
When you create or modify records in a batch, before you perform the CKModifyRecordsOperation, set the batchOrderId field to match the order of the records: set the batchOrderId for the first record to 0, the second to 1, and so on. The previous value does not matter; the only point of this field is to retain the order of the records in this batch.
Then when you need to sort your fetched records, you add another NSSortDescription using the batchOrderId:
query.sortDescriptors = [NSSortDescriptor(key: "modificationDate", ascending: true), NSSortDescriptor(key: “batchOrderId”, ascending: true)]
This will sort the records first by modification date, and then the records that have the same modification date will be sorted using the batchOrderId. Because we ensured that the value for that field was in the same order as the records in the batch, those records will be returned to the app properly sorted.
If you need to reverse the sort, simply switch the ascending value on both NSSortDescriptors to false.
What if I need to sort by both?
Most apps will probably only need to sort by one of either the creation date or modification date, and usually the latter. If that’s all you need, you’re done.
If you need to sort by both, you may still have a little additional work. The first solution above works without modification, but if you use the second one, though, you will need to add and keep track of a createBatchOrderId and modifyBatchOrderId in your schema for that record type.
You would only set the modifyBatchOrderId when you’re modifying a record, and you would set both when creating a record, because technically, a record creation is both a creation and a modification. If you’re creating and modifying records in the same CKModifyRecordsOperation, take care that you’re setting the createBatchOrderId on new records only. A simple way to do that would be by checking whether its creationDate field for nil.
When you’re working with CloudKit, or maybe even another database technology you use for your apps, remember these techniques to fix any thorny data ordering problem you run into.
Comments are closed.