One of the most time-consuming operations on a CI server is running autotests. There are many ways to speed them up, for example, parallelizing execution across several CI agents and / or emulators, full emulation of the external environment (backend / Google services / web sockets), fine-tuning emulators (Disabling animation / Headless builds / disabling snapshots) and so on . Today we will talk about impact analysis or running only those tests that are related to the latest changes in the code. I will tell you what steps are needed for impact analysis and how we implemented it in our project.

Step one: get the diff of the changes.

The easiest way to achieve this is with the built-in Git tools . We wrapped the impact analysis work in a Gradle plugin and use a Java wrapper over Git - JGit . For the merge request, we use premerge builds (this is when the target branch is merged first, it is used to quickly detect conflicts), so it’s enough to get the diff of the last commit:

    val objectReader = git.repository.newObjectReader()    val oldTreeIterator = CanonicalTreeParser()    val oldTree = git.repository.resolve("HEAD^^{tree}")    oldTreeIterator.reset(objectReader, oldTree)    val newTreeIterator = CanonicalTreeParser()    val newTree = git.repository.resolve("HEAD^{tree}")    newTreeIterator.reset(objectReader, newTree)
    val formatter = DiffFormatter(DisabledOutputStream.INSTANCE)    formatter.setRepository(git.repository)    val diffEntries = formatter.scan(oldTree, newTree)    val files = HashSet<File>()    diffEntries.forEach { diff ->        files.add(git.repository.directory.parentFile.resolve(diff.oldPath))        files.add(git.repository.directory.parentFile.resolve(diff.newPath))    }    return files

But nothing prevents you from collecting all the commits between two branches:

    val oldTree = treeParser(git.repository, previousBranchRef)    val newTree = treeParser(git.repository, branchRef)    val diffEntries = git.diff().setOldTree(oldTree).setNewTree(newTree).call()    val files = HashSet<File>()    diffEntries.forEach { diff ->        files.add(git.repository.directory.parentFile.resolve(diff.oldPath))        files.add(git.repository.directory.parentFile.resolve(diff.newPath))    }    return files

private fun treeParser(repository: Repository, ref: String): AbstractTreeIterator {    val head = repository.exactRef(ref)    RevWalk(repository).use { walk ->        val commit = walk.parseCommit(head.objectId)        val tree = walk.parseTree(commit.tree.id)        val treeParser = CanonicalTreeParser()        repository.newObjectReader().use { reader ->            treeParser.reset(reader, tree.id)        }        walk.dispose()        return treeParser    }
}

Step two: build the source code dependency tree.

The detailing of the tree depends on the amount of code and autotests. The more detailed, the higher the accuracy of isolating only the necessary tests, but the tree assembly is slower. Now we are building a dependency tree at the module level, and looking at the level of individual classes.

List of modules in the project:

private fun findModules(projectRootDirectory: File): List<Module> {    val modules = ArrayList<Module>()    projectRootDirectory.traverse { file ->        if (file.list()?.contains("build.gradle") == true) {            val name = file.path                .removePrefix(projectRootDirectory.absolutePath)                .replace("/", ":")            val pathToBuildGradle = "${file.path}/build.gradle"            val manifestFile = File("${file.path}/$ANDROID_MANIFEST_PATH")            if (manifestFile.exists()) {                if (modulePackage != null) {                    modules.add(Module(name))                }            }        }    }
    return modules
}

We connect the nodes by parsing the build.gradle file. Also, the dependency tree can be generated not automatically, but assembled once by hand and reused. Advantage - detailing at any level without affecting the time of work, disadvantage - someone will have to manually maintain the graph as the project develops.

Step three: select all the affected nodes of the dependency tree.

We take the changes from the first step, match them with the nodes from the second, and with a simple breadth-first traversal find all affected nodes.

private fun findAllDependentModules(origin: Module, links: Set<Link>): Set<Module> {    val queue = LinkedList<Module>()    val visited = HashSet<Module>()    queue.add(origin)    val result = HashSet<Module>()    while (queue.isNotEmpty()) {        val module = queue.poll()        if (visited.contains(module)) {            continue        }        visited.add(module)        result.add(module)        queue.addAll(links.filter { it.to == module }.map { it.from })    }    return result
}

Step four: we collect a list of tests associated with the affected nodes of the dependency tree.

At this stage, we need to somehow connect autotests with the nodes of the dependency tree from the second step. There are many ways to do this (for example, communication through custom annotations), but for a reliable and always up-to-date state, it is better to parse the source code of the autotests themselves. We use the Kaspresso framework , and to link tests with the dependency tree, we parse the tests with the Kotlin compiler itself. We collect the dependency tree of the form test cases -> scripts -> page descriptions ( Page Object ) -> dependency nodes from the second step, then by the reverse pass we get a list of all the necessary tests.

implementation("org.jetbrains.kotlin:kotlin-compiler-embeddable:1.5.10")

private fun readUiTestsMetaData(modules: List<Module>): List<UiTestMetaData> {
    val testRootDirectory = rootDirectory.get().resolve(TEST_ROOT_PATH)
    val ktFiles = kotlinFiles(testRootDirectory)
    val pageObjects = ktFiles.mapNotNull { parsePageObjectMetaData(it, modules) }
        .sortedBy { it.name }
    val scenarioObjects = ktFiles.map { parseScenarioObjects(it, pageObjects) }.flatten()
    val scenarios = buildScenarioMetaData(scenarioObjects, pageObjects)
    return ktFiles.map { parseUiTestMetaData(it, scenarios, pageObjects) }
        .flatten()
        .sortedBy { it.name }
}

Step five: run the necessary tests.

Android's stock test runner allows you to filter tests by name, package, or associated annotations. We use Marathon to run autotests , which has a wider filtering functionality . In Teamcity, at the impact analysis stage, our Gradle plugin collects all autotests from the fourth step, rips out the test ID from them and writes to a file. After that, when preparing Marathon, we feed it all these identifiers and get only the necessary tests to run from all existing ones.

Now a full run of all tests takes about 30 minutes, and the impact analysis saves us 10 minutes. With the further development of the project and the addition of new modules / autotests, the time saved will only increase. I hope the article was useful to you, and stay tuned folks :)