Persistent Tasks

Motivation

Julia 1.10 and higher wait for all running Tasks to finish before writing out the precompiled (cached) version of the package. One consequence is that a package that launches Tasks in its __init__ function may precompile successfully, but block precompilation of any packages that depend on it.

Example

Let's create a dummy package, PkgA, that launches a persistent Task:

module PkgA
const t = Ref{Any}()   # to prevent the Timer from being garbage-collected
__init__() = t[] = Timer(0.1; interval=1)   # create a persistent `Timer` `Task`
end

PkgA will precompile successfully, because PkgA.__init__() does not run when PkgA is precompiled. However,

module PkgB
using PkgA
end

fails to precompile: using PkgA runs PkgA.__init__(), which leaves the Timer Task running, and that causes precompilation of PkgB to hang.

How the test works

This test works by launching a Julia process that tries to precompile a dummy package similar to PkgB above, modified to signal back to Aqua when PkgA has finished loading. The test fails if the gap between loading PkgA and finishing precompilation exceeds time tmax.

How to fix failing packages

Often, the easiest fix is to modify the __init__ function to check whether the Julia process is precompiling some other package; if so, don't launch the persistent Tasks.

function __init__()
    # Other setup code here
    if ccall(:jl_generating_output, Cint, ()) == 0   # if we're not precompiling...
        # launch persistent tasks here
    end
end

In more complex cases, you may need to set up independently-callable functions to launch the tasks and set conditions that allow them to cleanly exit.

Test functions

Aqua.test_persistent_tasks — Function

Aqua.test_persistent_tasks(package)

Test whether loading package creates persistent Tasks which may block precompilation of dependent packages. See also Aqua.find_persistent_tasks_deps.

On Julia version 1.9 and before, this test always succeeds.

Arguments

package: a top-level Module or Base.PkgId.

Keyword Arguments

broken::Bool = false: If true, it uses @test_broken instead of @test.
tmax::Real = 5: the maximum time (in seconds) to wait after loading the package before forcibly shutting down the precompilation process (triggering a test failure).

source

Aqua.find_persistent_tasks_deps — Function

Aqua.find_persistent_tasks_deps(package; broken = Dict{String,Bool}(), kwargs...)

Test all the dependencies of package with Aqua.test_persistent_tasks. On Julia 1.10 and higher, it returns a list of all dependencies failing the test. These are likely the ones blocking precompilation of your package.

Any additional kwargs (e.g., tmax) are passed to Aqua.test_persistent_tasks.

source