Task

[bjornbytes]

This is an experiment to add a coroutine-based task system to LÖVR, with the main goal of making it easier for projects to scale to multiple CPU cores and run expensive work in the background.

Design

The basic idea is to create lots of Task objects, which are thin wrappers around Lua coroutines. Each Task represents an independent function doing work, and lots of tasks can all run more or less at the same time, cooperatively.

Normally, Lua coroutines are still single threaded, and don't allow for parallelism. However, Tasks in LÖVR can achieve parallelism by running asynchronous functions.

When a Task calls an asynchronous function, it will start work on a thread pool, yield itself, and wake up automatically once the work is complete. While the Task is yielded, other Tasks get a chance to run.

In this way, many Tasks (coroutines) can be running "at the same time", with all of them doing work on a thread pool, making use of all of the CPU cores.

API

Everything is in a new module, lovr.task:

lovr.task.newTask(function) --> create task
lovr.task.poll() --> iterate over ready tasks (for task in lovr.task.poll() do ... end)
lovr.task.wait(...tasks) --> wait for tasks to finish.  returns results/error

Task:resume(...args) --> run the task.  returns success, ...results
Task:isReady() --> whether the task is ready (can be resumed), or if it's still waiting for something
Task:isComplete()
Task:getResults()
Task:getError()

-- New callback: called when a task is ready
-- Default behavior is to just resume the task and propagate errors,
-- but you can override it to implement custom scheduling/error behavior
function lovr.taskready(task)
  assert(task:resume())
end

Notes:

• Task = lovr.task.newTask(function) creates a new Task. You give it a function you want the task to run. Internally, it gets turned into a coroutine, similar to coroutine.create.
  • Note that this is a regular function! It shares the global scope with the caller, as well as any other tasks. You don't have to use Channel to communicate with the task.
• for task in lovr.task.poll() do end. Iterates over any tasks that are ready to run. The default lovr.run calls this for you, and just passes the result to a new lovr.taskready callback.
• success, ...results = lovr.task.wait(...tasks). Blocks until all the tasks are complete, returning their results or the first error that was encountered.
  • Handling multiple return values: Results are collapsed similar to Lua's existing convention: All tasks return their first return value, except for the last task, which returns all of its results. So if you have 3 tasks that return 1, 2, 3, lovr.task.wait(a, b, c) returns 1, 1, 1, 2, 3.
  • Note that this is an asynchronous function! This means:
    • If you call it from within a task, it will yield, and become ready once the tasks are complete.
    • If you call from outside a task, this function will block, running tasks and jobs until all the tasks finish.
  • Note that if you wait on a task, it will not show up in lovr.taskready anymore, because it's already finished. You can choose to "fire and forget" a task and let lovr.taskready take care of running it on a future frame, or explicitly wait on the task if you need its results sooner.
• success, ...results = task:resume(...args). Resumes a task, if possible, otherwise returns false and an error.
  • You'll get an error if the task is already finished, has an error, or isn't ready to run yet.
  • If this is the first time running the task, or if the task wasn't waiting on anything and just yielded with a regular call to coroutine.yield, then args will get passed to the task. If the task yielded due to an asynchronous function, args are ignored and the results from the async operation are given to the task instead.
  • If the task ran successfully and yielded/finished, success will be true and you'll get any yield/return values.
  • Note that if you resume a task manually, it won't show up in lovr.taskready, since you already ran it.
• ready = task:isReady() returns true if the task can be resumed, or false if it's finished or still waiting on the result of an async function.
• complete = task:isComplete() returns true if the task is finished (its function returned or errored)
• ...results = task:getResults() returns the values the task returned with, or nil if the task hasn't returned yet.
• error = task:getError() returns the task's error

Asynchronous Functions

Asynchronous functions are where the magic happens. Asynchronous functions behave differently, depending on whether they are called inside of a Task or not:

• When run outside a task, the asynchronous function behaves in a blocking manner, exactly how it would today.
• When called from inside a task, asynchronous functions will yield the task immediately, running the work in the background. When the work finishes, the task will become ready, and it will show up in lovr.taskready on the next frame (unless it gets run sooner than that).

Any LÖVR function can become asynchronous. Let's take lovr.data.newImage, which I've been using as a test async function on this branch:

image = lovr.data.newImage('file.png')

Even though this is an async function, it behaves exactly the same as it does today, and you don't need to change any code. But if you run it on a task, it yields the task:

task = lovr.task.newTask(function()
  return lovr.data.newImage('file.png')
end)

-- runs the task.  it calls newImage and yields
task:resume()

print(task:isReady()) --> false, still loading image

-- wait until image is loaded
local image = lovr.task.wait(task)

By itself this isn't interesting. But there are interesting things you can do with asynchronicity and parallelism.

Here's an example that loads an image in the background:

function lovr.load()
  local task = lovr.task.newTask(function()
    local image = lovr.data.newImage('file.png')
    texture = lovr.graphics.newTexture(image)
    loaded = true
  end)

  -- Note: task will automatically resume again once it becomes ready, via lovr.taskready
  task:resume()
end

function lovr.draw(pass)
  if loaded then
    pass:draw(texture, 0, 2, -3)
  else
    pass:text('Loading...', 0, 2, -3)
  end
end

Here's an example that loads 100 images on multiple threads:

local tasks = {}
for i = 1, 100 do
  local task = lovr.task.newTask(function(file)
    return lovr.data.newImage(file)
  end)
  task:resume('file' .. i .. '.png')
  table.insert(tasks, task)
end
local images = lovr.task.wait(tasks)

Discussion

• Why not use regular coroutines?
  • I spent a lot of time going back and forth between regular coroutines vs. wrapping them in a Task object. Regular coroutines are nice because they compose with other Lua code, and they are a little more lightweight. However Task has some advantages: the API ends up more clean and lovely, some things are more convenient/possible (being able to retrieve the error/results whenever you want), and it simplified the implementation.
• Why isn't taskready just a regular event?
  • Had to remove tasks from queue if they are resumed/waited before being consumed. Felt weird to remove events like this.
• Why doesn't the task start automatically? Why do I have to resume it?
  • You might want to create the task and start it later (e.g. a task you resume once per frame).
• Which functions should/could be asynchronous?
  • [Will fill in later]
• The job system is good for compute tasks, but not "IO" tasks. May need separate thread pool or async IO solution.
• Could this completely replace the Readback object? What if Texture:getPixels was async?
• Will Thread continue to exist?
  • Yes, but it will be much more niche.

TODO

• Make .wait work with tables too
• Add more async functions
• Consider adding a way to sleep a task, like lovr.task.sleep (async, wakes up task later, could cause problems if you want the wait time to be based on dt?) Maybe just lovr.timer.sleep is async
• Add an async function that runs Lua code on the thread pool
• Add a parallel-for helper (potentially both task-based and thread-based)
• Profiling/testing
• Add a way of getting the underlying coroutine for a task? Could be useful for using the debug library with the task to inspect its state.

[shakesoda]

for my use cases i tend to want a parallel for (often 2d/3d, too), fwiw

[bjornbytes]

There will be a (async) function to run Lua code on a worker thread, and likely a parallel-for helper on top of this. I haven't figured out the specific API for the parallel-for function yet though.