AsyncIO

Stardag natively supports async implementation of Tasks and Targets.

A task can implement either - or both - of the methods def run(self) and async def run_aio(self). Which method is used depends on what's implemented and the build logic used, which is discussed in the next section Build & Execution.

Stardag's built-in FileSystemTargets (for, for example, local disk, S3 and Modal volumes) already implement complete async interfaces. Even if your task only implements the sync def run(self) method, the async implementation of targets can provide significant (order(s) of magnitude) speedup in traversing DAGs with a large number of dependencies.

---

🚧 Work in progress 🚧

This documentation is still taking shape. It should soon provide more details and examples of how to fully leverage asyncio for your DAGs.