A practical guide to asynchronous programming in Python

Basics of asynchronous programming

Python’s asyncio module provides the infrastructure for writing concurrent code, using the async/await syntax to make asynchronous code as readable as synchronous code.

Core concepts

• Event Loop: event loop, execution engine of asynchronous programs
• Coroutine: Coroutine, using functions defined by async def
• Task: Task, an encapsulation of coroutines, used for concurrent scheduling
• Future: future results, indicating unfinished operations

Basic usage

import asyncio

async def fetch_data(url: str) -> str:
    """模拟异步 HTTP 请求"""
    await asyncio.sleep(1)
    return f"Data from {url}"

async def main():
    # 串行执行：耗时 3 秒
    result1 = await fetch_data("https://api1.com")
    result2 = await fetch_data("https://api2.com")
    result3 = await fetch_data("https://api3.com")

    # 并发执行：耗时 1 秒
    results = await asyncio.gather(
        fetch_data("https://api1.com"),
        fetch_data("https://api2.com"),
        fetch_data("https://api3.com"),
    )

asyncio.run(main())

Concurrency control: Semaphore

When you need to limit the number of concurrencies, asyncio.Semaphore is the best choice:

async def crawl_with_limit(urls: list[str], limit: int = 10):
    semaphore = asyncio.Semaphore(limit)

    async def fetch(url: str):
        async with semaphore:
            return await fetch_data(url)

    tasks = [fetch(url) for url in urls]
    return await asyncio.gather(*tasks)

Practical application

Asynchronous programming is particularly useful in the following scenarios:

1. Web Crawler: Large number of concurrent HTTP requests
2. API call: Aggregating multiple external services
3. File IO: A large number of file read and write operations
4. Database Query: Execute multiple queries concurrently

Tip: Not all scenarios are suitable for asynchronous, CPU-intensive tasks should use multi-process.

Summarize

Mastering asynchronous programming in Python can significantly improve the performance of IO-intensive applications. The key is to understand how the event loop works, use Semaphore appropriately to control concurrency, and choose the appropriate concurrency mode.