def multistream_max_mem_parallel_ops() -> int:
    """
    Maximum number of parallel operators used in memory planning
    for simple multi-stream mode.
    Larger value imply higher level of possible parallelism, but
    higher memory allocations.

    This option is independent from AIT_MULTISTREAM_EXTRA_STREAMS.

    For example, say, there are 100 ops that can be run in parallel.

    Example 1: AIT_MULTISTREAM_EXTRA_STREAMS=4 and AIT_MULTISTREAM_MAX_MEM_PARALLEL_OPS=100.
    In this case 5 streams will be used (1 base and 4 extra),
    every stream gets 20 operators and no inter-stream barriers are used.
    Memory planning is done for 100 parallel ops.

    Example 2: AIT_MULTISTREAM_EXTRA_STREAMS=4 and AIT_MULTISTREAM_MAX_MEM_PARALLEL_OPS=5.
    In this case 5 streams will be used (1 base and 4 extra),
    there will be 20 waves separated by inter-stream barriers,
    every stream gets 1 operator for every wave.
    Memory planning is done for 20 waves of 5 parallel ops each.
    """
    # unlimited by default
    return int(os.getenv("AIT_MULTISTREAM_MAX_MEM_PARALLEL_OPS", "99999999"))
Maximum number of parallel operators used in memory planning
for simple multi-stream mode.
Larger value imply higher level of possible parallelism, but
higher memory allocations.

This option is independent from AIT_MULTISTREAM_EXTRA_STREAMS.

For example, say, there are 100 ops that can be run in parallel.

Example 1: AIT_MULTISTREAM_EXTRA_STREAMS=4 and AIT_MULTISTREAM_MAX_MEM_PARALLEL_OPS=100.
In this case 5 streams will be used (1 base and 4 extra),
every stream gets 20 operators and no inter-stream barriers are used.
Memory planning is done for 100 parallel ops.

Example 2: AIT_MULTISTREAM_EXTRA_STREAMS=4 and AIT_MULTISTREAM_MAX_MEM_PARALLEL_OPS=5.
In this case 5 streams will be used (1 base and 4 extra),
there will be 20 waves separated by inter-stream barriers,
every stream gets 1 operator for every wave.
Memory planning is done for 20 waves of 5 parallel ops each.
python/aitemplate/utils/environ.py