ParallelDispatch¶

class continual.ParallelDispatch(dispatch_mapping)[source]¶

Reorder, copy, and group streams from parallel streams.

Reorder example:

net = co.Sequential(
    co.Broadcast(2),
    co.Parallel(co.Add(1), co.Identity()),
    co.ParallelDispatch([1,0]),  # Reorder stream 0 and 1
    co.Parallel(co.Identity(), co.Add(2)),
    co.Reduce("max"),
)

assert torch.equal(net(torch.tensor([0])), torch.tensor([3]))

Depiction of the reorder example:

       | -> co.Add(1)     \ / -> co.Identity() |
[0] -> |                   X                   | -> max -> [3]
       | -> co.Identity() /  \ -> co.Add(2)    |

Copy example:

net = co.Sequential(
    co.Broadcast(2),
    co.Parallel(co.Add(1), co.Identity()),
    co.ParallelDispatch([0, 0, 1]),  # Copy stream 0
    co.Parallel(co.Identity(), co.Add(2), co.Identity()),
    co.Reduce("max"),
)

assert torch.equal(net(torch.tensor([0])), torch.tensor([3]))

Depiction of the copy example:

       | -> co.Add(1)  -> | -> co.Identity() -> |
[0] -> |                  | -> co.Add(2)     -> | -> max -> [3]
       | -> co.Identity() ------> co.Add(1)  -> |

Group example:

net = co.Sequential(
    co.Broadcast(2),
    co.Parallel(co.Add(2), co.Identity()),
    co.ParallelDispatch([[0, 0], 1]),  # Copy and group stream 0
    co.Parallel(co.Reduce("sum"), co.Identity()),
    co.Reduce("max"),
)

assert torch.equal(net(torch.tensor([0])), torch.tensor([4]))

Depiction of the group example:

                         | -> |
       | -> co.Add(2) -> |    | ->    sum     -> |
[0] -> |                 | -> |                  | -> max -> [4]
       | -> co.Identity() ----> co.Identity() -> |

Parameters:

dispatch_mapping (Sequence[Union[int, Sequence[int]]]) –

input-to-output mapping, where the integers signify the input stream ordering and the positions denote corresponding output ordering. Examples:

[1,0] to shuffle order of streams.
[0,1,1] to copy stream 1 onto a new stream.
[[0,1],2] to group stream 0 and 1 while keeping stream 2 separate.