qgym.envs.scheduling.scheduling module

This module contains an environment for training an RL agent on the quantum operation scheduling problem of OpenQL. The quantum operations scheduling problem is aimed at finding the shortest possible schedules of operations defined by a quantum circuit, whilst taking hardware constraints and commutation rules into account.

Circuits:

The quantum operations scheduling problem consist of scheduling the operations described by a quantum circuit. For the Scheduling environment in this module, a quantum circuit is defined as a list of gates, where a gate is a namedtuple with a ‘name’, ‘q1’ and ‘q2’. For example, a c-not with control qubit 2 and target qubit 5 would be Gate("cnot", 2, 5) and an X gate acting on qubit 4 would be Gate("x", 4, 4). In this representation, the circuit below can be created by:

>>> from qgym.custom_types import Gate
>>> circuit = [Gate("x", 1, 1), Gate("cnot", 0, 1), Gate("x", 0, 0), Gate("h",1,1)]

          QUANTUM CIRCUIT
       ┌───┐   ┌───┐   ┌───┐
|q1>───┤ Y ├───┤ X ├───┤ H ├──
       └───┘   └─┬─┘   └───┘
                 │     ┌───┐
|q0>─────────────┴─────┤ X ├──
                       └───┘

Hardware specifications:

Different operations defined by a quantum circuit have different operation times. These operation times are defined in the unit of machine cycles. For example, an X gate could take 2 machine cycles, whereas a measurement could take 15 cycles. These different operation times must be taken into account, because the hardware (the quantum computers) cannot perform multiple operations on the same qubit at the same time.

Finding an optimal schedule with these restriction is already a difficult task. However, to complicate matters there can be more limitations defined by the hardware. To be more concrete, the Scheduling environment can take two more types of limitations into account:

Some gates must start at the same time, or must wait till the previous one is finished. This is typically the case for measurements.
Some gates can’t be executed in the same cycle. This for example happens often with X, Y and Z gates.

The goal of the quantum operations scheduling problem is to find an optimal schedule, which takes these hardware limitations into account, as well as some optional commutation rules.

State space:

The state space is described by a dictionary with the following structure:

machine_properties: MachineProperties object containing machine properties and limitations.
utils: SchedulingUtils dataclass with a circuit generator, commutation rulebook and a gate encoder.
gates: Dictionary with gate names as keys and GateInfo dataclasses as values.
steps_done: Number of steps done since the last reset.
cycle: Current ‘machine’ cycle.
busy: Used internally for the hardware limitations.
circuit_info: CircuitInfo dataclass containing the encoded circuit and attributes used to update the state.

Observation space:

Each element in the observation space is a dictionary with 4 entries:

gate_names: Gate names of the encoded circuit.
acts_on: q1 and q2 of each gate.
dependencies: Shows the first \(n\) gates that must be scheduled before this gate.
legal_actions: List of legal actions. If the value at index \(i\) determines if gate number \(i\) can be scheduled or not.

Action Space:

Performing a quantum operation takes a certain amount of time, which is measured in (machine) cycles. Therefore, this environment aims to produce a schedule in terms of cycles. To do this, the environment schedules the circuit from right to left, so cycle zero in the schedule is the last operation to be performed in that schedule. Based on this idea, a valid action is then a tuple of an integer \(i\in\{0,1,\ldots,max\_gates\}\) and a binary value \(j\). The integer value \(i\) schedules gate \(i\) in the current cycle, and the binary value \(j\) increment the cycle number.

If the action is illegal (for example when we try to schedule a gate that has been scheduled already), then the environment will do nothing, and the rewarder should give a penalty for this.

Example 1:

In this example we want to create an environment for a machine with the following properties:

The machine has two qubits.
The machine supports the X, Y, C-NOT and Measure gates.
Multiple Measure gates should start in the same cycle, or wait till the previous one is done.
The X and Y gates cannot be performed in the same cycle.

The code block below shows how a Scheduling environment for such a machine, were the environment only allows for circuit with a maximum length of 5:

from qgym.envs.scheduling import Scheduling, MachineProperties
from qgym.generators import WorkshopCircuitGenerator

# Set up the hardware specifications
hardware_spec = MachineProperties(n_qubits=2)
hardware_spec.add_gates({"x": 2, "y": 2, "cnot": 4, "measure": 10})
hardware_spec.add_same_start(["measure"])
hardware_spec.add_not_in_same_cycle([("x", "y")])

# Initialize the environment
env = Scheduling(
    hardware_spec,
    max_gates=5,
    circuit_generator=WorkshopCircuitGenerator()
)

Example 2:

This example shows how to add a specified commutation rulebook, which can contain costum commutation rules. For more information on implementing custom cummutation rules, see the documentation of CommutationRulebook. This example uses the same machine properties as example 1. Custom commutation rules can be added as shown in the code block below:

from qgym.envs.scheduling import Scheduling, MachineProperties
from qgym.envs.scheduling.rulebook import CommutationRulebook
from qgym.generators import WorkshopCircuitGenerator

# Set up the hardware specifications
hardware_spec = MachineProperties(n_qubits=2)
hardware_spec.add_gates({"x": 2, "y": 2, "cnot": 4, "measure": 10})
hardware_spec.add_same_start(["measure"])
hardware_spec.add_not_in_same_cycle([("x", "y")])

# Setup the rulebook
rulebook = CommutationRulebook()

# Initialize the environment
env_com = Scheduling(
                    hardware_spec,
                    max_gates=5,
                    rulebook=rulebook,
                    circuit_generator=WorkshopCircuitGenerator(),
                )

class qgym.envs.scheduling.scheduling.Scheduling(machine_properties, *, max_gates=200, dependency_depth=1, circuit_generator=None, rulebook=None, rewarder=None, render_mode=None)[source]

Bases: Environment[Dict[str, ndarray[Any, dtype[int32]] | ndarray[Any, dtype[int8]]], ndarray[Any, dtype[int32]]]

RL environment for the scheduling problem.

__init__(machine_properties, *, max_gates=200, dependency_depth=1, circuit_generator=None, rulebook=None, rewarder=None, render_mode=None)[source]

Initialize the action space, observation space, and initial states for the scheduling environment.

Parameters:

machine_properties (Mapping[str, Any] | str | MachineProperties) – A MachineProperties object, a Mapping containing machine properties or a string with a filename for a file containing the machine properties.
max_gates (int) – Maximum number of gates allowed in a circuit. Defaults to 200.
dependency_depth (int) – Number of dependencies given in the observation. Determines the shape of the dependencies observation, which has the shape (dependency_depth, max_gates). Defaults to 1.
circuit_generator (CircuitGenerator | None) – Generator class for generating circuits for training.
rulebook (CommutationRulebook | None) – CommutationRulebook describing the commutation rules. If None (default) is given, a default CommutationRulebook will be used. (See CommutationRulebook for more info on the default rules.)
rewarder (Rewarder | None) – Rewarder to use for the environment. If None (default), then a default BasicRewarder is used.
render_mode (str | None) – If "human" open a pygame screen visualizing the step. If "rgb_array", return an RGB array encoding of the rendered frame on each render call.

get_circuit(mode='human')[source]

Return the quantum circuit of this episode.

Parameters:: mode (str) – Choose from be "human" or "encoded". Defaults to "human".
Raises:: ValueError – If an unsupported mode is provided.
Return type:: list[Gate]
Returns:: Human or encoded quantum circuit.

reset(*, seed=None, options=None)[source]

Reset the state, action space and load a new (random) initial state.

To be used after an episode is finished.

Parameters:

seed (int | None) – Seed for the random number generator, should only be provided (optionally) on the first reset call, i.e., before any learning is done.
return_info – Whether to receive debugging info.
options (Mapping[str, Any] | None) – Mapping with keyword arguments with additional options for the reset. Keywords can be found in the description of SchedulingState.reset.
_kwargs – Additional options to configure the reset.

Return type:

tuple[dict[str, ndarray[Any, dtype[int32]] | ndarray[Any, dtype[int8]]], dict[str, Any]]

Returns:

Initial observation and debugging info.