1. A method for performing fast reroute (FRR) operations at the edge of a network, the network having first and second edge devices coupled to a neighboring routing domain, the method comprising:
A. detecting a loss of communication between the first edge device and the neighboring routing domain;
B. receiving a data packet at the first edge device, the received data packet containing a destination address that is reachable via the neighboring routing domain;
C. determining whether the received data packet was previously rerouted in accordance with FRR operations; and
D. rerouting, in response to determining that the received data packet was not previously rerouted, the data packet to the second edge device for forwarding to the neighboring routing domain.
2. The method of claim 1, further comprising:
dropping the received data packet at the first edge device if the received data packet is determined to have been previously rerouted in accordance with FRR operations.
3. The method of claim 1, wherein the step of determining whether the received data packet was previously rerouted in accordance with FRR operations further comprises:
determining whether the received data packet was previously rerouted in accordance with FRR operations based on FRR status information included in the packet.
4. The method of claim 1, wherein the step of determining whether the received data packet was previously rerouted in accordance with FRR operations further comprises:
determining whether the received data packet was previously rerouted in accordance with FRR operations based on FRR status information previously communicated to the first edge device via an out-of-band signaling mechanism.
5. The method of claim 1, further comprising:
starting a timer in response to detecting the loss of communication between the first edge device and the neighboring routing domain; and
performing steps B-D until the timer expires.
6. The method of claim 1, further comprising:
automatically detecting, by the first edge device, that the second edge device is coupled to the neighboring routing domain.
7. The method of claim 6, wherein the first edge devices automatically detects that the second edge device is coupled to the neighboring routing domain by:
receiving a first destination address prefix in a first message advertised from an edge device in the neighboring routing domain;
receiving a second destination address prefix in a second message advertised from the second edge device; and
determining whether the first and second destination address prefixes are equal;
detecting, in response to determining that the first and second destination address prefixes are equal, that the second edge device is coupled to the neighboring routing domain.
8. The method of claim 1, further comprising:
communicating the received data packet from the first edge device through a network tunnel to the second edge device.
9. The method of claim 8, wherein the network tunnel is a Multi-Protocol Label Switching (MPLS) tunnel.
10. The method of claim 8, wherein the network tunnel is an Internet Protocol (IP) tunnel.
11. The method of claim 1, further comprising:
determining whether the received data packet is permitted to be rerouted in accordance with FRR operations; and
rerouting the received data packet to the second edge device only after determining that the received data packet is permitted to be rerouted.
12. The method of claim 11, wherein the step of determining whether the received data packet is permitted to be rerouted in accordance with FRR operations further comprises:
applying local policy to a destination address contained in the received data packet.
13. The method of claim 11, wherein the step of determining whether the received data packet is permitted to be rerouted in accordance with FRR operations further comprises:
identifying a destination address contained in the received data packet; and
statically configuring the first edge device not to reroute data packets containing the identified destination address.
14. A network node configured to perform fast reroute (FRR) operations at the edge of a network, the network node comprising:
a first network interface adapted to communicate with a neighboring routing domain;
means for detecting a loss of communication over the first network interface;
a second network interface adapted to receive a data packet containing a destination address that is reachable via the neighboring routing domain;
means for determining whether the received data packet was previously rerouted in accordance with FRR operations; and
means for rerouting, in response to determining that the received data packet was not previously rerouted, the data packet to a second network node coupled to the neighboring routing domain.
15. The network node of claim 14, further comprising:
means for dropping the received data packet if the received data packet is determined to have been previously rerouted in accordance with FRR operations.
16. The network node of claim 14, wherein means for determining whether the received data packet was previously rerouted in accordance with FRR operations further comprises:
means for determining whether the received data packet was previously rerouted in accordance with FRR operations based on FRR status information included in the packet.
17. The network node of claim 14, wherein the means for determining whether the received data packet was previously rerouted in accordance with FRR operations further comprises:
means for determining whether the received data packet was previously rerouted in accordance with FRR operations based on FRR status information previously communicated to the network node via an out-of-band signaling mechanism.
18. The network node of claim 14, further comprising:
means for starting a timer in response to detecting the loss of communication over the first network interface.
19. The network node of claim 14, further comprising:
means for automatically detecting that the second network node is coupled to the neighboring routing domain.
20. The network node of claim 19, wherein the means for automatically detecting that the second network node is coupled to the neighboring routing domain further comprises:
means for determining whether a first destination address prefix advertised from the neighboring routing domain and a second destination address prefix advertised from the second network node are equal;
means for detecting, in response to determining that the first and second destination address prefixes are equal, that the second network node is coupled to the neighboring routing domain.
21. The network node of claim 14, further comprising:
means for communicating the received data packet through a network tunnel to the second network node.
22. The network node of claim 14, further comprising:
means for determining whether the received data packet is permitted to be rerouted in accordance with FRR operations; and
means for rerouting the received data packet to the second network node only after determining that the received data packet is permitted to be rerouted.
23. The network node of claim 22, wherein the means for determining whether the received data packet is permitted to be rerouted in accordance with FRR operations further comprises:
means for applying local policy to a destination address contained in the received data packet.
24. The network node of claim 22, wherein the means for determining whether the received data packet is permitted to be rerouted in accordance with FRR operations further comprises:
means for identifying a destination address contained in the received data packet; and
means for statically configuring the network node not to reroute data packets containing the identified destination address.
25. A computer-readable medium storing instructions for execution on a processor for the practice of a method of performing fast reroute (FRR) operations at the edge of a network, the network having first and second edge devices coupled to a neighboring routing domain, the method comprising:
detecting a loss of communication between the first edge device and the neighboring routing domain;
receiving a data packet at the first edge device, the received data packet containing a destination address that is reachable via the neighboring routing domain;
determining whether the received data packet was previously rerouted in accordance with FRR operations; and
rerouting, in response to determining that the received data packet was not previously rerouted, the data packet to the second edge device for forwarding to the neighboring routing domain.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A computer-implemented method for a computer-aided control andor regulation of a technical system, comprising:
representing in a plurality of data sets based on observed data for the technical system a dynamic behavior of the technical system for a plurality of different points in time by a state of the technical system and an action executed on the technical system, with a respective action at a respective time leading to a follow-up state of the technical system at a next point in time;
implementing reinforcement learning via a neural network executed on a processor of a computer to derive an optimum action selection rule, the reinforcement learning implemented on the plurality of data sets, each data set including the state at a respective point in time, the action executed in the state at the point in time, and the follow-up state and whereby each data set is assigned an evaluation, the reinforcement learning of the optimum action selection rule based on rewards that depend on a quality function for the state and action and on a value function for the follow-up state, comprising:
(a) modeling of the quality function by the neural network reflecting a quality of an action for the plurality of states and the plurality of actions of the technical system, and
(b) determining parameters of the neural network by reinforced learning of the neural network on the basis of an optimality criterion that depends on the plurality of evaluations of the plurality of data sets and the quality function; and
regulating andor controlling the technical system by selecting the plurality actions to be carried out on the technical system using the learned optimum action selection rule based on the learned neural network.
2. The method as claimed in claim 1, wherein the quality function is modeled by the neural network such that an evaluation function is adapted to the plurality of evaluations of the plurality of data sets.
3. The method as claimed in claim 1, wherein during the learning, the action is selected in a respective state for which a highest value of the quality function will be created by the neural network.
4. The method as claimed in claim 1,
wherein the quality function is modeled with a plurality neural networks,
wherein each network of the plurality of neural networks forms a feed-forward network with an input layer including the respective state of the technical system, a hidden layer and an output layer which includes the quality function, and
wherein each neural network parameterizes the action to be carried out in the respective state.
5. The method as claimed in claim 1,
wherein the quality function is modeled by a single neural network, and
wherein the neural network forms a feed-forward network with the input layer including the respective state of the technical system and the action to be executed in the respective state, a hidden layer and the output layer which includes the quality function.
6. The method as claimed in claim 1, wherein a back-propagation method is used for the learning of the neural network.
7. The method as claimed in claim 1, wherein the optimality criterion is selected such that an optimum dynamic behavior of the technical system is parameterized.
8. The method as claimed in claim 1, wherein the optimality criterion is a minimization of a Bellmann residuum.
9. The method as claimed in claim 1, wherein the optimality criterion is reaching the checkpoint of the Bellmann iteration.
10. The method as claimed in claim 1,
wherein the optimality criterion includes a selectable parameter, and
wherein by modifying the selectable parameter, the optimality criterion is adapted.
11. The method as claimed in claim 1, wherein the state of the technical system includes a first variable, andor an action to be carried out on the technical system includes an action variable.
12. The method as claimed in claim 11, wherein the first variable is an observed state variable of the technical system.
13. The method as claimed in claim 1,
wherein the plurality of states in the plurality data sets are hidden states of the technical system that are generated by a recurrent neural network with an aid of a plurality of source data sets, and
wherein each source data set includes an observed state of the technical system, the action carried out in the observed state, and the follow-up state resulting from the action.
14. The method as claimed in claim 13,
wherein the dynamic behavior of the technical system is modeled by the recurrent neural network, and
wherein the recurrent neural network is formed by the input layer including the plurality of observed states of the technical system and the plurality of actions executed on the technical system, the hidden recurrent layer which includes the plurality of hidden states, and the output layer which also includes the plurality of observed states.
15. The method as claimed in 14, wherein the recurrent neural network is learned using a learning method.
16. The method as claimed in claim 15, wherein the learning method is a back-propagation method.
17. The method as claimed in claim 1, wherein the technical system is a turbine.
18. The method as claimed in claim 16, wherein the turbine is a gas turbine.
19. The method as claimed in claim 18,
wherein the gas turbine is regulated andor controlled with the method,
wherein the plurality of the states of the technical system andor the actions to be performed in the respective states includes a second variable selected from the group consisting of an overall power of the gas turbine, a pressure andor a temperature in the gas turbine or in a vicinity of the gas turbine, combustion chamber accelerations in the gas turbine, a setting parameter at the gas turbine, and a combination thereof, and
wherein the setting parameter may be a valve setting andor a fuel ratio andor an inlet guide vane position.
20. A computer program product with program code stored on a non-transitory machine-readable medium, when the program executes on a processor of a computer, the program comprising:
representing in a plurality of data sets based on observed data for the technical system a dynamic behavior of the technical system for a plurality of different points in time by a state of the technical system and an action executed on the technical system, with a respective action at a respective time leading to a follow-up state of the technical system at a next point in time;
implementing reinforcement learning via a neural network executed on a processor of a computer to derive an optimum action selection rule, the reinforcement learning implemented on the plurality of data sets, each data set including the state at a respective point in time, the action executed in the state at the point in time, and the follow-up state and whereby each data set is assigned an evaluation, the reinforcement learning of the optimum action selection rule based on rewards that depend on a quality function for the state and action and on a value function for the follow-up state, comprising:
(a) modeling of the quality function by the neural network reflecting a quality of an action for the plurality of states and the plurality of actions of the technical system, and
(b) determining parameters of the neural network by reinforced learning of the neural network on the basis of an optimality criterion that depends on the plurality of evaluations of the plurality of data sets and the quality function; and
regulating andor controlling the technical system by selecting the plurality actions to be carried out on the technical system using the learned optimum action selection rule based on the learned neural network.