1. A data processing device comprising:
one or more processors operable to:
calculate a state value having a predetermined state of a state transition model set as a reference, wherein a value of the state value increases as much as a state with a high transition probability to another state close to the predetermined state, wherein a state in the state transition model is transited by an action performed by an agent, and wherein the state value is calculated for each state of the state transition model based on the state transition model of each action;
calculate an action value, for each state of the state transition model and each action that the agent can perform, based on the state transition model and the state value having the predetermined state set as the reference, wherein a value of the action value increases as a transition probability to a state with a high state value having the predetermined state set as the reference increases;
set a state with a variance in the action value higher than a predetermined threshold among states of the state transition model to a target state, the target state being a target to reach by an action performed by the agent; and
select an action of the agent so as to move toward the target state.
2. The data processing device according to claim 1, wherein the one or more processors are operable to:
recognize a current state which is a state where an observation value, which is observed by the agent from the outside, is observed among states of the state transition model based on the observation value, wherein the predetermined state is the current state, and
calculate a state value having the current state set as a reference, of which the value increases as much as a state with a high transition probability to a state close to the current state.
3. The data processing device according to claim 2, wherein the one or more processors are further operable to:
calculate a state value having the target state set as a reference, of which the value increases as a state with a high transition probability to a state close to the target state, for each state of the state transition model based on the state transition model;
calculate an action value of which the value increases as high as a transition probability to a state with a high state value having the target state set as a reference for each state of the state transition model and each action that the agent can perform based on the state transition model and the state value having the target state set as a reference; and
select an action of the agent so as to move toward the target state based on an action value of the current state.
4. The data processing device according to claim 3, wherein the one or more processors are further operable to: update a state transition model for an action of the agent, in which state transition to the current state occurs, based on the state transition to the current state.
5. The data processing device according to claim 4, wherein the state transition model for a predetermined action indicates a frequency of transition to a second state by the predetermined action by the agent in a first state, and wherein the one or processors are further operable to update the state transition model by increasing the frequency.
6. The data processing device according to claim 5, wherein the agent acts in an action environment where the agent acts, assuming a predetermined space as the action environment, and observes a position of the agent in the action environment as the observation value, and wherein the state indicates a small area obtained by dividing the action environment into such small areas.
7. The data processing device according to claim 6, wherein the one or more processors are further operable to determine whether or not the current state coincides with the target state, and select an action of the agent so as to move toward the target state based on an action value of the current state when the current state does not coincide with the target state.
8. The data processing device according to claim 7, wherein, when the current state coincides with the target state, the one or more processors are further operable to:
re-calculate a state value having the current state set as a reference based on the state transition model;
re-calculate the action value based on the state transition model and the state value having the current state set as a reference; and
re-set the target state based on the action value.
9. The data processing device according to claim 2, wherein the one or more processors are operable to obtain a variance of the action value for each state of the state transition model, and set a state to be reached from the current state by state transitions within a predetermined number of times among states, in which a variance of the action value is equal to or higher than a predetermined threshold value, to the target state.
10. The data processing device according to claim 2, wherein the one or more processors are further operable to select an action of the agent so as to move toward the target state based on the action value of the current state with a 6-greedy method or a softmax method.
11. A data processing method, comprising:
in a data processing device:
calculating a state value having a predetermined state of a state transition model set as a reference, wherein a value of the state value increases as much as a state with a high transition probability to another state close to the predetermined state, wherein a state in the state transition model is transited by an action performed by an agent, and wherein the state value is calculated for each state of the state transition model based on the state transition model of each action;
calculating an action value, for each state of the state transition model and each action that the agent can perform, based on the state transition model and the state value having the predetermined state set as the reference, wherein a value of the action value increases as a transition probability to a state with a high state value having the predetermined state set as the reference increases;
setting a state with a variance in the action value higher than a predetermined threshold among states of the state transition model to a target state, the target state being a target to reach by an action performed by the agent, based on the action value; and
selecting an action of the agent so as to move toward the target state.
12. A non-transitory computer-readable storage medium having stored thereon, a computer program having at least one code section for processing, the at least one code section being executable by a computer for causing the computer to perform steps comprising:
calculating a state value having a predetermined state of a state transition model set as a reference, wherein a value of the state value increases as much as a state with a high transition probability to another state close to the predetermined state, wherein a state in the state transition model is transited by an action performed by an agent, and wherein the state value is calculated for each state of the state transition model based on the state transition model of each action;
calculating an action value, for each state of the state transition model and each action that the agent can perform, based on the state transition model and the state value having the predetermined state set as the reference, wherein a value of the action value increases as a transition probability to a state with a high state value having the predetermined state set as the reference increases;
setting a state with a variance in the action value higher than a predetermined threshold among states of the state transition model to a target state, the target state being a target to reach by action performed by the agent, based on the action value; and
selecting an action of the agent so as to move toward the target state.
13. The data processing method according to claim 11, further comprising:
recognizing a current state which is a state where an observation value, which is observed by the agent from the outside, is observed among states of the state transition model based on the observation value, wherein the predetermined state is the current state; and
calculating a state value having the current state set as a reference, of which the value increases as much as a state with a high transition probability to a state close to the current state.
14. The data processing method according to claim 13, further comprising:
calculating a state value having the target state set as a reference, of which the value increases as a state with a high transition probability to a state close to the target state, for each state of the state transition model based on the state transition model;
calculating an action value of which the value increases as high as a transition probability to a state with a high state value having the target state set as a reference for each state of the state transition model and each action that the agent can perform based on the state transition model and the state value having the target state set as a reference; and
selecting an action of the agent so as to move toward the target state based on an action value of the current state.
15. The data processing device according to claim 14, further comprising:
updating a state transition model for an action of the agent, in which state transition to the current state occurs, based on the state transition to the current state.
16. The data processing method according to claim 13, further comprising obtaining a variance of the action value for each state of the state transition model, and setting a state to be reached from the current state by state transitions within a predetermined number of times among states, in which a variance of the action value is equal to or higher than a predetermined threshold value, to the target state.
17. The data processing method according to claim 13, further comprising selecting an action of the agent so as to move toward the target state based on the action value of the current state with a 6-greedy method or a softmax method.
18. A data processing device comprising:
one or more processors operable to:
calculate a state value having a predetermined state of a state transition model set as a reference, wherein a value of the state value increases as much as a state with a high transition probability to another state close to the predetermined state, wherein a state in the state transition model is transited by an action performed by an agent and wherein the state value is calculated for each state of the state transition model based on the state transition model of each action;
calculate an action value, for each state of the state transition model and each action that the agent can perform, based on the state transition model and the state value having the predetermined state set as the reference, wherein a value of the action value increases as a transition probability to a state with a high state value having the predetermined state set as the reference increases;
set a state among states of the state transition model with a variance in the action value higher than a predetermined threshold to a target state, based on the action value, the target state being a target to reach by an action performed by the agent;
select an action of the agent so as to move toward the target state;
recognize a current state which is a state where an observation value, which is observed by the agent from the outside, is observed among states of the state transition model based on the observation value, wherein the predetermined state is the current state;
calculate a state value having the current state set as a reference, of which the value increases as much as a state with a high transition probability to a state close to the current state
calculate a state value having the target state set as a reference, of which the value increases as a state with a high transition probability to a state close to the target state, for each state of the state transition model based on the state transition model;
calculate an action value of which the value increases as high as a transition probability to a state with a high state value having the target state set as a reference for each state of the state transition model and each action that the agent can perform based on the state transition model and the state value having the target state set as a reference; and
select an action of the agent so as to move toward the target state based on an action value of the current state.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A method of controlling elevator car position, comprising:
determining that an elevator car requires re-leveling;
determining elevator car dynamics information associated with a current position of the elevator car; and
adjusting a gain for controlling operation of a motor responsible for moving the elevator car to accomplish the re-leveling while the elevator car is stopped at a landing based on the determined elevator car dynamics information.
2. The method of claim 1, comprising
generating a motor torque signal for controlling the motor for moving the elevator car to accomplish the required re-leveling using the adjusted gain.
3. The method of claim 1, comprising
using the adjusted gain when moving the elevator car during re-leveling; and
using a different, default gain during other elevator operation conditions.
4. The method of claim 1, comprising
determining elevator car dynamics information corresponding to each of a plurality of elevator car positions;
storing the determined elevator car dynamics information in association with an indication of the corresponding elevator car positions; and
determining the elevator car dynamics information by selecting the stored information associated with the current position of the elevator car.
5. The method of claim 1, wherein the gain has a default value and adjusting the gain comprises increasing the gain above the default value.
6. The method of claim 1, wherein the gain is a proportional integral gain of a velocity servo associated with the motor.
7. The method of claim 1, wherein the elevator car dynamics information comprises information regarding dynamics of a roping arrangement used to support the elevator car.
8. The method of claim 1, wherein the elevator car dynamics information comprises at least a first parameter indicative of a frequency of a bounce mode of the elevator car and a second parameter indicative of an attenuation factor.
9. The method of claim 8, wherein the first parameter comprises the frequency squared.
10. The method of claim 1, comprising using the determined elevator car dynamics information in a second order notch filter to obtain an adjustment factor for the adjusting.
11. An elevator system, comprising:
a device for controlling an elevator motor including
a velocity servo having a gain with a set value; and
a dynamic compensation module that selectively adjusts the gain of the velocity servo from the set value for controlling operation of the elevator motor for moving an elevator car to re-level the elevator car while the elevator car is stopped at a landing based on elevator car dynamics information associated with a current position of an elevator car.
12. The elevator system of claim 11, wherein the dynamic compensation module increases the gain above the set value.
13. The elevator system of claim 12, comprising an electronic storage of predetermined elevator car dynamics information associated with each of a plurality of elevator car positions and wherein the dynamic compensation module determines the elevator car dynamics information from the storage based on a current position of the elevator car.
14. The elevator system of claim 11, wherein the velocity servo generates a motor torque signal using the adjusted gain.
15. The elevator system of claim 14, wherein the velocity servo generates the motor torque signal using the adjusted gain for re-leveling an elevator car and otherwise uses the set value of the gain.
16. The elevator system of claim 11, wherein the gain is a proportional integral gain of the velocity servo.
17. The elevator system of claim 11, wherein the elevator car dynamics information comprises information regarding dynamics of a roping arrangement used to support the elevator car.
18. The elevator system of claim 11, wherein the elevator car dynamics information comprises at least a first parameter indicative of a frequency of a bounce mode of the elevator car and a second parameter indicative of an attenuation factor.
19. The elevator system of claim 18, wherein the first parameter comprises the frequency squared.
20. The elevator system of claim 11, wherein the dynamic compensation module comprises a second order notch filter that provides an adjustment factor for adjusting the gain.
21. The elevator system of claim 11, comprising:
an elevator car;
a roping arrangement secured to the elevator car;
a motor for moving the roping arrangement to cause movement of the elevator car; and
a motor controller for controlling the motor, the motor controller comprising the device.
22. A method of controlling elevator car position, comprising:
determining that an elevator car requires re-leveling;
determining elevator car dynamics information associated with a current position of the elevator car; and
adjusting a gain for controlling operation of a motor responsible for moving the elevator car for the re-leveling based on the determined elevator car dynamics information, wherein the elevator car dynamics information comprises at least a first parameter indicative of a frequency of a bounce mode of the elevator car and a second parameter indicative of an attenuation factor.
23. An elevator system, comprising:
a device for controlling an elevator motor including
a velocity servo having a gain with a set value; and
a dynamic compensation module that selectively adjusts the gain of the velocity servo from the set value based on elevator car dynamics information associated with a current position of an elevator car, wherein the elevator car dynamics information comprises at least a first parameter indicative of a frequency of a bounce mode of the elevator car and a second parameter indicative of an attenuation factor.