1. A method of credit assignment for an artificial spiking network comprising a plurality of units, the method comprising:
operating said network in accordance with reinforcement learning process capable of generating a network output;
determining a credit based on relating said network output to a contribution of a unit of said plurality of units; and
adjusting a learning parameter associated with said unit based at least in part on said credit;
wherein said contribution of said unit is determined based at least in part on an eligibility associated with said unit.
2. The method of claim 1, wherein:
said operating said network in accordance with said reinforcement learning process is based at least in part on at least one of: a unit input; a unit output; andor a unit state; and
said credit is determined for individual ones of said plurality of units based at least in part on any of: said unit input; (ii) said unit output; and (iii) said unit state.
3. The method of claim 1, wherein:
said learning parameter comprises a synaptic weight; and
said adjusting is configured to increase said weight based on a positive correlation between said network output and said contribution.
4. A computer-implemented method of operating a plurality of data interfaces in a computerized network comprising a plurality of nodes, the method comprising:
determining a network output based at least in part on individual contributions of said plurality of nodes;
based at least in part on a reinforcement indication:
determining an eligibility associated with individual ones of said plurality of data interfaces; and
adjusting a learning parameter associated with said individual ones of said plurality of data interfaces, said adjustment based at least in part on a combination of said output and said eligibility.
5. The method of claim 4, wherein:
said network is operable in accordance with a reinforcement learning process characterized by said reinforcement indication, said learning parameter, and a process performance;
said output is generated based at least in part on an input provided to said network;
said process performance is configured based at least in part on a quantity capable of being determined based on said input and said output; and
said adjusting said learning parameter causes generation of another network output, the another output characterized by a reduced value of said quantity for said input.
6. The method of claim 5, wherein said adjusting is configured to apply the reinforcement indication to the said learning parameter based on the unit output that is consistent with the network output.
7. The method of claim 5, wherein:
said reinforcement indication is configured based at least in part on said process performance; and
said adjusting comprises improving said process performance.
8. The method of claim 4, wherein said eligibility is configured based at least in part on a temporary record of one or more data events associated with at least one interface of said plurality of data interfaces, said temporary record being characterized by a time interval prior to said reinforcement indication.
9. The method of claim 8, wherein:
said at least one interface comprises a connection between a pre-synaptic node and a post-synaptic node of said plurality of nodes, said pre-synaptic node and a post-synaptic nodes being operable in accordance with a reinforcement learning process capable of causing generation of a node response; and
said one or more data events comprise one or more responses generated by said pre-synaptic node andor said post-synaptic node.
10. The method of claim 9, wherein:
said eligibility comprises a trace configured to decrease exponentially with time during at least said interval;
one or more of said individual contributions of said plurality of nodes comprise one or more of said responses by said post-synaptic neuron;
said output comprises a weighted average of said individual contributions; and
said combination corresponding to said connection is determined based on a product of (i) said eligibility trace associated with said connection; and (ii) a rate of change of said network output.
11. The method of claim 10, wherein said combination is determined based on a product of (i) said eligibility trace associated with said connection; (ii) a rate of change of said network output; and (iii) a partial derivative of said network output determined with respect to said eligibility trace.
12. The method of claim 10, wherein said combination is set to zero if said rate of change is negative.
13. The method of claim 10, wherein said interval is characterized by a decrease of said trace by a factor of about exp(1) within a duration of said interval.
14. The method of claim 4, wherein: said combination corresponding to said each interface is determined based on a product of (i) said eligibility trace of said each interface; and (ii) a sign of a rate of change of said network output.
15. The method of claim 4, wherein:
said each data interface comprises a synaptic connection;
said learning parameter comprises a weight associated with said connection; and
said adjustment is configured to increase said weight based on a positive correlation of a rate of change of said network output with said eligibility.
16. The method of claim 4, wherein:
said each data interface comprises a synaptic connection;
said learning parameter comprises a weight associated with said connection; and
said adjustment is configured to decrease said weight based on any of (i) a negative correlation of a rate of change of said network output with said eligibility; and (ii) a sign of a rate of change of said network output being opposite to sign of a derivative of said network output with respect to said eligibility.
17. The method of claim 4, wherein said combination comprises a sigmoidal function of a rate of change of said network output.
18. The method of claim 4, wherein:
said each data interface comprises a synaptic connection;
said learning parameter comprises efficacy associated with said connection; and
said adjustment is configured to increase said efficacy when a sign of a rate of change of said network output matches a sign of a derivative of said network output with respect to said eligibility.
19. The method of claim 4, wherein:
said efficacy comprises by a synaptic weight; and
increasing said weight is characterized by a time-dependent function having at least a time window associated therewith.
20. The method of claim 19, wherein:
said individual ones of said plurality of data interfaces are capable of providing an input signal to a node of said plurality of nodes, said input characterized by input time;
said reinforcement signal is characterized by reinforcement time;
said time window is selected based at least in part on said input time and said reinforcement time; and
integration of said time-dependent function over said window is capable of generating a positive value.
21. The method of claim 19, wherein:
said individual ones of said plurality of data interfaces are capable of providing an input signal to a node of said plurality of nodes, said input characterized by input time;
said reinforcement signal is characterized by reinforcement time;
said node of said plurality of nodes is capable of generating an output, based at least in part on said input, said output characterized by an output time;
said time windows is selected based at least in part on said input time, said output time, and said reinforcement time; and
integration of said time-dependent function over said window is capable of generating a positive value.
22. A computerized robotic system, comprising:
one or more processors configured to execute computer program modules, wherein execution of the computer program modules causes the one or more processors to implement a spiking neuron network utilizing a reinforcement learning process that is configured to:
determine a performance of said process based at least in part on a process output being generated based on an input; and
based on at least said performance, provide a reinforcement signal to said process, said reinforcement signal configured to cause update of at least one learning parameter associated with said process;
wherein:
said process output is based on a plurality of outputs by a plurality of nodes of the network, individual ones of the plurality of outputs being generated based on at least a part of the input; and
said update is configured based on a comparison of said process output with individual ones of the plurality of outputs.
23. A method of operating a neural network having a plurality of neurons and connections, the method comprising:
operating the network using a first subset of the plurality of neurons and connections in a first learning mode; and
operating the network using a second subset of the plurality of neurons and connections in a second learning mode, the second subset being larger in number than the first subset, the operation of the network using the second subset in a second operating mode increasing the learning rate of the network over operation of the network using the second subset in the first mode.
24. The method of claim 24, wherein the first learning mode comprises a global reinforcement signal, and the second mode comprises a reinforcement signal that is at least in part correlated to the performance of one or more individual neurons of the plurality.
25. The method of claim 24, wherein the second subset comprises a subset of sufficiently large number such that the global reinforcement signal would be substantially unrelated to the performance of any single neuron of the plurality if operated in the first mode.
26. A method of enhancing the learning performance of a neural network having a plurality of neurons, the method comprising attributing one or more reinforcement signals to appropriate individual ones of the plurality of neurons using a prescribed learning rule that accounts for at least an eligibility of the individual ones of the neurons for the reinforcement signals.
27. The method of claim 26, wherein the plurality of neurons is sufficiently large in number such that a global reinforcement signal would be inapplicable to at least a portion of the individual ones of the neurons.
28. Robotic apparatus capable of accelerated learning performance, the apparatus comprising:
a neural network having a plurality of neurons; and
logic in signal communication with the neural network, the logic configured to attribute one or more reinforcement signals to appropriate individual ones of the plurality of neurons of the network using a prescribed learning rule, the rule configured to account for at least an eligibility of the individual ones of the neurons for the reinforcement signals.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A subsea apparatus manipulating a wellbore component comprising:
a generally cylindrical body having a first, second and third chamber;
said first chamber having at least one load disbursing member;
said second chamber having at least one second load disbursing member, said chamber communicatively associated with the first chamber;
a freely translatable translational member disposed between said disbursing members said translational member being opposably actuated by said load disbursing members;
a longitudinal member having a first portion disposed within said first chamber for receiving force; and
a second portion of said longitudinal member disposed in said third chamber isolated from said first and second chambers;
wherein at least one of the load disbursing members is capable of overcoming at least one other load disbursing member to cause the translational member to dampen movement of said longitudinal member.
2. The subsea apparatus of claim 1 further comprising a bladder member operatively associated with the at least one second chamber for manipulating fluid communication within the at least one second chamber.
3. The subsea apparatus of claim 1 further comprising a chamber in fluid communication with the bladder member for supplying and disposing fluid into said second chamber.
4. The subsea apparatus of claim 1, further comprising at least one dampening mechanism operatively disposed about the at least one first longitudinal member for lessening a load displaced upon at least one of said first and second load disbursing members.
5. The subsea apparatus of claim 4, further comprising at least one additional load disbursing member in said first chamber for lessening a load disposed upon at least one of said first and second load disbursing members.
6. The subsea apparatus of claim 1 further comprising a port in said third chamber for fluid communication about said second portion of said longitudinal member.
7. A gasless subsea accumulator comprising:
an elongated body having two ends and containing a first, second and third chamber wherein said first and second chambers are operatively engaged by a cross-shaped member having opposing stair step protrusions for engagement of springs;
said first chamber having first and second springs each engaged to one side of said stair step protrusions and biased against one end of said first chamber of said elongated body;
said second chamber having first and second springs each engaged to said opposing side of stair step protrusions and biased against an end of said second chamber;
a longitudinal member disposed within said first chamber engaged to said springs in said first chamber for receiving force said cross-shaped member opposably actuated by said springs in said first and second chambers;
a portion of said longitudinal member disposed in said third chamber, said chamber sealed from said first and second chambers.
8. The gasless subsea accumulator of claim 7 wherein said first spring in said first chamber is capable of overcoming said second spring in said first chamber to cause the cross-shaped member to dampen movement of said longitudinal member by engagement to said second springs of said second chamber.
9. The gasless subsea accumulator of claim 7 further comprising a bladder containing fluid for operable engagement to one of said chambers.
10. The gasless subsea accumulator of claim 7 further comprising a port operably engaged to one of said chambers.
11. The gasless subsea accumulator of claim 7 wherein said protrusions on said cross shaped member are cylindrical for engagement to said springs.
12. The gasless subsea accumulator of claim 7 further comprising a port to said third sealed chamber about the circumference of said longitudinal member.
13. The gasless subsea accumulator of claim 12 further comprising a port operably engaged to said third chamber,
14. The gasless subsea accumulator of claim 9 wherein said bladder comprises at least two flexible chambers.
15. The gasless subsea accumulator of claim 9 wherein said bladder is operably connected to a reservoir of pressurized fluid.
16. A gasless subsea accumulator comprising:
an elongated body having two ends and containing a first and second chamber operatively engaged by a T-shaped member having opposing stair step circular protrusions for engagement of springs;
said first chamber having two springs each engaged to one side of said stair step protrusions and biased against one end of said first chamber;
said second chamber having two springs each engaged to said opposing side of stair step protrusions and biased against an end of said second chamber;
a longitudinal member having two ends, where said member is disposed within said first chamber and engaged to said springs on said first end in said first chamber for receiving force said T-shaped member opposably actuated by said springs in said first and second chambers; and
a third chamber sealed from said first two chambers about a portion of said longitudinal member.
17. The gasless subsea accumulator of claim 16 wherein said springs in said first chamber are of different force.
18. The gasless subsea accumulator of claim 16 wherein said springs in said second chamber are of different force.
19. The gasless subsea accumulator of claim 16 wherein said elongated body has a port disposed in said third chamber of said elongated body.