1. In a computer system having a plurality of users, a method for generating a graphical user interface (GUI) to be displayed to a user, wherein the GUI indicates activities of other users within the computer system, said method comprising:
storing recent posts of life stories by other users of the computer system;
storing recent comments made to life stories of said user by other users of the computer system; and
generating a GUI including images and text associated with a predetermined number of said recent posts and said recent comments and transmitting the GUI for display.
2. The method according to claim 1, further comprising receiving a request from a user for said GUI, and in response to said request, searching for the predetermined number of said recent posts and said recent comments prior to the storing.
3. The method according to claim 1, further comprising configuring a queue for storing posts of life stories of other users of the computer system and comments made to life stories of said user by other users of the computer system, wherein new posts and comments are added to the front of the queue.
4. The method according to claim 1, wherein the queue includes a first queue for storing posts of life stories of other users of the computer system and a second queue for storing comments made to life stories of said user by other users of the computer system, wherein new posts are added to the front of the first queue and new comments are added to the front of the second queue.
5. The method according to claim 4, further comprising receiving a request from a user for said GUI, wherein, in response to said request, the GUI is generated from the posts stored in the first queue and the comments stored in the second queue.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A method of processing network metadata generated on a network transmitting network traffic using one or more network protocols, the network including devices at least some of which receive network traffic through an ingress interface and transmit network traffic through an egress interface, the method comprising the steps of:
receiving network metadata from a plurality of sources in a data processing system, in at least one data format;
determining the type or character of said network metadata;
processing said network metadata to extract useful information therefrom; and
converting at least a portion of said network metadata into one or more different data formats that are used in said data processing system for other system metadata, in response, at least in part, to the results of said determining step.
2. The method as set forth in claim 1, wherein said processing step is performed while said network metadata is in transition on said network between a network device that generated said network metadata and a device that is able to store said network metadata.
3. The method as set forth in claim 2, wherein said processing step is achieved by applying at least one policy governing network metadata processing.
4. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of detecting traffic on said network indicative of a potential security threat.
5. The method as set forth in claim 4, wherein said at least one policy includes the steps of:
comparing the source of incoming network traffic to a predefined list of monitored off-limit devices on said network;
if the destination IP address is on a predefined list of off-limit devices, storing the source IPport, as well as the destination IPport in a potential alert list, along with the number of bytes and packets reported in the ingress NetFlow record;
examining output records to determine if the source IPport and the destination IPport match an entry in the potential alert list;
if a match is found, treating such match as an indication that an internal host replied to an outside peer request; and
generating an alert message in a timely manner to inform of a potential botnet infection.
6. The method as set forth in claim 4, wherein said at least one policy includes the steps of:
collecting information about internal network devices communicating with external network devices; said information comprising a list of communicating devices’ IP addresses and protocols;
applying a streaming cluster analysis method to determine patterns in communications between the communicating devices;
identifying abnormal instances of communications between the communicating devices by the presence of small disjoint patterns in the set of all computed patterns;
generating an alert message in a timely manner to inform of a potential botnet infection if at least one abnormal communication pattern is identified.
7. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of identifying traffic spikes on said network.
8. The method as set forth in claim 7, wherein said at least one policy includes the steps of:
comparing the source of incoming network traffic to a predefined list of monitored devices on said network;
if the source IP address is on a predefined list of devices, storing the source IP in an in-memory database along with the number of bytes and packets reported in the ingress NetFlow record;
at predefined intervals examining the in-memory database and identifying devices which exceeded a threshold set forth by a network manager;
generating an alert message in a timely manner to inform of a traffic spike.
9. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of improving the reliability of capturing NetFlow traffic on said network.
10. The method as set forth in claim 9, wherein said at least one policy includes the steps of:
receiving one or a plurality of NetFlow messages over an unreliable network transport protocol;
optionally converting said received NetFlow messages into one or a plurality of formats other than NetFlow, different flavor of NetFlow or same format as the received NetFlow;
forwarding said received or converted messages to a plurality of end points over an unreliable network transport protocol, to at least one end point over a reliable network transport protocol, or to at least one end point over an unreliable network transport protocol and to at least one end point over a reliable network transport protocol.
11. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of reducing the number of NetFlow messages that arrives at the device that stores said network metadata on said network.
12. The method as set forth in claim 11, wherein said at least one policy includes the steps of:
receiving a plurality of NetFlow messages;
identifying similarities in said received NetFlow messages;
consolidating information in the received NetFlow messages identified as similar into one or a plurality of NetFlow messages;
discarding NetFlow messages for which information was consolidated; and
forwarding consolidated NetFlow messages to the device that stores network metadata on said network.
13. The method as set forth in claim 12, wherein said step of forwarding consolidated NetFlow messages is performed after the passage of a configurable time period.
14. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of detecting when a network interface on said network becomes inoperable.
15. The method as set forth in claim 14, wherein said at least one policy includes the steps of:
establishing a threshold time period during which the absence of an interface identifier in the incoming network metadata signifies said network interface’s inoperable state,
said established threshold time period being variable depending on the time and geographic criteria;
monitoring network metadata related to said network interface;
generating an alert message in a timely manner to inform of an inoperable state of said network interface if network metadata related to said network interface fails to be detected for a period in excess of said threshold time period.
16. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of measuring network latency using network metadata on said network.
17. The method as set forth in claim 16, wherein said at least one policy includes the steps of:
establishing a threshold time period during which a network device which requested services of a Domain Name Server (DNS) must receive a response from said DNS service;
monitoring network metadata describing communications between network devices and DNS services;
computing time elapsed between said network device request for a DNS service and receiving a response from that DNS service;
comparing said elapsed time with said threshold time period; and
generating an alert message to inform of excessive network latency.
18. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of providing real time network visibility per application type across a plurality of devices on said network.
19. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of producing a list of most active hosts over a specified time period on said network.
20. The method as set forth in claim 19, wherein said at least one policy includes the steps of:
receiving a plurality of network metadata messages;
identifying network metadata related to individual network hosts;
aggregating network metadata for each identified network host;
selecting a configurable size list of the network hosts which produced most network connections; and
reporting said list of the network hosts which produced most network connections.
21. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of producing a list of devices on the network with the most access control lists violations over a specified period of time.
22. The method as set forth in claim 21, wherein said at least one policy includes the steps of:
receiving a plurality of messages containing network metadata; said metadata containing information about access control lists violations;
selecting a subset of said messages such that each message reports at least one access control list violation;
aggregating said subset of messages per reported network device;
sorting said subset of aggregated messages in descending order so that network devices with the most access control lists violations are at the head of the list;
retrieving a configurable number of entries from the head of said sorted list; and
generating an alert message in a timely manner to inform of the network devices with the most access control list violations found in said entries at the head of said list.
23. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of calculating the average size of network packets which passed through a network device via a specifiable ingress interface and egress interface over a specifiable time interval.
24. The method as set forth in claim 23, wherein said at least one policy includes the steps of:
receiving a plurality of messages containing network metadata; said metadata containing information about the number of packets and the number of bytes which passed between a distinct ingress interface and a distinct egress interface;
aggregating said information per distinct pairs of interfaces;
computing running average size of the network packets passed through each distinct pair of interfaces;
reporting in a timely fashion information about average size of the network packets which passed between at least a portion of said distinct pairs of network interfaces.
25. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of identifying the devices on the network that are the top consumers of the bandwidth of the network over a specifiable time interval.
26. The method as set forth in claim 25, wherein said at least one policy includes the steps of:
receiving a plurality of network metadata messages;
identifying network metadata related to individual network hosts;
aggregating network metadata for each identified network host;
selecting a configurable size list of the network hosts which produced most network traffic; and
reporting said list of the network hosts which produced most network traffic.
27. The method as set forth in claim 26, wherein said at least one policy is applied for the purpose of identifying the devices on the network that are the top consumers of the bandwidth of the network, by network device by network protocol, over a specifiable time interval.
28. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of identifying activity on the network is above a baseline level of such activity.
29. The method as set forth in claim 28, wherein said at least one policy includes the steps of operating for a specifiable period of time in a learning mode to establish a baseline of presumably normal parameters with respect to a characteristic of network conditions of interest to a manager of said network, and operating in a monitoring mode to report to said manager when activity on said network exceeds said baseline by pre-definable levels.
30. The method as set forth in claim 29, wherein said baseline comprises network information relating to one or more network attributes from the group consisting of:
number of concurrent flows, global packets rate, global throughput,
new flows creation rate, number of SYN packets sent, number of SYN packets received,
number of connection resets, average flow duration, flow duration dispersion and time of day.
31. The method as set forth in claim 3, wherein said at least one policy is applied for the purpose of deduplicating flow information reported by flow exporters.
32. The method as set forth in claim 31, wherein said at least one policy includes the steps of:
receiving a flow record containing IP addresses of communicating endpoints and the IP address of a device which provided the flow record;
noting the time when said record was received;
maintaining in memory information about the IP addresses of communicating endpoints, IP addresses of flow reporting devices and the time of last observed communication; said information being an indicator of the frequency of at least some communication paths utilized by said flow reporting device;
selecting based upon said memory information a flow reporting device with the highest frequency of use among other flow reporting devices which report said flow and designating said highest frequency flow reporting device as an authoritative source of information about communications between said network endpoints;
in the event that more than one flow reporting device has same highest frequency of use reflected in said memory information, further disambiguating the identity of said authoritative source of information about communications between said network endpoints based upon criteria selected form the group consisting of, the time when flow reporting devices reported said flow, a Time-To-Live counter reported in the flow records submitted by the flow reporting devices, and the next hop IP address reported in the flow records submitted by the flow reporting devices;
forwarding for further processing a flow record about communication between said communicating endpoints received from the authoritative flow reporting device; and
discarding flow records about communication between said communicating endpoints received from other flow reporting devices.
33. The method as set forth in claim 31, wherein said at least one policy includes the steps of:
collecting a list of adjacencies relating to IP addresses and interface indices from the network metadata traversing said network;
building a computer model representing the devices and topology of said network from said list of adjacencies;
statistically identifying devices that originally author network metadata relating to a particular network traffic flow on said network; and
deduplicating network metadata information that may be generated by downstream network devices to reflect the flow of the same network traffic through said downstream devices.
34. The method as set forth in claim 1, further comprising the steps of:
forwarding the received network metadata to a downstream consumer or collector of network metadata while preserving the at least one data format thereof.
35. The method as set forth in claim 1, further comprising the steps of:
forwarding converted network metadata to a downstream consumer or collector of network metadata of said one or more different data formats.
36. The method as set forth in claim 1, wherein said step of determining comprises confirming at least one assertion relating to the validity of the received network metadata.
37. The method as set forth in claim 1 wherein said step of converting comprises converting said received network metadata into a common data format.
38. The method as set forth in claim 1 wherein said determining step comprises a lookup of a transient identifier assigned to said network metadata.
39. The method as set forth in claim 38 wherein a transient network metadata identifier is dynamically associated with a permanent identifier assigned to said network metadata.
40. The method as set forth in claim 39 wherein a permanent network metadata identifier is further associated with one or a plurality of policies.
41. The method as set forth in claim 40 wherein said processing step comprises application of one or a plurality of policies to said network metadata.
42. The method as set forth in claim 1 wherein said determining step comprises classifying said network metadata.
43. The method as set forth in claim 42 wherein said classifying step is achieved by applying at least one policy governing network metadata processing.
44. The method as set forth in claim 43 wherein said at least one policy is based upon the content of said network metadata.
45. The method as set forth in claim 43 wherein said at least one policy is time based.
46. The method as set forth in claim 43 wherein said at least one policy is implemented by incorporating a dynamically loadable software module.
47. The method as set forth in claim 46 wherein said dynamically loadable software module is a binary executable module.
48. The method as set forth in claim 43 wherein the application of said at least one policy produces additional network metadata derived from the processed network metadata.
49. The method as set forth in claim 48 wherein said additional network metadata is in the same format as the processed network metadata.
50. The method as set forth in claim 48 wherein said additional network metadata is in a format other than the format of the processed network metadata.
51. The method as set forth in claim 43 wherein application of said at least one policy transforms network metadata content.
52. The method as set forth in claim 1 wherein said step of converting network metadata comprises applying a conversion rule.
53. The method as set forth in claim 52 wherein the conversion rule is based upon the content of said network metadata.
54. The method as set forth in claim 52 wherein the conversion rule is a default conversion rule.
55. The method as set forth in claim 52 wherein the conversion rule is implemented by incorporating a dynamically loadable software module.
56. The method as set forth in claim 55 wherein the dynamically loadable software module is a binary executable module.
57. The method as set forth in claim 2 wherein said step of forwarding received network metadata changes an indicator of origin of the network metadata.
58. The method as set forth in claim 2 wherein said step of forwarding received network metadata does not change the indicator of origin of said network metadata.
59. The method as set forth in claim 3 further comprising the step of authenticating forwarded converted network metadata.
60. The method as set forth in claim 59 wherein said authenticating step implements authentication that is cryptographically strong.
61. The method as set forth in claim 59 wherein said authenticating step implements authentication that is not cryptographically strong.
62. A method of classifying network metadata comprising the steps of:
computing an invariant network metadata definition unique identifier;
associating at least one processing rule with said network metadata unique identifier;
storing said network metadata definition unique identifier in a first information container;
storing said at least one processing rule in said first information container;
associating said at least one processing rule with said network metadata definition unique identifier in said first information container;
receiving a first packet of information containing a network metadata definition and a first transient network metadata identifier from a uniquely identified source;
locating invariant network metadata definition in said first packet;
computing an invariant network metadata definition unique identifier corresponding to the network metadata definition in said first packet;
locating said network metadata definition unique identifier in the first information container;
storing said first transient network metadata identifier and said source identifier in a second information container;
associating said first transient network metadata identifier in the second information container with said network metadata definition unique identifier in the first information container;
receiving a second packet of information containing network metadata, said first transient network metadata identifier and said source identifier;
locating said first transient network metadata identifier and said source identifier in the second information container;
using said association in the second information container to locate network metadata definition unique identifier in the first information container;
using said association of network metadata definition unique identifier in the first information container to locate said processing rule; and
executing said processing rule to classify network metadata.
63. The method as set forth in claim 62, further comprising:
receiving a third packet of information containing a network metadata definition and a second transient network metadata identifier from said uniquely identified source;
locating invariant network metadata definition in said third packet;
computing an invariant network metadata definition unique identifier corresponding to the network metadata definition in said third packet;
locating said network metadata definition unique identifier in the first information container;
storing said second transient network metadata identifier and said source identifier in a second information container;
associating said second transient network metadata identifier in the second information container with said network metadata definition unique identifier in the first information container;
locating said first transient network metadata identifier using said source identifier in the second information container;
removing said first transient network metadata identifier and said source identifier from the second container;
receiving a fourth packet of information containing network metadata, said second transient network metadata identifier and said source identifier;
locating said second transient network metadata identifier and said source identifier in the second information container;
using said association in the second information container to locate network metadata definition unique identifier in the first information container;
using said association of network metadata definition unique identifier in the first information container to locate said processing rule; and
executing said processing rule to classify network metadata
64. The method as set forth in claim 62 wherein said step of removing said first transient network metadata identifier and said source identifier from the second container is performed by applying an aging algorithm to the elements stored in the second container.
65. The method as set forth in claim 62 wherein said step of computing an invariant network metadata definition unique identifier comprises computing an invariant network metadata definition hash value.
66. The method as set forth in claim 65 wherein said hash value is computed using a cryptographically strong algorithm.
67. The method as set forth in claim 65 wherein said hash value is computed using a non-cryptographically strong algorithm.
68. The method as set forth in claim 62 wherein said processing rule is a policy governing network metadata processing.
69. The method as set forth in claim 68 wherein the policy is based at least in part upon the content of said network metadata.
70. The method as set forth in claim 68 wherein the policy is time based.
71. A method of selectively reducing the volume of network metadata supplied to a SIEM system on a network, comprising the steps of:
identifying network metadata packets that contain at least partially redundant metadata; and
processing said network metadata packets that contain at least partially redundant metadata to aggregate said packets into a smaller number of packets while preserving the metadata of interest.
72. A system for processing network metadata comprising the steps of:
a network node adapted for receiving network metadata from a plurality of sources in at least one data format;
a processing module for determining the type or character of said network metadata;
a processing module for processing said network metadata to extract useful information therefrom; and
a processing module for converting at least a portion of said network metadata into one or more different data formats in response, at least in part, to the results of said determining step.
73. A system for classifying network metadata comprising the steps of:
a processor for computing an invariant network metadata definition unique identifier;
a processor for associating at least one processing rule with said network metadata unique identifier;
a processor for storing said network metadata definition unique identifier in a first information container;
a processor for storing said at least one processing rule in said first information container;
a processor for associating said at least one processing rule with said network metadata definition unique identifier in said first information container;
a processor for receiving a first packet of information containing a network metadata definition and a transient network metadata identifier;
a processor for locating invariant network metadata definition in said first packet;
a processor for computing an invariant network metadata definition unique identifier corresponding to the network metadata definition in said first packet;
a processor for locating said network metadata definition unique identifier in the first information container;
a processor for storing said transient network metadata identifier in a second information container;
a processor for associating said transient network metadata identifier in the second information container with said network metadata definition unique identifier in the first information container;
a processor for receiving a second packet of information containing network metadata and said transient network metadata identifier;
a processor for locating said transient network metadata identifier in the second information container;
a processor for using said association in the second information container to locate network metadata definition unique identifier in the first information container;
a processor for using said association of network metadata definition unique identifier in the first information container to locate said processing rule; and
a processor for executing said processing rule to classify network metadata.
74. A networked system comprising:
a local network comprising
a plurality of devices on said network that generate a first collection of network metadata in syslog format;
a network monitoring facility that is adapted to receive syslog data;
a plurality of devices on said network that generate a second collection of network metadata in a format other than that of syslog data;
a first network metadata processing facility capable of converting at least portions of said second collection of network metadata into syslog format; and
communication media for supplying the converted output of said first network metadata processing facility as input to said network monitoring facility.
75. The networked system as set forth in claim 74, wherein said first network metadata processing facility comprises a process for processing at least portions of said second collection of network metadata, prior to converting said at least portions of said second collection of network metadata into syslog format.
76. The networked system as set forth in claim 75, wherein said processor comprises a classifier that is able to determine the type of messages included within said second collection of network metadata.
77. The networked system as set forth in claim 75, wherein said processor comprises a policy module that is able to selectively process each message within said second collection of network metadata based at least in part upon the type determination of said classifier.
78. The networked system as set forth in claim 77, wherein said policy module applies one or more policies to each message based at least in part upon the type determination of said classifier.
79. The networked system as set forth in claim 74, further comprising a remote subnet that comprises:
a plurality of devices on said remote subnet that generate a third collection of network metadata in syslog format;
a plurality of devices on said subnet that generate a fourth collection of network metadata in a format other than that of syslog data; and
a second network metadata processing facility capable of converting at least portions of said fourth collection of network metadata into syslog format; and
communication media for supplying the converted output of said second network metadata processing facility as input to said network monitoring facility.
80. A method of enhancing reliability of transmission of NetFlow messages over a WAN from a branch network to a primary network, comprising the steps of:
converting at least a portion of the NetFlow messages generated on said branch network into syslog messages; and
utilizing a network protocol more reliable than UDP to transfer said converted syslog messages from said branch network to a server on said primary network.