1. A system for managing a document conversion environment, comprising:
at least one server computer communicatively connected to one or more devices on a network and running an isolation service, wherein when a document having a native document type is ready for conversion, the isolation service determines whether a native application associated with the native document type is executing and
when the native application is executing, the isolation service causes the executing native application to load and convert the document from the native document type to a common document format,
when the native application is not executing, the isolation service executes the native application and causes the executing native application to load and convert the document to the common document format,
wherein by first determining whether the native application associated with the native document type is already executing prior to causing the native application to load and convert the document to the common document format, the isolation service ensures only a single instance of the native application executes for the conversion,
wherein the isolation service monitors and manages the conversion of the document by the single instance of the native application according to at least one specified document conversion operation parameter, condition, or setting.
2. The system of claim 1 wherein the isolation service restarts the single instance of the native application when it does not comply with the at least one specified document conversion operation parameter, condition, or setting.
3. A method performed by an application isolation system for managing document conversion in a document conversion environment, the method comprising:
the application isolation system determining that a document having a native document type is awaiting to be converted, the application isolation system running on a computer communicatively connected to one or more devices on a network;
the application isolation System identifying a native application corresponding to the native document type;
the application isolation system determining whether the identified native application is executing;
when the identified native application is executing, the application isolation system commanding the executing native application to load and convert the document;
when the identified native application is not executing, the application isolation system starting the native application and commanding the started native application to load and convert the document;
wherein by first determining whether the native application associated with the native document type is already executing prior to causing the native application to load and convert the document, the application isolation system ensures only a single instance of the native application executes for the conversion; and
the application isolation system monitoring and managing the conversion of the document by the identified native application to according to at least one specified document conversion operation parameter, condition, or setting.
4. The method of claim 3 further comprising restarting the native application when it does not comply with the at least one specified document conversion operation parameter, condition, or setting.
5. The method of claim 3 wherein the commanding the native application to convert the document includes commanding the native application to print the document.
6. The method of claim 3 wherein the commanding the native application to convert the document includes commanding the native application to print the document using a specified printer driver.
7. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a computer, cause the computer to perform an isolation service including:
determining that a document having a native document type is awaiting to be converted; identifying a native application corresponding to the native document type; determining whether the identified native application is executing;
when the identified native application is executing, commanding the executing native application to load and convert the document;
when the identified native application is not executing, starting the native application and commanding the started native application to load and convert the document;
wherein by first determining whether the native application associated with the native document type is already executing prior to causing the native application to load and convert the document, the isolation service ensures only a single instance of the native application executes for the conversion; and
monitoring and managing the conversion of the document by the identified native application according to one or more specified document conversion operation parameters, conditions, or settings.
8. The non-transitory computer-readable medium of claim 7, wherein the computer-executable instructions when executed by the computer further cause the computer to perform restarting the native application when it does not comply with the one or more specified document conversion operation parameters, conditions, or settings.
9. The non-transitory computer-readable medium of claim 7, wherein the commanding the native application to convert the document includes commanding the native application to print the document.
10. The non-transitory computer-readable medium of claim 9, wherein the commanding the native application to convert the document includes commanding the native application to print the document using a specified printer driver.
11. The non-transitory computer-readable medium of claim 7, wherein the computer-executable instructions when executed by the computer further cause the computer to perform balancing application management with speed of the native application.
12. The system of claim 1, wherein the at least one specified document conversion operation parameter, condition, or setting comprises an amount of memory allocation and wherein the isolation service running on the at least one server computer is configured to:
allocate the amount of memory to the native application when the isolation service executes the native application; and
restart the native application when a process associated with the native application requests additional memory.
13. The system of claim 1, wherein the at least one specified document conversion operation parameter, condition, or setting comprises a maximum time for conversion and wherein the isolation service running on the at least one server computer is configured to:
detect that the native application is taking more time to convert the document than the maximum time for conversion; and
restart the native application and a document conversion process for the document.
14. The system of claim 1, wherein the at least one specified document conversion operation parameter, condition, or setting comprises a maximum number of attempts for conversion and wherein the isolation service running on the at least one server computer is configured to:
detect an error condition in the conversion of the document;
attempt to perform the conversion until the maximum number of attempts for conversion is reached; and
when the maximum number of attempts for conversion is reached, restart the native application.
15. The method of claim 3, wherein the at least one specified document conversion operation parameter, condition, or setting comprises an amount of memory allocation, the method further comprising:
the application isolation system allocating the amount of memory to the native application when the application isolation system starts the native application; and
the application isolation system restarting the native application when a process associated with the native application requests additional memory.
16. The method of claim 3, wherein the at least one specified document conversion operation parameter, condition, or setting comprises a maximum time for conversion, the method further comprising:
the application isolation system detecting that the native application is taking more time to convert the document than the maximum time for conversion; and
the application isolation system restarting the native application and a document conversion process for the document.
17. The method of claim 3, wherein the at least one specified document conversion operation parameter, condition, or setting comprises a maximum number of attempts for conversion, the method further comprising:
the application isolation system detecting an error condition in the conversion of the document;
the application isolation system attempting to perform the conversion until the maximum number of attempts for conversion is reached; and
when the maximum number of attempts for conversion is reached, the application isolation system restarting the native application.
18. The non-transitory computer-readable medium of claim 7, wherein the one or more specified document conversion operation parameters, conditions, or settings comprise an amount of memory allocation and wherein the computer-executable instructions when executed by the computer further cause the computer to perform:
allocating the amount of memory to the native application when starting the native application; and
restarting the native application when a process associated with the native application requests additional memory.
19. The non-transitory computer-readable medium of claim 7, wherein the one or more specified document conversion operation parameters, conditions, or settings comprise a maximum time for conversion and wherein the computer-executable instructions when executed by the computer further cause the computer to perform:
detecting that the native application is taking more time to convert the document than the maximum time for conversion; and
restarting the native application and a document conversion process for the document.
20. The non-transitory computer-readable medium of claim 7, wherein the one or more specified document conversion operation parameters, conditions, or settings comprise a maximum number of attempts for conversion and wherein the computer-executable instructions when executed by the computer further cause the computer to perform:
detecting an error condition in the conversion of the document;
attempting to perform the conversion until the maximum number of attempts for conversion is reached; and
when the maximum number of attempts for conversion is reached, restarting the native application.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A computer-implemented method of preventing a detection of web crawling, comprising:
receiving, by a randomizing HTTP proxy server coupled to a web crawling module, a first request from said web crawling module to scan a target website provided by a web server;
forwarding, by said randomizing HTTP proxy server, said first request to a first HTTP proxy computing unit of a plurality of HTTP proxy computing units coupled to said randomizing HTTP proxy server via a network, said first HTTP proxy computing unit utilizing a first router of a plurality of routers to forward said first request to said web server, said first router utilizing a first instance of a network address translation (NAT) algorithm that associates a first source Internet Protocol (IP) address with said first request;
randomly selecting, by said randomizing HTTP proxy server, a second HTTP proxy computing unit of said plurality of HTTP proxy computing units, said second HTTP proxy computing unit being different from said first HTTP proxy computing unit;
receiving, by said randomizing HTTP proxy server, a second request from said web crawling module to scan said target website; and
forwarding, by said randomizing HTTP proxy server, said second request to said second HTTP proxy computing unit, said second HTTP proxy computing unit utilizing a second router of said plurality of routers to forward said second request to said web server, said second router utilizing a second instance of said NAT algorithm to associate a second source IP address with said second request,
wherein said forwarding said first request and said forwarding said second request facilitate an identification by said web server of a difference between said first source IP address and said second source IP address.
2. The method of claim 1, further comprising:
establishing a first Transfer Control Protocol (TCP) connection between said web crawling module and said randomizing HTTP proxy server;
randomly selecting, by said randomizing HTTP proxy server and in response to said establishing said first TCP connection, said first HTTP proxy computing unit; and
establishing, subsequent to said randomly selecting said first HTTP proxy computing unit, a second TCP connection between said randomizing HTTP proxy server and said first HTTP proxy computing unit.
3. The method of claim 1, further comprising establishing, subsequent to said randomly selecting said second HTTP proxy computing unit, a TCP connection between said randomizing HTTP proxy server and said second HTTP proxy computing unit.
4. The method of claim 1, further comprising:
preventing an identification by said web server of said second request being included in a set of multiple requests to scan said target website, each request of said multiple requests being initiated by said web crawling module and having said web server as a destination, said set of multiple requests including said first request and said second request, wherein said preventing said identification includes utilizing said identification of said difference.
5. The method of claim 1, further comprising:
statically assigning an IP address of a plurality of IP addresses in a specified range to each HTTP proxy computing unit of said plurality of HTTP proxy computing units; and
generating, within each HTTP proxy computing unit of said plurality of HTTP proxy computing units, a static routing table of a plurality of static routing tables, said generating including dividing an IP address space into L segments, wherein L is greater than or equal to M, wherein M is a total number of IP addresses included in said plurality of IP addresses.
6. The method of claim 5, further comprising:
mapping said plurality of IP addresses to a first segment of said L segments, said first segment including a first plurality of publicly routable IP addresses; and
mapping said plurality of IP addresses to a second segment of said L segments, said second segment including a second plurality of publicly routable IP addresses,
wherein said first source IP address is included in said first segment and said second source IP address is included in said second segment,
wherein said mapping said plurality of IP addresses to said first segment includes using said first instance of said NAT algorithm, and
wherein said mapping said plurality of IP addresses to said second segment includes using said second instance of said NAT algorithm.
7. The method of claim 5, further comprising:
providing said IP address space by a plurality of Internet service providers (ISPs),
wherein each router of said plurality of routers provides routing to a different segment of said L segments, and
wherein each router of said plurality of routers is associated with a different ISP of said plurality of ISPs.
8. The method of claim 7, further comprising:
configuring a set of IP addresses associated with said plurality of routers to belong to a single segment of non-routable IP addresses, wherein said plurality of routers are in communication with said plurality of HTTP proxy computing units via a local area network (LAN), and wherein each IP address of said set of IP addresses is associated with an interface of a router of said plurality of routers, said interface being a LAN-facing interface with respect to said LAN,
wherein said first request is associated with a first default gateway designated by a first IP address of said set of IP addresses, and
wherein said second request is associated with a second default gateway designated by a second IP address of said set of IP addresses.
9. The method of claim 1, further comprising:
receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said first request, a first response to said first request via a communication between said first HTTP proxy computing unit and said web server;
sending, by said randomizing HTTP proxy server and in response to said receiving said first response, said first response to said web crawling module;
receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said second request, a second response to said second request via a communication between said second HTTP proxy computing unit and said web server; and
sending, by said randomizing HTTP proxy server and in response to said receiving said second response, said second response to said web crawling module.
10. A system for preventing a detection of web crawling in a computing environment, comprising:
means for receiving, by a randomizing HTTP proxy server coupled to a web crawling module, a first request from said web crawling module to scan a target website provided by a web server;
means for forwarding, by said randomizing HTTP proxy server, said first request to a first HTTP proxy computing unit of a plurality of HTTP proxy computing units coupled to said randomizing HTTP proxy server via a network, said first HTTP proxy computing unit utilizing a first router of a plurality of routers to forward said first request to said web server, said first router utilizing a first instance of a network address translation (NAT) algorithm that associates a first source Internet Protocol (IP) address with said first request;
means for randomly selecting, by said randomizing HTTP proxy server, a second HTTP proxy computing unit of said plurality of HTTP proxy computing units, said second HTTP proxy computing unit being different from said first HTTP proxy computing unit;
means for receiving, by said randomizing HTTP proxy server, a second request from said web crawling module to scan said target website; and
means for forwarding, by said randomizing HTTP proxy server, said second request to said second HTTP proxy computing unit, said second HTTP proxy computing unit utilizing a second router of said plurality of routers to forward said second request to said web server, said second router utilizing a second instance of said NAT algorithm to associate a second source IP address with said second request,
wherein said means for forwarding said first request and said means for forwarding said second request facilitate an identification by said web server of a difference between said first source IP address and said second source IP address.
11. The system of claim 10, further comprising:
means for establishing a first Transfer Control Protocol (TCP) connection between said web crawling module and said randomizing HTTP proxy server;
means for randomly selecting, by said randomizing HTTP proxy server and in response to said establishing said first TCP connection, said first HTTP proxy computing unit; and
means for establishing, subsequent to said randomly selecting said first HTTP proxy computing unit, a second TCP connection between said randomizing HTTP proxy server and said first HTTP proxy computing unit.
12. The system of claim 10, further comprising means for establishing, subsequent to said randomly selecting said second HTTP proxy computing unit, a TCP connection between said randomizing HTTP proxy server and said second HTTP proxy computing unit.
13. The system of claim 10, further comprising:
means for preventing an identification by said web server of said second request being included in a set of multiple requests to scan said target website, each request of said multiple requests being initiated by said web crawling module and having said web server as a destination, said set of multiple requests including said first request and said second request, wherein said means for preventing said identification includes means for utilizing said identification of said difference.
14. The system of claim 10, further comprising:
means for statically assigning an IP address of a plurality of IP addresses in a specified range to each HTTP proxy computing unit of said plurality of HTTP proxy computing units; and
means for generating, within each HTTP proxy computing unit of said plurality of HTTP proxy computing units, a static routing table of a plurality of static routing tables, said means for generating including means for dividing an IP address space into L segments, wherein L is greater than or equal to M, wherein M is a total number of IP addresses included in said plurality of IP addresses.
15. The system of claim 14, further comprising:
means for mapping said plurality of IP addresses to a first segment of said L segments, said first segment including a first plurality of publicly routable IP addresses; and
means for mapping said plurality of IP addresses to a second segment of said L segments, said second segment including a second plurality of publicly routable IP addresses,
wherein said first source IP address is included in said first segment and said second source IP address is included in said second segment,
wherein said means for mapping said plurality of IP addresses to said first segment includes means for using said first instance of said NAT algorithm, and
wherein said means for mapping said plurality of IP addresses to said second segment includes means for using said second instance of said NAT algorithm.
16. The system of claim 14, further comprising:
means for providing said IP address space by a plurality of Internet service providers (ISPs),
wherein each router of said plurality of routers provides routing to a different segment of said L segments, and
wherein each router of said plurality of routers is associated with a different ISP of said plurality of ISPs.
17. The system of claim 16, further comprising:
means for configuring a set of IP addresses associated with said plurality of routers to belong to a single segment of non-routable IP addresses, wherein said plurality of routers are in communication with said plurality of HTTP proxy computing units via a local area network (LAN), and wherein each IP address of said set of IP addresses is associated with an interface of a router of said plurality of routers, said interface being a LAN-facing interface with respect to said LAN,
wherein said first request is associated with a first default gateway designated by a first IP address of said set of IP addresses, and
wherein said second request is associated with a second default gateway designated by a second IP address of said set of IP addresses.
18. The system of claim 10, further comprising:
means for receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said first request, a first response to said first request via a communication between said first HTTP proxy computing unit and said web server;
means for sending, by said randomizing HTTP proxy server and in response to said receiving said first response, said first response to said web crawling module;
means for receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said second request, a second response to said second request via a communication between said second HTTP proxy computing unit and said web server; and
means for sending, by said randomizing HTTP proxy server and in response to said receiving said second response, said second response to said web crawling module.
19. A computer program product comprising a computer-usable medium including computer-usable program code for preventing a detection of web crawling in a computing environment, comprising:
computer-usable code for receiving, by a randomizing HTTP proxy server coupled to a web crawling module, a first request from said web crawling module to scan a target website provided by a web server;
computer-usable code for forwarding, by said randomizing HTTP proxy server, said first request to a first HTTP proxy computing unit of a plurality of HTTP proxy computing units coupled to said randomizing HTTP proxy server via a network, said first HTTP proxy computing unit utilizing a first router of a plurality of routers to forward said first request to said web server, said first router utilizing a first instance of a network address translation (NAT) algorithm that associates a first source Internet Protocol (IP) address with said first request;
computer-usable code for randomly selecting, by said randomizing HTTP proxy server, a second HTTP proxy computing unit of said plurality of HTTP proxy computing units, said second HTTP proxy computing unit being different from said first HTTP proxy computing unit;
computer-usable code for receiving, by said randomizing HTTP proxy server, a second request from said web crawling module to scan said target website; and
computer-usable code for forwarding, by said randomizing HTTP proxy server, said second request to said second HTTP proxy computing unit, said second HTTP proxy computing unit utilizing a second router of said plurality of routers to forward said second request to said web server, said second router utilizing a second instance of said NAT algorithm to associate a second source IP address with said second request,
wherein said computer-usable code for forwarding said first request and said computer-usable code for forwarding said second request facilitate an identification by said web server of a difference between said first source IP address and said second source IP address.
20. The program product of claim 19, further comprising:
computer-usable code for establishing a first Transfer Control Protocol (TCP) connection between said web crawling module and said randomizing HTTP proxy server;
computer-usable code for randomly selecting, by said randomizing HTTP proxy server and in response to said establishing said first TCP connection, said first HTTP proxy computing unit; and
computer-usable code for establishing, subsequent to said randomly selecting said first HTTP proxy computing unit, a second TCP connection between said randomizing HTTP proxy server and said first HTTP proxy computing unit.
21. The program product of claim 19, further comprising computer-usable code for establishing, subsequent to said randomly selecting said second HTTP proxy computing unit, a TCP connection between said randomizing HTTP proxy server and said second HTTP proxy computing unit.
22. The program product of claim 19, further comprising:
computer-usable code for preventing an identification by said web server of said second request being included in a set of multiple requests to scan said target website, each request of said multiple requests being initiated by said web crawling module and having said web server as a destination, said set of multiple requests including said first request and said second request, wherein said computer-usable code for preventing said identification includes computer-usable code for utilizing said identification of said difference.
23. The program product of claim 19, further comprising:
computer-usable code for statically assigning an IP address of a plurality of IP addresses in a specified range to each HTTP proxy computing unit of said plurality of HTTP proxy computing units; and
computer-usable code for generating, within each HTTP proxy computing unit of said plurality of HTTP proxy computing units, a static routing table of a plurality of static routing tables, said computer-usable code for generating including computer-usable code for dividing an IP address space into L segments, wherein L is greater than or equal to M, wherein M is a total number of IP addresses included in said plurality of IP addresses.
24. The program product of claim 23, further comprising:
computer-usable code for mapping said plurality of IP addresses to a first segment of said L segments, said first segment including a first plurality of publicly routable IP addresses; and
computer-usable code for mapping said plurality of IP addresses to a second segment of said L segments, said second segment including a second plurality of publicly routable IP addresses,
wherein said first source IP address is included in said first segment and said second source IP address is included in said second segment,
wherein said computer-usable code for mapping said plurality of IP addresses to said first segment includes computer-usable code for using said first instance of said NAT algorithm, and
wherein said computer-usable code for mapping said plurality of IP addresses to said second segment includes computer-usable code for using said second instance of said NAT algorithm.
25. The program product of claim 23, further comprising:
computer-usable code for providing said IP address space by a plurality of Internet service providers (ISPs),
wherein each router of said plurality of routers provides routing to a different segment of said L segments, and
wherein each router of said plurality of routers is associated with a different ISP of said plurality of ISPs.
26. The program product of claim 25, further comprising:
computer-usable code for configuring a set of IP addresses associated with said plurality of routers to belong to a single segment of non-routable IP addresses, wherein said plurality of routers are in communication with said plurality of HTTP proxy computing units via a local area network (LAN), and wherein each IP address of said set of IP addresses is associated with an interface of a router of said plurality of routers, said interface being a LAN-facing interface with respect to said LAN,
wherein said first request is associated with a first default gateway designated by a first IP address of said set of IP addresses, and
wherein said second request is associated with a second default gateway designated by a second IP address of said set of IP addresses.
27. The program product of claim 19, further comprising:
computer-usable code for receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said first request, a first response to said first request via a communication between said first HTTP proxy computing unit and said web server;
computer-usable code for sending, by said randomizing HTTP proxy server and in response to said receiving said first response, said first response to said web crawling module;
computer-usable code for receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said second request, a second response to said second request via a communication between said second HTTP proxy computing unit and said web server; and
computer-usable code for sending, by said randomizing HTTP proxy server and in response to said receiving said second response, said second response to said web crawling module.
28. A process for supporting computing infrastructure, said process comprising providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in a computing system, wherein the code in combination with the computing system is capable of performing a method of preventing a detection of web crawling in a computing environment, comprising:
receiving, by a randomizing HTTP proxy server coupled to a web crawling module, a first request from said web crawling module to scan a target website provided by a web server;
forwarding, by said randomizing HTTP proxy server, said first request to a first HTTP proxy computing unit of a plurality of HTTP proxy computing units coupled to said randomizing HTTP proxy server via a network, said first HTTP proxy computing unit utilizing a first router of a plurality of routers to forward said first request to said web server, said first router utilizing a first instance of a network address translation (NAT) algorithm that associates a first source Internet Protocol (IP) address with said first request;
randomly selecting, by said randomizing HTTP proxy server, a second HTTP proxy computing unit of said plurality of HTTP proxy computing units, said second HTTP proxy computing unit being different from said first HTTP proxy computing unit;
receiving, by said randomizing HTTP proxy server, a second request from said web crawling module to scan said target website; and
forwarding, by said randomizing HTTP proxy server, said second request to said second HTTP proxy computing unit, said second HTTP proxy computing unit utilizing a second router of said plurality of routers to forward said second request to said web server, said second router utilizing a second instance of said NAT algorithm to associate a second source IP address with said second request,
wherein said forwarding said first request and said forwarding said second request facilitate an identification by said web server of a difference between said first source IP address and said second source IP address.
29. The process of claim 28, wherein said method further comprises:
establishing a first Transfer Control Protocol (TCP) connection between said web crawling module and said randomizing HTTP proxy server;
randomly selecting, by said randomizing HTTP proxy server and in response to said establishing said first TCP connection, said first HTTP proxy computing unit; and
establishing, subsequent to said randomly selecting said first HTTP proxy computing unit, a second TCP connection between said randomizing HTTP proxy server and said first HTTP proxy computing unit.
30. The process of claim 28, wherein said method further comprises establishing, subsequent to said randomly selecting said second HTTP proxy computing unit, a TCP connection between said randomizing HTTP proxy server and said second HTTP proxy computing unit.
31. The process of claim 28, wherein said method further comprises:
preventing an identification by said web server of said second request being included in a set of multiple requests to scan said target website, each request of said multiple requests being initiated by said web crawling module and having said web server as a destination, said set of multiple requests including said first request and said second request, wherein said preventing said identification includes utilizing said identification of said difference.
32. The process of claim 28, wherein said method further comprises:
statically assigning an IP address of a plurality of IP addresses in a specified range to each HTTP proxy computing unit of said plurality of HTTP proxy computing units; and
generating, within each HTTP proxy computing unit of said plurality of HTTP proxy computing units, a static routing table of a plurality of static routing tables, said generating including dividing an IP address space into L segments, wherein L is greater than or equal to M wherein M is a total number of IP addresses included in said plurality of IP addresses.
33. The process of claim 32, wherein said method further comprises:
mapping said plurality of IP addresses to a first segment of said L segments, said first segment including a first plurality of publicly routable IP addresses; and
mapping said plurality of IP addresses to a second segment of said L segments, said second segment including a second plurality of publicly routable IP addresses,
wherein said first source IP address is included in said first segment and said second source IP address is included in said second segment,
wherein said mapping said plurality of IP addresses to said first segment includes using said first instance of said NAT algorithm, and
wherein said mapping said plurality of IP addresses to said second segment includes using said second instance of said NAT algorithm.
34. The process of claim 32, wherein said method further comprises:
providing said IP address space by a plurality of Internet service providers (ISPs),
wherein each router of said plurality of routers provides routing to a different segment of said L segments, and
wherein each router of said plurality of routers is associated with a different ISP of said plurality of ISPs.
35. The process of claim 34, wherein said method further comprises:
configuring a set of IP addresses associated with said plurality of routers to belong to a single segment of non-routable IP addresses, wherein said plurality of routers are in communication with said plurality of HTTP proxy computing units via a local area network (LAN), and wherein each IP address of said set of IP addresses is associated with an interface of a router of said plurality of routers, said interface being a LAN-facing interface with respect to said LAN,
wherein said first request is associated with a first default gateway designated by a first IP address of said set of IP addresses, and
wherein said second request is associated with a second default gateway designated by a second IP address of said set of IP addresses.
36. The process of claim 28, wherein said method further comprises:
receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said first request, a first response to said first request via a communication between said first HTTP proxy computing unit and said web server;
sending, by said randomizing HTTP proxy server and in response to said receiving said first response, said first response to said web crawling module;
receiving, at said randomizing HTTP proxy server and subsequent to said forwarding said second request, a second response to said second request via a communication between said second HTTP proxy computing unit and said web server; and
sending, by said randomizing HTTP proxy server and in response to said receiving said second response, said second response to said web crawling module.