Apache HTTP Server Version 2.5
Description: | Multiplexes all I/O to a given file or pipe. |
---|---|
Status: | Extension |
Module Identifier: | firehose_module |
Source File: | mod_firehose.c |
mod_firehose
provides a mechanism to record data
being passed between the httpd server and the client at the raw
connection level to either a file or a pipe in such a way that the
data can be analysed or played back to the server at a future date.
It can be thought of as "tcpdump for httpd".
Connections are recorded after the SSL has been stripped, and can be used for forensic debugging.
The firehose
tool can be used to demultiplex
the recorded stream back into individual files for analysis, or
playback using a tool like netcat
.
To enable the module, it should be compiled and loaded in to your running Apache configuration, and the directives below used to record the data you are interested in.
It is possible to record both incoming and outgoing data to the same filename if desired, as the direction of flow is recorded within each fragment.
It is possible to write to both normal files and fifos (pipes). In the case of fifos, mod_firehose ensures that the packet size is no larger than PIPE_BUF to ensure writes are atomic.
If a pipe is being used, something must be reading from the pipe before httpd is started for the pipe to be successfully opened for write. If the request to open the pipe fails, mod_firehose will silently stand down and not record anything, and the server will keep running as normal.
By default, all attempts to write will block the server. If the webserver has been built against APR v2.0 or later, and an optional "nonblock" parameter is specified all file writes will be non blocking, and buffer overflows will cause debugging data to be lost. In this case it is possible to prioritise the running of the server over the recording of firehose data.
The server typically serves multiple connections simultaneously, and as a result requests and responses need to be multiplexed before being written to the firehose.
The fragment format is designed as clear text, so that a firehose
can be opened with and inspected by a normal text editor.
Alternatively, the firehose
tool can be used to
demultiplex the firehose back into individual requests or
connections.
The size of the multiplexed fragments is governed by PIPE_BUF, the maximum size of write the system is prepared to perform atomically. By keeping the multiplexed fragments below PIPE_BUF in size, the module guarantees that data from different fragments does not interleave. The size of PIPE_BUF varies on different operating systems.
The BNF for the fragment format is as follows:
stream = 0*(fragment) fragment = header CRLF body CRLF header = length SPC timestamp SPC ( request | response ) SPC uuid SPC count length = <up to 16 byte hex fragment length> timestamp = <up to 16 byte hex timestamp microseconds since 1970> request = "<" response = ">" uuid = <formatted uuid of the connection> count = <hex fragment number in the connection> body = <the binary content of the fragment> SPC = <a single space> CRLF = <a carriage return, followed by a line feed>
All fragments for a connection or a request will share the same UUID, depending on whether connections or requests are being recorded. If connections are being recorded, multiple requests may appear within a connection. A fragment with a zero length indicates the end of the connection.
Fragments may go missing or be dropped if the process reading the fragments is too slow. If this happens, gaps will exist in the connection counter numbering. A warning will be logged in the error log to indicate the UUID and counter of the dropped fragment, so it can be confirmed the fragment was dropped.
It is possible that the terminating empty fragment may not appear, caused by the httpd process crashing, or being terminated ungracefully. The terminating fragment may be dropped if the process reading the fragments is not fast enough.
Description: | Capture traffic coming into the server on each connection |
---|---|
Syntax: | FirehoseConnectionInput [ block | nonblock ] filename |
Default: | none |
Context: | server config |
Status: | Extension |
Module: | mod_firehose |
Compatibility: | FirehoseConnectionInput is only available in Apache 2.5.0 and later. |
Capture traffic coming into the server on each connection. Multiple requests will be captured within the same connection if keepalive is present.
FirehoseConnectionInput connection-input.firehose
Description: | Capture traffic going out of the server on each connection |
---|---|
Syntax: | FirehoseConnectionOutput [ block | nonblock ] filename |
Default: | none |
Context: | server config |
Status: | Extension |
Module: | mod_firehose |
Compatibility: | FirehoseConnectionOutput is only available in Apache 2.5.0 and later. |
Capture traffic going out of the server on each connection. Multiple requests will be captured within the same connection if keepalive is present.
FirehoseConnectionOutput connection-output.firehose
Description: | Capture traffic coming into the back of mod_proxy |
---|---|
Syntax: | FirehoseProxyConnectionInput [ block | nonblock ] filename |
Default: | none |
Context: | server config |
Status: | Extension |
Module: | mod_firehose |
Compatibility: | FirehoseProxyConnectionInput is only available in Apache 2.5.0 and later. |
Capture traffic being received by mod_proxy.
FirehoseProxyConnectionInput proxy-input.firehose
Description: | Capture traffic sent out from the back of mod_proxy |
---|---|
Syntax: | FirehoseProxyConnectionOutput [ block | nonblock ] filename |
Default: | none |
Context: | server config |
Status: | Extension |
Module: | mod_firehose |
Compatibility: | FirehoseProxyConnectionOutput is only available in Apache 2.5.0 and later. |
Capture traffic being sent out by mod_proxy.
FirehoseProxyConnectionOutput proxy-output.firehose
Description: | Capture traffic coming into the server on each request |
---|---|
Syntax: | FirehoseRequestInput [ block | nonblock ] filename |
Default: | none |
Context: | server config |
Status: | Extension |
Module: | mod_firehose |
Compatibility: | FirehoseRequestInput is only available in Apache 2.5.0 and later. |
Capture traffic coming into the server on each request. Requests will be captured separately, regardless of the presence of keepalive.
FirehoseRequestInput request-input.firehose
Description: | Capture traffic going out of the server on each request |
---|---|
Syntax: | FirehoseRequestOutput [ block | nonblock ] filename |
Default: | none |
Context: | server config |
Status: | Extension |
Module: | mod_firehose |
Compatibility: | FirehoseRequestOutput is only available in Apache 2.5.0 and later. |
Capture traffic going out of the server on each request. Requests will be captured separately, regardless of the presence of keepalive.
FirehoseRequestOutput request-output.firehose