httpclient 笔记

调试时想要打印出httpClient日志

httpClient 4.5.x

1
2
3
4
5
6
7
8
Context Logging    上下文日志
org.apache.http.impl.client

Wire Logging 传送和接收的数据日志
org.apache.http.wire

HTTP header Logging 请求头和响应头日志
org.apache.http.headers

参考httpcomponents-client-4.5.x logging.html

httpClient 3.x

1
2
3
4
5
6
7
8
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.showdatetime", "true");
// write logging
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire", "debug");
// header logging
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire.header", "debug");
// content logging
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.commons.httpclient", "debug");

参考httpclient-3.x logging.xml

使用slf4j+logback

1
2
3
4
<logger name="org.apache" level="DEBUG" />
<logger name="org.apache.http.client" level="DEBUG" />
<logger name="org.apache.http.wire" level="DEBUG" />
<logger name="org.apache.commons.httpclient" level="DEBUG" />

整个logback.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
<?xml version="1.0" encoding="UTF-8"?>

<configuration scan="true" scanPeriod="60 seconds" debug="false">
<!-- scan="true" scanPeriod="60 seconds" debug="false" -->
<!-- scan:当此属性设置为true时,配置文件如果发生改变,将会被重新加载,默认值为true。 -->
<!-- scanPeriod:设置监测配置文件是否有修改的时间间隔,如果没有给出时间单位,默认单位是毫秒当scan为true时,此属性生效。默认的时间间隔为1分钟。 -->
<!-- debug:当此属性设置为true时,将打印出logback内部日志信息,实时查看logback运行状态。默认值为false。 -->

<!--定义日志文件的存储地址 勿在 LogBack 的配置中使用相对路径-->
<property name="LOG_HOME" value="./logs"/>
<property name="LOG_LEVEL" value="DEBUG" />
<property name="appName" value="crawler"></property>

<!-- 控制台输出 -->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<!--格式化输出:%d表示日期,%thread表示线程名,%-5level:级别从左显示5个字符宽度%msg:日志消息,%n是换行符-->
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%logger{50}] [%line] - %msg%n</pattern>
</encoder>
</appender>

<!-- 按照每天生成日志文件 -->
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!--<file>${LOG_HOME}/${appName}log</file>-->
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!--日志文件输出的文件名 .%i -->
<FileNamePattern>${LOG_HOME}/${appName}.%d{yyyy-MM-dd}_%i.log</FileNamePattern>
<maxFileSize>10MB</maxFileSize>
<maxHistory>30</maxHistory>
<totalSizeCap>600MB</totalSizeCap>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<!--格式化输出:%d表示日期,%thread表示线程名,%-5level:级别从左显示5个字符宽度%msg:日志消息,%n是换行符-->
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level [%logger{50}] [%line] - %msg%n</pattern>
</encoder>
</appender>

<!-- 日志输出级别 -->
<root level="${LOG_LEVEL}">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE" />
</root>

<logger name="org.apache" level="DEBUG" />
<logger name="org.apache.http.client" level="DEBUG" />
<logger name="org.apache.http.wire" level="DEBUG" />
<logger name="org.apache.commons.httpclient" level="DEBUG" />

</configuration>

使用log4j

1
2
3
4
5
6
# httpclient 4.5.x
log4j.logger.org.apache.http.client=DEBUG
# httpclient 3.x
log4j.logger.org.apache.commons.httpclient=DEBUG
log4j.logger.httpclient.wire.header=DEBUG
log4j.logger.httpclient.wire=DEBUG

整个log4j.properties

1
2
3
4
5
6
7
8
9
10
11
log4j.rootLogger=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%c] %m%n

# httpclient 4.5.x
log4j.logger.org.apache.http.client=DEBUG
# httpclient 3.x
log4j.logger.org.apache.commons.httpclient=DEBUG
log4j.logger.httpclient.wire.header=DEBUG
log4j.logger.httpclient.wire=DEBUG

常见的Content-Type

1
2
3
4
Content-Type : application/x-www-form-urlencoded;charset=utf-8
Content-Type : multipart/form-data; boundary=----WebKitFormBoundaryrGKCBY7qhFd3TrwA
Content-Type : application/json; charset=UTF-8
Content-Type : text/xml

References

[1] office document
[2] source code
[3] 浅谈HttpClient
[4] Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
[5] 高并发场景下的httpClient优化使用
[6] 搜索系统17:HttpClient的网络连接是否被复用了
[7] httpclient 多线程高并发Get请求
[8] HttpClient模拟get,post请求并发送请求参数(json等)
[9] 常用的几种 Content-Type
[10] HTTP Content-Type常用一览表
[11] 关闭HttpClient控制台输出语句
[12] Spring boot 配置HttpClient 请求日志
[13] httpclient-3.x logging document
[14] httpcomponents-client-4.5.x logging document