Maven
Maven

Maven常见问题总结

IDEA中POM变更

IDEA中由于POM中未配置maven-compiler-plugin插件可能导致每次POM发生变更时，Language level变成5，从而导致编译时不成功。

1
2
3
4
5
6
7
8
9
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.8.0</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
    </configuration>
</plugin>
1
2
3
4
5
6
<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <java.version>8</java.version>
    <maven.compiler.source>8</maven.compiler.source>
    <maven.compiler.target>8</maven.compiler.target>
</properties>

Maven编译后文件损坏

在一般Maven项目中的build标签下通常会有如下配置：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<resources>
    <resource>
        <directory>src/main/resources</directory>
        <includes>
            <include>**/*.properties</include>
            <include>**/*.yml</include>
            <include>**/*.*</include>
        </includes>
        <excludes>
            <exclude>application-test.yml</exclude>
        </excludes>
        <filtering>true</filtering>
    </resource>
    <resource>
        <directory>src/main/java</directory>
        <includes>
            <include>**/*.xml</include>
        </includes>
        <filtering>true</filtering>
    </resource>
</resources>

以上的配置本身是没有问题的，但在某些需要需要将一些文件存放在项目中直接提供下载时，以上配置就回有问题，如果是将文件放到resources目录下，由于filtering设置为true，开启了过滤，会用指定的参数替换directory下的文件中的参数(eg. ${name})。从而导致文件可能会损坏。

若配置文件需要传参数将filtering设置为false显然是不行的。使用exclude排除想下载的文件打包时都不会将文件打到jar包中，显然也是不行的。然后尝试做了如下调整，但并没有生效：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<resource>
    <directory>src/main/resources</directory>
    <includes>
        <include>**/*.properties</include>
        <include>**/*.yml</include>
        <include>**/*.*</include>
    </includes>
    <excludes>
        <exclude>application-test.yml</exclude>
        <exclude>src/main/resources/download</exclude>
    </excludes>
    <filtering>true</filtering>
</resource>
<resource>
    <directory>src/main/resources/download</directory>
    <includes>
        <include>**/*.*</include>
    </includes>
    <filtering>false</filtering>
</resource>

最终只能将下载的文件单独放到其他目录，然后做如下配置：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<resources>
    <resource>
        <directory>src/main/resources</directory>
        <includes>
            <include>**/*.properties</include>
            <include>**/*.yml</include>
            <include>**/*.*</include>
        </includes>
        <excludes>
            <exclude>application-test.yml</exclude>
        </excludes>
        <filtering>true</filtering>
    </resource>
    <resource>
        <directory>download</directory>
        <includes>
            <include>**/*.*</include>
        </includes>
        <filtering>false</filtering>
    </resource>
    <resource>
        <directory>src/main/java</directory>
        <includes>
            <include>**/*.xml</include>
        </includes>
        <filtering>true</filtering>
    </resource>
</resources>

HBase依赖冲突

远程连接

maven依赖：

1
2
3
4
5
<dependency>
    <groupId>org.apache.hbase</groupId>
    <artifactId>hbase-client</artifactId>
    <version>2.2.0</version>
</dependency>

连接代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
@Bean
public Connection hbaseConnection() throws IOException {
    org.apache.hadoop.conf.Configuration conf = HBaseConfiguration.create();
    conf.set("hbase.zookeeper.quorum", "127.0.0.1");
    conf.set("hbase.zookeeper.property.clientPort", "2181");
    conf.set("zookeeper.znode.parent", "/hbase-unsecure");
    Connection connection = ConnectionFactory.createConnection(conf);
    return connection;
}

public String query(String tableName, String familyName, String columnName, String qualifier) throws Exception {
    TableName name = TableName.valueOf(tableName);
    Table table = hbaseConnection.getTable(name);
    Get get = new Get(qualifier.getBytes());
    get.addColumn(Bytes.toBytes(familyName), Bytes.toBytes(columnName));
    Result rs = table.get(get);
    return Bytes.toString(rs.value());
}

hive依赖包冲突

在项目中同时存在hive和hbase时，hive中引用的guava与hbase中引用的guava版本冲突，从而导致执行hbase查询时报错。

1
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator

Stopwatch在google的guava包下，hbase1.1.2只在guava12-16下能正常运行。guava17开始，出现以上异常。尝试着将guava版本降低，后启动项目后报如下错误：

1
2
3
An attempt was made to call the method com.google.common.base.Splitter.splitToList(Ljava/lang/CharSequence;)Ljava/lang/List; but it does not exist. Its class, com.google.common.base.Splitter, is available from the following locations:
	jar:file:/mvnRespo/org/apache/hive/hive-exec/1.2.1/hive-exec-1.2.1.jar!/com/google/common/base/Splitter.class
    jar:file:/mvnRespo/com/google/guava/guava/16.0/guava-16.0.jar!/com/google/common/base/Splitter.class

由此可见hive必须依赖高版本的guava才行，然后就尝试着将hbase的依赖版本升级到1.3.0，问题得到了解决。

tablestore依赖包冲突

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
@Value(value = "${aliyun.endpoint:http}")
private String endPoint;
@Value(value = "${aliyun.accessKeyId:test}")
private String accessKeyId;
@Value(value = "${aliyun.accessSecret:test}")
private String accessSecret;
@Value(value = "${aliyun.instanceName:test}")
private String instanceName;

private SyncClient syncClient;

@PostConstruct
public void init() {
    syncClient = new SyncClient(endPoint, accessKeyId, accessSecret, instanceName);
}

public String query(String key, String value, String tableName, String columnName) {
    PrimaryKeyBuilder primaryKeyBuilder = PrimaryKeyBuilder.createPrimaryKeyBuilder();
    primaryKeyBuilder.addPrimaryKeyColumn(key, PrimaryKeyValue.fromString(value));
    PrimaryKey primaryKey = primaryKeyBuilder.build();
    GetRowRequest request = new GetRowRequest();
    SingleRowQueryCriteria singleRowQueryCriteria = new SingleRowQueryCriteria(tableName, primaryKey);
    singleRowQueryCriteria.setMaxVersions(1);
    request.setRowQueryCriteria(singleRowQueryCriteria);
    GetRowResponse response = syncClient.getRow(request);
    Row row = response.getRow();
    return row.getLatestColumn(columnName).getValue().asString();
}

在项目中同时存在tablestore和hbase时，tablestore中引用的protobuf版本2.4.1与hbase中引用的protobuf版本2.5.0版本冲突，从而导致执行hbase查询时报错。

1
Caused by: java.lang.VerifyError: class org.apache.hadoop.hbase.protofuf.generated.ClientProtos$Result overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;

首先想到的是引入高版本的protobuf。但是引入高版本的protobuf后查询阿里的彩虹表tablestore时会报另外的错误。

1
java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.

尝试着将hbase的版本升高到2.2.0，然后发现问题解决了，然后继续查看了一下protobuf依赖冲突版本，发现多了一个hbase-shaded-protobuf,原来在该版本中使用了hbase-shaded , 用来更改hbase中的一些报名，解决protobuf的冲突问题。