Java Client
Java client library to communicate with a DB server through its protocols. The current implementation only supports the HTTP interface. The library provides its own API to send requests to a server. The library also provides tools to work with different binary data formats (RowBinary* & Native*).
Setup
- Maven Central (project web page): https://mvnrepository.com/artifact/com.clickhouse/client-v2
- Nightly builds (repository link): https://s01.oss.sonatype.org/content/repositories/snapshots/com/clickhouse/
- Maven
- Gradle (Kotlin)
- Gradle
Initialization
The Client object is initialized by com.clickhouse.client.api.Client.Builder#build(). Each client has its own context and no objects are shared between them.
The Builder has configuration methods for convenient setup.
Example:
Client is AutoCloseable and should be closed when not needed anymore.
Authentication
Authentication is configured per client at the initialization phase. There are three authentication methods supported: by password, by access token, by SSL Client Certificate.
Authentication by a password requires setting user name password by calling setUsername(String) and setPassword(String):
Authentication by an access token requires setting access token by calling setAccessToken(String):
Authentication by a SSL Client Certificate require setting username, enabling SSL Authentication, setting a client sertificate and a client key by calling setUsername(String), useSSLAuthentication(boolean), setClientCertificate(String) and setClientKey(String) accordingly:
SSL Authentication may be hard to troubleshoot on production because many errors from SSL libraries provide not enough information. For example, if client certificate and key do not match then server will terminate connection immediately (in case of HTTP it will be connection initiation stage where no HTTP requests are send so no response is sent).
Please use tools like openssl to verify certificates and keys:
- check key integrity:
openssl rsa -in [key-file.key] -check -noout - check client certificate has matching CN for a user:
- get CN from an user certificate -
openssl x509 -noout -subject -in [user.cert] - verify same value is set in database
select name, auth_type, auth_params from system.users where auth_type = 'ssl_certificate'(query will outputauth_paramswith something like{"common_names":["some_user"]})
- get CN from an user certificate -
Configuration
All settings are defined by instance methods (a.k.a configuration methods) that make the scope and context of each value clear. Major configuration parameters are defined in one scope (client or operation) and do not override each other.
Configuration is defined during client creation. See com.clickhouse.client.api.Client.Builder.
Client Configuration
| Configuration Method | Arguments | Description |
|---|---|---|
addEndpoint(String endpoint) | - enpoint - URL formatted a server address. | Adds a server endpoint to list of available servers. Currently only one endpoint is supported. Default: none Enum: none Key: none |
addEndpoint(Protocol protocol, String host, int port, boolean secure) | - protocol - connection protocol com.clickhouse.client.api.enums.Protocol#HTTP.- host - IP or hostname of a server.- secure - if communication should use secure version of the protocol (HTTPS) | Adds a server endpoint to list of available servers. Currently only one endpoint is supported. Default: none Enum: none Key: none |
setOption(String key, String value) | - key - String key of the client configuration option.- value - String value of the option | Sets raw value of client options. Useful when reading configuration from properties files. |
setUsername(String username) | - username - User's username to use while authentication | Sets username for an authentication method that is selected by further configuration Default: default Enum: ClientConfigProperties.USER Key: user |
setPassword(String password) | - password - secret value for password authentication | Sets a secret for password authentication and effectively selects as authentication method Default: - Enum: ClientConfigProperties.PASSWORD Key: password |
setAccessToken(String accessToken) | - accessToken - String representation of an access token | Sets an access token to authenticate with a sets corresponding authentication method Default: - Enum: ClientConfigProperties.ACCESS_TOKEN Key: access_token |
useSSLAuthentication(boolean useSSLAuthentication) | - useSSLAuthentication - flag that indicates if SSL auth should be used | Sets SSL Client Certificate as an authentication method. Default: - Enum: ClientConfigProperties.SSL_AUTH Key: ssl_authentication |
enableConnectionPool(boolean enable) | - enable - flag that indicates if the option should be enabled | Sets if a connection pool is enabled Default: true Enum: ClientConfigProperties.CONNECTION_POOL_ENABLED Key: connection_pool_enabled |
setConnectTimeout(long timeout, ChronoUnit unit) | - timeout - timeout in some time unit.- unit - time unit of the timeout | Sets connection initiation timeout for any outgoing connection. This affects time wait on getting socket connect. Default: - Enum: ClientConfigProperties.CONNECTION_TIMEOUT Key: connection_timeout |
setConnectionRequestTimeout(long timeout, ChronoUnit unit) | - timeout - timeout in some time unit.- unit - time unit of the timeout | Sets connection request timeout. This take effect only for getting connection from a pool. Default: 10000 Enum: ClientConfigProperties.CONNECTION_REQUEST_TIMEOUT Key: connection_request_timeout |
setMaxConnections(int maxConnections) | - maxConnections - number of connections | Sets how many connections can a client open to each server endpoint. Default: 10 Enum: ClientConfigProperties.HTTP_MAX_OPEN_CONNECTIONS Key: max_open_connections |
setConnectionTTL(long timeout, ChronoUnit unit) | - timeout - timeout in some time unit.- unit - time unit of the timeout | Sets connection TTL after which connection will be considered as not active Default: -1 Enum: ClientConfigProperties.CONNECTION_TTL Key: connection_ttl |
setKeepAliveTimeout(long timeout, ChronoUnit unit) | - timeout - timeout in some time unit.- unit - time unit of the timeout | Sets HTTP connection keep-alive timeout. This option may be used to disable Keep-Alive by setting timeout to zero - 0 Default: - Enum: ClientConfigProperties.HTTP_KEEP_ALIVE_TIMEOUT Key: http_keep_alive_timeout |
setConnectionReuseStrategy(ConnectionReuseStrategy strategy) | - strategy - enum com.clickhouse.client.api.ConnectionReuseStrategy constant | Selects which strategy connection pool should use: LIFO if connection should be reused as soon as they are returned to a pool or FIFO to use connection in the order they become available (returned connection are not used immediately). Default: FIFO Enum: ClientConfigProperties.CONNECTION_REUSE_STRATEGY Key: connection_reuse_strategy |
setSocketTimeout(long timeout, ChronoUnit unit)` | - timeout - timeout in some time unit.- unit - time unit of the timeout | Sets socket timeout that affects read and write operations Default: 0 Enum: ClientConfigProperties.SOCKET_OPERATION_TIMEOUT Key: socket_timeout |
setSocketRcvbuf(long size) | - size - size in bytes | Sets TCP socket receive buffer. This buffer out of the JVM memory. Default: 8196 Enum: ClientConfigProperties.SOCKET_RCVBUF_OPT Key: socket_rcvbuf |
setSocketSndbuf(long size) | - size - size in bytes | Sets TCP socket receive buffer. This buffer out of the JVM memory. Default: 8196 Enum: ClientConfigProperties.SOCKET_SNDBUF_OPT Key: socket_sndbuf |
setSocketKeepAlive(boolean value) | - value - flag that indicates if option should be enabled. | Sets option SO_KEEPALIVE for every TCP socket created by the client. TCP Keep Alive enables mechanism that will check liveness of the connection and will help to detect abruptly terminated ones. Default: - Enum: ClientConfigProperties.SOCKET_KEEPALIVE_OPT Key: socket_keepalive |
setSocketTcpNodelay(boolean value) | - value - flag that indicates if option should be enabled. | Sets option SO_NODELAY for every TCP socket created by the client. This TCP option will make socket to push data as soon as possible. Default: - Enum: ClientConfigProperties.SOCKET_TCP_NO_DELAY_OPT Key: socket_tcp_nodelay |
setSocketLinger(int secondsToWait) | - secondsToWait - number of seconds. | Set linger time for every TCP socket created by the client. Default: - Enum: ClientConfigProperties.SOCKET_LINGER_OPT Key: socket_linger |
compressServerResponse(boolean enabled) | - enabled - flag that indicates if the option should be enabled | Sets if server should compress its responses. Default: true Enum: ClientConfigProperties.COMPRESS_SERVER_RESPONSE Key: compress |
compressClientRequest(boolean enabled) | - enabled - flag that indicates if the option should be enabled | Sets if client should compress its requests. Default: false Enum: ClientConfigProperties.COMPRESS_CLIENT_REQUEST Key: decompress |
useHttpCompression(boolean enabled) | - enabled - flag that indicates if the option should be enabled | Sets if HTTP compression should be used for client/server communications if corresponding options are enabled |
appCompressedData(boolean enabled) | - enabled - flag that indicates if the option should be enabled | Tell client that compression will be handled by application. Default: false Enum: ClientConfigProperties.APP_COMPRESSED_DATA Key: app_compressed_data |
setLZ4UncompressedBufferSize(int size) | - size - size in bytes | Sets size of a buffer that will receive uncompressed portion of a data stream. If buffer is underestimated - a new one will be created and corresponding warning will be present in logs. Default: 65536 Enum: ClientConfigProperties.COMPRESSION_LZ4_UNCOMPRESSED_BUF_SIZE Key: compression.lz4.uncompressed_buffer_size |
disableNativeCompression | - disable - flag that indicates if the option should be disabled | Disable native compression. If set to true then native compression will be disabled. Default: false Enum: ClientConfigProperties.DISABLE_NATIVE_COMPRESSION Key: disable_native_compression |
setDefaultDatabase(String database) | - database - name of a database | Sets default database. Default: default Enum: ClientConfigProperties.DATABASE Key: database |
addProxy(ProxyType type, String host, int port) | - type - proxy type.- host - proxy host name or IP Address.- port - proxy port | Sets proxy to be used for communication with a server. Setting proxy is required if proxy requires authentication. Default: - Enum: ClientConfigProperties.PROXY_TYPE Key: proxy_type Default: - Enum: ClientConfigProperties.PROXY_HOST Key: proxy_host Default: - Enum: ClientConfigProperties.PROXY_PORT Key: proxy_port |
setProxyCredentials(String user, String pass) | - user - proxy username.- pass - password | Sets user credentials to authenticate with a proxy. Default: - Enum: ClientConfigProperties.PROXY_USER Key: proxy_user Default: - Enum: ClientConfigProperties.PROXY_PASSWORD Key: proxy_password |
setExecutionTimeout(long timeout, ChronoUnit timeUnit) | - timeout - timeout in some time unit.- timeUnit - time unit of the timeout | Sets maximum execution timeout for queries Default: 0 Enum: ClientConfigProperties.MAX_EXECUTION_TIME Key: max_execution_time |
setHttpCookiesEnabled(boolean enabled) | enabled - flag that indicates if the option should be enabled | Set if HTTP cookies should be remembered and sent to server back. |
setSSLTrustStore(String path) | path - file path on local (client side) system | Sets if client should use SSL truststore for server host validation. Default: - Enum: ClientConfigProperties.SSL_TRUST_STORE Key: trust_store |
setSSLTrustStorePassword(String password) | password - secret value | Sets password to be used to unlock SSL truststore specified by setSSLTrustStore(String path) Default: - Enum: ClientConfigProperties.SSL_KEY_STORE_PASSWORD Key: key_store_password |
setSSLTrustStoreType(String type) | type - truststore type name | Sets type of the truststore specified by setSSLTrustStore(String path). Default: - Enum: ClientConfigProperties.SSL_KEYSTORE_TYPE Key: key_store_type |
setRootCertificate(String path) | path - file path on local (client side) system | Sets if client should use specified root (CA) certificate for server host to validation. Default: - Enum: ClientConfigProperties.CA_CERTIFICATE Key: sslrootcert |
setClientCertificate(String path) | path - file path on local (client side) system | Sets client certificate path to be used while initiating SSL connection and to be used by SSL authentication. Default: - Enum: ClientConfigProperties.SSL_CERTIFICATE Key: sslcert |
setClientKey(String path) | path - file path on local (client side) system | Sets client private key to be used for encrypting SSL communication with a server. Default: - Enum: ClientConfigProperties.SSL_KEY Key: ssl_key |
useServerTimeZone(boolean useServerTimeZone) | useServerTimeZone - flag that indicates if the option should be enabled | Sets if client should use server timezone when decoding DateTime and Date column values. If enabled then server timezone should be set by setServerTimeZone(String timeZone) Default: true Enum: ClientConfigProperties.USE_SERVER_TIMEZONE Key: use_server_time_zone |
useTimeZone(String timeZone) | timeZone - string value of java valid timezone ID (see java.time.ZoneId) | Sets if specified timezone should be used when decoding DateTime and Date column values. Will override server timezone. Default: - Enum: ClientConfigProperties.USE_TIMEZONE Key: use_time_zone |
setServerTimeZone(String timeZone) | timeZone - string value of java valid timezone ID (see java.time.ZoneId) | Sets server side timezone. UTC timezone will be used by default. Default: UTC Enum: ClientConfigProperties.SERVER_TIMEZONE Key: server_time_zone |
useAsyncRequests(boolean async) | async - flag that indicates if the option should be enabled. | Sets if client should execute request in a separate thread. Disabled by default because application knows better how to organize multi-threaded tasks and running tasks in separate thread do not help with performance. Default: false Enum: ClientConfigProperties.ASYNC_OPERATIONS Key: async |
setSharedOperationExecutor(ExecutorService executorService) | executorService - instance of executor service. | Sets executor service for operation tasks. Default: none Enum: none Key: none |
setClientNetworkBufferSize(int size) | - size - size in bytes | Sets size of a buffer in application memory space that is used to copy data back-and-forth between socket and application. Greater reduces system calls to TCP stack, but affects how much memory is spent on every connection. This buffer is also subject for GC because connections are shortlive. Also keep in mind that allocating big continious block of memory might be a problem. Default: 300000 Enum: ClientConfigProperties.CLIENT_NETWORK_BUFFER_SIZE Key: client_network_buffer_size |
retryOnFailures(ClientFaultCause ...causes) | - causes - enum constant of com.clickhouse.client.api.ClientFaultCause | Sets recoverable/retriable fault types. Default: NoHttpResponse,ConnectTimeout,ConnectionRequestTimeout Enum: ClientConfigProperties.CLIENT_RETRY_ON_FAILURE Key: client_retry_on_failures |
setMaxRetries(int maxRetries) | - maxRetries - number of retries | Sets maximum number of retries for failures defined by retryOnFailures(ClientFaultCause ...causes) Default: 3 Enum: ClientConfigProperties.RETRY_ON_FAILURE Key: retry |
allowBinaryReaderToReuseBuffers(boolean reuse) | - reuse - flag that indicates if the option should be enabled | Most datasets contain numeric data encoded as small byte sequences. By default reader will allocate required buffer, read data into it and then transform into a target Number class. That may cause significant GC preasure because of many small objects are being allocated and released. If this option is enabled then reader will use preallocated buffers to do numbers transcoding. It is safe because each reader has own set of buffers and readers are used by one thread. |
httpHeader(String key, String value) | - key - HTTP header key.- value - string value of the header. | Sets value for a single HTTP header. Previous value is overridden. Default: none Enum: none Key: none |
httpHeader(String key, Collection values) | - key - HTTP header key.- values - list of string values. | Sets values for a single HTTP header. Previous value is overridden. Default: none Enum: none Key: none |
httpHeaders(Map headers) | - header - map with HTTP headers and their values. | Sets multiple HTTP header values at a time. Default: none Enum: none Key: none |
serverSetting(String name, String value) | - name - name of a query level setting.- value - string value of the setting. | Sets what settings to pass to server along with each query. Individual operation settings may override it. The List of settings Default: none Enum: none Key: none |
serverSetting(String name, Collection values) | - name - name of a query level setting.- values - string values of the setting. | Sets what settings to pass to server along with each query. Individual operation settings may override it. The List of settings. This method is useful to set settings with multiple values, for example roles Default: none Enum: none Key: none |
columnToMethodMatchingStrategy(ColumnToMethodMatchingStrategy strategy) | - strategy - implementation of a column-field matching strategy | Sets custom strategy to be used for matching DTO class fields and DB columns when registering DTO. Default: none Enum: none Key: none |
useHTTPBasicAuth(boolean useBasicAuth) | - useBasicAuth - flag that indicates if the option should be enabled | Sets if basic HTTP authentication should be used for user-password authentication. Default is enabled. Using this type of authentication resolves issues with passwords containing special characters that cannot be transferred over HTTP headers. Default: true Enum: ClientConfigProperties.HTTP_USE_BASIC_AUTH Key: http_use_basic_auth |
setClientName(String clientName) | - clientName - a string representing application name | Sets additional information about calling application. This string will be passed to server as a client name. In case of HTTP protocol it will be passed as a User-Agent header. Default: - Enum: ClientConfigProperties.CLIENT_NAME Key: client_name |
useBearerTokenAuth(String bearerToken) | - bearerToken - an encoded bearer token | Specifies whether to use Bearer Authentication and what token to use. The token will be sent as is, so it should be encoded before passing to this method. Default: - Enum: ClientConfigProperties.BEARERTOKEN_AUTH Key: bearer_token |
Server Settings
Server side settings can be set on the client level once while creation (see serverSetting method of the Builder) and on operation level (see serverSetting for operation settings class).
When options are set via setOption method (either the Client.Builder or operation settings class) then server settings name should be prefixed with clickhouse_setting_. The com.clickhouse.client.api.ClientConfigProperties#serverSetting() may be handy in this case.
Custom HTTP Header
Custom HTTP headers can be set for all operations (client level) or a single one (operation level).
When options are set via setOption method (either the Client.Builder or operation settings class) then custom header name should be prefixed with http_header_. Method com.clickhouse.client.api.ClientConfigProperties#httpHeader() may be handy in this case.
Common Definitions
ClickHouseFormat
Enum of supported formats. It includes all formats that ClickHouse supports.
raw- user should transcode raw datafull- the client can transcode data by itself and accepts a raw data stream-- operation not supported by ClickHouse for this format
This client version supports:
Insert API
insert(String tableName, InputStream data, ClickHouseFormat format)
Accepts data as an InputStream of bytes in the specified format. It is expected that data is encoded in the format.
Signatures
Parameters
tableName - a target table name.
data - an input stream of an encoded data.
format - a format in which the data is encoded.
settings - request settings.
Return value
Future of InsertResponse type - result of the operation and additional information like server side metrics.
Examples
insert(String tableName, List<?> data, InsertSettings settings)
Sends a write request to database. The list of objects is converted into an efficient format and then is sent to a server. The class of the list items should be registed up-front using register(Class, TableSchema) method.
Signatures
Parameters
tableName - name of the target table.
data - collection DTO (Data Transfer Object) objects.
settings - request settings.
Return value
Future of InsertResponse type - the result of the operation and additional information like server side metrics.
Examples
InsertSettings
Configuration options for insert operations.
Configuration methods
| Method | Description |
|---|---|
setQueryId(String queryId) | Sets query ID that will be assigned to the operation. Default: null. |
setDeduplicationToken(String token) | Sets the deduplication token. This token will be sent to the server and can be used to identify the query. Default: null. |
setInputStreamCopyBufferSize(int size) | Copy buffer size. The buffer is used during write operations to copy data from user-provided input stream to an output stream. Default: 8196. |
serverSetting(String name, String value) | Sets individual server settings for an operation. |
serverSetting(String name, Collection values) | Sets individual server settings with multiple values for an operation. Items of the collection should be String values. |
setDBRoles(Collection dbRoles) | Sets DB roles to be set before executing an operation. Items of the collection should be String values. |
setOption(String option, Object value) | Sets a configuration option in raw format. This is not a server setting. |
InsertResponse
Response object that holds result of insert operation. It is only available if the client got response from a server.
This object should be closed as soon as possible to release a connection because the connection cannot be re-used until all data of previous response is fully read.
| Method | Description |
|---|---|
OperationMetrics getMetrics() | Returns object with operation metrics. |
String getQueryId() | Returns query ID assigned for the operation by the application (through operation settings or by server). |
Query API
query(String sqlQuery)
Sends sqlQuery as is. Response format is set by query settings. QueryResponse will hold a reference to the response stream that should be consumed by a reader for the supportig format.
Signatures
Parameters
sqlQuery - a single SQL statement. The Query is sent as is to a server.
settings - request settings.
Return value
Future of QueryResponse type - a result dataset and additional information like server side metrics. The Response object should be closed after consuming the dataset.
Examples
query(String sqlQuery, Map<String, Object> queryParams, QuerySettings settings)
Sends sqlQuery as is. Additionally will send query parameters so the server can compile the SQL expression.
Signatures
Parameters
sqlQuery - sql expression with placeholders {}.
queryParams - map of variables to complete the sql expression on server.
settings - request settings.
Return value
Future of QueryResponse type - a result dataset and additional information like server side metrics. The Response object should be closed after consuming the dataset.
Examples
queryAll(String sqlQuery)
Queries a data in RowBinaryWithNamesAndTypes format. Returns the result as a collection. Read performance is the same as with the reader but more memory is required to hold the whole dataset.
Signatures
Parameters
sqlQuery - sql expression to query data from a server.
Return value
Complete dataset represented by a list of GenericRecord objects that provide access in row style for the result data.
Examples
QuerySettings
Configuration options for query operations.
Configuration methods
| Method | Description |
|---|---|
setQueryId(String queryId) | Sets query ID that will be assigned to the operation. |
setFormat(ClickHouseFormat format) | Sets response format. See RowBinaryWithNamesAndTypes for the full list. |
setMaxExecutionTime(Integer maxExecutionTime) | Sets operation execution time on server. Will not affect read timeout. |
waitEndOfQuery(Boolean waitEndOfQuery) | Requests the server to wait for the end of the query before sending a response. |
setUseServerTimeZone(Boolean useServerTimeZone) | Server timezone (see client config) will be used to parse date/time types in the result of an operation. Default false. |
setUseTimeZone(String timeZone) | Requests server to use timeZone for time conversion. See session_timezone. |
serverSetting(String name, String value) | Sets individual server settings for an operation. |
serverSetting(String name, Collection values) | Sets individual server settings with multiple values for an operation. Items of the collection should be String values. |
setDBRoles(Collection dbRoles) | Sets DB roles to be set before executing an operation. Items of the collection should be String values. |
setOption(String option, Object value) | Sets a configuration option in raw format. This is not a server setting. |
QueryResponse
Response object that holds result of query execution. It is only available if the client got a response from a server.
This object should be closed as soon as possible to release a connection because the connection cannot be re-used until all data of previous response is fully read.
| Method | Description |
|---|---|
ClickHouseFormat getFormat() | Returns a format in which data in the response is encoded. |
InputStream getInputStream() | Returns uncompressed byte stream of data in the specified format. |
OperationMetrics getMetrics() | Returns object with operation metrics. |
String getQueryId() | Returns query ID assigned for the operation by the application (through operation settings or by server). |
TimeZone getTimeZone() | Returns timezone that should be used for handling Date/DateTime types in the response. |
Examples
- Example code is available in repo
- Reference Spring Service implementation
Common API
getTableSchema(String table)
Fetches table schema for the table.
Signatures
Parameters
table - table name for which schema data should be fetched.
database - database where the target table is defined.
Return value
Returns a TableSchema object with list of table columns.
getTableSchemaFromQuery(String sql)
Fetches schema from a SQL statement.
Signatures
Parameters
sql - "SELECT" SQL statement which schema should be returned.
Return value
Returns a TableSchema object with columns matching the sql expression.
TableSchema
register(Class<?> clazz, TableSchema schema)
Compiles serialization and deserialization layer for the Java Class to use for writing/reading data with schema. The method will create a serializer and deserializer for the pair getter/setter and corresponding column.
Column match is found by extracting its name from a method name. For example, getFirstName will be for the column first_name or firstname.
Signatures
Parameters
clazz - Class representing the POJO used to read/write data.
schema - Data schema to use for matching with POJO properties.
Examples
Usage Examples
Complete examples code is stored in the repo in a 'example` folder:
- client-v2 - main set of examples.
- demo-service - example of how to use the client in a Spring Boot application.
- demo-kotlin-service - example of how to use the client in Ktor (Kotlin) application.
