Trouble-Shooting
This section describes some common errors with this library and grpc in general, and how to solve them. Please note that this page can never cover all cases, please also search the existing issues/PRs (both opened and closed ones) for related topics. If a corresponding topic already exists, leave us a comment / info so that we know that you are also affected. If there is no such topic, feel free to open a new one as described at the bottom of this page.
Table of Contents
- NoClassDefFoundError, ClassNotFoundException, NoSuchMethodError, AbstractMethodError
- Transport failed
- First received frame was not SETTINGS
- Network closed for unknown reason
- Could not find TLS ALPN provider
- Dismatching certificates
- Untrusted certificates
- Server port already in use
- Client fails to resolve domain name
- Creating issues / asking questions
NoClassDefFoundError, ClassNotFoundException, NoSuchMethodError, AbstractMethodError
Example
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'client' defined in file [~/.../MyGrpcClient.class]: Initialization of bean failed; nested exception is java.lang.NoClassDefFoundError: io/grpc/TlsChannelCredentials$Feature
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:602)
[...]
Caused by: java.lang.NoClassDefFoundError: io/grpc/TlsChannelCredentials$Feature
at io.grpc.netty.ProtocolNegotiators.<clinit>(ProtocolNegotiators.java:92)
The Problem
The server/client does not start because some class or method is missing.
This is usually the case if the grpc-libraries use slightly different versions.
The solution
Make sure to use exactly the same version for all grpc-java
versions.
Add the following entry to your dependencyManagement
section of your project:
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-bom</artifactId>
<version>${grpcVersion}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
You can use a similar approach for gradle:
dependencyManagement {
imports {
mavenBom "io.grpc:grpc-bom:${grpcVersion}"
Note: grpc-spring-boot-starter isn’t strictly bound to a specific version of grpc-java, so you can also use this to change the version of grpc-java you are using in your project.
See also Could not find TLS ALPN provider
Transport failed
Server-side
2019-07-07 10:05:46.217 INFO 6552 --- [-worker-ELG-3-5] i.g.n.s.i.g.n.N.connections : Transport failed
io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception: HTTP/2 client preface string missing or corrupt. Hex dump for received bytes: 16030100820100007e0303aae6126974cbb4638b325d6bdb
at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception.connectionError(Http2Exception.java:85) ~[grpc-netty-shaded-1.21.0.jar:1.21.0]
at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.readClientPrefaceString(Http2ConnectionHandler.java:318) ~[grpc-netty-shaded-1.21.0.jar:1.21.0]
at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(Http2ConnectionHandler.java:251) ~[grpc-netty-shaded-1.21.0.jar:1.21.0]
at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:450) [grpc-netty-shaded-1.21.0.jar:1.21.0]
Client-side
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:235)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:216)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:141)
at net.devh.boot.grpc.examples.lib.SimpleGrpc$SimpleBlockingStub.sayHello(SimpleGrpc.java:178)
[...]
Caused by: io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 00001204000000000000037fffffff000400100000000600002000000004080000000000000f0001
at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1204)
at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1272)
at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502)
The Problem
The server runs in PLAINTEXT
mode, but the client tries to connect it in TLS
(default) mode.
The simple solution
a.k.a.: Configure the client to connect in PLAINTEXT
mode (Not recommended for production).
Add the following entry to your client side application config:
grpc.client.__name__.negotiationType=PLAINTEXT
The better solution
a.k.a.: Configure the server to run in TLS
mode (Recommended).
Add the following entry to your sever side application config:
grpc.server.security.enabled=true
grpc.server.security.certificateChain=file:certificates/server.crt
grpc.server.security.privateKey=file:certificates/server.key
First received frame was not SETTINGS
Client-side
Status{code=INTERNAL, description=http2 exception, cause=...Http2Exception: First received frame was not SETTINGS. Hex dump for first 5 bytes: 485454502f
The Problem
485454502f
-> HTTP/
from HTTP/1.1
You are probably connecting to a normal web server and not a HTTP/2 grpc server or proxy.
The Solution
Check your address configuration. Make sure that you are connecting to the correct port.
e.g. dns:///my-service.grpc.example
-> dns:///my-service.grpc.example:12345
Network closed for unknown reason
Client-side
io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
The Problem
You are either (1) trying to connect to an grpc-server in TLS
mode using a PLAINTEXT
client
or (2) the target is not a grpc-server (e.g. a web-server).
The Solution
-
Configure your client to use
TLS
mode.grpc.client.__name__.negotiationType=TLS
or remove the
negotiationType
config completely asTLS
is the default. -
Validate that the configured server is running and is a grpc-server using
grpcurl
or a similar tool.
Could not find TLS ALPN provider
Server-side
org.springframework.context.ApplicationContextException: Failed to start bean 'nettyGrpcServerLifecycle'; nested exception is java.lang.IllegalStateException: Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
[...]
Caused by: java.lang.IllegalStateException: Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available
at io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:258) ~[grpc-netty-1.21.0.jar:1.21.0]
at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171) ~[grpc-netty-1.21.0.jar:1.21.0]
at io.grpc.netty.GrpcSslContexts.forServer(GrpcSslContexts.java:130) ~[grpc-netty-1.21.0.jar:1.21.0]
[...]
Client-side
[...]
Caused by: java.lang.IllegalStateException: Failed to create channel: <name>
at net.devh.boot.grpc.client.inject.GrpcClientBeanPostProcessor.processInjectionPoint(GrpcClientBeanPostProcessor.java:118) ~[grpc-client-spring-boot-autoconfigure-2.4.0.RELEASE.jar:2.4.0.RELEASE]
at net.devh.boot.grpc.client.inject.GrpcClientBeanPostProcessor.postProcessBeforeInitialization(GrpcClientBeanPostProcessor.java:77)
[...]
Caused by: java.lang.IllegalStateException: Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available
at io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:258) ~[grpc-netty-1.21.0.jar:1.21.0]
at io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171) ~[grpc-netty-1.21.0.jar:1.21.0]
at io.grpc.netty.GrpcSslContexts.forClient(GrpcSslContexts.java:120) ~[grpc-netty-1.21.0.jar:1.21.0]
[...]
Both sides
AbstractMethodError: io.netty.internal.tcnative.SSL.readFromSSL()
The Problem
There is no (compatible) netty TLS implementation available on the classpath.
The Solution
Either switch from grpc-netty
to grpc-netty-shaded
or add a dependency to netty-tcnative-boringssl-static
(Please use the exact same (compatible) versions that are listed in the table in grpc-java’s netty security section.
Note: You need a 64bit Java JVM.
Dismatching certificates
Client-side
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
[...]
Caused by: javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem
[...]
Caused by: java.security.cert.CertificateException: No subject alternative names present
or
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
[...]
Caused by: javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem
[...]
Caused by: java.security.cert.CertificateException: No name matching <name> found
The Problem
The certificate does not match the target’s address/name.
The Solution
Configure an override for the name comparison by adding the following to your client config:
grpc.client.__name__.security.authorityOverride=<authority>
Untrusted certificates
Client-side
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
[...]
Caused by: javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem
[...]
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
[...]
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
The Problem
The certificate used by the server is not in the trust store of the client.
The Solution
Either add the certificate to java’s truststore by using java’s keytool
or configure the client to use a custom trusted certificate file:
grpc.client.__name__.security.trustCertCollection=file:certificates/trusted-servers-collection.crt.list
Note: Both stores are currently read only at creation time and updates won’t be picked up.
Server port already in use
Server-side
Caused by: java.lang.IllegalStateException: Failed to start the grpc server
at net.devh.boot.grpc.server.serverfactory.GrpcServerLifecycle.start(GrpcServerLifecycle.java:51) ~[grpc-server-spring-boot-autoconfigure-2.4.0.RELEASE.jar:2.4.0.RELEASE]
[...]
Caused by: java.io.IOException: Failed to bind
at io.grpc.netty.shaded.io.grpc.netty.NettyServer.start(NettyServer.java:246) ~[grpc-netty-shaded-1.21.0.jar:1.21.0]
at io.grpc.internal.ServerImpl.start(ServerImpl.java:177) ~[grpc-core-1.21.0.jar:1.21.0]
at io.grpc.internal.ServerImpl.start(ServerImpl.java:85) ~[grpc-core-1.21.0.jar:1.21.0]
at net.devh.boot.grpc.server.serverfactory.GrpcServerLifecycle.createAndStartGrpcServer(GrpcServerLifecycle.java:90) ~[grpc-server-spring-boot-autoconfigure-2.4.0.RELEASE.jar:2.4.0.RELEASE]
at net.devh.boot.grpc.server.serverfactory.GrpcServerLifecycle.start(GrpcServerLifecycle.java:49) ~[grpc-server-spring-boot-autoconfigure-2.4.0.RELEASE.jar:2.4.0.RELEASE]
... 13 common frames omitted
Caused by: java.net.BindException: Address already in use: bind
The Problem
The port the grpc server is trying to use is already used.
There are four common cases where this error might occur.
- The application is already running
- Another application is using that port
- The grpc server uses a port that is already used for something else (e.g. spring-web)
- You are running tests and spring didn’t shutdown the grpc-server after each test
The Solution
- Try searching for the application using the task manager or
jps
- Try searching for the port using
netstat
- Check/Change your configuration. This library uses port
9090
by default - Adding
@DirtiesContext
to your test classes and methods Please note that the error will only occur from the second test onwards, so you have to annotate the first one as well!
Client fails to resolve domain name
Client-side
WARN io.grpc.internal.ManagedChannelImpl - [Failed to resolve name. status=Status{code=UNAVAILABLE, description=No servers found for `discovery-server:443`}
ERROR n.d.b.g.c.n.DiscoveryClientNameResolver - No servers found for `discovery-server:443`
The Problem
The discovery service library or it’s configuration failed to specify the scheme how discovery-server:443
should be
resolved. If you don’t have a service discovery, then the default is dns
, but if you use a discovery service, then
that will be the default and thus failing to resolve that address.
The same applies to other libraries, such as tracing or reporting libraries, which report their results via grpc to an external server.
The Solution
- Configure the (discovery service) library to specify the
dns
scheme: e.g.dns:///discovery-server:443
- Search for invocations of
ManagedChannelBuilder#forTarget(String)
orNettyChannelBuilder#forTarget(String)
(or similar methods) and make sure they use thedns
scheme. - Disable the service discovery for grpc services:
spring.autoconfigure.exclude=net.devh.boot.grpc.client.autoconfigure.GrpcDiscoveryClientAutoConfiguration
- or create a custom
NameResolverRegistry
bean
See also client target configuration.
Creating issues
Creating issues/asking questions on GitHub isn’t hard, but with a little bit of your effort you can help us solving your issues faster.
If your issue/question is about grpc in general consider asking it over at grpc-java.
Use the provided templates to create new issues, these contain sections for the required/helpful information we need.
In general, you should include the following information in your issue:
- What type of request do you have?
- Question
- Bug-report
- Feature-Request
- What do you wish to achieve?
- What’s The Problem? What’s not working? What’s missing and why do you need it?
- Any relevant stacktraces/logs (very important)
- Which versions do you use?
- Spring (boot)
- grpc-java
- grpc-spring-boot-starter
- Other relevant libraries
- Additional context
- Did it ever work before?
- How can we reproduce it?
- Do you have a demo?