OOM 3.0.2-ONAP: PolicyPKIX path validation failed #oom #policy


jkzcristiano
 

Dear all,

Policy is no longer able to get events. From debug.log this is the error (the same for other topics):

[2019-06-17T09:44:45.253+00:00|ERROR|InlineBusTopicSink|DMAAP-source-unauthenticated.DCAE_CL_OUTPUT] SingleThreadedDmaapTopicSource [userName=null, password=-, getTopicCommInfrastructure()=DMAAP, toString()=SingleThreadedBusTopicSource [consumerGroup=dcae.policy.shared, consumerInstance=dev-policy-drools-0, fetchTimeout=15000, fetchLimit=100, consumer=CambriaConsumerWrapper [fetchTimeout=15000], alive=true, locked=false, uebThread=Thread[DMAAP-source-unauthenticated.DCAE_CL_OUTPUT,5,main], topicListeners=1, toString()=BusTopicBase [apiKey=, apiSecret=, useHttps=true, allowSelfSignedCerts=false, toString()=TopicBase [servers=[message-router], topic=unauthenticated.DCAE_CL_OUTPUT, #recentEvents=0, locked=false, #topicListeners=1]]]]: cannot fetch because of
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1964)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:328)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1614)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:987)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at com.att.nsa.apiClient.http.HttpClient.runCall(HttpClient.java:708)
        at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:384)
        at com.att.nsa.apiClient.http.HttpClient.get(HttpClient.java:368)
        at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:87)
        at com.att.nsa.cambria.client.impl.CambriaConsumerImpl.fetch(CambriaConsumerImpl.java:64)
        at org.onap.policy.common.endpoints.event.comm.bus.internal.BusConsumer$CambriaConsumerWrapper.fetch(BusConsumer.java:172)
        at org.onap.policy.common.endpoints.event.comm.bus.internal.SingleThreadedBusTopicSource.run(SingleThreadedBusTopicSource.java:224)
        at java.lang.Thread.run(Thread.java:748)
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:362)
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:270)
        at sun.security.validator.Validator.validate(Validator.java:260)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1596)
        ... 26 common frames omitted
Caused by: java.security.cert.CertPathValidatorException: validity check failed
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:233)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:141)
        at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:80)
        at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:357)
        ... 32 common frames omitted
Caused by: java.security.cert.CertificateExpiredException: NotAfter: Fri May 31 15:56:05 UTC 2019
        at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:274)
        at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:629)
        at sun.security.provider.certpath.BasicChecker.verifyValidity(BasicChecker.java:190)
        at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144)
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
        ... 37 common frames omitted
[2019-06-17T09:44:45.253+00:00|INFO|CambriaConsumerImpl|DMAAP-source-unauthenticated.DCAE_CL_OUTPUT] UEB GET /events/unauthenticated.DCAE_CL_OUTPUT/dcae.policy.shared/dev-policy-drools-0?timeout=15000&limit=100
[2019-06-17T09:44:45.253+00:00|WARN|HostSelector|DMAAP-source-unauthenticated.DCAE_CL_OUTPUT] All hosts were blacklisted; reverting to full set of hosts.
[2019-06-17T09:44:45.253+00:00|INFO|HttpClient|DMAAP-source-unauthenticated.DCAE_CL_OUTPUT] GET https://message-router:3905/events/unauthenticated.DCAE_CL_OUTPUT/dcae.policy.shared/dev-policy-drools-0?timeout=15000&limit=100 (anonymous) ...
[2019-06-17T09:44:45.272+00:00|WARN|HttpClient|DMAAP-source-unauthenticated.DCAE_CL_OUTPUT] Error executing HTTP request. sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed; blacklisting for 2 minutes
[2019-06-17T09:44:45.272+00:00|ERROR|BusConsumer$CambriaConsumerWrapper|DMAAP-source-unauthenticated.DCAE_CL_OUTPUT] CambriaConsumerWrapper [fetchTimeout=15000]: cannot fetch because of sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed - backoff for 15000 ms.


Thus, closed loop operations cannot be processed. It seems a certificate issue (change date time and it works). Is there any workaround for this? These are logs from a fresh 3.0.2-ONAP install with AAF, POLICY, DMAAP and ROBOT (all runinng).

I hope you can help!
Xoan

Join onap-discuss@lists.onap.org to automatically receive all group messages.