FW: Blocker- ONAP casablanca deployment issue with AAI


Michael O'Brien <frank.obrien@...>
 

Adding list (aai and oom team) – I don’t represent the OOM team – I am only a contributing member J

I would recommend you post mails to the list above – to get the widest audience.

 

I recommend keeping pod releases in sync – there are dependencies especially with the aaf-aai and aaf-dmaap chains – if you move from 3.0.1-ONAP – use the 3.0.2-ONAP tag when it is cut (not likely until thu or next week) or the casablanca branch for now – which is essentially 3.0.2

 

That issue is specific to the container – I recommend raising a jira against that particular component.

Thank you

 

/michael

 

 

From: Pranjal Dadhich <pranjal0491@...>
Sent: Wednesday, April 3, 2019 9:43 AM
To: Michael O'Brien <Frank.Obrien@...>; onap-discuss@...
Subject: Blocker- ONAP casablanca deployment issue with AAI

 

Hello Michael,

We have below environment set up in our lab using Rancher,kubernetes/helm and openstack. We are stuck with the deployment,need an immediate help to resolve the errors mentioned below.

Details as follows:

Environment : 14+1 Cluster , deployed Casablanca (3.0.0)

Issue : DBschema job within AAI fails to run sucessfuly , thus impacting the AAI services .

 

Request to help address this issue . Is this a bug , if so can we upgrade to 3.0.1 of Casablanca . If we were to upgrade to 3.0.1 , would you recommend upgrading just the AAI components or the whole ONAP installation .

Would you consider 3.0.1 , stable enough for an upgrade or is it work in progress .

 

Please find attached screenshot of the error message from Kubectl as well as logs from the aai-db-schema job (below screenshot) . Please let us know , if further information is required .

Kubectl logs: (attached)

 Logs from   aai-graphadmin-job in onap-aai-aai-graphadmin-create-db-schema-pj5hr

 

 

Project Build Version: 1.0.1

chown: changing ownership of '/opt/app/aai-graphadmin/resources/localhost-access-logback.xml': Read-only file system

chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/auth/aai_keystore': Read-only file system

chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/appprops/aaiconfig.properties': Read-only file system

chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/appprops/janusgraph-cached.properties': Read-only file system

chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/appprops/janusgraph-realtime.properties': Read-only file system

chown: changing ownership of '/opt/app/aai-graphadmin/resources/application.properties': Read-only file system

chown: changing ownership of '/opt/app/aai-graphadmin/resources/logback.xml': Read-only file system

Wed Apr 3 12:36:23 UTC 2019 Starting /opt/app/aai-graphadmin/bin/createDBSchema.sh

---- NOTE --- about to open graph (takes a little while)--------;

Exception in thread "main" java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)

at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)

at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)

at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:595)

Caused by: java.lang.ExceptionInInitializerError

at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:103)

at org.onap.aai.schema.GenTester.main(GenTester.java:126)

... 8 more

Caused by: java.lang.RuntimeException: Failed to instantiate graphs

at org.onap.aai.dbmap.AAIGraph.&lt;init&gt;(AAIGraph.java:85)

at org.onap.aai.dbmap.AAIGraph.&lt;init&gt;(AAIGraph.java:57)

at org.onap.aai.dbmap.AAIGraph$Helper.&lt;clinit&gt;(AAIGraph.java:90)

... 10 more

Caused by: org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception

at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)

at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159)

at org.janusgraph.diskstorage.configuration.backend.KCVSConfiguration.get(KCVSConfiguration.java:100)

at org.janusgraph.diskstorage.configuration.BasicConfiguration.isFrozen(BasicConfiguration.java:106)

Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend

at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getNamesSlice(AstyanaxKeyColumnValueStore.java:161)

at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getNamesSlice(AstyanaxKeyColumnValueStore.java:115)

at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getSlice(AstyanaxKeyColumnValueStore.java:104)

at org.janusgraph.diskstorage.configuration.backend.KCVSConfiguration$1.call(KCVSConfiguration.java:103)

at org.janusgraph.diskstorage.configuration.backend.KCVSConfiguration$1.call(KCVSConfiguration.java:100)

at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148)

at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162)

at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)

... 22 more

Caused by: com.netflix.astyanax.connectionpool.exceptions.TokenRangeOfflineException: TokenRangeOfflineException: [host=10.42.94.176(10.42.94.176):9160, latency=2(2), attempts=1]UnavailableException()

at com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:165)

at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65)

at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:28)

at com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:153)

at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:119)

at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:352)

at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4.execute(ThriftColumnFamilyQueryImpl.java:538)

at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getNamesSlice(AstyanaxKeyColumnValueStore.java:159)

... 29 more

Caused by: UnavailableException()

at org.apache.cassandra.thrift.Cassandra$multiget_slice_result$multiget_slice_resultStandardScheme.read(Cassandra.java:14687)

at org.apache.cassandra.thrift.Cassandra$multiget_slice_result$multiget_slice_resultStandardScheme.read(Cassandra.java:14633)

at org.apache.cassandra.thrift.Cassandra$multiget_slice_result.read(Cassandra.java:14559)

at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)

at org.apache.cassandra.thrift.Cassandra$Client.recv_multiget_slice(Cassandra.java:741)

at org.apache.cassandra.thrift.Cassandra$Client.multiget_slice(Cassandra.java:725)

at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4$1.internalExecute(ThriftColumnFamilyQueryImpl.java:544)

at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4$1.internalExecute(ThriftColumnFamilyQueryImpl.java:541)

at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60)

... 35 more

Failed to run the tool /opt/app/aai-graphadmin/bin/createDBSchema.sh successfully

Failed to run the createDBSchema.sh

 

 

Please help us to resolve this issue, unable to proceed because of it.

 

 

 

Regards,

Pranjal

 

 

This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service

Join onap-discuss@lists.onap.org to automatically receive all group messages.