Re: [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case


Taka Cho
 

Not sure why tenant is null and saw 404 Error below. check your openstack compute api and the appc.properties inside appc container, and your payload.

 

2020-05-21T16:23:52,220 | INFO  | appc-dispatcher-2 |                                  | 482 - appc-iaas-adapter - 1.6.4 | About to login principal [admin] to provider [OpenStackProvider] on tenant [null]

2020-05-21T16:23:52,295 | INFO  | appc-dispatcher-2 | Keystone                         | 482 - appc-iaas-adapter - 1.6.4 | 1 * Sending client request on thread appc-dispatcher-2

1 > POST http://192.168.9.75:5000/v2.0/tokens

1 > Accept: application/json

1 > Content-Type: application/json

{

  "auth" : {

    "tenantName" : "admin",

    "passwordCredentials" : {

      "username" : "admin",

      "password" : "********"

    }

  }

}

 

2020-05-21T16:23:52,303 | INFO  | appc-dispatcher-2 | Keystone                         | 482 - appc-iaas-adapter - 1.6.4 | 1 * Client response received on thread appc-dispatcher-2

1 < 404

1 < Connection: Keep-Alive

1 < Content-Length: 233

1 < Content-Type: text/html

1 < Date: Thu, 21 May 2020 16:23:52 GMT

1 < Keep-Alive: timeout=5, max=99

1 < Server: Apache/2.4.29 (Ubuntu)

1 < Vary: X-Auth-Token

1 < x-openstack-request-id: req-76b7a584-4ce1-45af-b945-f1532590ad1c

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">

<title>404 Not Found</title>

<h1>Not Found</h1>

<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>

 

From: Devangam Manjunatha <Devangam.Manjunatha@...>
Sent: Thursday, May 21, 2020 1:17 PM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...; CHO, TAKAMUNE <tc012c@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Hi Taka,

 

W.r.t one of the mail which you replied earlier, Where you suggested us to add non-encrypted password to the parameter "openStackEncryptedPassword" in openstack.yaml file(i.e. https://gerrit.onap.org/r/gitweb?p=oom.git;a=blob;f=kubernetes/onap/resources/overrides/openstack.yaml;h=5e09d7cfe9ee3bcc98c1ce9efedad70d1016d721;hb=935623ebd592070fc55fe5dc1ebc3bad07fcc714).

 

We re-deployed ONAP with above changes. I can see in the APPC log that token generation is successful as shown below.

 

 

2020-05-21T16:23:51,455 | INFO  | appc-dispatcher-2 | Keystone                         | 482 - appc-iaas-adapter - 1.6.4 | 1 * Sending client request on thread appc-dispatcher-2

1 > Accept: application/json

1 > Content-Type: application/json

{

  "auth" : {

    "identity" : {

      "methods" : [ "password" ],

      "password" : {

        "user" : {

          "domain" : {

            "name" : "Default"

          },

          "name" : "admin",

          "password" : "********"

        }

      }

    },

    "scope" : {

      "project" : {

        "id" : "70968f3b213c4c8d81d92c06a9dddf7c"

      }

    }

  }

}

 

2020-05-21T16:23:51,858 | INFO  | appc-dispatcher-2 | Keystone                         | 482 - appc-iaas-adapter - 1.6.4 | 1 * Client response received on thread appc-dispatcher-2

1 < 201

1 < Connection: Keep-Alive

1 < Content-Length: 6013

1 < Content-Type: application/json

1 < Date: Thu, 21 May 2020 16:23:51 GMT

1 < Keep-Alive: timeout=5, max=100

1 < Server: Apache/2.4.29 (Ubuntu)

1 < Vary: X-Auth-Token

1 < X-Distribution: Ubuntu

1 < x-openstack-request-id: req-7d191d0c-3017-4de4-9622-ef272157b91f

1 < X-Subject-Token: gAAAAABexqsXyk_OzG5JH74PXlE66Mk3qA4G4ocgNVuV6sc-kBL3-S9whRB0nkwQaegdHNkWnzgfb-LQNLnCgktCYAcJg0XmXTLiq6YbDV_-dLpMRUa6OLzD73wvEw55gFqzs9ShRx8Bjnq1uEAHocn2-x7mnsmMlgukO7VI7o93sJJiJLNr0uk

{"token": {"is_domain": false, "methods": ["password" : "********"}}

 

 

 

After above authentication we are getting below error. Request to let us know how to fix below issue. for more details please find the attached logs and properties files etc.

I think we are missing some configuration.

 

 

2020-05-21T16:23:52,304 | ERROR | appc-dispatcher-2 | ProviderOperation                | 482 - appc-iaas-adapter - 1.6.4 | APPC0054E Connection to provider http://192.168.9.75:5000/v3 at identity http://192.168.9.75:5000/v3 using tenant name admin (id 70968f3b213c4c8d81d92c06a9dddf7c) failed, reason=The allocator failed to allocate a new context to extend the pool., retrying in 30 seconds, attempt 1 of 10.

org.onap.appc.pool.PoolExtensionException: The allocator failed to allocate a new context to extend the pool.

at org.onap.appc.pool.Pool.extend(Pool.java:336) ~[?:?]

at org.onap.appc.pool.Pool.reserve(Pool.java:257) ~[?:?]

at org.onap.appc.adapter.iaas.provider.operation.impl.base.ProviderOperation.getContext(ProviderOperation.java:373) ~[?:?]

at org.onap.appc.adapter.iaas.provider.operation.impl.RestartServer.restartServer(RestartServer.java:206) ~[?:?]

 

 

Regards,

Manjunath.

 

 

 


From: FREEMAN, BRIAN D <bf1936@...>
Sent: Friday, May 15, 2020 6:15 PM
To: Devangam Manjunatha <Devangam.Manjunatha@...>; onap-discuss@... <onap-discuss@...>; CHO, TAKAMUNE <tc012c@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: RE: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

I [would recommend you] ….

 

Brian

 

 

From: Devangam Manjunatha <Devangam.Manjunatha@...>
Sent: Friday, May 15, 2020 5:10 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...; CHO, TAKAMUNE <tc012c@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Sure Brian, Let me know once it is done.

 

Regards,

Manjunath.

 


From: FREEMAN, BRIAN D <bf1936@...>
Sent: Thursday, May 14, 2020 9:50 PM
To: Devangam Manjunatha <Devangam.Manjunatha@...>; onap-discuss@... <onap-discuss@...>; CHO, TAKAMUNE <tc012c@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: RE: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

I would start with the default odlPassword to make sure your install works and then look at how to change the password. AAF needs to be  updated with the new password and all the clients not just APPC so I would get the flow working first then you can do the work to change the password.

 

Brian

 

 

From: Devangam Manjunatha <Devangam.Manjunatha@...>
Sent: Thursday, May 14, 2020 11:22 AM
To: onap-discuss@...; CHO, TAKAMUNE <tc012c@...>; FREEMAN, BRIAN D <bf1936@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Hi Brian,

 

As shown below, Shall i need re-deploy APPC component by changing  "odlPassword"  property in  values.yaml to 'root123'.

Please correct me if i am wrong.

 

 

root@rndelato-nfs:~/oom/kubernetes/appc# grep -r 'odlPassword'

resources/config/appc/opt/onap/appc/data/properties/appc.properties:appc.demo.provider.pass={{.Values.config.odlPassword}}

resources/config/appc/opt/onap/appc/data/properties/appc.properties:appc.provider.vfodl.url=http://{{.Values.config.odlUser|urlquery}}:{{.Values.config.odlPassword|urlquery}}@{{.Values.service.name}}:{{.Values.service.externalPort}}/restconf/config/network-topology:network-topology/topology/topology-netconf/node/NODE_NAME/yang-ext:mount/stream-count:stream-count/streams/

resources/config/appc/opt/onap/appc/data/properties/appc.properties:appc.LCM.provider.pass={{.Values.config.odlPassword}}

resources/config/appc/opt/onap/appc/data/properties/appc.properties:restconf.pass={{.Values.config.odlPassword}}

resources/config/appc/opt/onap/appc/data/properties/appc.properties:appc.OAM.provider.pass={{.Values.config.odlPassword}}

values.yaml:  odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

root@rndelato-nfs:~/oom/kubernetes/appc#

 

 

I can see "odlPassword" in othe components to.

 

 

root@rndelato-nfs:~/oom/kubernetes# grep -r 'Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U'

aaf/charts/aaf-sms/resources/config/has.json:                    "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U"

vid/charts/vid-mariadb-galera/values.yaml:  userPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

vid/values.yaml:  vidmysqlpassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

cds/charts/cds-blueprints-processor/resources/config/application.properties:blueprintsprocessor.restclient.sdncodl.password=Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

cds/charts/cds-blueprints-processor/resources/config/application.properties:blueprintsprocessor.restclient.sdnc.password=Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

cds/charts/cds-sdc-listener/resources/config/application.yaml:    password: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U #SDC-password

multicloud/charts/multicloud-k8s/resources/config/config.json:                "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U",

multicloud/charts/multicloud-starlingx/resources/config/log/config.json:                "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U",

multicloud/charts/multicloud-windriver/resources/config/log/config.json:                "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U",

dcaegen2/charts/dcae-servicechange-handler/values.yaml:  password: !!str Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

oof/charts/oof-has/resources/config/conductor.conf:password = Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

policy/charts/drools/values.yaml:  password: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

policy/charts/policy-distribution/resources/config/config.json:                "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U",

sdc/charts/sdc-wfd-be/values.yaml:  sdcExternalUserPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/resources/config/bin/startODL.sh:ODL_ADMIN_PASSWORD=${ODL_ADMIN_PASSWORD:-Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U}

sdnc/charts/ueb-listener/resources/config/ueb-listener.properties:org.onap.ccsdk.sli.northbound.uebclient.password=Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/charts/ueb-listener/values.yaml:  odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/charts/sdnc-portal/values.yaml:  odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/charts/dmaap-listener/resources/config/aai.properties:sdnc.odl.password=Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/charts/dmaap-listener/values.yaml:  odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/values.yaml:  odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/values.yaml:    odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/values.yaml:    odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/values.yaml:    odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

sdnc/sdnc-prom/values.yaml:    password: "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U"

appc/values.yaml:  odlPassword: Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U

robot/values.yaml:sdncPassword: "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U"

root@rndelato-nfs:~/oom/kubernetes#

 

 

Regards,

Manjunath.

 

 

 


From: onap-discuss@... <onap-discuss@...> on behalf of Brian Freeman via lists.onap.org <bf1936=att.com@...>
Sent: Thursday, May 14, 2020 7:32 PM
To: Devangam Manjunatha <Devangam.Manjunatha@...>; onap-discuss@... <onap-discuss@...>; CHO, TAKAMUNE <tc012c@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Not sure how root123 is being handled by the APPC/AAF interaction.

 

I suspect the 401 is appropriately telling you that the mapping from APPC to AAF is telling you your password is invalid but I’m not sure I am current on the APPC authentication mechanism.  I doubt it is a simple update to the ODL idm file.

 

Brian

 

 

From: Devangam Manjunatha <Devangam.Manjunatha@...>
Sent: Thursday, May 14, 2020 9:07 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...; CHO, TAKAMUNE <tc012c@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Hi Taka, Hi Brian,

 

I had added dns resolver(i.e. 192.168.9.75   controller) in /etc/hosts file of APPC pod. 

Following self link URL tried with "curl" command in APPC pod, request reaching controller, there we are getting "Unauthorized".

If host name is not resolved, we will get "No route to host" error.

 

bash-4.4$ curl http://controller:8774/v2.1/70968f3b213c4c8d81d92c06a9dddf7c/servers/f78915a3-8b6e-4225-86f0-76810fb5ed7e

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}bash-4.4$ 

bash-4.4$

 

Also i tied with below options in APPC pod.

 

ubuntu@rndelato-nfs:~$

ubuntu@rndelato-nfs:~$ kubectl -it exec rnd-appc-appc-0 bash

Defaulting container name to appc.

Use 'kubectl describe pod/rnd-appc-appc-0 -n onap' to see all of the containers in this pod.

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}bash-4.4$

bash-4.4$

bash-4.4$

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}bash-4.4$

bash-4.4$

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}bash-4.4$

bash-4.4$

bash-4.4$

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}bash-4.4$

bash-4.4$

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}bash-4.4$

bash-4.4$

 

W.r.t above curl commands, our controller password is "root123",  I encrypted same password (i.e. "ZW5jOnJvb3QxMjM=" encrypted value of "enc:root123" where "enc" is prefixed to my password.) with Base64 algorithm, even i am getting "Unauthorized"

 

Can you please clarify below open points.

 

1) As per below mail Error-1 & Error-2 can we ignore ? If i can't ignore, please let me know how to fix below errors.

 

2) I am using "identity-url" with the version "v2.0" (i.e. http://192.168.9.75:5000/v2.0) which is matching with "provider1.identity" (i.e. http://192.168.9.75:5000/v2.0) property in "appc.properties" file.

    Please confirm which version we need to use in the above URLs "v2.0" or "v3" ?

    

3) Is it possible to restart vGMuX using "self.link" URL via curl command? will you please share sample payload and URL ?

 

 

Currently our goal is to restart vGMuX with self.link url from APPC pod.

 

 

Regards,

Manjunath.

 


From: onap-discuss@... <onap-discuss@...> on behalf of Taka Cho via lists.onap.org <tc012c=att.com@...>
Sent: Thursday, May 14, 2020 12:34 AM
To: Devangam Manjunatha <Devangam.Manjunatha@...>; FREEMAN, BRIAN D <bf1936@...>; onap-discuss@... <onap-discuss@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

[http://controller:8774/v2.1/70968f3b213c4c8d81d92c06a9dddf7c/servers/f78915a3-8b6e-4225-86f0-76810fb5ed7e]

 

Is “controller” correct? Sounds like this is coming from your request to APPC. Try the IP address

 

Taka

 

From: Devangam Manjunatha <Devangam.Manjunatha@...>
Sent: Wednesday, May 13, 2020 1:27 PM
To: CHO, TAKAMUNE <tc012c@...>; FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Hi Taka,

 

I redeployed APPC component alone by changing "openStackEncryptedPassword"  with un-encrypted password.

 

Now i can see 2 below errors in  "karaf.log"

 

 

Error-1:

 

2020-05-13T17:04:18,841 | INFO  | appc-dispatcher-9 | DmaapUtil                        | 444 - appc-common-bundle - 1.6.4 | Loading Default DMaaP Profile

2020-05-13T17:04:18,842 | ERROR | appc-dispatcher-9 | ExecuteNodeExecutor              | 428 - wrap_file__opt_opendaylight_system_org_onap_ccsdk_sli_core_sli-provider-base_0.5.4_sli-provider-base-0.5.4.jar - 0.0.0 | Could not execute plugin. SvcLogic status will be set to failure.

java.lang.NoClassDefFoundError: com/att/nsa/mr/client/impl/MRConsumerImpl

at com.att.nsa.mr.client.MRClientFactory.createInternalBatchingPublisher(MRClientFactory.java:495) ~[?:?]

at com.att.nsa.mr.client.MRClientFactory.createBatchingPublisher(MRClientFactory.java:437) ~[?:?]

at com.att.nsa.mr.client.MRClientFactory.createBatchingPublisher(MRClientFactory.java:457) ~[?:?]

at org.onap.appc.adapter.messaging.dmaap.impl.DmaapProducerImpl.getClients(DmaapProducerImpl.java:126) ~[?:?]

at org.onap.appc.adapter.messaging.dmaap.impl.DmaapProducerImpl.post(DmaapProducerImpl.java:159) ~[?:?]

at org.onap.appc.adapter.messaging.dmaap.impl.EventSenderDmaapImpl.sendEvent(EventSenderDmaapImpl.java:114) ~[?:?]

at Proxy61d007f1_2cc3_46fe_b19f_278fc6f7406c.sendEvent(Unknown Source) ~[?:?]

at Proxyd4b6e43b_c068_40cf_97a5_1604c4357070.sendEvent(Unknown Source) ~[?:?]

at org.onap.appc.dg.common.impl.DCAEReporterPluginImpl.reportLegacy(DCAEReporterPluginImpl.java:145) ~[?:?]

at org.onap.appc.dg.common.impl.DCAEReporterPluginImpl.report(DCAEReporterPluginImpl.java:67) ~[?:?]

at Proxy7bb21108_fab0_4081_ab15_00204ea4b6c3.report(Unknown Source) ~[?:?]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]

at java.lang.reflect.Method.invoke(Method.java:498) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.ExecuteNodeExecutor.execute(ExecuteNodeExecutor.java:99) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.SvcLogicServiceImplBase.executeNode(SvcLogicServiceImplBase.java:147) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.BlockNodeExecutor.execute(BlockNodeExecutor.java:62) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.SvcLogicServiceImplBase.executeNode(SvcLogicServiceImplBase.java:147) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.ForNodeExecutor.execute(ForNodeExecutor.java:94) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.SvcLogicServiceImplBase.executeNode(SvcLogicServiceImplBase.java:147) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.BlockNodeExecutor.execute(BlockNodeExecutor.java:62) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.SvcLogicServiceImplBase.executeNode(SvcLogicServiceImplBase.java:147) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.BlockNodeExecutor.execute(BlockNodeExecutor.java:62) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.SvcLogicServiceImplBase.executeNode(SvcLogicServiceImplBase.java:147) ~[?:?]

at org.onap.ccsdk.sli.core.sli.provider.base.SvcLogicServiceImplBase.execute(SvcLogicServiceImplBase.java:117) ~[?:?]

..........................

2020-05-13T17:04:18,847 | INFO  | appc-dispatcher-9 | BlockNodeExecutor                | 428 - wrap_file__opt_opendaylight_system_org_onap_ccsdk_sli_core_sli-provider-base_0.5.4_sli-provider-base-0.5.4.jar - 0.0.0 | Block - stopped executing nodes due to failure status

2020-05-13T17:04:18,847 | INFO  | appc-dispatcher-9 | BlockNodeExecutor                | 428 - wrap_file__opt_opendaylight_system_org_onap_ccsdk_sli_core_sli-provider-base_0.5.4_sli-provider-base-0.5.4.jar - 0.0.0 | Block - stopped executing nodes due to failure status

2020-05-13T17:04:18,859 | INFO  | appc-dispatcher-9 | TransactionRecorderImpl          | 473 - transaction-recorder - 1.6.4 | No of Rows in Transactions Table with TRANSACTION_ID: null~f186df77-d464-40d4-8f88-9d0c538f77d8 and REQUEST_ID 49e15d63-c89b-40f9-9f52-0ae8c1618d4c is: 0

2020-05-13T17:04:18,863 | INFO  | appc-dispatcher-9 | audit                            | 444 - appc-common-bundle - 1.6.4 | APPC0090A Operation "App-C IaaS Adapter:Restart" for VNF type "9c04c768-9284-43a2-9e2b-1a22966bc8ad" from Source "49e15d63-c89b-40f9-9f52-0ae8c1618d4c" with RequestID "49e15d63-c89b-40f9-9f52-0ae8c1618d4c" was started at "2020-05-13T17:04:18Z" and ended at "2020-05-13T17:04:18Z" with status code "401"

 

 

In the above i can see status code "401"  i.e. "Unauthorized".  Not sure why i am getting.

 

Error-2:

 

Service request state context:

 PROVIDER=OpenStackProvider, PRINCIPAL=admin, METHOD=init, LINE_NUMBER=159, CLASS=org.onap.appc.adapter.iaas.impl.ServiceCatalogV2, TENANT=70968f3b213c4c8d81d92c06a9dddf7c, THREAD=appc-dispatcher-5

at com.att.cdp.openstack.util.ExceptionMapper.mapResponseException(ExceptionMapper.java:122) ~[?:?]

at com.att.cdp.openstack.util.ExceptionMapper.mapException(ExceptionMapper.java:75) ~[?:?]

at org.onap.appc.adapter.iaas.impl.ServiceCatalogV2.init(ServiceCatalogV2.java:178) ~[?:?]

at org.onap.appc.adapter.iaas.impl.TenantCache.initialize(TenantCache.java:171) ~[?:?]

at org.onap.appc.adapter.iaas.impl.ProviderCache.addTenant(ProviderCache.java:154) ~[?:?]

at org.onap.appc.adapter.iaas.provider.operation.impl.base.ProviderOperation.getContext(ProviderOperation.java:355) ~[?:?]

at org.onap.appc.adapter.iaas.provider.operation.impl.RestartServer.restartServer(RestartServer.java:206) ~[?:?]

at org.onap.appc.adapter.iaas.provider.operation.impl.RestartServer.executeProviderOperation(RestartServer.java:247) ~[?:?]

at org.onap.appc.adapter.iaas.provider.operation.impl.base.ProviderOperation.doOperation(ProviderOperation.java:430) ~[?:?]

at org.onap.appc.adapter.iaas.impl.ProviderAdapterImpl.restartServer(ProviderAdapterImpl.java:115) ~[?:?]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]

at java.lang.reflect.Method.invoke(Method.java:498) ~[?:?]

......................

2020-05-13T16:39:58,753 | ERROR | appc-dispatcher-5 | TenantCache                      | 482 - appc-iaas-adapter - 1.6.4 | APPC0055E Connection to provider http://192.168.9.75:5000/v2.0 at service http://192.168.9.75:5000/v2.0 failed after all retry attempts.

2020-05-13T16:39:58,753 | ERROR | appc-dispatcher-5 | ProviderOperation                | 482 - appc-iaas-adapter - 1.6.4 | APPC0043E No server found in provider with self-link URL [http://controller:8774/v2.1/70968f3b213c4c8d81d92c06a9dddf7c/servers/f78915a3-8b6e-4225-86f0-76810fb5ed7e]

 

 

I tried in APPC pod by using curl command, I ca see below error.

 

bash-4.4$ curl http://controller:8774/v2.1/70968f3b213c4c8d81d92c06a9dddf7c/servers/f78915a3-8b6e-4225-86f0-76810fb5ed7e

{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}bash-4.4$ 

bash-4.4$ 

 

 

We are using identity-url = "http://192.168.9.75:5000/v2.0" and "http://192.168.9.75:5000/v2.0" both are matching.

 

Please find the "provider1.identity" property from "appc.properties" below.

 

 

provider1.type=OpenStackProvider                                                                                                                      

provider1.name=OpenStack                                                                                                                              

# provider1.identity=http://192.168.9.75:5000/v3                                                                                                      

provider1.identity=http://192.168.9.75:5000/v2.0                                                                                                      

provider1.tenant1.name=admin                                                                                                                          

provider1.tenant1.domain=Default                                                                                                                      

provider1.tenant1.userid=admin                                                                                                                        

provider1.tenant1.password=root123

 

 

For more details please find the attached trace.

 

 

 

 

Regards,

Manjunath.

 


From: CHO, TAKAMUNE <tc012c@...>
Sent: Wednesday, May 13, 2020 7:35 PM
To: Devangam Manjunatha <Devangam.Manjunatha@...>; FREEMAN, BRIAN D <bf1936@...>; onap-discuss@... <onap-discuss@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: RE: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

I do not know what your env is. I just gave you a sample. I can’t help set up your env.

 

Basically, do a helm undeploy, and helm list check your helm status, then helm deploy, try google helm and ONAP OOM page

 

Taka

 

From: Devangam Manjunatha <Devangam.Manjunatha@...>
Sent: Wednesday, May 13, 2020 9:59 AM
To: CHO, TAKAMUNE <tc012c@...>; FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Hi Taka,

 

I think we need to change "openStackEncryptedPassword" in 3 places in "openstack.yaml" file.

After changing the "openstack.yaml" file with non-encrypted password, We need to re-deploy only APPC component right ?

I hope re-deployment procedure is same using "redeploy-module.sh" script.

 

 

 

Regards,

Manjunath.

 


From: CHO, TAKAMUNE <tc012c@...>
Sent: Wednesday, May 13, 2020 7:10 PM
To: Devangam Manjunatha <Devangam.Manjunatha@...>; FREEMAN, BRIAN D <bf1936@...>; onap-discuss@... <onap-discuss@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: RE: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Try non-encrypted openstack password first.

 

Here is a sample to override the helm value:

 

https://gerrit.onap.org/r/gitweb?p=oom.git;a=blob;f=kubernetes/onap/resources/overrides/openstack.yaml;h=5e09d7cfe9ee3bcc98c1ce9efedad70d1016d721;hb=935623ebd592070fc55fe5dc1ebc3bad07fcc714

 

In Frankfurt, there is a fix for iaas encrypted method: https://gerrit.onap.org/r/c/appc/+/98825

 

Taka

 

From: Devangam Manjunatha <Devangam.Manjunatha@...>
Sent: Wednesday, May 13, 2020 6:02 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...; CHO, TAKAMUNE <tc012c@...>
Cc: JOMY JOSE <Jomy.Jose@...>
Subject: Re: [onap-discuss] [El-Alto][vCPE]- APPC not reading DMAAP messages "closed loop" event in vCPE use case

 

Hi Brian, Hi Taka,

 

You are right, I enabled logs of 3 APPC pods, i can see messages from Policy are picked by "rnd-appc-appc-2" pod. 

I hope we are close to vGMuX restart as part of vCPE Closed loop testing. 

 

In APPC log i can see following two errors.

 

Error-1: Request from APPC to Keystone failing with "404 Not Found" error:

 

2020-05-13T05:45:18,840 | INFO  | appc-dispatcher-4 | Keystone                         | 482 - appc-iaas-adapter - 1.6.4 | 1 * Sending client request on thread appc-dispatcher-4

1 > Accept: application/json

1 > Content-Type: application/json

{

  "auth" : {

    "tenantName" : "admin",

    "passwordCredentials" : {

      "username" : "admin",

      "password" : "********"

    }

  }

}

 

2020-05-13T05:45:18,866 | INFO  | appc-dispatcher-4 | Keystone                         | 482 - appc-iaas-adapter - 1.6.4 | 1 * Client response received on thread appc-dispatcher-4

1 < 404

1 < Connection: Keep-Alive

1 < Content-Length: 233

1 < Content-Type: text/html

1 < Date: Wed, 13 May 2020 05:45:18 GMT

1 < Keep-Alive: timeout=5, max=100

1 < Server: Apache/2.4.29 (Ubuntu)

1 < Vary: X-Auth-Token

1 < x-openstack-request-id: req-f8b8830c-b091-4a48-9aaa-5fa6f7451abd

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">

<title>404 Not Found</title>

<h1>Not Found</h1>

<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p></span

Join onap-discuss@lists.onap.org to automatically receive all group messages.