Date   

[CVC]Call for Agenda for Next Weeky CVC Joint meeting( March 4th)

Gaoweitao(Victor)
 

Dear VNFSDK/VVP/Dovetail Developers,

 

Here is the proposed agenda wiki for March 4th Joint CVC Meeting: https://wiki.onap.org/display/DW/CVC+Joint+Meeting+03-04-2019

 

Feel free to add your topics.

 

         we already finished all the task we figured out on Paris event and starting coding part. Do we still need the CVC joint meeting?

 

BR

Victor

 


Event: #SO Team #so #cal-invite

onap-discuss@lists.onap.org Calendar <onap-discuss@...>
 

#SO Team

When:
Wednesday, 6 March 2019
6:00am to 7:30am
(GMT-08:00) America/Los Angeles
Repeats: Weekly on Wednesday

Where:
https://zoom.us/j/947366983

Organizer:
seshu.kumar.m@... Bridge: ONAP11

Description:

ONAP Meeting 11 is inviting you to a scheduled Zoom meeting.
 
Topic: #SO Team Meeting
Time: Mar 6, 2019 6:30 AM Pacific Time (US and Canada)
 
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/947366983/ics?icsToken=04c464b216534215674efa5a341ee343b32f6a3eeda6c9d5b2a7869bdc9c990a
 
Join Zoom Meeting
https://zoom.us/j/947366983
 
One tap mobile
+16699006833,,947366983# US (San Jose)
+16465588656,,947366983# US (New York)
 
Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        877 369 0926 US Toll-free
        855 880 1246 US Toll-free
Meeting ID: 947 366 983
Find your local number: https://zoom.us/u/absKHfBYrv
 


moderated ONAP-DISCUSS List Moderators

Kenny Paul
 
Edited

Moderators: Alla Goldner, Michael O'Brien, Michela Bevilacqua, Pam Dragosh, Seshu Kumar, Timo Perala
Calendar:  maintained in the onap-meetings group.
Bridge: the following discuss list meetings are assigned as indicated.
ONAP2 #doc
ONAP3 #ccsdk #cds
ONAP5 #extapi #vnfpkg
ONAP6 #vfc #logging #policy
ONAP7 #aaf #coe #multicloud #pomba
ONAP8 #dmaap #appc
ONAP9 #clamp #vvp #vnfrqts
ONAP10 #university #aai #sdnr
ONAP11 #so #portal #usecaseui #dcae #cia
ONAP12 #oom #vid

Moderators are responsible for either releasing or rejecting held messages, including messages (cross)posted by non-members and messages that exceed globally defined thresholds for message size, number of recipients and/or content type. Anyone can be a Moderator. You do not need to be a TSC member, PTL, Committer code contributor, as long as you are active in the Community

. . . and be subscribed to the list you are moderating.

 

Below are the basic rules for mailing list management across all LF projects:

- A minimum of 3 members from the Community are required as Moderators for all mailing lists

There are have never been any moderators from the Community for either of these lists

 

If the number of Moderators should ever fall below 3 a mailing list will be locked and made read only

I hope that you have enjoyed getting a free ride in that regard for over a year now ;-)

 

Moderators are not granted permissions to modify any list configuration variables. 

I've made a ton of adjustments to both of these lists to try and accommodate your usage patterns as much as I can.


Updated Event: CIA Container Images (updated Mar. 3, 2019) #cal-invite

onap-discuss@lists.onap.org Calendar <onap-discuss@...>
 

CIA Container Images (updated Mar. 3, 2019)

When:
Friday, 8 March 2019
6:00am to 7:00am
(GMT-08:00) America/Los Angeles
Repeats: Weekly on Friday

Where:
https://zoom.us/j/821673222

Organizer:
adolfo.perez-duran@... Bridge: ONAP6

Description:

ONAP6 https://zoom.us/j/821673222
One tap mobile +16699006833,,821673222# US (San Jose) . +16465588656,,821673222# US (New York)
Dial by your location  +1 669 900 6833 US (San Jose)   +1 646 558 8656 US (New York)  +1 855 880 1246 US Toll-free +1 877 369 0926 US Toll-free
Meeting ID: 821 673 222
Find your local number: https://zoom.us/u/aH7nSlgHI

updated 12/4/2018 tic# 64568


Updated Event: #coe Team #coe #cal-invite

onap-discuss@lists.onap.org Calendar <onap-discuss@...>
 

#coe Team

When:
Tuesday, 5 March 2019
1:00pm to 2:00pm
(GMT-08:00) America/Los Angeles
Repeats: Weekly on Tuesday

Where:
https://zoom.us/j/961993336

Organizer:
ritu.sood@... Bridge ONAP5

Description:

iPhone one-tap :US: +16465588656,,961993336# or +16699006833,,961993336# 
Or Telephone: US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
Meeting ID: 961 993 336


Re: Casablanca vFW deployment failed with MSO error request status 404

MALINCONICO ANIELLO PAOLO
 

The keystone identity url should be with the openstack version as http://xx.xx.xx.xx:5000/v2.0 .
Try to override the value by adding the OS keystone version to the identity url.

My override file for Service Orchestrator:

dmaapTopic: "AUTO"
  # openstack configuration
  openStackKeyStoneUrl: "http://xx:xx:xx:xx:5000/v2.0"
  openStackKeystoneAPIVersion: "v2.0"
  openStackPublicNetId: "2c921f24-7e22-40f0-af62-e6801bff0ae1"
  openStackTenantId: "34f1fe41d1a0483dbd1aa94c26dc5545"
  openStackUserName: "dc1"
  openStackServiceTenantName: "service"
  openStackEncryptedPasswordHere: "3xxxxxxxx0"
  openStackRegion: "RegionOne"
  openStackProjectName: "datacenter1"


Aniello Paolo Malinconico


Re: Casablanca vFW deployment failed with MSO error request status 404

Vivekanandan Muthukrishnan
 

Hi Aniello,

Yes, we did select VNF-API (old api)  in the VID home screen. 

Do you know, how to get detailed exception trace in SO openstack adapter debug.log file. I am not able to get the reason why does it fail connect with openstack?

Regards
Vivek


On Sun, Mar 3, 2019 at 11:22 PM MALINCONICO ANIELLO PAOLO <aniello.malinconico@...> wrote:

Hi, Did you make sure to select the VNF-API (old api) (insted of GR-API (new api))box on the front page of the VID screen ?


Aniello Paolo Malinconico

 


Re: Casablanca vFW deployment failed with MSO error request status 404

MALINCONICO ANIELLO PAOLO
 

Hi, Did you make sure to select the VNF-API (old api) (insted of GR-API (new api))box on the front page of the VID screen ?


Aniello Paolo Malinconico

 


Casablanca vFW deployment failed with MSO error request status 404

Vivekanandan Muthukrishnan
 

Hi All,

We tried to deploy vFW from VID with SDNC preload. The VNF module deployment fails with request status 404.

VID UI log

"requestStatus": "Received vfModuleException from VnfAdapter: category='INTERNAL' message='404 Not Found: The resource could not be found.' 

It seems like SO Opentack adapter is not able to connect with openstack. I am attaching here with is the dev-so-so-openstack-adapter.log for your reference. Kindly note that we are using Openstack OCATA and it is accessible from openstack CLI from all k8s nodes.

Did any one face this issue ? Are we missing some SO configurations? We would really appreciate your support.

so-openstack-adapter log snippets

2019-03-03T15:34:24.282Z|c4c8b981-f31f-4da8-b468-1a65dfd1fc94| org.onap.so.openstack.utils.MsoHeatUtils - Found: CloudSite_.._jvstc55_31[regionId=regionOne,identityServiceId=DEFAULT_KEYSTONE,cloudVersion=2.5,clli=regionOne,cloudifyId=<null>,platform=<null>,orchestrator=<null>]
2019-03-03T15:34:24.283Z|c4c8b981-f31f-4da8-b468-1a65dfd1fc94| org.onap.so.openstack.utils.MsoHeatUtils - Found: CloudIdentity[id=DEFAULT_KEYSTONE,identityUrl=http://192.168.37.13:5000,msoId=admin,adminTenant=service,memberRole=admin,tenantMetadata=true,identityServerType=KEYSTONE,identityAuthenticationType=USERNAME_PASSWORD]
2019-03-03T15:34:24.284Z|c4c8b981-f31f-4da8-b468-1a65dfd1fc94| org.onap.so.openstack.utils.MsoHeatUtils - keystoneUrl=http://192.168.37.13:5000
2019-03-03T15:34:24.403Z|c4c8b981-f31f-4da8-b468-1a65dfd1fc94| org.onap.so.openstack.utils.MsoCommonUtils - RA_CONNECTION_EXCEPTION
2019-03-03T15:34:24.404Z|c4c8b981-f31f-4da8-b468-1a65dfd1fc94| org.onap.so.openstack.utils.MsoHeatUtils - RA_CONNECTION_EXCEPTION
2019-03-03T15:34:24.415Z|c4c8b981-f31f-4da8-b468-1a65dfd1fc94| org.onap.so.adapters.vnf.MsoVnfAdapterImpl - RA_QUERY_VNF_ERR
org.onap.so.openstack.exceptions.MsoOpenstackException: The resource could not be found.

Screen Shot

image.png


Casablanca APPC Readiness probe failed: APPC is not healthy for more than 2 hours

Vivekanandan Muthukrishnan
 

Hi All,

It seems like APPC POD dev-appc-appc-0 (Container aapc) is taking more time to install KARAF bundles. And dev-appc-appc-ansible-server is keep getting restarted.

It this expected ? Are there any workaround to offload KARAF packages from local maven repository?

# All APPC PODs
$ kubectl get pods -n onap | grep appc
dev-appc-appc-0                                               1/2       Running            0          2h
dev-appc-appc-ansible-server-6877b497df-j544r                 0/1       Init:0/1           3          37m
dev-appc-appc-cdt-77bccf4847-fmtpw                            1/1       Running            0          2h
dev-appc-appc-db-0                                            1/1       Running            1          2h
dev-appc-appc-db-1                                            1/1       Running            0          2h
dev-appc-appc-db-2                                            1/1       Running            1          2h
dev-appc-appc-dgbuilder-f7565468-fnrz6                        1/1       Running            0          2h


# It seems like ODL KARAF features are still getting installed and it is taking more time
$ kubectl logs -n onap dev-appc-appc-0 -c appc
Adding feature url mvn:org.onap.appc/onap-appc-design-services/1.4.4/xml/features
Archive:  /opt/onap/appc/features/appc-interfaces-service/appc-interfaces-service-1.4.4.zip
   creating: /opt/opendaylight/system/com/google/code/gson/gson/2.8.0/
   creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/
   creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/1.4.4/
   creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/
   creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/1.4.4/
   creating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/
   creating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/1.4.4/
  inflating: /opt/opendaylight/system/com/google/code/gson/gson/2.8.0/_remote.repositories  
  inflating: /opt/opendaylight/system/com/google/code/gson/gson/2.8.0/gson-2.8.0.jar  
  inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/1.4.4/appc-interfaces-service-model-1.4.4.jar  
  inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/1.4.4/_remote.repositories  
  inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/maven-metadata-local.xml  
  inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/1.4.4/appc-interfaces-service-bundle-1.4.4.jar  
  inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/1.4.4/_remote.repositories  
  inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/maven-metadata-local.xml  
  inflating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/1.4.4/_remote.repositories  
  inflating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/1.4.4/onap-appc-interfaces-service-1.4.4-features.xml  
  inflating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/maven-metadata-local.xml  
Adding feature url mvn:org.onap.appc/onap-appc-interfaces-service/1.4.4/xml/features
Installing onap-appc-core
Install of onap-appc-core took 1802 seconds
Sleep Finished
Installing onap-appc-metric

$ kubectl describe pod -n onap dev-appc-appc-0
Name:           dev-appc-appc-0
Namespace:      onap
Node:           casablanca03/192.168.122.233
Start Time:     Sun, 03 Mar 2019 08:39:03 +0000
Labels:         app=appc
                controller-revision-hash=dev-appc-appc-69d746947b
                release=dev-appc
Annotations:    <none>
Status:         Running
IP:             10.42.135.188
Controlled By:  StatefulSet/dev-appc-appc
Init Containers:
  appc-readiness:
    Container ID:  docker://219c491421ce1c5f83539e84508a642c0726930fe4078a72fbf0de65d17aa236
    Image:         oomk8s/readiness-check:2.0.0
    Image ID:      docker://sha256:867cb038e1d2445a6e5aedc3b5f970dacc8249ab119d6c2e088e10df886ff51f
    Port:          <none>
    Host Port:     <none>
    Command:
      /root/ready.py
    Args:
      --container-name
      appc-db
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sun, 03 Mar 2019 08:40:03 +0000
      Finished:     Sun, 03 Mar 2019 08:45:25 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      NAMESPACE:  onap (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-lw9wt (ro)
Containers:
  appc:
    Container ID:  docker://f1c058a8fb540c20beaffcad0349191877527824a0b6e11d55150694c65d6427
    Ports:         8181/TCP, 1830/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /opt/appc/bin/startODL.sh
    State:          Running
      Started:      Sun, 03 Mar 2019 08:48:29 +0000
    Ready:          False
    Restart Count:  0
    Readiness:      exec [/opt/appc/bin/health_check.sh] delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      MYSQL_ROOT_PASSWORD:  <set to the key 'db-root-password' in secret 'dev-appc-appc'>  Optional: false
      SDNC_CONFIG_DIR:      /opt/onap/appc/data/properties
      APPC_CONFIG_DIR:      /opt/onap/appc/data/properties
      DMAAP_TOPIC_ENV:      SUCCESS
      ENABLE_AAF:           true
      ENABLE_ODL_CLUSTER:   false
      APPC_REPLICAS:        1
    Mounts:
      /etc/localtime from localtime (ro)
      /opt/onap/appc/bin/health_check.sh from onap-appc-bin (rw)
      /opt/onap/appc/bin/installAppcDb.sh from onap-appc-bin (rw)
      /opt/onap/appc/bin/startODL.sh from onap-appc-bin (rw)
      /opt/onap/appc/data/properties/aaa-app-config.xml from onap-appc-data-properties (rw)
      /opt/onap/appc/data/properties/aaiclient.properties from onap-appc-data-properties (rw)
      /opt/onap/appc/data/properties/appc.properties from onap-appc-data-properties (rw)
      /opt/onap/appc/data/properties/cadi.properties from onap-appc-data-properties (rw)
      /opt/onap/appc/data/properties/dblib.properties from onap-appc-data-properties (rw)
      /opt/onap/appc/data/properties/svclogic.properties from onap-appc-data-properties (rw)
      /opt/onap/appc/svclogic/bin/showActiveGraphs.sh from onap-appc-svclogic-bin (rw)
      /opt/onap/appc/svclogic/config/svclogic.properties from onap-appc-svclogic-config (rw)
      /opt/onap/ccsdk/bin/installSdncDb.sh from onap-sdnc-bin (rw)
      /opt/onap/ccsdk/bin/startODL.sh from onap-sdnc-bin (rw)
      /opt/onap/ccsdk/data/properties/aaiclient.properties from onap-sdnc-data-properties (rw)
      /opt/onap/ccsdk/data/properties/dblib.properties from onap-sdnc-data-properties (rw)
      /opt/onap/ccsdk/data/properties/svclogic.properties from onap-sdnc-data-properties (rw)
      /opt/onap/ccsdk/svclogic/bin/showActiveGraphs.sh from onap-sdnc-svclogic-bin (rw)
      /opt/onap/ccsdk/svclogic/config/svclogic.properties from onap-sdnc-svclogic-config (rw)
      /opt/opendaylight/current/daexim from dev-appc-appc-data (rw)
      /opt/opendaylight/current/etc/org.ops4j.pax.logging.cfg from log-config (rw)
      /var/log/onap from logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-lw9wt (ro)
  filebeat-onap:
    Container ID:   docker://daf7ddc4a3e4945a1a4cab940906022248696e72c90bb15fa01144cacd3a1833
    Image:          docker.elastic.co/beats/filebeat:5.5.0
    Image ID:       docker://sha256:b61327632415b6d374b9f34cea71cb14f9c352e5259140ce6e3c8eaf8becaa1b
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sun, 03 Mar 2019 08:48:30 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /usr/share/filebeat/data from data-filebeat (rw)
      /usr/share/filebeat/filebeat.yml from filebeat-conf (rw)
      /var/log/onap from logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-lw9wt (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  dev-appc-appc-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  dev-appc-appc-data-dev-appc-appc-0
    ReadOnly:   false
  localtime:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/localtime
    HostPathType:  
  filebeat-conf:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-filebeat
    Optional:  false
  log-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-logging-cfg
    Optional:  false
  logs:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:  
  data-filebeat:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:  
  onap-appc-data-properties:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-appc-data-properties
    Optional:  false
  onap-appc-svclogic-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-appc-svclogic-config
    Optional:  false
  onap-appc-svclogic-bin:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-appc-svclogic-bin
    Optional:  false
  onap-appc-bin:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-appc-bin
    Optional:  false
  onap-sdnc-data-properties:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-sdnc-data-properties
    Optional:  false
  onap-sdnc-svclogic-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-sdnc-svclogic-config
    Optional:  false
  onap-sdnc-svclogic-bin:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-sdnc-svclogic-bin
    Optional:  false
  onap-sdnc-bin:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      dev-appc-appc-onap-sdnc-bin
    Optional:  false
  default-token-lw9wt:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-lw9wt
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age               From                   Message
  ----     ------     ----              ----                   -------
  Warning  Unhealthy  26m (x9 over 1h)  kubelet, casablanca03  Readiness probe failed: APPC is not healthy.
++ ps -e
++ grep startODL
++ wc -l
+ startODL_status=1
++ grep Waiting
++ wc -l
++ /opt/opendaylight/current/bin/client bundle:list
+ waiting_bundles=0
++ /opt/opendaylight/current/bin/client system:start-level
+ run_level='Level 100'
+ '[' 'Level 100' == 'Level 100' ']'
+ '[' 1 -lt 1 ']'
+ echo APPC is not healthy.
+ exit 1
  Warning  Unhealthy  20m (x212 over 1h)  kubelet, casablanca03  (combined from similar events): Readiness probe failed: APPC is not healthy.
++ ps -e
++ grep startODL
++ wc -l
+ startODL_status=1
++ grep Waiting
++ /opt/opendaylight/current/bin/client bundle:list
++ wc -l
+ waiting_bundles=0
++ /opt/opendaylight/current/bin/client system:start-level
+ run_level='Level 100'
+ '[' 'Level 100' == 'Level 100' ']'
+ '[' 1 -lt 1 ']'
+ echo APPC is not healthy.
+ exit 1
  Warning  Unhealthy  16m  kubelet, casablanca03  Readiness probe failed: APPC is not healthy.
++ wc -l
++ ps -e
++ grep startODL
+ startODL_status=1
++ /opt/opendaylight/current/bin/client bundle:list
++ grep Waiting
++ wc -l
+ waiting_bundles=0
++ /opt/opendaylight/current/bin/client system:start-level
+ run_level='Level 100'
+ '[' 'Level 100' == 'Level 100' ']'
+ '[' 1 -lt 1 ']'
+ echo APPC is not healthy.
+ exit 1
  Warning  Unhealthy  11m (x203 over 1h)  kubelet, casablanca03  Readiness probe failed: APPC is not healthy.
++ ps -e
++ grep startODL
++ wc -l
+ startODL_status=1
++ /opt/opendaylight/current/bin/client bundle:list
++ grep Waiting
++ wc -l
+ waiting_bundles=0
++ /opt/opendaylight/current/bin/client system:start-level
+ run_level='Level 100'
+ '[' 'Level 100' == 'Level 100' ']'
+ '[' 1 -lt 1 ']'
+ echo APPC is not healthy.
+ exit 1
  Warning  Unhealthy  5m (x12 over 1h)  kubelet, casablanca03  Readiness probe failed: APPC is not healthy.
++ ps -e
++ wc -l
++ grep startODL
+ startODL_status=1
++ /opt/opendaylight/current/bin/client bundle:list
++ wc -l
++ grep Waiting
+ waiting_bundles=0
++ /opt/opendaylight/current/bin/client system:start-level
+ run_level='Level 100'
+ '[' 'Level 100' == 'Level 100' ']'
+ '[' 1 -lt 1 ']'
+ echo APPC is not healthy.
+ exit 1
  Warning  Unhealthy  1m (x31 over 1h)  kubelet, casablanca03  Readiness probe failed: APPC is not healthy.
++ ps -e
++ wc -l
++ grep startODL
+ startODL_status=1
++ /opt/opendaylight/current/bin/client bundle:list
++ grep Waiting
++ wc -l
+ waiting_bundles=0
++ /opt/opendaylight/current/bin/client system:start-level
+ run_level='Level 100'
+ '[' 'Level 100' == 'Level 100' ']'
+ '[' 1 -lt 1 ']'
+ echo APPC is not healthy.
+ exit 1


=== We can access the karaf

$ kubectl exec -it -n onap dev-appc-appc-0 -c appc -- /bin/bash
root@dev-appc-appc-0:/# ls
bin   dev  home  lib64  mnt  proc  run   srv  tmp  var
boot  etc  lib   media  opt  root  sbin  sys  usr
root@dev-appc-appc-0:/# ps -ef | grep java
root       217   156 15 08:48 ?        00:21:44 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Djava.security.properties=/opt/opendaylight/etc/odl.java.security -Xms128M -Xmx2048m -XX:+UnlockDiagnosticVMOptions -XX:+HeapDumpOnOutOfMemoryError -Dcom.sun.management.jmxremote -Djava.security.egd=file:/dev/./urandom -Djava.endorsed.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed:/usr/lib/jvm/java-8-openjdk-amd64/lib/endorsed:/opt/opendaylight/lib/endorsed -Djava.ext.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/lib/jvm/java-8-openjdk-amd64/lib/ext:/opt/opendaylight/lib/ext -Dkaraf.instances=/opt/opendaylight/instances -Dkaraf.home=/opt/opendaylight -Dkaraf.base=/opt/opendaylight -Dkaraf.data=/opt/opendaylight/data -Dkaraf.etc=/opt/opendaylight/etc -Dkaraf.restart.jvm.supported=true -Djava.io.tmpdir=/opt/opendaylight/data/tmp -Djava.util.logging.config.file=/opt/opendaylight/etc/java.util.logging.properties -Dkaraf.startLocalConsole=false -Dkaraf.startRemoteShell=true -classpath /opt/opendaylight/lib/boot/org.apache.karaf.diagnostic.boot-4.1.5.jar:/opt/opendaylight/lib/boot/org.apache.karaf.jaas.boot-4.1.5.jar:/opt/opendaylight/lib/boot/org.apache.karaf.main-4.1.5.jar:/opt/opendaylight/lib/boot/org.osgi.core-6.0.0.jar org.apache.karaf.main.Main
root     10908  1660  0 10:53 ?        00:00:04 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dkaraf.instances=/opt/opendaylight/instances -Dkaraf.home=/opt/opendaylight -Dkaraf.base=/opt/opendaylight -Dkaraf.etc=/opt/opendaylight/etc -Djava.io.tmpdir=/opt/opendaylight/data/tmp -Djava.util.logging.config.file=/opt/opendaylight/etc/java.util.logging.properties -classpath /opt/opendaylight/system/org/apache/karaf/org.apache.karaf.client/4.1.5/org.apache.karaf.client-4.1.5.jar:/opt/opendaylight/system/org/apache/sshd/sshd-core/1.6.0/sshd-core-1.6.0.jar:/opt/opendaylight/system/org/fusesource/jansi/jansi/1.17/jansi-1.17.jar:/opt/opendaylight/system/org/jline/jline/3.6.0/jline-3.6.0.jar:/opt/opendaylight/system/org/slf4j/slf4j-api/1.7.12/slf4j-api-1.7.12.jar org.apache.karaf.client.Main feature:install -r onap-appc-metric
root     25586 25509  0 11:10 ?        00:00:00 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dkaraf.instances=/opt/opendaylight/current/instances -Dkaraf.home=/opt/opendaylight/current -Dkaraf.base=/opt/opendaylight/current -Dkaraf.etc=/opt/opendaylight/current/etc -Djava.io.tmpdir=/opt/opendaylight/current/data/tmp -Djava.util.logging.config.file=/opt/opendaylight/current/etc/java.util.logging.properties -classpath /opt/opendaylight/current/system/org/apache/karaf/org.apache.karaf.client/4.1.5/org.apache.karaf.client-4.1.5.jar:/opt/opendaylight/current/system/org/apache/sshd/sshd-core/1.6.0/sshd-core-1.6.0.jar:/opt/opendaylight/current/system/org/fusesource/jansi/jansi/1.17/jansi-1.17.jar:/opt/opendaylight/current/system/org/jline/jline/3.6.0/jline-3.6.0.jar:/opt/opendaylight/current/system/org/slf4j/slf4j-api/1.7.12/slf4j-api-1.7.12.jar org.apache.karaf.client.Main system:start-level
root     25637 25428  0 11:10 ?        00:00:00 grep --color=auto java









Casablanca SDNC health check failed for controller-blueprints-db-0

Vivekanandan Muthukrishnan
 

Hi All,

We deployed ONAP Casablanca  and we see that SDNC health check is failing for dev-sdnc-controller-blueprints-db-0.

Kindly note that all SNDC PODs came up except sdnc-controller-blueprints and  sdnc-controller-blueprints-db-0.

Did any one face this issue?  Please let me know, If you have any workaround for this one.

Regards
Vivek

$ kubectl describe pod -n onap dev-sdnc-controller-blueprints-db-0
Events:
  Type     Reason     Age                From                   Message
  ----     ------     ----               ----                   -------
  Warning  Unhealthy  1m (x680 over 1h)  kubelet, casablanca02  Readiness probe failed: ERROR 1045 (28000): Access denied for user 'readinessProbe'@'localhost' (using password: YES)



$ kubectl get pods -n onap | grep sdnc
dev-sdnc-controller-blueprints-54dc68ddcd-76vjz               0/1       Init:0/1     7          1h
dev-sdnc-controller-blueprints-db-0                           0/1       Running      1          1h
dev-sdnc-nengdb-0                                             1/1       Running      4          1h
dev-sdnc-network-name-gen-64fbd6cdfb-gxc7w                    1/1       Running      0          1h
dev-sdnc-sdnc-0                                               2/2       Running      0          1h
dev-sdnc-sdnc-ansible-server-7fc98c97f4-64wfv                 1/1       Running      0          1h
dev-sdnc-sdnc-db-0                                            2/2       Running      0          1h
dev-sdnc-sdnc-dgbuilder-754b86c668-jnj9v                      1/1       Running      0          1h
dev-sdnc-sdnc-dmaap-listener-75f6b78686-ckkgx                 1/1       Running      0          1h
dev-sdnc-sdnc-portal-794556b78d-72wmf                         1/1       Running      0          1h
dev-sdnc-sdnc-ueb-listener-857dffb89b-khqzl                   1/1       Running      2          1h
dev-so-so-sdnc-adapter-568b59d568-nrx94                       1/1       Running      0          1h
dev-vfc-vfc-zte-sdnc-driver-dbcf67f5b-z658s                   1/1       Running      0          1h



Re: SDNC ansible-server container error - Casablanca

Darosz, Piotr (Nokia - PL/Wroclaw)
 

Hi,

this issue is tracked under https://jira.onap.org/browse/SDNC-639

There are some workarounds described for APPC (might work for SDNC too): https://jira.onap.org/browse/OOM-1685  

 

Piotr Darosz
Software Developer, NOKIA


From: onap-discuss@... <onap-discuss@...> on behalf of kranthi guttikonda via Lists.Onap.Org <kranthi.guttikonda=b-yond.com@...>
Sent: Saturday, March 2, 2019 12:47:20 AM
To: onap-discuss@...
Subject: [onap-discuss] SDNC ansible-server container error - Casablanca
 

Hi Team,

 

I am seeing the following error with SDNC-ansible-server container. Here are the logs. Any help would be appreciated.

 

docker logs -f k8s_sdnc-ansible-server_sdnc-sdnc-ansible-server-5868848678-h6rgr_onap_77fbe9ed-3c75-11e9-b47f-02175f16d4b2_10

Collecting PyMySQL

  Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB)

Installing collected packages: PyMySQL

Successfully installed PyMySQL-0.9.3

You are using pip version 8.1.1, however version 19.0.3 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Collecting cherrypy<18.0.0

  Downloading https://files.pythonhosted.org/packages/de/62/caa09bdd89ff16ef0c6caf9453ef45e28a4a0cb88739267fbb14a16336a8/CherryPy-17.4.1-py2.py3-none-any.whl (421kB)

Collecting cheroot>=6.2.4 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/89/10/96d1db9062afd476e6d63895dd3e41ced836228f904532bc640106be20ef/cheroot-6.5.4-py2.py3-none-any.whl (74kB)

Collecting contextlib2 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl

Collecting more-itertools (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/dd/f4/cdfbb6f07f767e0cd8a11b16adfecc735b6eb87dbebda026ce8898e1cc22/more-itertools-6.0.0.tar.gz (68kB)

Collecting portend>=2.1.1 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/81/43/21afd5914b74d4271184ee76f4093b45aa6a580dc6627d72dfc33664c6ac/portend-2.3-py2.py3-none-any.whl

Collecting zc.lockfile (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/58/c2/d7c89bdad237b4b7837609172be3e8bf5630796c0020494a15b97ece8eb1/zc.lockfile-1.4.tar.gz

Collecting six>=1.11.0 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl

Collecting backports.functools-lru-cache (from cheroot>=6.2.4->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/03/8e/2424c0e65c4a066e28f539364deee49b6451f8fcd4f718fefa50cc3dcf48/backports.functools_lru_cache-1.5-py2.py3-none-any.whl

Collecting tempora>=1.8 (from portend>=2.1.1->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/6a/73/22900a52243fdcb2251a10bdb7c6a75fc8d40ab59ec25c01e26823af5126/tempora-1.14-py2.py3-none-any.whl

Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from zc.lockfile->cherrypy<18.0.0)

Collecting pytz (from tempora>=1.8->portend>=2.1.1->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl (510kB)

Collecting jaraco.functools>=1.20 (from tempora>=1.8->portend>=2.1.1->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/12/a4/3e7366d0f5e75dcad7be88524c8cbd0f3a9fb1db243269550981740c57fe/jaraco.functools-2.0-py2.py3-none-any.whl

Building wheels for collected packages: more-itertools, zc.lockfile

  Running setup.py bdist_wheel for more-itertools: started

  Running setup.py bdist_wheel for more-itertools: finished with status 'done'

  Stored in directory: /root/.cache/pip/wheels/d7/8a/37/026dec21baebd47a0c08276bf3a4be178889e0856bc59fa17f

  Running setup.py bdist_wheel for zc.lockfile: started

  Running setup.py bdist_wheel for zc.lockfile: finished with status 'done'

  Stored in directory: /root/.cache/pip/wheels/09/27/a8/323eeff503e981b09d45ca3a0f085eb03c8a249e87157dd23f

Successfully built more-itertools zc.lockfile

Installing collected packages: more-itertools, six, backports.functools-lru-cache, cheroot, contextlib2, pytz, jaraco.functools, tempora, portend, zc.lockfile, cherrypy

Successfully installed backports.functools-lru-cache-1.5 cheroot-6.5.4 cherrypy-17.4.1 contextlib2-0.5.5 jaraco.functools-2.0 more-itertools-6.0.0 portend-2.3 pytz-2018.9 six-1.12.0 tempora-1.14 zc.lockfile-1.4

You are using pip version 8.1.1, however version 19.0.3 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Collecting requests

  Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl (57kB)

Collecting certifi>=2017.4.17 (from requests)

  Downloading https://files.pythonhosted.org/packages/9f/e0/accfc1b56b57e9750eba272e24c4dddeac86852c2bebd1236674d7887e8a/certifi-2018.11.29-py2.py3-none-any.whl (154kB)

Collecting chardet<3.1.0,>=3.0.2 (from requests)

  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)

Collecting idna<2.9,>=2.5 (from requests)

  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)

Collecting urllib3<1.25,>=1.21.1 (from requests)

  Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)

Installing collected packages: certifi, chardet, idna, urllib3, requests

Successfully installed certifi-2018.11.29 chardet-3.0.4 idna-2.8 requests-2.21.0 urllib3-1.24.1

You are using pip version 8.1.1, however version 19.0.3 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Reading package lists...

Building dependency tree...

Reading state information...

The following additional packages will be installed:

  apt-utils cron dh-python distro-info-data gir1.2-glib-2.0 iso-codes

  libapt-inst2.0 libdbus-glib-1-2 libgirepository-1.0-1 libpython3-stdlib

  lsb-release python-apt-common python3 python3-apt python3-dbus python3-gi

  python3-minimal python3-pycurl python3-software-properties python3.5

  python3.5-minimal unattended-upgrades

Suggested packages:

  anacron logrotate checksecurity exim4 | postfix | mail-transport-agent

  isoquery lsb python3-doc python3-tk python3-venv python3-apt-dbg

  python-apt-doc python-dbus-doc python3-dbus-dbg libcurl4-gnutls-dev

  python-pycurl-doc python3-pycurl-dbg python3.5-venv python3.5-doc

  binfmt-support bsd-mailx mail-transport-agent

The following NEW packages will be installed:

  apt-utils cron dh-python distro-info-data gir1.2-glib-2.0 iso-codes

  libapt-inst2.0 libdbus-glib-1-2 libgirepository-1.0-1 libpython3-stdlib

  lsb-release python-apt-common python3 python3-apt python3-dbus python3-gi

  python3-minimal python3-pycurl python3-software-properties python3.5

  python3.5-minimal software-properties-common unattended-upgrades

0 upgraded, 23 newly installed, 0 to remove and 2 not upgraded.

Need to get 5256 kB of archives.

After this operation, 31.7 MB of additional disk space will be used.

Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 cron amd64 3.0pl1-128ubuntu2 [68.4 kB]

Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3.5-minimal amd64 3.5.2-2ubuntu0~16.04.5 [1598 kB]

Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-minimal amd64 3.5.1-3 [23.3 kB]

Get:4 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3.5 amd64 3.5.2-2ubuntu0~16.04.5 [165 kB]

Get:5 http://archive.ubuntu.com/ubuntu xenial/main amd64 libpython3-stdlib amd64 3.5.1-3 [6818 B]

Get:6 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 dh-python all 2.20151103ubuntu1.1 [74.1 kB]

Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3 amd64 3.5.1-3 [8710 B]

Err:8 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libapt-inst2.0 amd64 1.2.29

  404  Not Found [IP: 91.189.88.161 80]

Err:9 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-utils amd64 1.2.29

  404  Not Found [IP: 91.189.88.161 80]

Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 distro-info-data all 0.28ubuntu0.9 [4534 B]

Get:11 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 lsb-release all 9.20160110ubuntu0.2 [11.8 kB]

Get:12 http://archive.ubuntu.com/ubuntu xenial/main amd64 libgirepository-1.0-1 amd64 1.46.0-3ubuntu1 [88.3 kB]

Get:13 http://archive.ubuntu.com/ubuntu xenial/main amd64 gir1.2-glib-2.0 amd64 1.46.0-3ubuntu1 [127 kB]

Get:14 http://archive.ubuntu.com/ubuntu xenial/main amd64 iso-codes all 3.65-1 [2268 kB]

Get:15 http://archive.ubuntu.com/ubuntu xenial/main amd64 libdbus-glib-1-2 amd64 0.106-1 [67.1 kB]

Get:16 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-apt-common all 1.1.0~beta1ubuntu0.16.04.2 [16.0 kB]

Get:17 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3-apt amd64 1.1.0~beta1ubuntu0.16.04.2 [137 kB]

Get:18 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-dbus amd64 1.2.0-3 [83.1 kB]

Get:19 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-gi amd64 3.20.0-0ubuntu1 [153 kB]

Get:20 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-pycurl amd64 7.43.0-1ubuntu1 [42.3 kB]

Get:21 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3-software-properties all 0.96.20.8 [20.2 kB]

Get:22 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 software-properties-common all 0.96.20.8 [9440 B]

Err:23 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 unattended-upgrades all 0.90ubuntu0.9

  404  Not Found [IP: 91.189.88.161 80]

Fetched 4972 kB in 1s (2635 kB/s)

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/a/apt/libapt-inst2.0_1.2.29_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

 

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/a/apt/apt-utils_1.2.29_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

 

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/u/unattended-upgrades/unattended-upgrades_0.90ubuntu0.9_all.deb  404  Not Found [IP: 91.189.88.161 80]

 

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

./startAnsibleServer.sh: line 11: apt-add-repository: command not found

Reading package lists...

Building dependency tree...

Reading state information...

The following additional packages will be installed:

  ieee-data libyaml-0-2 python-crypto python-ecdsa python-httplib2

  python-jinja2 python-markupsafe python-netaddr python-paramiko

  python-selinux python-six python-yaml

Suggested packages:

  sshpass python-crypto-dbg python-crypto-doc python-jinja2-doc ipython

  python-netaddr-docs

The following NEW packages will be installed:

  ansible ieee-data libyaml-0-2 python-crypto python-ecdsa python-httplib2

  python-jinja2 python-markupsafe python-netaddr python-paramiko

  python-selinux python-six python-yaml

0 upgraded, 13 newly installed, 0 to remove and 2 not upgraded.

Need to get 2906 kB of archives.

After this operation, 17.6 MB of additional disk space will be used.

Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libyaml-0-2 amd64 0.1.6-3 [47.6 kB]

Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-crypto amd64 2.6.1-6ubuntu0.16.04.3 [246 kB]

Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-markupsafe amd64 0.23-2build2 [15.5 kB]

Get:4 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-jinja2 all 2.8-1 [109 kB]

Get:5 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-six all 1.10.0-3 [10.9 kB]

Get:6 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-ecdsa all 0.13-2 [34.0 kB]

Get:7 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-paramiko all 1.16.0-1ubuntu0.2 [110 kB]

Get:8 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-yaml amd64 3.11-3build1 [105 kB]

Get:9 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-httplib2 all 0.9.1+dfsg-1 [34.2 kB]

Get:10 http://archive.ubuntu.com/ubuntu xenial/main amd64 ieee-data all 20150531.1 [830 kB]

Get:11 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-netaddr all 0.7.18-1 [174 kB]

Get:12 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 ansible all 2.0.0.2-2ubuntu1.1 [1018 kB]

Get:13 http://archive.ubuntu.com/ubuntu xenial/universe amd64 python-selinux amd64 2.4-3build2 [173 kB]

debconf: delaying package configuration, since apt-utils is not installed

Fetched 2906 kB in 1s (1937 kB/s)

Selecting previously unselected package libyaml-0-2:amd64.

(Reading database ... 30031 files and directories currently installed.)

Preparing to unpack .../libyaml-0-2_0.1.6-3_amd64.deb ...

Unpacking libyaml-0-2:amd64 (0.1.6-3) ...

Selecting previously unselected package python-crypto.

Preparing to unpack .../python-crypto_2.6.1-6ubuntu0.16.04.3_amd64.deb ...

Unpacking python-crypto (2.6.1-6ubuntu0.16.04.3) ...

Selecting previously unselected package python-markupsafe.

Preparing to unpack .../python-markupsafe_0.23-2build2_amd64.deb ...

Unpacking python-markupsafe (0.23-2build2) ...

Selecting previously unselected package python-jinja2.

Preparing to unpack .../python-jinja2_2.8-1_all.deb ...

Unpacking python-jinja2 (2.8-1) ...

Selecting previously unselected package python-six.

Preparing to unpack .../python-six_1.10.0-3_all.deb ...

Unpacking python-six (1.10.0-3) ...

Selecting previously unselected package python-ecdsa.

Preparing to unpack .../python-ecdsa_0.13-2_all.deb ...

Unpacking python-ecdsa (0.13-2) ...

Selecting previously unselected package python-paramiko.

Preparing to unpack .../python-paramiko_1.16.0-1ubuntu0.2_all.deb ...

Unpacking python-paramiko (1.16.0-1ubuntu0.2) ...

Selecting previously unselected package python-yaml.

Preparing to unpack .../python-yaml_3.11-3build1_amd64.deb ...

Unpacking python-yaml (3.11-3build1) ...

Selecting previously unselected package python-httplib2.

Preparing to unpack .../python-httplib2_0.9.1+dfsg-1_all.deb ...

Unpacking python-httplib2 (0.9.1+dfsg-1) ...

Selecting previously unselected package ieee-data.

Preparing to unpack .../ieee-data_20150531.1_all.deb ...

Unpacking ieee-data (20150531.1) ...

Selecting previously unselected package python-netaddr.

Preparing to unpack .../python-netaddr_0.7.18-1_all.deb ...

Unpacking python-netaddr (0.7.18-1) ...

Selecting previously unselected package ansible.

Preparing to unpack .../ansible_2.0.0.2-2ubuntu1.1_all.deb ...

Unpacking ansible (2.0.0.2-2ubuntu1.1) ...

Selecting previously unselected package python-selinux.

Preparing to unpack .../python-selinux_2.4-3build2_amd64.deb ...

Unpacking python-selinux (2.4-3build2) ...

Setting up libyaml-0-2:amd64 (0.1.6-3) ...

Setting up python-crypto (2.6.1-6ubuntu0.16.04.3) ...

Setting up python-markupsafe (0.23-2build2) ...

Setting up python-jinja2 (2.8-1) ...

Setting up python-six (1.10.0-3) ...

Setting up python-ecdsa (0.13-2) ...

Setting up python-paramiko (1.16.0-1ubuntu0.2) ...

Setting up python-yaml (3.11-3build1) ...

Setting up python-httplib2 (0.9.1+dfsg-1) ...

Setting up ieee-data (20150531.1) ...

Setting up python-netaddr (0.7.18-1) ...

Setting up ansible (2.0.0.2-2ubuntu1.1) ...

Setting up python-selinux (2.4-3build2) ...

Processing triggers for libc-bin (2.23-0ubuntu10) ...

Traceback (most recent call last):

  File "RestServer.py", line 34, in <module>

    import cherrypy

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/__init__.py", line 66, in <module>

    from ._cperror import (

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cperror.py", line 129, in <module>

    from more_itertools import always_iterable

  File "/usr/local/lib/python2.7/dist-packages/more_itertools/__init__.py", line 1, in <module>

    from more_itertools.more import *  # noqa

  File "/usr/local/lib/python2.7/dist-packages/more_itertools/more.py", line 329

    def _collate(*iterables, key=lambda a: a, reverse=False):

                               ^

SyntaxError: invalid syntax

 

 

Thanks,
Kranthi


SDNC ansible-server container error - Casablanca

kranthi guttikonda
 

Hi Team,

 

I am seeing the following error with SDNC-ansible-server container. Here are the logs. Any help would be appreciated.

 

docker logs -f k8s_sdnc-ansible-server_sdnc-sdnc-ansible-server-5868848678-h6rgr_onap_77fbe9ed-3c75-11e9-b47f-02175f16d4b2_10

Collecting PyMySQL

  Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB)

Installing collected packages: PyMySQL

Successfully installed PyMySQL-0.9.3

You are using pip version 8.1.1, however version 19.0.3 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Collecting cherrypy<18.0.0

  Downloading https://files.pythonhosted.org/packages/de/62/caa09bdd89ff16ef0c6caf9453ef45e28a4a0cb88739267fbb14a16336a8/CherryPy-17.4.1-py2.py3-none-any.whl (421kB)

Collecting cheroot>=6.2.4 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/89/10/96d1db9062afd476e6d63895dd3e41ced836228f904532bc640106be20ef/cheroot-6.5.4-py2.py3-none-any.whl (74kB)

Collecting contextlib2 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl

Collecting more-itertools (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/dd/f4/cdfbb6f07f767e0cd8a11b16adfecc735b6eb87dbebda026ce8898e1cc22/more-itertools-6.0.0.tar.gz (68kB)

Collecting portend>=2.1.1 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/81/43/21afd5914b74d4271184ee76f4093b45aa6a580dc6627d72dfc33664c6ac/portend-2.3-py2.py3-none-any.whl

Collecting zc.lockfile (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/58/c2/d7c89bdad237b4b7837609172be3e8bf5630796c0020494a15b97ece8eb1/zc.lockfile-1.4.tar.gz

Collecting six>=1.11.0 (from cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl

Collecting backports.functools-lru-cache (from cheroot>=6.2.4->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/03/8e/2424c0e65c4a066e28f539364deee49b6451f8fcd4f718fefa50cc3dcf48/backports.functools_lru_cache-1.5-py2.py3-none-any.whl

Collecting tempora>=1.8 (from portend>=2.1.1->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/6a/73/22900a52243fdcb2251a10bdb7c6a75fc8d40ab59ec25c01e26823af5126/tempora-1.14-py2.py3-none-any.whl

Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from zc.lockfile->cherrypy<18.0.0)

Collecting pytz (from tempora>=1.8->portend>=2.1.1->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl (510kB)

Collecting jaraco.functools>=1.20 (from tempora>=1.8->portend>=2.1.1->cherrypy<18.0.0)

  Downloading https://files.pythonhosted.org/packages/12/a4/3e7366d0f5e75dcad7be88524c8cbd0f3a9fb1db243269550981740c57fe/jaraco.functools-2.0-py2.py3-none-any.whl

Building wheels for collected packages: more-itertools, zc.lockfile

  Running setup.py bdist_wheel for more-itertools: started

  Running setup.py bdist_wheel for more-itertools: finished with status 'done'

  Stored in directory: /root/.cache/pip/wheels/d7/8a/37/026dec21baebd47a0c08276bf3a4be178889e0856bc59fa17f

  Running setup.py bdist_wheel for zc.lockfile: started

  Running setup.py bdist_wheel for zc.lockfile: finished with status 'done'

  Stored in directory: /root/.cache/pip/wheels/09/27/a8/323eeff503e981b09d45ca3a0f085eb03c8a249e87157dd23f

Successfully built more-itertools zc.lockfile

Installing collected packages: more-itertools, six, backports.functools-lru-cache, cheroot, contextlib2, pytz, jaraco.functools, tempora, portend, zc.lockfile, cherrypy

Successfully installed backports.functools-lru-cache-1.5 cheroot-6.5.4 cherrypy-17.4.1 contextlib2-0.5.5 jaraco.functools-2.0 more-itertools-6.0.0 portend-2.3 pytz-2018.9 six-1.12.0 tempora-1.14 zc.lockfile-1.4

You are using pip version 8.1.1, however version 19.0.3 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Collecting requests

  Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl (57kB)

Collecting certifi>=2017.4.17 (from requests)

  Downloading https://files.pythonhosted.org/packages/9f/e0/accfc1b56b57e9750eba272e24c4dddeac86852c2bebd1236674d7887e8a/certifi-2018.11.29-py2.py3-none-any.whl (154kB)

Collecting chardet<3.1.0,>=3.0.2 (from requests)

  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)

Collecting idna<2.9,>=2.5 (from requests)

  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)

Collecting urllib3<1.25,>=1.21.1 (from requests)

  Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)

Installing collected packages: certifi, chardet, idna, urllib3, requests

Successfully installed certifi-2018.11.29 chardet-3.0.4 idna-2.8 requests-2.21.0 urllib3-1.24.1

You are using pip version 8.1.1, however version 19.0.3 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Reading package lists...

Building dependency tree...

Reading state information...

The following additional packages will be installed:

  apt-utils cron dh-python distro-info-data gir1.2-glib-2.0 iso-codes

  libapt-inst2.0 libdbus-glib-1-2 libgirepository-1.0-1 libpython3-stdlib

  lsb-release python-apt-common python3 python3-apt python3-dbus python3-gi

  python3-minimal python3-pycurl python3-software-properties python3.5

  python3.5-minimal unattended-upgrades

Suggested packages:

  anacron logrotate checksecurity exim4 | postfix | mail-transport-agent

  isoquery lsb python3-doc python3-tk python3-venv python3-apt-dbg

  python-apt-doc python-dbus-doc python3-dbus-dbg libcurl4-gnutls-dev

  python-pycurl-doc python3-pycurl-dbg python3.5-venv python3.5-doc

  binfmt-support bsd-mailx mail-transport-agent

The following NEW packages will be installed:

  apt-utils cron dh-python distro-info-data gir1.2-glib-2.0 iso-codes

  libapt-inst2.0 libdbus-glib-1-2 libgirepository-1.0-1 libpython3-stdlib

  lsb-release python-apt-common python3 python3-apt python3-dbus python3-gi

  python3-minimal python3-pycurl python3-software-properties python3.5

  python3.5-minimal software-properties-common unattended-upgrades

0 upgraded, 23 newly installed, 0 to remove and 2 not upgraded.

Need to get 5256 kB of archives.

After this operation, 31.7 MB of additional disk space will be used.

Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 cron amd64 3.0pl1-128ubuntu2 [68.4 kB]

Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3.5-minimal amd64 3.5.2-2ubuntu0~16.04.5 [1598 kB]

Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-minimal amd64 3.5.1-3 [23.3 kB]

Get:4 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3.5 amd64 3.5.2-2ubuntu0~16.04.5 [165 kB]

Get:5 http://archive.ubuntu.com/ubuntu xenial/main amd64 libpython3-stdlib amd64 3.5.1-3 [6818 B]

Get:6 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 dh-python all 2.20151103ubuntu1.1 [74.1 kB]

Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3 amd64 3.5.1-3 [8710 B]

Err:8 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libapt-inst2.0 amd64 1.2.29

  404  Not Found [IP: 91.189.88.161 80]

Err:9 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-utils amd64 1.2.29

  404  Not Found [IP: 91.189.88.161 80]

Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 distro-info-data all 0.28ubuntu0.9 [4534 B]

Get:11 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 lsb-release all 9.20160110ubuntu0.2 [11.8 kB]

Get:12 http://archive.ubuntu.com/ubuntu xenial/main amd64 libgirepository-1.0-1 amd64 1.46.0-3ubuntu1 [88.3 kB]

Get:13 http://archive.ubuntu.com/ubuntu xenial/main amd64 gir1.2-glib-2.0 amd64 1.46.0-3ubuntu1 [127 kB]

Get:14 http://archive.ubuntu.com/ubuntu xenial/main amd64 iso-codes all 3.65-1 [2268 kB]

Get:15 http://archive.ubuntu.com/ubuntu xenial/main amd64 libdbus-glib-1-2 amd64 0.106-1 [67.1 kB]

Get:16 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-apt-common all 1.1.0~beta1ubuntu0.16.04.2 [16.0 kB]

Get:17 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3-apt amd64 1.1.0~beta1ubuntu0.16.04.2 [137 kB]

Get:18 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-dbus amd64 1.2.0-3 [83.1 kB]

Get:19 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-gi amd64 3.20.0-0ubuntu1 [153 kB]

Get:20 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-pycurl amd64 7.43.0-1ubuntu1 [42.3 kB]

Get:21 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3-software-properties all 0.96.20.8 [20.2 kB]

Get:22 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 software-properties-common all 0.96.20.8 [9440 B]

Err:23 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 unattended-upgrades all 0.90ubuntu0.9

  404  Not Found [IP: 91.189.88.161 80]

Fetched 4972 kB in 1s (2635 kB/s)

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/a/apt/libapt-inst2.0_1.2.29_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

 

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/a/apt/apt-utils_1.2.29_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

 

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/u/unattended-upgrades/unattended-upgrades_0.90ubuntu0.9_all.deb  404  Not Found [IP: 91.189.88.161 80]

 

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

./startAnsibleServer.sh: line 11: apt-add-repository: command not found

Reading package lists...

Building dependency tree...

Reading state information...

The following additional packages will be installed:

  ieee-data libyaml-0-2 python-crypto python-ecdsa python-httplib2

  python-jinja2 python-markupsafe python-netaddr python-paramiko

  python-selinux python-six python-yaml

Suggested packages:

  sshpass python-crypto-dbg python-crypto-doc python-jinja2-doc ipython

  python-netaddr-docs

The following NEW packages will be installed:

  ansible ieee-data libyaml-0-2 python-crypto python-ecdsa python-httplib2

  python-jinja2 python-markupsafe python-netaddr python-paramiko

  python-selinux python-six python-yaml

0 upgraded, 13 newly installed, 0 to remove and 2 not upgraded.

Need to get 2906 kB of archives.

After this operation, 17.6 MB of additional disk space will be used.

Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libyaml-0-2 amd64 0.1.6-3 [47.6 kB]

Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-crypto amd64 2.6.1-6ubuntu0.16.04.3 [246 kB]

Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-markupsafe amd64 0.23-2build2 [15.5 kB]

Get:4 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-jinja2 all 2.8-1 [109 kB]

Get:5 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-six all 1.10.0-3 [10.9 kB]

Get:6 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-ecdsa all 0.13-2 [34.0 kB]

Get:7 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-paramiko all 1.16.0-1ubuntu0.2 [110 kB]

Get:8 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-yaml amd64 3.11-3build1 [105 kB]

Get:9 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-httplib2 all 0.9.1+dfsg-1 [34.2 kB]

Get:10 http://archive.ubuntu.com/ubuntu xenial/main amd64 ieee-data all 20150531.1 [830 kB]

Get:11 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-netaddr all 0.7.18-1 [174 kB]

Get:12 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 ansible all 2.0.0.2-2ubuntu1.1 [1018 kB]

Get:13 http://archive.ubuntu.com/ubuntu xenial/universe amd64 python-selinux amd64 2.4-3build2 [173 kB]

debconf: delaying package configuration, since apt-utils is not installed

Fetched 2906 kB in 1s (1937 kB/s)

Selecting previously unselected package libyaml-0-2:amd64.

(Reading database ... 30031 files and directories currently installed.)

Preparing to unpack .../libyaml-0-2_0.1.6-3_amd64.deb ...

Unpacking libyaml-0-2:amd64 (0.1.6-3) ...

Selecting previously unselected package python-crypto.

Preparing to unpack .../python-crypto_2.6.1-6ubuntu0.16.04.3_amd64.deb ...

Unpacking python-crypto (2.6.1-6ubuntu0.16.04.3) ...

Selecting previously unselected package python-markupsafe.

Preparing to unpack .../python-markupsafe_0.23-2build2_amd64.deb ...

Unpacking python-markupsafe (0.23-2build2) ...

Selecting previously unselected package python-jinja2.

Preparing to unpack .../python-jinja2_2.8-1_all.deb ...

Unpacking python-jinja2 (2.8-1) ...

Selecting previously unselected package python-six.

Preparing to unpack .../python-six_1.10.0-3_all.deb ...

Unpacking python-six (1.10.0-3) ...

Selecting previously unselected package python-ecdsa.

Preparing to unpack .../python-ecdsa_0.13-2_all.deb ...

Unpacking python-ecdsa (0.13-2) ...

Selecting previously unselected package python-paramiko.

Preparing to unpack .../python-paramiko_1.16.0-1ubuntu0.2_all.deb ...

Unpacking python-paramiko (1.16.0-1ubuntu0.2) ...

Selecting previously unselected package python-yaml.

Preparing to unpack .../python-yaml_3.11-3build1_amd64.deb ...

Unpacking python-yaml (3.11-3build1) ...

Selecting previously unselected package python-httplib2.

Preparing to unpack .../python-httplib2_0.9.1+dfsg-1_all.deb ...

Unpacking python-httplib2 (0.9.1+dfsg-1) ...

Selecting previously unselected package ieee-data.

Preparing to unpack .../ieee-data_20150531.1_all.deb ...

Unpacking ieee-data (20150531.1) ...

Selecting previously unselected package python-netaddr.

Preparing to unpack .../python-netaddr_0.7.18-1_all.deb ...

Unpacking python-netaddr (0.7.18-1) ...

Selecting previously unselected package ansible.

Preparing to unpack .../ansible_2.0.0.2-2ubuntu1.1_all.deb ...

Unpacking ansible (2.0.0.2-2ubuntu1.1) ...

Selecting previously unselected package python-selinux.

Preparing to unpack .../python-selinux_2.4-3build2_amd64.deb ...

Unpacking python-selinux (2.4-3build2) ...

Setting up libyaml-0-2:amd64 (0.1.6-3) ...

Setting up python-crypto (2.6.1-6ubuntu0.16.04.3) ...

Setting up python-markupsafe (0.23-2build2) ...

Setting up python-jinja2 (2.8-1) ...

Setting up python-six (1.10.0-3) ...

Setting up python-ecdsa (0.13-2) ...

Setting up python-paramiko (1.16.0-1ubuntu0.2) ...

Setting up python-yaml (3.11-3build1) ...

Setting up python-httplib2 (0.9.1+dfsg-1) ...

Setting up ieee-data (20150531.1) ...

Setting up python-netaddr (0.7.18-1) ...

Setting up ansible (2.0.0.2-2ubuntu1.1) ...

Setting up python-selinux (2.4-3build2) ...

Processing triggers for libc-bin (2.23-0ubuntu10) ...

Traceback (most recent call last):

  File "RestServer.py", line 34, in <module>

    import cherrypy

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/__init__.py", line 66, in <module>

    from ._cperror import (

  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cperror.py", line 129, in <module>

    from more_itertools import always_iterable

  File "/usr/local/lib/python2.7/dist-packages/more_itertools/__init__.py", line 1, in <module>

    from more_itertools.more import *  # noqa

  File "/usr/local/lib/python2.7/dist-packages/more_itertools/more.py", line 329

    def _collate(*iterables, key=lambda a: a, reverse=False):

                               ^

SyntaxError: invalid syntax

 

 

Thanks,
Kranthi


Re: [aai] Standalone AAI UI

Arul Nambi
 

Hey Chandra,

In Casablanca yes it is available. Basically you need to disable portal profile(sprin-boot) which can be found in application.properties. If you are running it locally, then it must be disabled by default.

Regards

Arul

 

From: Chandrashekhar Thakare <CT00548828@...>
Sent: Thursday, February 28, 2019 11:44 AM
To: Arul Nambi <Arul.Nambi@...>; onap-discuss@...
Subject: RE: [onap-discuss] [aai] Standalone AAI UI

 

Hi Arul,

Is the standalone UI is available for AAI in Casablanca? Or it is still under development  for Dublin?

 

From: Arul Nambi [mailto:Arul.Nambi@...]
Sent: 06 November 2018 19:47
To: Chandrashekhar Thakare <CT00548828@...>; onap-discuss@...
Subject: RE: [onap-discuss] [aai] Standalone AAI UI

 

Hi Chandra,

Yes you are right, you cannot use sparky without portal in Beijing L

Regards

Arul

 

From: Chandrashekhar Thakare [mailto:CT00548828@...]
Sent: Tuesday, November 6, 2018 3:44 AM
To: onap-discuss@...; Arul Nambi <Arul.Nambi@...>
Subject: RE: [onap-discuss] [aai] Standalone AAI UI

 

Thanks Arul.

Understood that the portal redirection can not be disabled  but does this mean that standalone AAI application UI cannot be accessed in Beijing and we will have to use Casablanca or Master for same?

 

 

 

 

From: onap-discuss@... [mailto:onap-discuss@...] On Behalf Of Arul Nambi
Sent: 05 November 2018 19:01
To: Chandrashekhar Thakare <CT00548828@...>; onap-discuss@...
Subject: Re: [onap-discuss] [aai] Standalone AAI UI

 

Hey Chandra,

Sorry unfortunately we were in the middle of configuration changes in Beijing and there is no way to disable portal in Beijing. Can you move to master/Casablanca? It is stable at this point in time.

Regards

Arul

 

From: Chandrashekhar Thakare [mailto:CT00548828@...]
Sent: Monday, November 5, 2018 4:33 AM
To: Arul Nambi <Arul.Nambi@...>; onap-discuss@...
Subject: RE: [aai] Standalone AAI UI

 

Hi Arul,

Thanks for information.

I am currently using the  following and using Beijing installation. Its heat based installation.

nexus3.onap.org:10001/onap/sparky-be             1.2.1

 

Where exactly I can locate this file ? Inside any specific docker ? I tried to search but could not get the portal profile related details inside aai-resources/aai-traversal.

 

From: Arul Nambi [mailto:Arul.Nambi@...]
Sent: 02 November 2018 16:00
To: onap-discuss@...; Chandrashekhar Thakare <CT00548828@...>
Subject: RE: [aai] Standalone AAI UI

 

Hi Chandra,

Are you which version of sparky are you using? If you using the one from master, then you should be able to disable the portal profile from application.properties in backend. This will make sure that you are not redirected to portal for authentication.

Regards

Arul

 

From: onap-discuss@... [mailto:onap-discuss@...] On Behalf Of Chandra
Sent: Friday, November 2, 2018 5:21 AM
To: onap-discuss@...
Subject: [onap-discuss] [aai] Standalone AAI UI

 

Hi,

I am trying to  access the standalone UI of AAI with below URL-

 

http://<AAI IP address>:9517/services/aai/webapp/index.html

 

However; it is pointing to

 

http://portal.api.simpledemo.onap.org:8989/ONAPPORTAL/login.htm

 

Is there any way to disable this redirection?

============================================================================================================================

Disclaimer:  This message and the information contained herein is proprietary and confidential and subject to the Tech Mahindra policy statement, you may review the policy at http://www.techmahindra.com/Disclaimer.html externally http://tim.techmahindra.com/tim/disclaimer.html internally within TechMahindra.

============================================================================================================================

“Amdocs’ email platform is based on a third-party, worldwide, cloud-based system. Any emails sent to Amdocs will be processed and stored using such system and are accessible by third party providers of such system on a limited basis. Your sending of emails to Amdocs evidences your consent to the use of such system and such processing, storing and access”.

“Amdocs’ email platform is based on a third-party, worldwide, cloud-based system. Any emails sent to Amdocs will be processed and stored using such system and are accessible by third party providers of such system on a limited basis. Your sending of emails to Amdocs evidences your consent to the use of such system and such processing, storing and access”.

“Amdocs’ email platform is based on a third-party, worldwide, cloud-based system. Any emails sent to Amdocs will be processed and stored using such system and are accessible by third party providers of such system on a limited basis. Your sending of emails to Amdocs evidences your consent to the use of such system and such processing, storing and access”.

This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service


Re: dmaap-message-router NodePort not recheable

Brian Freeman
 

I’d do a helm delete dev-sdc –purge

Delete /dockerdata-nfs/dev-so

Configm pv/pvc/pod are gone

Then

 

helm deploy dev-sdc local/onap -f /root/oom/kubernetes/onap/resources/environments/public-cloud.yaml -f /root/integration-override.yaml --namespace onap  --verbose

 

(or whatever your override files are)

 

Looks like SDC came up before dmaap and is confused.

 

There are some less intrusive things to try but you need SDC to Pass Health Check (with DMaaP Up from its perspective)

Basic SDC Health Check                                                (DMaaP:UP)| PASS |

 

 

Brian

 

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:30 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

It works.

 

curl -X POST http://138.132.168.85:30227/events/TEST_TOPIC -H 'cache-control: no-cache'   -H 'content-type: application/json'  -H 'postman-token: 1c679102-85e8-f1a2-e708-3e6d84f8ea06' -d '{ "test": "success",                "timestamp": "1/1/2020" }'

{

    "serverTimeMs": 1,

    "count": 1

 

curl -X GET 'http://138.132.168.85:30227/events/TEST_TOPIC/g1/c3?timeout=5000' -H 'accept: application/json'  -H 'cache-control: no-cache'  -H 'postman-token: 04778117-fd44-0cac-b70c-ef2a2c3024af'                       

["{\"test\":\"success\",\"timestamp\":\"1/1/2020\"}"]

 

Agos.

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 15:21
A: Calamita Agostino <agostino.calamita@...>; onap-discuss@...
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Casablanca.

 

OK

 

Use curl or POSTMAN to write to a TEST_TOPIC (unauthenticated topics are created on demand)

(replace 10.12.5.13 with one of your k8 host IPs) – dont need the postman-token and modify for your environment and preferences etc.

 

curl -X POST \

  http://10.12.5.13:30227/events/TEST_TOPIC \

  -H 'cache-control: no-cache' \

  -H 'content-type: application/json' \

  -H 'postman-token: 1c679102-85e8-f1a2-e708-3e6d84f8ea06' \

  -d '{ "test": "success",

               "timestamp": "1/1/2020"

}'

 

The do a GET

 

curl -X GET \

  'http://10.12.5.13:30227/events/TEST_TOPIC/g1/c3?timeout=5000' \

  -H 'accept: application/json' \

  -H 'cache-control: no-cache' \

  -H 'postman-token: 04778117-fd44-0cac-b70c-ef2a2c3024af'

 

 

You should get the test/timestamp object back on the GET (have to execute the POST/GET twice on the initial topic create)

 

This is to confirm that Mesage Router is internally talking to itself correctly.

 

 

Brian

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:13 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

This is the output:

 

Executing robot tests at log level TRACE

[ ERROR ] Suite 'Testsuites' contains no tests with tag 'healthmr'.

 

Try --help for usage information.

command terminated with exit code 252

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 15:12
A: Calamita Agostino <agostino.calamita@...>; onap-discuss@...
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Please try ./ete-k8s.sh onap healthmr

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:09 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I didn’t find healthmr test but only health.

(

./ete-k8s.sh onap

Usage: ete-k8s.sh [namespace] [ health | healthdist | distribute | instantiate | instantiateVFWCL | instantiateDemoVFWCL |  | portal ] )

 

The command ./ete-k8s.sh onap health reports the list below (  Basic DMAAP Message Router Health Check = PASS )                            

In my environment there are some PODs not in Running state:

 

dev-aai-aai-data-router-5d55646cdc-cc62v                      1/2       CrashLoopBackOff   1084       4d        10.42.79.203    onapkm3   <none>

dev-appc-appc-ansible-server-76fcf9454d-8km9d                 0/1       CrashLoopBackOff   1656       6d        10.42.212.202   onapkm0   <none>

dev-oof-oof-has-api-585497f5-ktjsv                            0/1       Init:0/3           1085       8d        10.42.86.82     onapkm0   <none>

dev-oof-oof-has-controller-9469b9ff8-td4k9                    0/1       Init:1/3           945        8d        10.42.5.110     onapkm2   <none>

dev-oof-oof-has-data-d559897dc-4lmkt                          0/1       Init:1/4           1091       8d        10.42.199.220   onapkm3   <none>

dev-oof-oof-has-healthcheck-jq9xq                             0/1       Init:0/1           1092       8d        10.42.242.145   onapkm3   <none>

dev-oof-oof-has-reservation-868c7c88ff-pv79n                  0/1       Init:1/4           1081       8d        10.42.176.61    onapkm1   <none>

dev-oof-oof-has-solver-6f8bc6fdf4-tw4cj                       0/1       Init:1/4           1084       8d        10.42.29.154    onapkm0   <none>

dev-sdnc-sdnc-ansible-server-7c76f965c6-hqtzl                 0/1       CrashLoopBackOff   1844       8d        10.42.202.36    onapkm3   <none>

dev-sdnc-sdnc-ueb-listener-6d74459c6-tdqhc                    0/1       CrashLoopBackOff   542        1d        10.42.219.51    onapkm2   <none>

 

and multicloud is not deployed.

 

==============================================================================

Testsuites

==============================================================================

Testsuites.Health-Check :: Testing ecomp components are available via calls.

==============================================================================

Basic A&AI Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic AAF Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic AAF SMS Health Check                                            | PASS |

------------------------------------------------------------------------------

Basic APPC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic CLI Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic CLAMP Health Check                                              | PASS |

------------------------------------------------------------------------------

Basic DCAE Health Check                                               [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8102250>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a8d0>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a850>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

| FAIL |

ConnectionError: HTTPConnectionPool(host='dcae-healthcheck.onap', port=80): Max retries exceeded with url: /healthcheck (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a5d0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic DMAAP Data Router Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic DMAAP Message Router Health Check                               | PASS |

------------------------------------------------------------------------------

Basic External API NBI Health Check                                   [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80d6c50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80c4e10>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8199f50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

| FAIL |

ConnectionError: HTTPConnectionPool(host='nbi.onap', port=8080): Max retries exceeded with url: /nbi/api/v3/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80e5410>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic Log Elasticsearch Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic Log Kibana Health Check                                         | PASS |

------------------------------------------------------------------------------

Basic Log Logstash Health Check                                       | PASS |

------------------------------------------------------------------------------

Basic Microservice Bus Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic Multicloud API Health Check                                     | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-ocata API Health Check                               | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-pike API Health Check                                | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-titanium_cloud API Health Check                      | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-vio API Health Check                                 | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic OOF-Homing Health Check                                         | FAIL |

Test timeout 10 seconds exceeded.

------------------------------------------------------------------------------

Basic OOF-SNIRO Health Check                                          | PASS |

------------------------------------------------------------------------------

Basic OOF-CMSO Health Check                                           | PASS |

------------------------------------------------------------------------------

Basic Policy Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Pomba AAI-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba SDC-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba Network-discovery-context-builder Health Check            | PASS |

------------------------------------------------------------------------------

Basic Portal Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic SDC Health Check                                                (DMaaP:None)| PASS |

------------------------------------------------------------------------------

Basic SDNC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic SO Health Check                                                 | PASS |

------------------------------------------------------------------------------

Basic UseCaseUI API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC catalog API Health Check                                    | PASS |

------------------------------------------------------------------------------

Basic VFC emsdriver API Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic VFC gvnfmdriver API Health Check                                | PASS |

------------------------------------------------------------------------------

Basic VFC huaweivnfmdriver API Health Check                           | PASS |

------------------------------------------------------------------------------

Basic VFC jujuvnfmdriver API Health Check                             | PASS |

------------------------------------------------------------------------------

Basic VFC multivimproxy API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nokiavnfmdriver API Health Check                            | PASS |

------------------------------------------------------------------------------

Basic VFC nokiav2driver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nslcm API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC resmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnflcm API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfres API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC workflow API Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic VFC ztesdncdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC ztevnfmdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VID Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic VNFSDK Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Holmes Rule Management API Health Check                         | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Holmes Engine Management API Health Check                       | FAIL |

502 != 200

------------------------------------------------------------------------------

Testsuites.Health-Check :: Testing ecomp components are available ... | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Testsuites                                                            | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Output:  /share/logs/0001_ete_health/output.xml

Log:     /share/logs/0001_ete_health/log.html

Report:  /share/logs/0001_ete_health/report.html

command terminated with exit code 10

 

 

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 14:49
A: onap-discuss@...; Calamita Agostino <agostino.calamita@...>
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Try a POST to make sure you can write to message router.

I doubt its connectivity.

 

If you are on master branch – try ./ete-k8s.sh onap healthmr to test a write/read to a test topic.

 

(do it twice since the first time it creates a test topic and kafka doesnt forward the message till both the publisher and the subscriber have connected)

 

Brian

 

 

From: onap-discuss@... <onap-discuss@...> On Behalf Of Calamita Agostino
Sent: Friday, March 01, 2019 4:30 AM
To: onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I tried to execute a wget command from sdc-be POD to message-router REST API and I see that dmaap-message-router is reacheable from sdc-be.

 

This is the result:

 

# kubectl exec -it  dev-sdc-sdc-be-656bd64b9b-jh57x  -n onap -- /bin/bash

 

bash-4.4# wget "http://message-router:3904/topics"

Connecting to message-router:3904 (10.43.1.20:3904)

topics               100% |*******************************|   131   0:00:00 ETA

bash-4.4# cat topics

{"topics": [

    "__consumer_offsets",

    "champRawEvents",

    "SDC-DISTR-NOTIF-TOPIC-AUTO",

    "org.onap.dmaap.mr.PNF_READY"

]}bash-4.4#

 

But audit.log of sdc-be, after “Distribution Service” action from Portal , says:

 

2019-03-01T08:32:07.986Z        [qtp215145189-323354]   INFO    o.o.sdc.be.filters.BeServletFilter     

ResponseCode=500        InstanceUUID=null       RequestId=d2f65e19-b07b-4266-8be2-f170aba42fb1  AlertSeverity=0 ElapsedTime=3  

EndTimestamp=2019-03-01 08:32:07.986Z   PartnerName=op0001      auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      

StatusCode=ERROR        TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate       

timer=3 CustomField2=500        AuditBeginTimestamp=2019-03-01 08:32:07.983Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate      

ServiceInstanceId=null   ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

 

In the same log file I found a lot of messages like this one:

 

2019-03-01T09:21:31.850Z        [qtp215145189-399996]   INFO    o.o.sdc.be.filters.BeServletFilter      AuditMessage=ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"  ResponseCode=500        InstanceUUID=aai-ml     RequestId=7f01a5b2-ee38-42c9-b7a4-330f020a4134 AlertSeverity=0  ElapsedTime=169 EndTimestamp=2019-03-01 09:21:31.850Z   PartnerName=Apache-HttpClient/4.5.6 (Java/1.8.0_171)    auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR        TargetEntity=registerInDistributionEngine       CustomField1=POST: https://sdc-be.onap:8443/sdc/v1/registerForDistribution      timer=169       CustomField2=500        AuditBeginTimestamp=2019-03-01 09:21:31.681Z    RemoteHost=10.42.209.109        ErrorCategory=ERROR     ServerIPAddress=10.42.179.134   ServiceName=/v1/registerForDistribution ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error       ErrorCode=500   ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"

 

Thanks.

 

Da: onap-discuss@... [mailto:onap-discuss@...] Per conto di Calamita Agostino
Inviato: giovedì 28 febbraio 2019 16:13
A: onap-discuss@...
Oggetto: [onap-discuss] dmaap-message-router NodePort not recheable

 

Hi all,

I have an issue releted to connectivity from sdc-be pod and dmaap-message-router.

My installation is Casablanca 3.0.0 on 7 kubernetes VM cluster.

 

All dmaap pods are up and running:

 

dev-dmaap-dbc-pg-0                                            1/1       Running            0          1d        10.42.173.158   onapkm5   <none>

dev-dmaap-dbc-pg-1                                            1/1       Running            0          1d        10.42.188.140   onapkm2   <none>

dev-dmaap-dbc-pgpool-7b748d5894-mr2m9                         1/1       Running            0          1d        10.42.237.193   onapkm3   <none>

dev-dmaap-dbc-pgpool-7b748d5894-n6dks                         1/1       Running            0          1d        10.42.192.244   onapkm2   <none>

dev-dmaap-dmaap-bus-controller-6757c4c86-8rq5p                1/1       Running            0          1d        10.42.185.132   onapkm1   <none>

dev-dmaap-dmaap-dr-db-bb4c67cfd-tm7td                         1/1       Running            0          1d        10.42.152.59    onapkm1   <none>

dev-dmaap-dmaap-dr-node-66c8749959-tpdtf                      1/1       Running            0          1d        10.42.216.13    onapkm2   <none>

dev-dmaap-dmaap-dr-prov-5c766b8d69-qzqn2                      1/1       Running            0          1d        10.42.115.247   onapkm6   <none>

dev-dmaap-message-router-fb9f4bc7d-5z52j                      1/1       Running            0          6h        10.42.138.31    onapkm3   <none>

dev-dmaap-message-router-kafka-5fbc897f48-4bpb6               1/1       Running            0          1d        10.42.78.141    onapkm4   <none>

dev-dmaap-message-router-zookeeper-557954854-8d6p9            1/1       Running            0          1d        10.42.169.205   onapkm1   <none>

 

but when I try to distribute a service, from SDC Portal, I got “Internal Server Error”.

 

SDC-BE log file traces:

 

2019-02-28T08:50:35.318Z        [qtp215145189-159837]   INFO    o.o.sdc.be.filters.BeServletFilter      ResponseCode=500       

InstanceUUID=null RequestId=dab0fd50-b06e-4a65-b4a8-7d7edeae3e01   AlertSeverity=0 ElapsedTime=99  EndTimestamp=2019-02-28 08:50:35.318Z PartnerName=op0001      auditOn=true       ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR       

TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

timer=99        CustomField2=500   AuditBeginTimestamp=2019-02-28 08:50:35.219Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter     ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

Also SDC healthcheck reports that U-EB Cluster is DOWN.

 

Inside SDC-BE POD, I tried to make a traceroute to “message-router-zookeeper” and to “message-router”.

 

This is the result ( the first is OK, the second one NOT OK ):

 

bash-4.4# traceroute  message-router-zookeeper

traceroute to message-router-zookeeper (10.42.169.205), 30 hops max, 46 byte packets

1  10.42.7.46 (10.42.7.46)  0.213 ms  0.005 ms  0.005 ms

2  10.42.190.179 (10.42.190.179)  0.194 ms  0.145 ms  0.135 ms

3  10.42.169.205 (10.42.169.205)  0.461 ms  0.160 ms  0.134 ms

 

bash-4.4# traceroute  message-router

traceroute to message-router (10.43.1.20), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.009 ms  0.005 ms  0.005 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.344 ms  2.211 ms  1.910 ms     ß 138.132.168.X  is VM public network

 3  138.132.169.2 (138.132.169.2)  5.063 ms  3.859 ms  3.934 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

traceroute to message-router-kafka (10.43.148.154), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.006 ms  0.005 ms  0.004 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.391 ms  0.337 ms  0.314 ms

3  138.132.169.2 (138.132.169.2)  0.803 ms  0.748 ms  0.807 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

It seems that I cannot reach NodePort or ClusterIP inside a POD. This is routing table inside POD:

 

bash-4.4# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         10.42.0.1       0.0.0.0         UG        0 0          0 eth0

10.42.0.0       0.0.0.0         255.255.0.0     U         0 0          0 eth0

 

What can I check on Kubernetes Cluster ?

 

Thanks.

Agostino.

 

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/


R: [onap-discuss] dmaap-message-router NodePort not recheable

Calamita Agostino
 

It works.

 

curl -X POST http://138.132.168.85:30227/events/TEST_TOPIC -H 'cache-control: no-cache'   -H 'content-type: application/json'  -H 'postman-token: 1c679102-85e8-f1a2-e708-3e6d84f8ea06' -d '{ "test": "success",                "timestamp": "1/1/2020" }'

{

    "serverTimeMs": 1,

    "count": 1

 

curl -X GET 'http://138.132.168.85:30227/events/TEST_TOPIC/g1/c3?timeout=5000' -H 'accept: application/json'  -H 'cache-control: no-cache'  -H 'postman-token: 04778117-fd44-0cac-b70c-ef2a2c3024af'                       

["{\"test\":\"success\",\"timestamp\":\"1/1/2020\"}"]

 

Agos.

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 15:21
A: Calamita Agostino <agostino.calamita@...>; onap-discuss@...
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Casablanca.

 

OK

 

Use curl or POSTMAN to write to a TEST_TOPIC (unauthenticated topics are created on demand)

(replace 10.12.5.13 with one of your k8 host IPs) – dont need the postman-token and modify for your environment and preferences etc.

 

curl -X POST \

  http://10.12.5.13:30227/events/TEST_TOPIC \

  -H 'cache-control: no-cache' \

  -H 'content-type: application/json' \

  -H 'postman-token: 1c679102-85e8-f1a2-e708-3e6d84f8ea06' \

  -d '{ "test": "success",

               "timestamp": "1/1/2020"

}'

 

The do a GET

 

curl -X GET \

  'http://10.12.5.13:30227/events/TEST_TOPIC/g1/c3?timeout=5000' \

  -H 'accept: application/json' \

  -H 'cache-control: no-cache' \

  -H 'postman-token: 04778117-fd44-0cac-b70c-ef2a2c3024af'

 

 

You should get the test/timestamp object back on the GET (have to execute the POST/GET twice on the initial topic create)

 

This is to confirm that Mesage Router is internally talking to itself correctly.

 

 

Brian

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:13 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

This is the output:

 

Executing robot tests at log level TRACE

[ ERROR ] Suite 'Testsuites' contains no tests with tag 'healthmr'.

 

Try --help for usage information.

command terminated with exit code 252

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 15:12
A: Calamita Agostino <agostino.calamita@...>; onap-discuss@...
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Please try ./ete-k8s.sh onap healthmr

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:09 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I didn’t find healthmr test but only health.

(

./ete-k8s.sh onap

Usage: ete-k8s.sh [namespace] [ health | healthdist | distribute | instantiate | instantiateVFWCL | instantiateDemoVFWCL |  | portal ] )

 

The command ./ete-k8s.sh onap health reports the list below (  Basic DMAAP Message Router Health Check = PASS )                            

In my environment there are some PODs not in Running state:

 

dev-aai-aai-data-router-5d55646cdc-cc62v                      1/2       CrashLoopBackOff   1084       4d        10.42.79.203    onapkm3   <none>

dev-appc-appc-ansible-server-76fcf9454d-8km9d                 0/1       CrashLoopBackOff   1656       6d        10.42.212.202   onapkm0   <none>

dev-oof-oof-has-api-585497f5-ktjsv                            0/1       Init:0/3           1085       8d        10.42.86.82     onapkm0   <none>

dev-oof-oof-has-controller-9469b9ff8-td4k9                    0/1       Init:1/3           945        8d        10.42.5.110     onapkm2   <none>

dev-oof-oof-has-data-d559897dc-4lmkt                          0/1       Init:1/4           1091       8d        10.42.199.220   onapkm3   <none>

dev-oof-oof-has-healthcheck-jq9xq                             0/1       Init:0/1           1092       8d        10.42.242.145   onapkm3   <none>

dev-oof-oof-has-reservation-868c7c88ff-pv79n                  0/1       Init:1/4           1081       8d        10.42.176.61    onapkm1   <none>

dev-oof-oof-has-solver-6f8bc6fdf4-tw4cj                       0/1       Init:1/4           1084       8d        10.42.29.154    onapkm0   <none>

dev-sdnc-sdnc-ansible-server-7c76f965c6-hqtzl                 0/1       CrashLoopBackOff   1844       8d        10.42.202.36    onapkm3   <none>

dev-sdnc-sdnc-ueb-listener-6d74459c6-tdqhc                    0/1       CrashLoopBackOff   542        1d        10.42.219.51    onapkm2   <none>

 

and multicloud is not deployed.

 

==============================================================================

Testsuites

==============================================================================

Testsuites.Health-Check :: Testing ecomp components are available via calls.

==============================================================================

Basic A&AI Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic AAF Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic AAF SMS Health Check                                            | PASS |

------------------------------------------------------------------------------

Basic APPC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic CLI Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic CLAMP Health Check                                              | PASS |

------------------------------------------------------------------------------

Basic DCAE Health Check                                               [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8102250>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a8d0>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a850>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

| FAIL |

ConnectionError: HTTPConnectionPool(host='dcae-healthcheck.onap', port=80): Max retries exceeded with url: /healthcheck (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a5d0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic DMAAP Data Router Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic DMAAP Message Router Health Check                               | PASS |

------------------------------------------------------------------------------

Basic External API NBI Health Check                                   [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80d6c50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80c4e10>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8199f50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

| FAIL |

ConnectionError: HTTPConnectionPool(host='nbi.onap', port=8080): Max retries exceeded with url: /nbi/api/v3/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80e5410>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic Log Elasticsearch Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic Log Kibana Health Check                                         | PASS |

------------------------------------------------------------------------------

Basic Log Logstash Health Check                                       | PASS |

------------------------------------------------------------------------------

Basic Microservice Bus Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic Multicloud API Health Check                                     | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-ocata API Health Check                               | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-pike API Health Check                                | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-titanium_cloud API Health Check                      | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-vio API Health Check                                 | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic OOF-Homing Health Check                                         | FAIL |

Test timeout 10 seconds exceeded.

------------------------------------------------------------------------------

Basic OOF-SNIRO Health Check                                          | PASS |

------------------------------------------------------------------------------

Basic OOF-CMSO Health Check                                           | PASS |

------------------------------------------------------------------------------

Basic Policy Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Pomba AAI-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba SDC-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba Network-discovery-context-builder Health Check            | PASS |

------------------------------------------------------------------------------

Basic Portal Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic SDC Health Check                                                (DMaaP:None)| PASS |

------------------------------------------------------------------------------

Basic SDNC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic SO Health Check                                                 | PASS |

------------------------------------------------------------------------------

Basic UseCaseUI API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC catalog API Health Check                                    | PASS |

------------------------------------------------------------------------------

Basic VFC emsdriver API Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic VFC gvnfmdriver API Health Check                                | PASS |

------------------------------------------------------------------------------

Basic VFC huaweivnfmdriver API Health Check                           | PASS |

------------------------------------------------------------------------------

Basic VFC jujuvnfmdriver API Health Check                             | PASS |

------------------------------------------------------------------------------

Basic VFC multivimproxy API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nokiavnfmdriver API Health Check                            | PASS |

------------------------------------------------------------------------------

Basic VFC nokiav2driver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nslcm API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC resmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnflcm API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfres API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC workflow API Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic VFC ztesdncdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC ztevnfmdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VID Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic VNFSDK Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Holmes Rule Management API Health Check                         | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Holmes Engine Management API Health Check                       | FAIL |

502 != 200

------------------------------------------------------------------------------

Testsuites.Health-Check :: Testing ecomp components are available ... | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Testsuites                                                            | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Output:  /share/logs/0001_ete_health/output.xml

Log:     /share/logs/0001_ete_health/log.html

Report:  /share/logs/0001_ete_health/report.html

command terminated with exit code 10

 

 

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 14:49
A: onap-discuss@...; Calamita Agostino <agostino.calamita@...>
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Try a POST to make sure you can write to message router.

I doubt its connectivity.

 

If you are on master branch – try ./ete-k8s.sh onap healthmr to test a write/read to a test topic.

 

(do it twice since the first time it creates a test topic and kafka doesnt forward the message till both the publisher and the subscriber have connected)

 

Brian

 

 

From: onap-discuss@... <onap-discuss@...> On Behalf Of Calamita Agostino
Sent: Friday, March 01, 2019 4:30 AM
To: onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I tried to execute a wget command from sdc-be POD to message-router REST API and I see that dmaap-message-router is reacheable from sdc-be.

 

This is the result:

 

# kubectl exec -it  dev-sdc-sdc-be-656bd64b9b-jh57x  -n onap -- /bin/bash

 

bash-4.4# wget "http://message-router:3904/topics"

Connecting to message-router:3904 (10.43.1.20:3904)

topics               100% |*******************************|   131   0:00:00 ETA

bash-4.4# cat topics

{"topics": [

    "__consumer_offsets",

    "champRawEvents",

    "SDC-DISTR-NOTIF-TOPIC-AUTO",

    "org.onap.dmaap.mr.PNF_READY"

]}bash-4.4#

 

But audit.log of sdc-be, after “Distribution Service” action from Portal , says:

 

2019-03-01T08:32:07.986Z        [qtp215145189-323354]   INFO    o.o.sdc.be.filters.BeServletFilter     

ResponseCode=500        InstanceUUID=null       RequestId=d2f65e19-b07b-4266-8be2-f170aba42fb1  AlertSeverity=0 ElapsedTime=3  

EndTimestamp=2019-03-01 08:32:07.986Z   PartnerName=op0001      auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      

StatusCode=ERROR        TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate       

timer=3 CustomField2=500        AuditBeginTimestamp=2019-03-01 08:32:07.983Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate      

ServiceInstanceId=null   ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

 

In the same log file I found a lot of messages like this one:

 

2019-03-01T09:21:31.850Z        [qtp215145189-399996]   INFO    o.o.sdc.be.filters.BeServletFilter      AuditMessage=ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"  ResponseCode=500        InstanceUUID=aai-ml     RequestId=7f01a5b2-ee38-42c9-b7a4-330f020a4134 AlertSeverity=0  ElapsedTime=169 EndTimestamp=2019-03-01 09:21:31.850Z   PartnerName=Apache-HttpClient/4.5.6 (Java/1.8.0_171)    auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR        TargetEntity=registerInDistributionEngine       CustomField1=POST: https://sdc-be.onap:8443/sdc/v1/registerForDistribution      timer=169       CustomField2=500        AuditBeginTimestamp=2019-03-01 09:21:31.681Z    RemoteHost=10.42.209.109        ErrorCategory=ERROR     ServerIPAddress=10.42.179.134   ServiceName=/v1/registerForDistribution ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error       ErrorCode=500   ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"

 

Thanks.

 

Da: onap-discuss@... [mailto:onap-discuss@...] Per conto di Calamita Agostino
Inviato: giovedì 28 febbraio 2019 16:13
A: onap-discuss@...
Oggetto: [onap-discuss] dmaap-message-router NodePort not recheable

 

Hi all,

I have an issue releted to connectivity from sdc-be pod and dmaap-message-router.

My installation is Casablanca 3.0.0 on 7 kubernetes VM cluster.

 

All dmaap pods are up and running:

 

dev-dmaap-dbc-pg-0                                            1/1       Running            0          1d        10.42.173.158   onapkm5   <none>

dev-dmaap-dbc-pg-1                                            1/1       Running            0          1d        10.42.188.140   onapkm2   <none>

dev-dmaap-dbc-pgpool-7b748d5894-mr2m9                         1/1       Running            0          1d        10.42.237.193   onapkm3   <none>

dev-dmaap-dbc-pgpool-7b748d5894-n6dks                         1/1       Running            0          1d        10.42.192.244   onapkm2   <none>

dev-dmaap-dmaap-bus-controller-6757c4c86-8rq5p                1/1       Running            0          1d        10.42.185.132   onapkm1   <none>

dev-dmaap-dmaap-dr-db-bb4c67cfd-tm7td                         1/1       Running            0          1d        10.42.152.59    onapkm1   <none>

dev-dmaap-dmaap-dr-node-66c8749959-tpdtf                      1/1       Running            0          1d        10.42.216.13    onapkm2   <none>

dev-dmaap-dmaap-dr-prov-5c766b8d69-qzqn2                      1/1       Running            0          1d        10.42.115.247   onapkm6   <none>

dev-dmaap-message-router-fb9f4bc7d-5z52j                      1/1       Running            0          6h        10.42.138.31    onapkm3   <none>

dev-dmaap-message-router-kafka-5fbc897f48-4bpb6               1/1       Running            0          1d        10.42.78.141    onapkm4   <none>

dev-dmaap-message-router-zookeeper-557954854-8d6p9            1/1       Running            0          1d        10.42.169.205   onapkm1   <none>

 

but when I try to distribute a service, from SDC Portal, I got “Internal Server Error”.

 

SDC-BE log file traces:

 

2019-02-28T08:50:35.318Z        [qtp215145189-159837]   INFO    o.o.sdc.be.filters.BeServletFilter      ResponseCode=500       

InstanceUUID=null RequestId=dab0fd50-b06e-4a65-b4a8-7d7edeae3e01   AlertSeverity=0 ElapsedTime=99  EndTimestamp=2019-02-28 08:50:35.318Z PartnerName=op0001      auditOn=true       ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR       

TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

timer=99        CustomField2=500   AuditBeginTimestamp=2019-02-28 08:50:35.219Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter     ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

Also SDC healthcheck reports that U-EB Cluster is DOWN.

 

Inside SDC-BE POD, I tried to make a traceroute to “message-router-zookeeper” and to “message-router”.

 

This is the result ( the first is OK, the second one NOT OK ):

 

bash-4.4# traceroute  message-router-zookeeper

traceroute to message-router-zookeeper (10.42.169.205), 30 hops max, 46 byte packets

1  10.42.7.46 (10.42.7.46)  0.213 ms  0.005 ms  0.005 ms

2  10.42.190.179 (10.42.190.179)  0.194 ms  0.145 ms  0.135 ms

3  10.42.169.205 (10.42.169.205)  0.461 ms  0.160 ms  0.134 ms

 

bash-4.4# traceroute  message-router

traceroute to message-router (10.43.1.20), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.009 ms  0.005 ms  0.005 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.344 ms  2.211 ms  1.910 ms     ß 138.132.168.X  is VM public network

 3  138.132.169.2 (138.132.169.2)  5.063 ms  3.859 ms  3.934 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

traceroute to message-router-kafka (10.43.148.154), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.006 ms  0.005 ms  0.004 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.391 ms  0.337 ms  0.314 ms

3  138.132.169.2 (138.132.169.2)  0.803 ms  0.748 ms  0.807 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

It seems that I cannot reach NodePort or ClusterIP inside a POD. This is routing table inside POD:

 

bash-4.4# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         10.42.0.1       0.0.0.0         UG        0 0          0 eth0

10.42.0.0       0.0.0.0         255.255.0.0     U         0 0          0 eth0

 

What can I check on Kubernetes Cluster ?

 

Thanks.

Agostino.

 

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/


Re: dmaap-message-router NodePort not recheable

UNNAVA, SUNIL
 

What is the dmaap zookeeper image used in the Helm charts.

 

Thanks,

Suni

 

From: onap-discuss@... <onap-discuss@...> On Behalf Of Calamita Agostino
Sent: Friday, March 01, 2019 9:13 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

This is the output:

 

Executing robot tests at log level TRACE

[ ERROR ] Suite 'Testsuites' contains no tests with tag 'healthmr'.

 

Try --help for usage information.

command terminated with exit code 252

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 15:12
A: Calamita Agostino <agostino.calamita@...>; onap-discuss@...
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Please try ./ete-k8s.sh onap healthmr

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:09 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I didn’t find healthmr test but only health.

(

./ete-k8s.sh onap

Usage: ete-k8s.sh [namespace] [ health | healthdist | distribute | instantiate | instantiateVFWCL | instantiateDemoVFWCL |  | portal ] )

 

The command ./ete-k8s.sh onap health reports the list below (  Basic DMAAP Message Router Health Check = PASS )                            

In my environment there are some PODs not in Running state:

 

dev-aai-aai-data-router-5d55646cdc-cc62v                      1/2       CrashLoopBackOff   1084       4d        10.42.79.203    onapkm3   <none>

dev-appc-appc-ansible-server-76fcf9454d-8km9d                 0/1       CrashLoopBackOff   1656       6d        10.42.212.202   onapkm0   <none>

dev-oof-oof-has-api-585497f5-ktjsv                            0/1       Init:0/3           1085       8d        10.42.86.82     onapkm0   <none>

dev-oof-oof-has-controller-9469b9ff8-td4k9                    0/1       Init:1/3           945        8d        10.42.5.110     onapkm2   <none>

dev-oof-oof-has-data-d559897dc-4lmkt                          0/1       Init:1/4           1091       8d        10.42.199.220   onapkm3   <none>

dev-oof-oof-has-healthcheck-jq9xq                             0/1       Init:0/1           1092       8d        10.42.242.145   onapkm3   <none>

dev-oof-oof-has-reservation-868c7c88ff-pv79n                  0/1       Init:1/4           1081       8d        10.42.176.61    onapkm1   <none>

dev-oof-oof-has-solver-6f8bc6fdf4-tw4cj                       0/1       Init:1/4           1084       8d        10.42.29.154    onapkm0   <none>

dev-sdnc-sdnc-ansible-server-7c76f965c6-hqtzl                 0/1       CrashLoopBackOff   1844       8d        10.42.202.36    onapkm3   <none>

dev-sdnc-sdnc-ueb-listener-6d74459c6-tdqhc                    0/1       CrashLoopBackOff   542        1d        10.42.219.51    onapkm2   <none>

 

and multicloud is not deployed.

 

==============================================================================

Testsuites

==============================================================================

Testsuites.Health-Check :: Testing ecomp components are available via calls.

==============================================================================

Basic A&AI Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic AAF Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic AAF SMS Health Check                                            | PASS |

------------------------------------------------------------------------------

Basic APPC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic CLI Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic CLAMP Health Check                                              | PASS |

------------------------------------------------------------------------------

Basic DCAE Health Check                                               [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8102250>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a8d0>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a850>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

| FAIL |

ConnectionError: HTTPConnectionPool(host='dcae-healthcheck.onap', port=80): Max retries exceeded with url: /healthcheck (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a5d0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic DMAAP Data Router Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic DMAAP Message Router Health Check                               | PASS |

------------------------------------------------------------------------------

Basic External API NBI Health Check                                   [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80d6c50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80c4e10>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8199f50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

| FAIL |

ConnectionError: HTTPConnectionPool(host='nbi.onap', port=8080): Max retries exceeded with url: /nbi/api/v3/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80e5410>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic Log Elasticsearch Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic Log Kibana Health Check                                         | PASS |

------------------------------------------------------------------------------

Basic Log Logstash Health Check                                       | PASS |

------------------------------------------------------------------------------

Basic Microservice Bus Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic Multicloud API Health Check                                     | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-ocata API Health Check                               | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-pike API Health Check                                | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-titanium_cloud API Health Check                      | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-vio API Health Check                                 | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic OOF-Homing Health Check                                         | FAIL |

Test timeout 10 seconds exceeded.

------------------------------------------------------------------------------

Basic OOF-SNIRO Health Check                                          | PASS |

------------------------------------------------------------------------------

Basic OOF-CMSO Health Check                                           | PASS |

------------------------------------------------------------------------------

Basic Policy Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Pomba AAI-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba SDC-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba Network-discovery-context-builder Health Check            | PASS |

------------------------------------------------------------------------------

Basic Portal Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic SDC Health Check                                                (DMaaP:None)| PASS |

------------------------------------------------------------------------------

Basic SDNC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic SO Health Check                                                 | PASS |

------------------------------------------------------------------------------

Basic UseCaseUI API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC catalog API Health Check                                    | PASS |

------------------------------------------------------------------------------

Basic VFC emsdriver API Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic VFC gvnfmdriver API Health Check                                | PASS |

------------------------------------------------------------------------------

Basic VFC huaweivnfmdriver API Health Check                           | PASS |

------------------------------------------------------------------------------

Basic VFC jujuvnfmdriver API Health Check                             | PASS |

------------------------------------------------------------------------------

Basic VFC multivimproxy API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nokiavnfmdriver API Health Check                            | PASS |

------------------------------------------------------------------------------

Basic VFC nokiav2driver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nslcm API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC resmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnflcm API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfres API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC workflow API Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic VFC ztesdncdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC ztevnfmdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VID Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic VNFSDK Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Holmes Rule Management API Health Check                         | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Holmes Engine Management API Health Check                       | FAIL |

502 != 200

------------------------------------------------------------------------------

Testsuites.Health-Check :: Testing ecomp components are available ... | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Testsuites                                                            | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Output:  /share/logs/0001_ete_health/output.xml

Log:     /share/logs/0001_ete_health/log.html

Report:  /share/logs/0001_ete_health/report.html

command terminated with exit code 10

 

 

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 14:49
A: onap-discuss@...; Calamita Agostino <agostino.calamita@...>
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Try a POST to make sure you can write to message router.

I doubt its connectivity.

 

If you are on master branch – try ./ete-k8s.sh onap healthmr to test a write/read to a test topic.

 

(do it twice since the first time it creates a test topic and kafka doesnt forward the message till both the publisher and the subscriber have connected)

 

Brian

 

 

From: onap-discuss@... <onap-discuss@...> On Behalf Of Calamita Agostino
Sent: Friday, March 01, 2019 4:30 AM
To: onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I tried to execute a wget command from sdc-be POD to message-router REST API and I see that dmaap-message-router is reacheable from sdc-be.

 

This is the result:

 

# kubectl exec -it  dev-sdc-sdc-be-656bd64b9b-jh57x  -n onap -- /bin/bash

 

bash-4.4# wget "http://message-router:3904/topics"

Connecting to message-router:3904 (10.43.1.20:3904)

topics               100% |*******************************|   131   0:00:00 ETA

bash-4.4# cat topics

{"topics": [

    "__consumer_offsets",

    "champRawEvents",

    "SDC-DISTR-NOTIF-TOPIC-AUTO",

    "org.onap.dmaap.mr.PNF_READY"

]}bash-4.4#

 

But audit.log of sdc-be, after “Distribution Service” action from Portal , says:

 

2019-03-01T08:32:07.986Z        [qtp215145189-323354]   INFO    o.o.sdc.be.filters.BeServletFilter     

ResponseCode=500        InstanceUUID=null       RequestId=d2f65e19-b07b-4266-8be2-f170aba42fb1  AlertSeverity=0 ElapsedTime=3  

EndTimestamp=2019-03-01 08:32:07.986Z   PartnerName=op0001      auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      

StatusCode=ERROR        TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate       

timer=3 CustomField2=500        AuditBeginTimestamp=2019-03-01 08:32:07.983Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate      

ServiceInstanceId=null   ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

 

In the same log file I found a lot of messages like this one:

 

2019-03-01T09:21:31.850Z        [qtp215145189-399996]   INFO    o.o.sdc.be.filters.BeServletFilter      AuditMessage=ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"  ResponseCode=500        InstanceUUID=aai-ml     RequestId=7f01a5b2-ee38-42c9-b7a4-330f020a4134 AlertSeverity=0  ElapsedTime=169 EndTimestamp=2019-03-01 09:21:31.850Z   PartnerName=Apache-HttpClient/4.5.6 (Java/1.8.0_171)    auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR        TargetEntity=registerInDistributionEngine       CustomField1=POST: https://sdc-be.onap:8443/sdc/v1/registerForDistribution      timer=169       CustomField2=500        AuditBeginTimestamp=2019-03-01 09:21:31.681Z    RemoteHost=10.42.209.109        ErrorCategory=ERROR     ServerIPAddress=10.42.179.134   ServiceName=/v1/registerForDistribution ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error       ErrorCode=500   ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"

 

Thanks.

 

Da: onap-discuss@... [mailto:onap-discuss@...] Per conto di Calamita Agostino
Inviato: giovedì 28 febbraio 2019 16:13
A: onap-discuss@...
Oggetto: [onap-discuss] dmaap-message-router NodePort not recheable

 

Hi all,

I have an issue releted to connectivity from sdc-be pod and dmaap-message-router.

My installation is Casablanca 3.0.0 on 7 kubernetes VM cluster.

 

All dmaap pods are up and running:

 

dev-dmaap-dbc-pg-0                                            1/1       Running            0          1d        10.42.173.158   onapkm5   <none>

dev-dmaap-dbc-pg-1                                            1/1       Running            0          1d        10.42.188.140   onapkm2   <none>

dev-dmaap-dbc-pgpool-7b748d5894-mr2m9                         1/1       Running            0          1d        10.42.237.193   onapkm3   <none>

dev-dmaap-dbc-pgpool-7b748d5894-n6dks                         1/1       Running            0          1d        10.42.192.244   onapkm2   <none>

dev-dmaap-dmaap-bus-controller-6757c4c86-8rq5p                1/1       Running            0          1d        10.42.185.132   onapkm1   <none>

dev-dmaap-dmaap-dr-db-bb4c67cfd-tm7td                         1/1       Running            0          1d        10.42.152.59    onapkm1   <none>

dev-dmaap-dmaap-dr-node-66c8749959-tpdtf                      1/1       Running            0          1d        10.42.216.13    onapkm2   <none>

dev-dmaap-dmaap-dr-prov-5c766b8d69-qzqn2                      1/1       Running            0          1d        10.42.115.247   onapkm6   <none>

dev-dmaap-message-router-fb9f4bc7d-5z52j                      1/1       Running            0          6h        10.42.138.31    onapkm3   <none>

dev-dmaap-message-router-kafka-5fbc897f48-4bpb6               1/1       Running            0          1d        10.42.78.141    onapkm4   <none>

dev-dmaap-message-router-zookeeper-557954854-8d6p9            1/1       Running            0          1d        10.42.169.205   onapkm1   <none>

 

but when I try to distribute a service, from SDC Portal, I got “Internal Server Error”.

 

SDC-BE log file traces:

 

2019-02-28T08:50:35.318Z        [qtp215145189-159837]   INFO    o.o.sdc.be.filters.BeServletFilter      ResponseCode=500       

InstanceUUID=null RequestId=dab0fd50-b06e-4a65-b4a8-7d7edeae3e01   AlertSeverity=0 ElapsedTime=99  EndTimestamp=2019-02-28 08:50:35.318Z PartnerName=op0001      auditOn=true       ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR       

TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

timer=99        CustomField2=500   AuditBeginTimestamp=2019-02-28 08:50:35.219Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter     ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

Also SDC healthcheck reports that U-EB Cluster is DOWN.

 

Inside SDC-BE POD, I tried to make a traceroute to “message-router-zookeeper” and to “message-router”.

 

This is the result ( the first is OK, the second one NOT OK ):

 

bash-4.4# traceroute  message-router-zookeeper

traceroute to message-router-zookeeper (10.42.169.205), 30 hops max, 46 byte packets

1  10.42.7.46 (10.42.7.46)  0.213 ms  0.005 ms  0.005 ms

2  10.42.190.179 (10.42.190.179)  0.194 ms  0.145 ms  0.135 ms

3  10.42.169.205 (10.42.169.205)  0.461 ms  0.160 ms  0.134 ms

 

bash-4.4# traceroute  message-router

traceroute to message-router (10.43.1.20), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.009 ms  0.005 ms  0.005 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.344 ms  2.211 ms  1.910 ms     ß 138.132.168.X  is VM public network

 3  138.132.169.2 (138.132.169.2)  5.063 ms  3.859 ms  3.934 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

traceroute to message-router-kafka (10.43.148.154), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.006 ms  0.005 ms  0.004 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.391 ms  0.337 ms  0.314 ms

3  138.132.169.2 (138.132.169.2)  0.803 ms  0.748 ms  0.807 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

It seems that I cannot reach NodePort or ClusterIP inside a POD. This is routing table inside POD:

 

bash-4.4# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         10.42.0.1       0.0.0.0         UG        0 0          0 eth0

10.42.0.0       0.0.0.0         255.255.0.0     U         0 0          0 eth0

 

What can I check on Kubernetes Cluster ?

 

Thanks.

Agostino.

 

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/


Re: dmaap-message-router NodePort not recheable

Brian Freeman
 

Casablanca.

 

OK

 

Use curl or POSTMAN to write to a TEST_TOPIC (unauthenticated topics are created on demand)

(replace 10.12.5.13 with one of your k8 host IPs) – dont need the postman-token and modify for your environment and preferences etc.

 

curl -X POST \

  http://10.12.5.13:30227/events/TEST_TOPIC \

  -H 'cache-control: no-cache' \

  -H 'content-type: application/json' \

  -H 'postman-token: 1c679102-85e8-f1a2-e708-3e6d84f8ea06' \

  -d '{ "test": "success",

               "timestamp": "1/1/2020"

}'

 

The do a GET

 

curl -X GET \

  'http://10.12.5.13:30227/events/TEST_TOPIC/g1/c3?timeout=5000' \

  -H 'accept: application/json' \

  -H 'cache-control: no-cache' \

  -H 'postman-token: 04778117-fd44-0cac-b70c-ef2a2c3024af'

 

 

You should get the test/timestamp object back on the GET (have to execute the POST/GET twice on the initial topic create)

 

This is to confirm that Mesage Router is internally talking to itself correctly.

 

 

Brian

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:13 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

This is the output:

 

Executing robot tests at log level TRACE

[ ERROR ] Suite 'Testsuites' contains no tests with tag 'healthmr'.

 

Try --help for usage information.

command terminated with exit code 252

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 15:12
A: Calamita Agostino <agostino.calamita@...>; onap-discuss@...
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Please try ./ete-k8s.sh onap healthmr

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:09 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I didn’t find healthmr test but only health.

(

./ete-k8s.sh onap

Usage: ete-k8s.sh [namespace] [ health | healthdist | distribute | instantiate | instantiateVFWCL | instantiateDemoVFWCL |  | portal ] )

 

The command ./ete-k8s.sh onap health reports the list below (  Basic DMAAP Message Router Health Check = PASS )                            

In my environment there are some PODs not in Running state:

 

dev-aai-aai-data-router-5d55646cdc-cc62v                      1/2       CrashLoopBackOff   1084       4d        10.42.79.203    onapkm3   <none>

dev-appc-appc-ansible-server-76fcf9454d-8km9d                 0/1       CrashLoopBackOff   1656       6d        10.42.212.202   onapkm0   <none>

dev-oof-oof-has-api-585497f5-ktjsv                            0/1       Init:0/3           1085       8d        10.42.86.82     onapkm0   <none>

dev-oof-oof-has-controller-9469b9ff8-td4k9                    0/1       Init:1/3           945        8d        10.42.5.110     onapkm2   <none>

dev-oof-oof-has-data-d559897dc-4lmkt                          0/1       Init:1/4           1091       8d        10.42.199.220   onapkm3   <none>

dev-oof-oof-has-healthcheck-jq9xq                             0/1       Init:0/1           1092       8d        10.42.242.145   onapkm3   <none>

dev-oof-oof-has-reservation-868c7c88ff-pv79n                  0/1       Init:1/4           1081       8d        10.42.176.61    onapkm1   <none>

dev-oof-oof-has-solver-6f8bc6fdf4-tw4cj                       0/1       Init:1/4           1084       8d        10.42.29.154    onapkm0   <none>

dev-sdnc-sdnc-ansible-server-7c76f965c6-hqtzl                 0/1       CrashLoopBackOff   1844       8d        10.42.202.36    onapkm3   <none>

dev-sdnc-sdnc-ueb-listener-6d74459c6-tdqhc                    0/1       CrashLoopBackOff   542        1d        10.42.219.51    onapkm2   <none>

 

and multicloud is not deployed.

 

==============================================================================

Testsuites

==============================================================================

Testsuites.Health-Check :: Testing ecomp components are available via calls.

==============================================================================

Basic A&AI Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic AAF Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic AAF SMS Health Check                                            | PASS |

------------------------------------------------------------------------------

Basic APPC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic CLI Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic CLAMP Health Check                                              | PASS |

------------------------------------------------------------------------------

Basic DCAE Health Check                                               [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8102250>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a8d0>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a850>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

| FAIL |

ConnectionError: HTTPConnectionPool(host='dcae-healthcheck.onap', port=80): Max retries exceeded with url: /healthcheck (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a5d0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic DMAAP Data Router Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic DMAAP Message Router Health Check                               | PASS |

------------------------------------------------------------------------------

Basic External API NBI Health Check                                   [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80d6c50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80c4e10>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8199f50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

| FAIL |

ConnectionError: HTTPConnectionPool(host='nbi.onap', port=8080): Max retries exceeded with url: /nbi/api/v3/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80e5410>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic Log Elasticsearch Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic Log Kibana Health Check                                         | PASS |

------------------------------------------------------------------------------

Basic Log Logstash Health Check                                       | PASS |

------------------------------------------------------------------------------

Basic Microservice Bus Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic Multicloud API Health Check                                     | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-ocata API Health Check                               | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-pike API Health Check                                | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-titanium_cloud API Health Check                      | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-vio API Health Check                                 | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic OOF-Homing Health Check                                         | FAIL |

Test timeout 10 seconds exceeded.

------------------------------------------------------------------------------

Basic OOF-SNIRO Health Check                                          | PASS |

------------------------------------------------------------------------------

Basic OOF-CMSO Health Check                                           | PASS |

------------------------------------------------------------------------------

Basic Policy Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Pomba AAI-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba SDC-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba Network-discovery-context-builder Health Check            | PASS |

------------------------------------------------------------------------------

Basic Portal Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic SDC Health Check                                                (DMaaP:None)| PASS |

------------------------------------------------------------------------------

Basic SDNC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic SO Health Check                                                 | PASS |

------------------------------------------------------------------------------

Basic UseCaseUI API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC catalog API Health Check                                    | PASS |

------------------------------------------------------------------------------

Basic VFC emsdriver API Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic VFC gvnfmdriver API Health Check                                | PASS |

------------------------------------------------------------------------------

Basic VFC huaweivnfmdriver API Health Check                           | PASS |

------------------------------------------------------------------------------

Basic VFC jujuvnfmdriver API Health Check                             | PASS |

------------------------------------------------------------------------------

Basic VFC multivimproxy API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nokiavnfmdriver API Health Check                            | PASS |

------------------------------------------------------------------------------

Basic VFC nokiav2driver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nslcm API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC resmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnflcm API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfres API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC workflow API Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic VFC ztesdncdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC ztevnfmdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VID Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic VNFSDK Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Holmes Rule Management API Health Check                         | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Holmes Engine Management API Health Check                       | FAIL |

502 != 200

------------------------------------------------------------------------------

Testsuites.Health-Check :: Testing ecomp components are available ... | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Testsuites                                                            | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Output:  /share/logs/0001_ete_health/output.xml

Log:     /share/logs/0001_ete_health/log.html

Report:  /share/logs/0001_ete_health/report.html

command terminated with exit code 10

 

 

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 14:49
A: onap-discuss@...; Calamita Agostino <agostino.calamita@...>
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Try a POST to make sure you can write to message router.

I doubt its connectivity.

 

If you are on master branch – try ./ete-k8s.sh onap healthmr to test a write/read to a test topic.

 

(do it twice since the first time it creates a test topic and kafka doesnt forward the message till both the publisher and the subscriber have connected)

 

Brian

 

 

From: onap-discuss@... <onap-discuss@...> On Behalf Of Calamita Agostino
Sent: Friday, March 01, 2019 4:30 AM
To: onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I tried to execute a wget command from sdc-be POD to message-router REST API and I see that dmaap-message-router is reacheable from sdc-be.

 

This is the result:

 

# kubectl exec -it  dev-sdc-sdc-be-656bd64b9b-jh57x  -n onap -- /bin/bash

 

bash-4.4# wget "http://message-router:3904/topics"

Connecting to message-router:3904 (10.43.1.20:3904)

topics               100% |*******************************|   131   0:00:00 ETA

bash-4.4# cat topics

{"topics": [

    "__consumer_offsets",

    "champRawEvents",

    "SDC-DISTR-NOTIF-TOPIC-AUTO",

    "org.onap.dmaap.mr.PNF_READY"

]}bash-4.4#

 

But audit.log of sdc-be, after “Distribution Service” action from Portal , says:

 

2019-03-01T08:32:07.986Z        [qtp215145189-323354]   INFO    o.o.sdc.be.filters.BeServletFilter     

ResponseCode=500        InstanceUUID=null       RequestId=d2f65e19-b07b-4266-8be2-f170aba42fb1  AlertSeverity=0 ElapsedTime=3  

EndTimestamp=2019-03-01 08:32:07.986Z   PartnerName=op0001      auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      

StatusCode=ERROR        TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate       

timer=3 CustomField2=500        AuditBeginTimestamp=2019-03-01 08:32:07.983Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate      

ServiceInstanceId=null   ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

 

In the same log file I found a lot of messages like this one:

 

2019-03-01T09:21:31.850Z        [qtp215145189-399996]   INFO    o.o.sdc.be.filters.BeServletFilter      AuditMessage=ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"  ResponseCode=500        InstanceUUID=aai-ml     RequestId=7f01a5b2-ee38-42c9-b7a4-330f020a4134 AlertSeverity=0  ElapsedTime=169 EndTimestamp=2019-03-01 09:21:31.850Z   PartnerName=Apache-HttpClient/4.5.6 (Java/1.8.0_171)    auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR        TargetEntity=registerInDistributionEngine       CustomField1=POST: https://sdc-be.onap:8443/sdc/v1/registerForDistribution      timer=169       CustomField2=500        AuditBeginTimestamp=2019-03-01 09:21:31.681Z    RemoteHost=10.42.209.109        ErrorCategory=ERROR     ServerIPAddress=10.42.179.134   ServiceName=/v1/registerForDistribution ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error       ErrorCode=500   ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"

 

Thanks.

 

Da: onap-discuss@... [mailto:onap-discuss@...] Per conto di Calamita Agostino
Inviato: giovedì 28 febbraio 2019 16:13
A: onap-discuss@...
Oggetto: [onap-discuss] dmaap-message-router NodePort not recheable

 

Hi all,

I have an issue releted to connectivity from sdc-be pod and dmaap-message-router.

My installation is Casablanca 3.0.0 on 7 kubernetes VM cluster.

 

All dmaap pods are up and running:

 

dev-dmaap-dbc-pg-0                                            1/1       Running            0          1d        10.42.173.158   onapkm5   <none>

dev-dmaap-dbc-pg-1                                            1/1       Running            0          1d        10.42.188.140   onapkm2   <none>

dev-dmaap-dbc-pgpool-7b748d5894-mr2m9                         1/1       Running            0          1d        10.42.237.193   onapkm3   <none>

dev-dmaap-dbc-pgpool-7b748d5894-n6dks                         1/1       Running            0          1d        10.42.192.244   onapkm2   <none>

dev-dmaap-dmaap-bus-controller-6757c4c86-8rq5p                1/1       Running            0          1d        10.42.185.132   onapkm1   <none>

dev-dmaap-dmaap-dr-db-bb4c67cfd-tm7td                         1/1       Running            0          1d        10.42.152.59    onapkm1   <none>

dev-dmaap-dmaap-dr-node-66c8749959-tpdtf                      1/1       Running            0          1d        10.42.216.13    onapkm2   <none>

dev-dmaap-dmaap-dr-prov-5c766b8d69-qzqn2                      1/1       Running            0          1d        10.42.115.247   onapkm6   <none>

dev-dmaap-message-router-fb9f4bc7d-5z52j                      1/1       Running            0          6h        10.42.138.31    onapkm3   <none>

dev-dmaap-message-router-kafka-5fbc897f48-4bpb6               1/1       Running            0          1d        10.42.78.141    onapkm4   <none>

dev-dmaap-message-router-zookeeper-557954854-8d6p9            1/1       Running            0          1d        10.42.169.205   onapkm1   <none>

 

but when I try to distribute a service, from SDC Portal, I got “Internal Server Error”.

 

SDC-BE log file traces:

 

2019-02-28T08:50:35.318Z        [qtp215145189-159837]   INFO    o.o.sdc.be.filters.BeServletFilter      ResponseCode=500       

InstanceUUID=null RequestId=dab0fd50-b06e-4a65-b4a8-7d7edeae3e01   AlertSeverity=0 ElapsedTime=99  EndTimestamp=2019-02-28 08:50:35.318Z PartnerName=op0001      auditOn=true       ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR       

TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

timer=99        CustomField2=500   AuditBeginTimestamp=2019-02-28 08:50:35.219Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter     ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

Also SDC healthcheck reports that U-EB Cluster is DOWN.

 

Inside SDC-BE POD, I tried to make a traceroute to “message-router-zookeeper” and to “message-router”.

 

This is the result ( the first is OK, the second one NOT OK ):

 

bash-4.4# traceroute  message-router-zookeeper

traceroute to message-router-zookeeper (10.42.169.205), 30 hops max, 46 byte packets

1  10.42.7.46 (10.42.7.46)  0.213 ms  0.005 ms  0.005 ms

2  10.42.190.179 (10.42.190.179)  0.194 ms  0.145 ms  0.135 ms

3  10.42.169.205 (10.42.169.205)  0.461 ms  0.160 ms  0.134 ms

 

bash-4.4# traceroute  message-router

traceroute to message-router (10.43.1.20), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.009 ms  0.005 ms  0.005 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.344 ms  2.211 ms  1.910 ms     ß 138.132.168.X  is VM public network

 3  138.132.169.2 (138.132.169.2)  5.063 ms  3.859 ms  3.934 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

traceroute to message-router-kafka (10.43.148.154), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.006 ms  0.005 ms  0.004 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.391 ms  0.337 ms  0.314 ms

3  138.132.169.2 (138.132.169.2)  0.803 ms  0.748 ms  0.807 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

It seems that I cannot reach NodePort or ClusterIP inside a POD. This is routing table inside POD:

 

bash-4.4# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         10.42.0.1       0.0.0.0         UG        0 0          0 eth0

10.42.0.0       0.0.0.0         255.255.0.0     U         0 0          0 eth0

 

What can I check on Kubernetes Cluster ?

 

Thanks.

Agostino.

 

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/


R: [onap-discuss] dmaap-message-router NodePort not recheable

Calamita Agostino
 

This is the output:

 

Executing robot tests at log level TRACE

[ ERROR ] Suite 'Testsuites' contains no tests with tag 'healthmr'.

 

Try --help for usage information.

command terminated with exit code 252

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 15:12
A: Calamita Agostino <agostino.calamita@...>; onap-discuss@...
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Please try ./ete-k8s.sh onap healthmr

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:09 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I didn’t find healthmr test but only health.

(

./ete-k8s.sh onap

Usage: ete-k8s.sh [namespace] [ health | healthdist | distribute | instantiate | instantiateVFWCL | instantiateDemoVFWCL |  | portal ] )

 

The command ./ete-k8s.sh onap health reports the list below (  Basic DMAAP Message Router Health Check = PASS )                            

In my environment there are some PODs not in Running state:

 

dev-aai-aai-data-router-5d55646cdc-cc62v                      1/2       CrashLoopBackOff   1084       4d        10.42.79.203    onapkm3   <none>

dev-appc-appc-ansible-server-76fcf9454d-8km9d                 0/1       CrashLoopBackOff   1656       6d        10.42.212.202   onapkm0   <none>

dev-oof-oof-has-api-585497f5-ktjsv                            0/1       Init:0/3           1085       8d        10.42.86.82     onapkm0   <none>

dev-oof-oof-has-controller-9469b9ff8-td4k9                    0/1       Init:1/3           945        8d        10.42.5.110     onapkm2   <none>

dev-oof-oof-has-data-d559897dc-4lmkt                          0/1       Init:1/4           1091       8d        10.42.199.220   onapkm3   <none>

dev-oof-oof-has-healthcheck-jq9xq                             0/1       Init:0/1           1092       8d        10.42.242.145   onapkm3   <none>

dev-oof-oof-has-reservation-868c7c88ff-pv79n                  0/1       Init:1/4           1081       8d        10.42.176.61    onapkm1   <none>

dev-oof-oof-has-solver-6f8bc6fdf4-tw4cj                       0/1       Init:1/4           1084       8d        10.42.29.154    onapkm0   <none>

dev-sdnc-sdnc-ansible-server-7c76f965c6-hqtzl                 0/1       CrashLoopBackOff   1844       8d        10.42.202.36    onapkm3   <none>

dev-sdnc-sdnc-ueb-listener-6d74459c6-tdqhc                    0/1       CrashLoopBackOff   542        1d        10.42.219.51    onapkm2   <none>

 

and multicloud is not deployed.

 

==============================================================================

Testsuites

==============================================================================

Testsuites.Health-Check :: Testing ecomp components are available via calls.

==============================================================================

Basic A&AI Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic AAF Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic AAF SMS Health Check                                            | PASS |

------------------------------------------------------------------------------

Basic APPC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic CLI Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic CLAMP Health Check                                              | PASS |

------------------------------------------------------------------------------

Basic DCAE Health Check                                               [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8102250>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a8d0>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a850>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

| FAIL |

ConnectionError: HTTPConnectionPool(host='dcae-healthcheck.onap', port=80): Max retries exceeded with url: /healthcheck (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a5d0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic DMAAP Data Router Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic DMAAP Message Router Health Check                               | PASS |

------------------------------------------------------------------------------

Basic External API NBI Health Check                                   [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80d6c50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80c4e10>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8199f50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

| FAIL |

ConnectionError: HTTPConnectionPool(host='nbi.onap', port=8080): Max retries exceeded with url: /nbi/api/v3/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80e5410>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic Log Elasticsearch Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic Log Kibana Health Check                                         | PASS |

------------------------------------------------------------------------------

Basic Log Logstash Health Check                                       | PASS |

------------------------------------------------------------------------------

Basic Microservice Bus Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic Multicloud API Health Check                                     | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-ocata API Health Check                               | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-pike API Health Check                                | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-titanium_cloud API Health Check                      | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-vio API Health Check                                 | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic OOF-Homing Health Check                                         | FAIL |

Test timeout 10 seconds exceeded.

------------------------------------------------------------------------------

Basic OOF-SNIRO Health Check                                          | PASS |

------------------------------------------------------------------------------

Basic OOF-CMSO Health Check                                           | PASS |

------------------------------------------------------------------------------

Basic Policy Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Pomba AAI-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba SDC-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba Network-discovery-context-builder Health Check            | PASS |

------------------------------------------------------------------------------

Basic Portal Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic SDC Health Check                                                (DMaaP:None)| PASS |

------------------------------------------------------------------------------

Basic SDNC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic SO Health Check                                                 | PASS |

------------------------------------------------------------------------------

Basic UseCaseUI API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC catalog API Health Check                                    | PASS |

------------------------------------------------------------------------------

Basic VFC emsdriver API Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic VFC gvnfmdriver API Health Check                                | PASS |

------------------------------------------------------------------------------

Basic VFC huaweivnfmdriver API Health Check                           | PASS |

------------------------------------------------------------------------------

Basic VFC jujuvnfmdriver API Health Check                             | PASS |

------------------------------------------------------------------------------

Basic VFC multivimproxy API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nokiavnfmdriver API Health Check                            | PASS |

------------------------------------------------------------------------------

Basic VFC nokiav2driver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nslcm API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC resmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnflcm API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfres API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC workflow API Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic VFC ztesdncdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC ztevnfmdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VID Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic VNFSDK Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Holmes Rule Management API Health Check                         | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Holmes Engine Management API Health Check                       | FAIL |

502 != 200

------------------------------------------------------------------------------

Testsuites.Health-Check :: Testing ecomp components are available ... | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Testsuites                                                            | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Output:  /share/logs/0001_ete_health/output.xml

Log:     /share/logs/0001_ete_health/log.html

Report:  /share/logs/0001_ete_health/report.html

command terminated with exit code 10

 

 

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 14:49
A: onap-discuss@...; Calamita Agostino <agostino.calamita@...>
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Try a POST to make sure you can write to message router.

I doubt its connectivity.

 

If you are on master branch – try ./ete-k8s.sh onap healthmr to test a write/read to a test topic.

 

(do it twice since the first time it creates a test topic and kafka doesnt forward the message till both the publisher and the subscriber have connected)

 

Brian

 

 

From: onap-discuss@... <onap-discuss@...> On Behalf Of Calamita Agostino
Sent: Friday, March 01, 2019 4:30 AM
To: onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I tried to execute a wget command from sdc-be POD to message-router REST API and I see that dmaap-message-router is reacheable from sdc-be.

 

This is the result:

 

# kubectl exec -it  dev-sdc-sdc-be-656bd64b9b-jh57x  -n onap -- /bin/bash

 

bash-4.4# wget "http://message-router:3904/topics"

Connecting to message-router:3904 (10.43.1.20:3904)

topics               100% |*******************************|   131   0:00:00 ETA

bash-4.4# cat topics

{"topics": [

    "__consumer_offsets",

    "champRawEvents",

    "SDC-DISTR-NOTIF-TOPIC-AUTO",

    "org.onap.dmaap.mr.PNF_READY"

]}bash-4.4#

 

But audit.log of sdc-be, after “Distribution Service” action from Portal , says:

 

2019-03-01T08:32:07.986Z        [qtp215145189-323354]   INFO    o.o.sdc.be.filters.BeServletFilter     

ResponseCode=500        InstanceUUID=null       RequestId=d2f65e19-b07b-4266-8be2-f170aba42fb1  AlertSeverity=0 ElapsedTime=3  

EndTimestamp=2019-03-01 08:32:07.986Z   PartnerName=op0001      auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      

StatusCode=ERROR        TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate       

timer=3 CustomField2=500        AuditBeginTimestamp=2019-03-01 08:32:07.983Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate      

ServiceInstanceId=null   ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

 

In the same log file I found a lot of messages like this one:

 

2019-03-01T09:21:31.850Z        [qtp215145189-399996]   INFO    o.o.sdc.be.filters.BeServletFilter      AuditMessage=ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"  ResponseCode=500        InstanceUUID=aai-ml     RequestId=7f01a5b2-ee38-42c9-b7a4-330f020a4134 AlertSeverity=0  ElapsedTime=169 EndTimestamp=2019-03-01 09:21:31.850Z   PartnerName=Apache-HttpClient/4.5.6 (Java/1.8.0_171)    auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR        TargetEntity=registerInDistributionEngine       CustomField1=POST: https://sdc-be.onap:8443/sdc/v1/registerForDistribution      timer=169       CustomField2=500        AuditBeginTimestamp=2019-03-01 09:21:31.681Z    RemoteHost=10.42.209.109        ErrorCategory=ERROR     ServerIPAddress=10.42.179.134   ServiceName=/v1/registerForDistribution ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error       ErrorCode=500   ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"

 

Thanks.

 

Da: onap-discuss@... [mailto:onap-discuss@...] Per conto di Calamita Agostino
Inviato: giovedì 28 febbraio 2019 16:13
A: onap-discuss@...
Oggetto: [onap-discuss] dmaap-message-router NodePort not recheable

 

Hi all,

I have an issue releted to connectivity from sdc-be pod and dmaap-message-router.

My installation is Casablanca 3.0.0 on 7 kubernetes VM cluster.

 

All dmaap pods are up and running:

 

dev-dmaap-dbc-pg-0                                            1/1       Running            0          1d        10.42.173.158   onapkm5   <none>

dev-dmaap-dbc-pg-1                                            1/1       Running            0          1d        10.42.188.140   onapkm2   <none>

dev-dmaap-dbc-pgpool-7b748d5894-mr2m9                         1/1       Running            0          1d        10.42.237.193   onapkm3   <none>

dev-dmaap-dbc-pgpool-7b748d5894-n6dks                         1/1       Running            0          1d        10.42.192.244   onapkm2   <none>

dev-dmaap-dmaap-bus-controller-6757c4c86-8rq5p                1/1       Running            0          1d        10.42.185.132   onapkm1   <none>

dev-dmaap-dmaap-dr-db-bb4c67cfd-tm7td                         1/1       Running            0          1d        10.42.152.59    onapkm1   <none>

dev-dmaap-dmaap-dr-node-66c8749959-tpdtf                      1/1       Running            0          1d        10.42.216.13    onapkm2   <none>

dev-dmaap-dmaap-dr-prov-5c766b8d69-qzqn2                      1/1       Running            0          1d        10.42.115.247   onapkm6   <none>

dev-dmaap-message-router-fb9f4bc7d-5z52j                      1/1       Running            0          6h        10.42.138.31    onapkm3   <none>

dev-dmaap-message-router-kafka-5fbc897f48-4bpb6               1/1       Running            0          1d        10.42.78.141    onapkm4   <none>

dev-dmaap-message-router-zookeeper-557954854-8d6p9            1/1       Running            0          1d        10.42.169.205   onapkm1   <none>

 

but when I try to distribute a service, from SDC Portal, I got “Internal Server Error”.

 

SDC-BE log file traces:

 

2019-02-28T08:50:35.318Z        [qtp215145189-159837]   INFO    o.o.sdc.be.filters.BeServletFilter      ResponseCode=500       

InstanceUUID=null RequestId=dab0fd50-b06e-4a65-b4a8-7d7edeae3e01   AlertSeverity=0 ElapsedTime=99  EndTimestamp=2019-02-28 08:50:35.318Z PartnerName=op0001      auditOn=true       ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR       

TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

timer=99        CustomField2=500   AuditBeginTimestamp=2019-02-28 08:50:35.219Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter     ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

Also SDC healthcheck reports that U-EB Cluster is DOWN.

 

Inside SDC-BE POD, I tried to make a traceroute to “message-router-zookeeper” and to “message-router”.

 

This is the result ( the first is OK, the second one NOT OK ):

 

bash-4.4# traceroute  message-router-zookeeper

traceroute to message-router-zookeeper (10.42.169.205), 30 hops max, 46 byte packets

1  10.42.7.46 (10.42.7.46)  0.213 ms  0.005 ms  0.005 ms

2  10.42.190.179 (10.42.190.179)  0.194 ms  0.145 ms  0.135 ms

3  10.42.169.205 (10.42.169.205)  0.461 ms  0.160 ms  0.134 ms

 

bash-4.4# traceroute  message-router

traceroute to message-router (10.43.1.20), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.009 ms  0.005 ms  0.005 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.344 ms  2.211 ms  1.910 ms     ß 138.132.168.X  is VM public network

 3  138.132.169.2 (138.132.169.2)  5.063 ms  3.859 ms  3.934 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

traceroute to message-router-kafka (10.43.148.154), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.006 ms  0.005 ms  0.004 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.391 ms  0.337 ms  0.314 ms

3  138.132.169.2 (138.132.169.2)  0.803 ms  0.748 ms  0.807 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

It seems that I cannot reach NodePort or ClusterIP inside a POD. This is routing table inside POD:

 

bash-4.4# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         10.42.0.1       0.0.0.0         UG        0 0          0 eth0

10.42.0.0       0.0.0.0         255.255.0.0     U         0 0          0 eth0

 

What can I check on Kubernetes Cluster ?

 

Thanks.

Agostino.

 

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/


Re: dmaap-message-router NodePort not recheable

Brian Freeman
 

Please try ./ete-k8s.sh onap healthmr

 

From: Calamita Agostino <agostino.calamita@...>
Sent: Friday, March 01, 2019 9:09 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I didn’t find healthmr test but only health.

(

./ete-k8s.sh onap

Usage: ete-k8s.sh [namespace] [ health | healthdist | distribute | instantiate | instantiateVFWCL | instantiateDemoVFWCL |  | portal ] )

 

The command ./ete-k8s.sh onap health reports the list below (  Basic DMAAP Message Router Health Check = PASS )                            

In my environment there are some PODs not in Running state:

 

dev-aai-aai-data-router-5d55646cdc-cc62v                      1/2       CrashLoopBackOff   1084       4d        10.42.79.203    onapkm3   <none>

dev-appc-appc-ansible-server-76fcf9454d-8km9d                 0/1       CrashLoopBackOff   1656       6d        10.42.212.202   onapkm0   <none>

dev-oof-oof-has-api-585497f5-ktjsv                            0/1       Init:0/3           1085       8d        10.42.86.82     onapkm0   <none>

dev-oof-oof-has-controller-9469b9ff8-td4k9                    0/1       Init:1/3           945        8d        10.42.5.110     onapkm2   <none>

dev-oof-oof-has-data-d559897dc-4lmkt                          0/1       Init:1/4           1091       8d        10.42.199.220   onapkm3   <none>

dev-oof-oof-has-healthcheck-jq9xq                             0/1       Init:0/1           1092       8d        10.42.242.145   onapkm3   <none>

dev-oof-oof-has-reservation-868c7c88ff-pv79n                  0/1       Init:1/4           1081       8d        10.42.176.61    onapkm1   <none>

dev-oof-oof-has-solver-6f8bc6fdf4-tw4cj                       0/1       Init:1/4           1084       8d        10.42.29.154    onapkm0   <none>

dev-sdnc-sdnc-ansible-server-7c76f965c6-hqtzl                 0/1       CrashLoopBackOff   1844       8d        10.42.202.36    onapkm3   <none>

dev-sdnc-sdnc-ueb-listener-6d74459c6-tdqhc                    0/1       CrashLoopBackOff   542        1d        10.42.219.51    onapkm2   <none>

 

and multicloud is not deployed.

 

==============================================================================

Testsuites

==============================================================================

Testsuites.Health-Check :: Testing ecomp components are available via calls.

==============================================================================

Basic A&AI Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic AAF Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic AAF SMS Health Check                                            | PASS |

------------------------------------------------------------------------------

Basic APPC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic CLI Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic CLAMP Health Check                                              | PASS |

------------------------------------------------------------------------------

Basic DCAE Health Check                                               [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8102250>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a8d0>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a850>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /healthcheck

| FAIL |

ConnectionError: HTTPConnectionPool(host='dcae-healthcheck.onap', port=80): Max retries exceeded with url: /healthcheck (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e817a5d0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic DMAAP Data Router Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic DMAAP Message Router Health Check                               | PASS |

------------------------------------------------------------------------------

Basic External API NBI Health Check                                   [ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80d6c50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80c4e10>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e8199f50>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /nbi/api/v3/status

| FAIL |

ConnectionError: HTTPConnectionPool(host='nbi.onap', port=8080): Max retries exceeded with url: /nbi/api/v3/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f03e80e5410>: Failed to establish a new connection: [Errno -2] Name or service not known',))

------------------------------------------------------------------------------

Basic Log Elasticsearch Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic Log Kibana Health Check                                         | PASS |

------------------------------------------------------------------------------

Basic Log Logstash Health Check                                       | PASS |

------------------------------------------------------------------------------

Basic Microservice Bus Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic Multicloud API Health Check                                     | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-ocata API Health Check                               | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-pike API Health Check                                | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-titanium_cloud API Health Check                      | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Multicloud-vio API Health Check                                 | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic OOF-Homing Health Check                                         | FAIL |

Test timeout 10 seconds exceeded.

------------------------------------------------------------------------------

Basic OOF-SNIRO Health Check                                          | PASS |

------------------------------------------------------------------------------

Basic OOF-CMSO Health Check                                           | PASS |

------------------------------------------------------------------------------

Basic Policy Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Pomba AAI-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba SDC-context-builder Health Check                          | PASS |

------------------------------------------------------------------------------

Basic Pomba Network-discovery-context-builder Health Check            | PASS |

------------------------------------------------------------------------------

Basic Portal Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic SDC Health Check                                                (DMaaP:None)| PASS |

------------------------------------------------------------------------------

Basic SDNC Health Check                                               | PASS |

------------------------------------------------------------------------------

Basic SO Health Check                                                 | PASS |

------------------------------------------------------------------------------

Basic UseCaseUI API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC catalog API Health Check                                    | PASS |

------------------------------------------------------------------------------

Basic VFC emsdriver API Health Check                                  | PASS |

------------------------------------------------------------------------------

Basic VFC gvnfmdriver API Health Check                                | PASS |

------------------------------------------------------------------------------

Basic VFC huaweivnfmdriver API Health Check                           | PASS |

------------------------------------------------------------------------------

Basic VFC jujuvnfmdriver API Health Check                             | PASS |

------------------------------------------------------------------------------

Basic VFC multivimproxy API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nokiavnfmdriver API Health Check                            | PASS |

------------------------------------------------------------------------------

Basic VFC nokiav2driver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC nslcm API Health Check                                      | PASS |

------------------------------------------------------------------------------

Basic VFC resmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnflcm API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfmgr API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC vnfres API Health Check                                     | PASS |

------------------------------------------------------------------------------

Basic VFC workflow API Health Check                                   | PASS |

------------------------------------------------------------------------------

Basic VFC ztesdncdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VFC ztevnfmdriver API Health Check                              | PASS |

------------------------------------------------------------------------------

Basic VID Health Check                                                | PASS |

------------------------------------------------------------------------------

Basic VNFSDK Health Check                                             | PASS |

------------------------------------------------------------------------------

Basic Holmes Rule Management API Health Check                         | FAIL |

502 != 200

------------------------------------------------------------------------------

Basic Holmes Engine Management API Health Check                       | FAIL |

502 != 200

------------------------------------------------------------------------------

Testsuites.Health-Check :: Testing ecomp components are available ... | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Testsuites                                                            | FAIL |

51 critical tests, 41 passed, 10 failed

51 tests total, 41 passed, 10 failed

==============================================================================

Output:  /share/logs/0001_ete_health/output.xml

Log:     /share/logs/0001_ete_health/log.html

Report:  /share/logs/0001_ete_health/report.html

command terminated with exit code 10

 

 

 

Da: FREEMAN, BRIAN D [mailto:bf1936@...]
Inviato: venerdì 1 marzo 2019 14:49
A: onap-discuss@...; Calamita Agostino <agostino.calamita@...>
Oggetto: RE: [onap-discuss] dmaap-message-router NodePort not recheable

 

Try a POST to make sure you can write to message router.

I doubt its connectivity.

 

If you are on master branch – try ./ete-k8s.sh onap healthmr to test a write/read to a test topic.

 

(do it twice since the first time it creates a test topic and kafka doesnt forward the message till both the publisher and the subscriber have connected)

 

Brian

 

 

From: onap-discuss@... <onap-discuss@...> On Behalf Of Calamita Agostino
Sent: Friday, March 01, 2019 4:30 AM
To: onap-discuss@...
Subject: R: [onap-discuss] dmaap-message-router NodePort not recheable

 

I tried to execute a wget command from sdc-be POD to message-router REST API and I see that dmaap-message-router is reacheable from sdc-be.

 

This is the result:

 

# kubectl exec -it  dev-sdc-sdc-be-656bd64b9b-jh57x  -n onap -- /bin/bash

 

bash-4.4# wget "http://message-router:3904/topics"

Connecting to message-router:3904 (10.43.1.20:3904)

topics               100% |*******************************|   131   0:00:00 ETA

bash-4.4# cat topics

{"topics": [

    "__consumer_offsets",

    "champRawEvents",

    "SDC-DISTR-NOTIF-TOPIC-AUTO",

    "org.onap.dmaap.mr.PNF_READY"

]}bash-4.4#

 

But audit.log of sdc-be, after “Distribution Service” action from Portal , says:

 

2019-03-01T08:32:07.986Z        [qtp215145189-323354]   INFO    o.o.sdc.be.filters.BeServletFilter     

ResponseCode=500        InstanceUUID=null       RequestId=d2f65e19-b07b-4266-8be2-f170aba42fb1  AlertSeverity=0 ElapsedTime=3  

EndTimestamp=2019-03-01 08:32:07.986Z   PartnerName=op0001      auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      

StatusCode=ERROR        TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate       

timer=3 CustomField2=500        AuditBeginTimestamp=2019-03-01 08:32:07.983Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate      

ServiceInstanceId=null   ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

 

In the same log file I found a lot of messages like this one:

 

2019-03-01T09:21:31.850Z        [qtp215145189-399996]   INFO    o.o.sdc.be.filters.BeServletFilter      AuditMessage=ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"  ResponseCode=500        InstanceUUID=aai-ml     RequestId=7f01a5b2-ee38-42c9-b7a4-330f020a4134 AlertSeverity=0  ElapsedTime=169 EndTimestamp=2019-03-01 09:21:31.850Z   PartnerName=Apache-HttpClient/4.5.6 (Java/1.8.0_171)    auditOn=true    ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR        TargetEntity=registerInDistributionEngine       CustomField1=POST: https://sdc-be.onap:8443/sdc/v1/registerForDistribution      timer=169       CustomField2=500        AuditBeginTimestamp=2019-03-01 09:21:31.681Z    RemoteHost=10.42.209.109        ErrorCategory=ERROR     ServerIPAddress=10.42.179.134   ServiceName=/v1/registerForDistribution ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter  ResponseDescription=Internal Server Error       ErrorCode=500   ACTION = "HttpAuthentication" URL = "v1/registerForDistribution" USER = "aai" AUTH_STATUS = "AUTH_SUCCESS" REALM = "ASDC"

 

Thanks.

 

Da: onap-discuss@... [mailto:onap-discuss@...] Per conto di Calamita Agostino
Inviato: giovedì 28 febbraio 2019 16:13
A: onap-discuss@...
Oggetto: [onap-discuss] dmaap-message-router NodePort not recheable

 

Hi all,

I have an issue releted to connectivity from sdc-be pod and dmaap-message-router.

My installation is Casablanca 3.0.0 on 7 kubernetes VM cluster.

 

All dmaap pods are up and running:

 

dev-dmaap-dbc-pg-0                                            1/1       Running            0          1d        10.42.173.158   onapkm5   <none>

dev-dmaap-dbc-pg-1                                            1/1       Running            0          1d        10.42.188.140   onapkm2   <none>

dev-dmaap-dbc-pgpool-7b748d5894-mr2m9                         1/1       Running            0          1d        10.42.237.193   onapkm3   <none>

dev-dmaap-dbc-pgpool-7b748d5894-n6dks                         1/1       Running            0          1d        10.42.192.244   onapkm2   <none>

dev-dmaap-dmaap-bus-controller-6757c4c86-8rq5p                1/1       Running            0          1d        10.42.185.132   onapkm1   <none>

dev-dmaap-dmaap-dr-db-bb4c67cfd-tm7td                         1/1       Running            0          1d        10.42.152.59    onapkm1   <none>

dev-dmaap-dmaap-dr-node-66c8749959-tpdtf                      1/1       Running            0          1d        10.42.216.13    onapkm2   <none>

dev-dmaap-dmaap-dr-prov-5c766b8d69-qzqn2                      1/1       Running            0          1d        10.42.115.247   onapkm6   <none>

dev-dmaap-message-router-fb9f4bc7d-5z52j                      1/1       Running            0          6h        10.42.138.31    onapkm3   <none>

dev-dmaap-message-router-kafka-5fbc897f48-4bpb6               1/1       Running            0          1d        10.42.78.141    onapkm4   <none>

dev-dmaap-message-router-zookeeper-557954854-8d6p9            1/1       Running            0          1d        10.42.169.205   onapkm1   <none>

 

but when I try to distribute a service, from SDC Portal, I got “Internal Server Error”.

 

SDC-BE log file traces:

 

2019-02-28T08:50:35.318Z        [qtp215145189-159837]   INFO    o.o.sdc.be.filters.BeServletFilter      ResponseCode=500       

InstanceUUID=null RequestId=dab0fd50-b06e-4a65-b4a8-7d7edeae3e01   AlertSeverity=0 ElapsedTime=99  EndTimestamp=2019-02-28 08:50:35.318Z PartnerName=op0001      auditOn=true       ServerFQDN=dev-sdc-sdc-be-656bd64b9b-jh57x      StatusCode=ERROR       

TargetEntity=Distribution Engine is DOWN       

CustomField1=POST: http://sdc-be.onap:8080/sdc2/rest/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

timer=99        CustomField2=500   AuditBeginTimestamp=2019-02-28 08:50:35.219Z    RemoteHost=10.42.194.84 ErrorCategory=ERROR    

ServerIPAddress=10.42.179.134   ServiceName=/v1/catalog/services/02e0c5a4-be65-4d09-9f1e-49a2dab0f865/distribution/PROD/activate  

ServiceInstanceId=null  ClassName=org.openecomp.sdc.be.filters.BeServletFilter     ResponseDescription=Internal Server Error      

ErrorCode=500   null

 

Also SDC healthcheck reports that U-EB Cluster is DOWN.

 

Inside SDC-BE POD, I tried to make a traceroute to “message-router-zookeeper” and to “message-router”.

 

This is the result ( the first is OK, the second one NOT OK ):

 

bash-4.4# traceroute  message-router-zookeeper

traceroute to message-router-zookeeper (10.42.169.205), 30 hops max, 46 byte packets

1  10.42.7.46 (10.42.7.46)  0.213 ms  0.005 ms  0.005 ms

2  10.42.190.179 (10.42.190.179)  0.194 ms  0.145 ms  0.135 ms

3  10.42.169.205 (10.42.169.205)  0.461 ms  0.160 ms  0.134 ms

 

bash-4.4# traceroute  message-router

traceroute to message-router (10.43.1.20), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.009 ms  0.005 ms  0.005 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.344 ms  2.211 ms  1.910 ms     ß 138.132.168.X  is VM public network

 3  138.132.169.2 (138.132.169.2)  5.063 ms  3.859 ms  3.934 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

traceroute to message-router-kafka (10.43.148.154), 30 hops max, 46 byte packets

1  10.42.0.1 (10.42.0.1)  0.006 ms  0.005 ms  0.004 ms

2  itpat1ng505.palermo.italtel.it (138.132.168.173)  0.391 ms  0.337 ms  0.314 ms

3  138.132.169.2 (138.132.169.2)  0.803 ms  0.748 ms  0.807 ms

4  *  *  *

5  *  *  *

6  *  *  *

 

It seems that I cannot reach NodePort or ClusterIP inside a POD. This is routing table inside POD:

 

bash-4.4# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         10.42.0.1       0.0.0.0         UG        0 0          0 eth0

10.42.0.0       0.0.0.0         255.255.0.0     U         0 0          0 eth0

 

What can I check on Kubernetes Cluster ?

 

Thanks.

Agostino.

 

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

Internet Email Confidentiality Footer ** La presente comunicazione, con le informazioni in essa contenute e ogni documento o file allegato, e' rivolta unicamente alla/e persona/e cui e' indirizzata ed alle altre da questa autorizzata/e a riceverla. Se non siete i destinatari/autorizzati siete avvisati che qualsiasi azione, copia, comunicazione, divulgazione o simili basate sul contenuto di tali informazioni e' vietata e potrebbe essere contro la legge vigente (ad es. art. 616 C.P., D.Lgs n. 196/2003 Codice Privacy, Regolamento Europeo n. 679/2016/GDPR). Se avete ricevuto questa comunicazione per errore, vi preghiamo di darne immediata notizia al mittente e di distruggere il messaggio originale e ogni file allegato senza farne copia alcuna o riprodurne in alcun modo il contenuto. Al link seguente e' disponibile l'informativa Privacy: http://www.italtel.com/it/about/privacy/ ** This e-mail and its attachments are intended for the addressee(s) only and are confidential and/or may contain legally privileged information. If you have received this message by mistake or are not one of the addressees above, you may take no action based on it, and you may not copy or show it to anyone; please reply to this e-mail and point out the error which has occurred. Click here to read your privacy notice: http://www.italtel.com/it/about/privacy/

7521 - 7540 of 23354