Topics

Dublin SO pods not running #so


Reshmasree
 

SO pods are not running properly when we try to deploy dublin SO using '4.0.0-ONAP' oom. Pods status is like below

so-so-5cd8c67bd6-6wmwl                                            0/1     CrashLoopBackOff   9          45m
so-so-bpmn-infra-846988b565-q6m74                                 0/1     CrashLoopBackOff   9          45m
so-so-catalog-db-adapter-74fffc4d66-cx5gn                         0/1     CrashLoopBackOff   9          45m
so-so-mariadb-config-job-z7mdl                                    0/1     Completed          0          45m
so-so-monitoring-847c66c547-fmplz                                 1/1     Running            0          45m
so-so-openstack-adapter-9cf7c5fdb-kcz67                           0/1     CrashLoopBackOff   9          45m
so-so-request-db-adapter-584ddc56c9-qmwqh                         0/1     CrashLoopBackOff   9          45m
so-so-sdc-controller-79769bcfd-bfhk6                              0/1     CrashLoopBackOff   9          45m
so-so-sdnc-adapter-66d5799fb5-7lwx7                               1/1     Running            0          45m
so-so-vfc-adapter-59b6f44b5c-7g7f8                                0/1     CrashLoopBackOff   9          45m
so-so-vnfm-adapter-58d5646878-k5xm8                               1/1     Running            0          45m


There no logs found for debugging. Need help in solving this issue.

Thanks,
Reshma.


Brian Freeman
 

When you do a describe pod for the catalog-db, requestdb adapter what reason is being given for the restart ?

 

Check the shared mariadb to make sure there are tables in the requestdb, catalogdb

 

If not I would helm delete dev-so –purge, run cleanup.sh , then helm deploy dev-so ... (from the wiki)

 

helm deploy dev-so local/onap -f /root/oom/kubernetes/onap/resources/environments/public-cloud.yaml -f /root/integration-override.yaml --namespace onap  --verbose

 

if you have a slow environment add a –timeout 1000

 

 

Brian

 

 

From: onap-users@... <onap-users@...> On Behalf Of Reshmasree via Lists.Onap.Org
Sent: Thursday, August 08, 2019 9:03 AM
To: onap-users@...
Cc: onap-users@...
Subject: [Onap-users] Dublin SO pods not running #so

 

SO pods are not running properly when we try to deploy dublin SO using '4.0.0-ONAP' oom. Pods status is like below

so-so-5cd8c67bd6-6wmwl                                            0/1     CrashLoopBackOff   9          45m
so-so-bpmn-infra-846988b565-q6m74                                 0/1     CrashLoopBackOff   9          45m
so-so-catalog-db-adapter-74fffc4d66-cx5gn                         0/1     CrashLoopBackOff   9          45m
so-so-mariadb-config-job-z7mdl                                    0/1     Completed          0          45m
so-so-monitoring-847c66c547-fmplz                                 1/1     Running            0          45m
so-so-openstack-adapter-9cf7c5fdb-kcz67                           0/1     CrashLoopBackOff   9          45m
so-so-request-db-adapter-584ddc56c9-qmwqh                         0/1     CrashLoopBackOff   9          45m
so-so-sdc-controller-79769bcfd-bfhk6                              0/1     CrashLoopBackOff   9          45m
so-so-sdnc-adapter-66d5799fb5-7lwx7                               1/1     Running            0          45m
so-so-vfc-adapter-59b6f44b5c-7g7f8                                0/1     CrashLoopBackOff   9          45m
so-so-vnfm-adapter-58d5646878-k5xm8                               1/1     Running            0          45m


There no logs found for debugging. Need help in solving this issue.

Thanks,
Reshma.


Reshmasree
 

'BackOff' was the reason.

  Type     Reason   Age                   From                 Message
  ----     ------   ----                  ----                 -------
  Normal   Pulled   53m (x4 over 54m)     kubelet, onap-k8s-8  Successfully pulled image "registry:5000/onap/so/request-db-adapter:2aug10"
  Normal   Created  53m (x4 over 54m)     kubelet, onap-k8s-8  Created container
  Normal   Started  53m (x4 over 54m)     kubelet, onap-k8s-8  Started container
  Normal   Pulling  42m (x8 over 54m)     kubelet, onap-k8s-8  pulling image "registry:5000/onap/so/request-db-adapter:2aug10"
  Warning  BackOff  2m9s (x238 over 53m)  kubelet, onap-k8s-8  Back-off restarting failed container

There is no catalog/requestdb created in the shared mariadb. What should I do now?


From: onap-users@... <onap-users@...> on behalf of Brian <bf1936@...>
Sent: Thursday, August 8, 2019 6:46 PM
To: Reshmasree Chamarthi (CTO Office) <reshmasree.chamarthi2@...>; onap-users@... <onap-users@...>
Subject: Re: [Onap-users] Dublin SO pods not running #so
 

** This mail has been sent from an external source **

When you do a describe pod for the catalog-db, requestdb adapter what reason is being given for the restart ?

 

Check the shared mariadb to make sure there are tables in the requestdb, catalogdb

 

If not I would helm delete dev-so –purge, run cleanup.sh , then helm deploy dev-so ... (from the wiki)

 

helm deploy dev-so local/onap -f /root/oom/kubernetes/onap/resources/environments/public-cloud.yaml -f /root/integration-override.yaml --namespace onap  --verbose

 

if you have a slow environment add a –timeout 1000

 

 

Brian

 

 

From: onap-users@... <onap-users@...> On Behalf Of Reshmasree via Lists.Onap.Org
Sent: Thursday, August 08, 2019 9:03 AM
To: onap-users@...
Cc: onap-users@...
Subject: [Onap-users] Dublin SO pods not running #so

 

SO pods are not running properly when we try to deploy dublin SO using '4.0.0-ONAP' oom. Pods status is like below

so-so-5cd8c67bd6-6wmwl                                            0/1     CrashLoopBackOff   9          45m
so-so-bpmn-infra-846988b565-q6m74                                 0/1     CrashLoopBackOff   9          45m
so-so-catalog-db-adapter-74fffc4d66-cx5gn                         0/1     CrashLoopBackOff   9          45m
so-so-mariadb-config-job-z7mdl                                    0/1     Completed          0          45m
so-so-monitoring-847c66c547-fmplz                                 1/1     Running            0          45m
so-so-openstack-adapter-9cf7c5fdb-kcz67                           0/1     CrashLoopBackOff   9          45m
so-so-request-db-adapter-584ddc56c9-qmwqh                         0/1     CrashLoopBackOff   9          45m
so-so-sdc-controller-79769bcfd-bfhk6                              0/1     CrashLoopBackOff   9          45m
so-so-sdnc-adapter-66d5799fb5-7lwx7                               1/1     Running            0          45m
so-so-vfc-adapter-59b6f44b5c-7g7f8                                0/1     CrashLoopBackOff   9          45m
so-so-vnfm-adapter-58d5646878-k5xm8                               1/1     Running            0          45m


There no logs found for debugging. Need help in solving this issue.

Thanks,
Reshma.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com


Brian Freeman
 

And remove dockerdata-nfs/dev-so

 

But the issue is what is it about your environment that is causing k8 to restart your pods.

 

Brian

 

 

From: onap-users@... <onap-users@...> On Behalf Of FREEMAN, BRIAN D
Sent: Thursday, August 08, 2019 9:17 AM
To: reshmasree.chamarthi2@...; onap-users@...
Subject: Re: [Onap-users] Dublin SO pods not running #so

 

***Security Advisory: This Message Originated Outside of AT&T ***
Reference http://cso.att.com/EmailSecurity/IDSP.html for more information.

When you do a describe pod for the catalog-db, requestdb adapter what reason is being given for the restart ?

 

Check the shared mariadb to make sure there are tables in the requestdb, catalogdb

 

If not I would helm delete dev-so –purge, run cleanup.sh , then helm deploy dev-so ... (from the wiki)

 

helm deploy dev-so local/onap -f /root/oom/kubernetes/onap/resources/environments/public-cloud.yaml -f /root/integration-override.yaml --namespace onap  --verbose

 

if you have a slow environment add a –timeout 1000

 

 

Brian

 

 

From: onap-users@... <onap-users@...> On Behalf Of Reshmasree via Lists.Onap.Org
Sent: Thursday, August 08, 2019 9:03 AM
To: onap-users@...
Cc: onap-users@...
Subject: [Onap-users] Dublin SO pods not running #so

 

SO pods are not running properly when we try to deploy dublin SO using '4.0.0-ONAP' oom. Pods status is like below

so-so-5cd8c67bd6-6wmwl                                            0/1     CrashLoopBackOff   9          45m
so-so-bpmn-infra-846988b565-q6m74                                 0/1     CrashLoopBackOff   9          45m
so-so-catalog-db-adapter-74fffc4d66-cx5gn                         0/1     CrashLoopBackOff   9          45m
so-so-mariadb-config-job-z7mdl                                    0/1     Completed          0          45m
so-so-monitoring-847c66c547-fmplz                                 1/1     Running            0          45m
so-so-openstack-adapter-9cf7c5fdb-kcz67                           0/1     CrashLoopBackOff   9          45m
so-so-request-db-adapter-584ddc56c9-qmwqh                         0/1     CrashLoopBackOff   9          45m
so-so-sdc-controller-79769bcfd-bfhk6                              0/1     CrashLoopBackOff   9          45m
so-so-sdnc-adapter-66d5799fb5-7lwx7                               1/1     Running            0          45m
so-so-vfc-adapter-59b6f44b5c-7g7f8                                0/1     CrashLoopBackOff   9          45m
so-so-vnfm-adapter-58d5646878-k5xm8                               1/1     Running            0          45m


There no logs found for debugging. Need help in solving this issue.

Thanks,
Reshma.


Brian Freeman
 

Strange image tag on the docker –

 

The root cause is probably the job to create the requestdb and catalog db is not running correctly so they aren’t there when the pods try to start.

 

Doing the clean and de-deploy is one method.

You can also copy the database scripts into the mariadb-0 container and run the 01-xxx script by hand to run the 2 sql script. (might have tweak the path)

 

Brian

 

 

From: reshmasree.chamarthi2@... <reshmasree.chamarthi2@...>
Sent: Thursday, August 08, 2019 9:32 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-users@...
Subject: Re: [Onap-users] Dublin SO pods not running #so

 

'BackOff' was the reason.


  Type     Reason   Age                   From                 Message

  ----     ------   ----                  ----                 -------

  Normal   Pulled   53m (x4 over 54m)     kubelet, onap-k8s-8  Successfully pulled image "registry:5000/onap/so/request-db-adapter:2aug10"

  Normal   Created  53m (x4 over 54m)     kubelet, onap-k8s-8  Created container

  Normal   Started  53m (x4 over 54m)     kubelet, onap-k8s-8  Started container

  Normal   Pulling  42m (x8 over 54m)     kubelet, onap-k8s-8  pulling image "registry:5000/onap/so/request-db-adapter:2aug10"

  Warning  BackOff  2m9s (x238 over 53m)  kubelet, onap-k8s-8  Back-off restarting failed container

 

There is no catalog/requestdb created in the shared mariadb. What should I do now?


From: onap-users@... <onap-users@...> on behalf of Brian <bf1936@...>
Sent: Thursday, August 8, 2019 6:46 PM
To: Reshmasree Chamarthi (CTO Office) <reshmasree.chamarthi2@...>; onap-users@... <onap-users@...>
Subject: Re: [Onap-users] Dublin SO pods not running #so

 

** This mail has been sent from an external source **

When you do a describe pod for the catalog-db, requestdb adapter what reason is being given for the restart ?

 

Check the shared mariadb to make sure there are tables in the requestdb, catalogdb

 

If not I would helm delete dev-so –purge, run cleanup.sh , then helm deploy dev-so ... (from the wiki)

 

helm deploy dev-so local/onap -f /root/oom/kubernetes/onap/resources/environments/public-cloud.yaml -f /root/integration-override.yaml --namespace onap  --verbose

 

if you have a slow environment add a –timeout 1000

 

 

Brian

 

 

From: onap-users@... <onap-users@...> On Behalf Of Reshmasree via Lists.Onap.Org
Sent: Thursday, August 08, 2019 9:03 AM
To: onap-users@...
Cc: onap-users@...
Subject: [Onap-users] Dublin SO pods not running #so

 

SO pods are not running properly when we try to deploy dublin SO using '4.0.0-ONAP' oom. Pods status is like below

so-so-5cd8c67bd6-6wmwl                                            0/1     CrashLoopBackOff   9          45m
so-so-bpmn-infra-846988b565-q6m74                                 0/1     CrashLoopBackOff   9          45m
so-so-catalog-db-adapter-74fffc4d66-cx5gn                         0/1     CrashLoopBackOff   9          45m
so-so-mariadb-config-job-z7mdl                                    0/1     Completed          0          45m
so-so-monitoring-847c66c547-fmplz                                 1/1     Running            0          45m
so-so-openstack-adapter-9cf7c5fdb-kcz67                           0/1     CrashLoopBackOff   9          45m
so-so-request-db-adapter-584ddc56c9-qmwqh                         0/1     CrashLoopBackOff   9          45m
so-so-sdc-controller-79769bcfd-bfhk6                              0/1     CrashLoopBackOff   9          45m
so-so-sdnc-adapter-66d5799fb5-7lwx7                               1/1     Running            0          45m
so-so-vfc-adapter-59b6f44b5c-7g7f8                                0/1     CrashLoopBackOff   9          45m
so-so-vnfm-adapter-58d5646878-k5xm8                               1/1     Running            0          45m


There no logs found for debugging. Need help in solving this issue.

Thanks,
Reshma.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com


Reshmasree
 

Thanks, Will try those methods.


From: onap-users@... <onap-users@...> on behalf of Brian <bf1936@...>
Sent: Thursday, August 8, 2019 7:12 PM
To: Reshmasree Chamarthi (CTO Office) <reshmasree.chamarthi2@...>; onap-users@... <onap-users@...>
Subject: Re: [Onap-users] Dublin SO pods not running #so
 

** This mail has been sent from an external source **

Strange image tag on the docker –

 

The root cause is probably the job to create the requestdb and catalog db is not running correctly so they aren’t there when the pods try to start.

 

Doing the clean and de-deploy is one method.

You can also copy the database scripts into the mariadb-0 container and run the 01-xxx script by hand to run the 2 sql script. (might have tweak the path)

 

Brian

 

 

From: reshmasree.chamarthi2@... <reshmasree.chamarthi2@...>
Sent: Thursday, August 08, 2019 9:32 AM
To: FREEMAN, BRIAN D <bf1936@...>; onap-users@...
Subject: Re: [Onap-users] Dublin SO pods not running #so

 

'BackOff' was the reason.


  Type     Reason   Age                   From                 Message

  ----     ------   ----                  ----                 -------

  Normal   Pulled   53m (x4 over 54m)     kubelet, onap-k8s-8  Successfully pulled image "registry:5000/onap/so/request-db-adapter:2aug10"

  Normal   Created  53m (x4 over 54m)     kubelet, onap-k8s-8  Created container

  Normal   Started  53m (x4 over 54m)     kubelet, onap-k8s-8  Started container

  Normal   Pulling  42m (x8 over 54m)     kubelet, onap-k8s-8  pulling image "registry:5000/onap/so/request-db-adapter:2aug10"

  Warning  BackOff  2m9s (x238 over 53m)  kubelet, onap-k8s-8  Back-off restarting failed container

 

There is no catalog/requestdb created in the shared mariadb. What should I do now?


From: onap-users@... <onap-users@...> on behalf of Brian <bf1936@...>
Sent: Thursday, August 8, 2019 6:46 PM
To: Reshmasree Chamarthi (CTO Office) <reshmasree.chamarthi2@...>; onap-users@... <onap-users@...>
Subject: Re: [Onap-users] Dublin SO pods not running #so

 

** This mail has been sent from an external source **

When you do a describe pod for the catalog-db, requestdb adapter what reason is being given for the restart ?

 

Check the shared mariadb to make sure there are tables in the requestdb, catalogdb

 

If not I would helm delete dev-so –purge, run cleanup.sh , then helm deploy dev-so ... (from the wiki)

 

helm deploy dev-so local/onap -f /root/oom/kubernetes/onap/resources/environments/public-cloud.yaml -f /root/integration-override.yaml --namespace onap  --verbose

 

if you have a slow environment add a –timeout 1000

 

 

Brian

 

 

From: onap-users@... <onap-users@...> On Behalf Of Reshmasree via Lists.Onap.Org
Sent: Thursday, August 08, 2019 9:03 AM
To: onap-users@...
Cc: onap-users@...
Subject: [Onap-users] Dublin SO pods not running #so

 

SO pods are not running properly when we try to deploy dublin SO using '4.0.0-ONAP' oom. Pods status is like below

so-so-5cd8c67bd6-6wmwl                                            0/1     CrashLoopBackOff   9          45m
so-so-bpmn-infra-846988b565-q6m74                                 0/1     CrashLoopBackOff   9          45m
so-so-catalog-db-adapter-74fffc4d66-cx5gn                         0/1     CrashLoopBackOff   9          45m
so-so-mariadb-config-job-z7mdl                                    0/1     Completed          0          45m
so-so-monitoring-847c66c547-fmplz                                 1/1     Running            0          45m
so-so-openstack-adapter-9cf7c5fdb-kcz67                           0/1     CrashLoopBackOff   9          45m
so-so-request-db-adapter-584ddc56c9-qmwqh                         0/1     CrashLoopBackOff   9          45m
so-so-sdc-controller-79769bcfd-bfhk6                              0/1     CrashLoopBackOff   9          45m
so-so-sdnc-adapter-66d5799fb5-7lwx7                               1/1     Running            0          45m
so-so-vfc-adapter-59b6f44b5c-7g7f8                                0/1     CrashLoopBackOff   9          45m
so-so-vnfm-adapter-58d5646878-k5xm8                               1/1     Running            0          45m


There no logs found for debugging. Need help in solving this issue.

Thanks,
Reshma.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com