1. onap-discuss@lists.onap.org
Topics

[ONAP Helpdesk #65809] Nexus3 slowdown 10X - docker pulls very slow in openlab, my laptop and aws #lfn

Kenny Paul via RT
 

I am looking into this server now .. I do not see anything unusual on memory and CPU usage
and I am still debugging the server. I had performed a restart to see if it helps on anything.

I am right now running a test with Pam's failed jobs.

Can you test and let me know if you see any improvement?

Thanks!
Jess

On Mon Dec 17 23:53:48 2018, Frank.Obrien@amdocs.com wrote:
Team,
I understand the LF is out for now - just passing on an issue that
started appearing this week.
Normally a full docker pull of the 40G of images for Casablanca
takes 25+ min. I am now seeing multiple hours for pulls - so far just
pulling the AAI images in manifest sequence is taking 3 hours - this
occurs even on fast 10G pipes at AWS not just on my windriver cluster.
On Sunday I was able to bring up dmaap in 12 min and multicloud in
2 on a clean cluster - as of today a clean install of just dmaap takes
40 min and multicloud took 35, OOF because of the size took 2 hours to
complete - these are running sequential helm deploys - 1 pod at a time
on 3.0.0-ONAP manifest images.
The long term solution is for me to setup a nexus3 proxy to buffer
the mostly unchanging images away from nexus3.onap.org
Keeping details on
https://jira.onap.org/browse/TSC-79

this is over the course of 4 hours
ubuntu@ip-172-31-16-86:~$ sudo docker images
REPOSITORY TAG IMAGE
ID CREATED SIZE
nexus3.onap.org:10001/onap/aaf/aaf_service 2.1.8
6eb295fed110 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_oauth 2.1.8
74dcdce76094 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_locate 2.1.8
2a4eaa6275ff 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_hello 2.1.8
495a01176053 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_gui 2.1.8
8caa6dc681f0 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_fs 2.1.8
3d663698534d 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_cm 2.1.8
0ba25c4ec3fb 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_agent 2.1.8
090b326a7f11 4 weeks ago 1.14 GB
nexus3.onap.org:10001/onap/aaf/aaf_config 2.1.8
6506ac785cb5 4 weeks ago 1.14 GB
nexus3.onap.org:10001/onap/aaf/aaf_cass 2.1.8
4b91e9b0b43f 4 weeks ago 323 MB
ubuntu@ip-172-31-16-86:~$ sudo docker images
REPOSITORY TAG
IMAGE ID CREATED SIZE
nexus3.onap.org:10001/onap/aaf/aaf_service 2.1.8
6eb295fed110 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_oauth 2.1.8
74dcdce76094 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_locate 2.1.8
2a4eaa6275ff 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_hello 2.1.8
495a01176053 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_gui 2.1.8
8caa6dc681f0 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_fs 2.1.8
3d663698534d 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_cm 2.1.8
0ba25c4ec3fb 4 weeks ago 1.16 GB
nexus3.onap.org:10001/onap/aaf/aaf_agent 2.1.8
090b326a7f11 4 weeks ago 1.14 GB
nexus3.onap.org:10001/onap/aaf/aaf_config 2.1.8
6506ac785cb5 4 weeks ago 1.14 GB
nexus3.onap.org:10001/onap/aaf/aaf_cass 2.1.8
4b91e9b0b43f 4 weeks ago 323 MB
nexus3.onap.org:10001/onap/aaf/smsquorumclient 3.0.1
f8cf701eadc3 6 weeks ago 18.2 MB
nexus3.onap.org:10001/onap/aaf/sms 3.0.1
02363fccc6c7 6 weeks ago 35.4 MB
nexus3.onap.org:10001/onap/aaf/testcaservice 3.0.0
fc717d0b071c 2 months ago 1.17 GB
nexus3.onap.org:10001/onap/aaf/abrmd 3.0.0
00a91d2dc09d 2 months ago 1.15 GB
nexus3.onap.org:10001/onap/aaf/distcenter 3.0.0
d8d9137ef2d3 2 months ago 1.09 GB
nexus3.onap.org:10001/onap/aai-cacher 1.0.0
8ec3df246a35 2 months ago 466 MB


thank you
/michael

This email and the information contained herein is proprietary and
confidential and subject to the Amdocs Email Terms of Service, which
you may review at https://www.amdocs.com/about/email-terms-of-service
<https://www.amdocs.com/about/email-terms-of-service>
1 - 1 of 1