Topics

Nexus3 slowdown 10X - docker pulls very slow in openlab, my laptop and aws #lfn


Michael O'Brien <frank.obrien@...>
 

Team,

    I understand the LF is out for now – just passing on an issue that started appearing this week.

   Normally a full docker pull of the 40G of images for Casablanca takes 25+ min.  I am now seeing multiple hours for pulls – so far just pulling the AAI images in manifest sequence is taking 3 hours – this occurs even on fast 10G pipes at AWS not just on my windriver cluster.

   On Sunday I was able to bring up dmaap in 12 min and multicloud in 2 on a clean cluster – as of today a clean install of just dmaap takes 40 min and multicloud took 35, OOF because of the size took 2 hours to complete – these are running sequential helm deploys – 1 pod at a time on 3.0.0-ONAP manifest images.

    The long term solution is for me to setup a nexus3 proxy to buffer the mostly unchanging images away from nexus3.onap.org

    Keeping details on

https://jira.onap.org/browse/TSC-79

 

this is over the course of 4 hours

ubuntu@ip-172-31-16-86:~$ sudo docker images

REPOSITORY                                   TAG                 IMAGE ID            CREATED             SIZE

nexus3.onap.org:10001/onap/aaf/aaf_service   2.1.8               6eb295fed110        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_oauth     2.1.8               74dcdce76094        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_locate    2.1.8               2a4eaa6275ff        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_hello     2.1.8               495a01176053        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_gui       2.1.8               8caa6dc681f0        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_fs        2.1.8               3d663698534d        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_cm        2.1.8               0ba25c4ec3fb        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_agent     2.1.8               090b326a7f11        4 weeks ago         1.14 GB

nexus3.onap.org:10001/onap/aaf/aaf_config    2.1.8               6506ac785cb5        4 weeks ago         1.14 GB

nexus3.onap.org:10001/onap/aaf/aaf_cass      2.1.8               4b91e9b0b43f        4 weeks ago         323 MB

ubuntu@ip-172-31-16-86:~$ sudo docker images

REPOSITORY                                       TAG                 IMAGE ID            CREATED             SIZE

nexus3.onap.org:10001/onap/aaf/aaf_service       2.1.8               6eb295fed110        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_oauth         2.1.8               74dcdce76094        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_locate        2.1.8               2a4eaa6275ff        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_hello         2.1.8               495a01176053        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_gui           2.1.8               8caa6dc681f0        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_fs            2.1.8               3d663698534d        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_cm            2.1.8               0ba25c4ec3fb        4 weeks ago         1.16 GB

nexus3.onap.org:10001/onap/aaf/aaf_agent         2.1.8               090b326a7f11        4 weeks ago         1.14 GB

nexus3.onap.org:10001/onap/aaf/aaf_config        2.1.8               6506ac785cb5        4 weeks ago         1.14 GB

nexus3.onap.org:10001/onap/aaf/aaf_cass          2.1.8               4b91e9b0b43f        4 weeks ago         323 MB

nexus3.onap.org:10001/onap/aaf/smsquorumclient   3.0.1               f8cf701eadc3        6 weeks ago         18.2 MB

nexus3.onap.org:10001/onap/aaf/sms               3.0.1               02363fccc6c7        6 weeks ago         35.4 MB

nexus3.onap.org:10001/onap/aaf/testcaservice     3.0.0               fc717d0b071c        2 months ago        1.17 GB

nexus3.onap.org:10001/onap/aaf/abrmd             3.0.0               00a91d2dc09d        2 months ago        1.15 GB

nexus3.onap.org:10001/onap/aaf/distcenter        3.0.0               d8d9137ef2d3        2 months ago        1.09 GB

nexus3.onap.org:10001/onap/aai-cacher            1.0.0               8ec3df246a35        2 months ago        466 MB

 

 

    thank you

    /michael

 

This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service