Skip to content
Snippets Groups Projects
Select Git revision
  • 15f2c97dc6691c455c1784d0347ce70dd290233d
  • collabora/staging default protected
  • master
  • wip/pvigo/add-oidc-compatibility
  • collabora/production protected
  • wip/refi64/update-aptly-with-split-reflists-2
  • wip/refi64/staging-rebase-with-changed-image-name
  • wip/refi64/staging-rebase
  • collabora/rebase-202305 protected
  • wip/refi64/staging-updates
  • backup-27022024-collabora/production
11 results

compress.go

  • Sjoerd Simons's avatar
    15f2c97d
    Use parallel gzip instead of gzip for compression · 15f2c97d
    Sjoerd Simons authored
    
    golangs compress/gzip isn't a parallel implementation, so it's quite a
    bit slower on most modern servers then pgzip. The below benchmark
    run shows that publishing a debian bullseye mirror snapshot (amd64, arm64,
    armhf, source) shows a gain of about 35% in publishing time (when skipping
    bz2 using MR #1081)
    
    ```
     hyperfine -w 1 -m 3 -L aptly aptly-nobz2,aptly-nobz2-pgzip -p "{aptly} -config aptly.conf publish drop bullseye || true" "{aptly} -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye"
    Benchmark 1: aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
      Time (mean ± σ):     35.548 s ±  0.378 s    [User: 39.465 s, System: 10.046 s]
      Range (min … max):   35.149 s … 35.902 s    3 runs
    
    Benchmark 2: aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
      Time (mean ± σ):     26.592 s ±  0.069 s    [User: 42.207 s, System: 9.676 s]
      Range (min … max):   26.521 s … 26.660 s    3 runs
    
    Summary
      'aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye' ran
        1.34 ± 0.01 times faster than 'aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye'
    ```
    
    Signed-off-by: default avatarSjoerd Simons <sjoerd@collabora.com>
    Upstream MR: https://github.com/aptly-dev/aptly/pull/1082
    15f2c97d
    History
    Use parallel gzip instead of gzip for compression
    Sjoerd Simons authored
    
    golangs compress/gzip isn't a parallel implementation, so it's quite a
    bit slower on most modern servers then pgzip. The below benchmark
    run shows that publishing a debian bullseye mirror snapshot (amd64, arm64,
    armhf, source) shows a gain of about 35% in publishing time (when skipping
    bz2 using MR #1081)
    
    ```
     hyperfine -w 1 -m 3 -L aptly aptly-nobz2,aptly-nobz2-pgzip -p "{aptly} -config aptly.conf publish drop bullseye || true" "{aptly} -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye"
    Benchmark 1: aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
      Time (mean ± σ):     35.548 s ±  0.378 s    [User: 39.465 s, System: 10.046 s]
      Range (min … max):   35.149 s … 35.902 s    3 runs
    
    Benchmark 2: aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
      Time (mean ± σ):     26.592 s ±  0.069 s    [User: 42.207 s, System: 9.676 s]
      Range (min … max):   26.521 s … 26.660 s    3 runs
    
    Summary
      'aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye' ran
        1.34 ± 0.01 times faster than 'aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye'
    ```
    
    Signed-off-by: default avatarSjoerd Simons <sjoerd@collabora.com>
    Upstream MR: https://github.com/aptly-dev/aptly/pull/1082