Skip to content
Snippets Groups Projects
  1. Aug 26, 2022
  2. Aug 25, 2022
    • Edmund Smith's avatar
      Add debug logging · 1d2d9375
      Edmund Smith authored
      These help significantly in debugging the late release of tokens which
      can cause querying to stall as the request volume grows.
      
      A previous iteration of the code held the Jobs endpoint open
      throughout its main loop, because it performed its work within a loop
      that only has one iteration. During this loop, the logging endpoint
      was called, which is also rate limited.
      
      Because there can be many jobs active, sharing the same rate limiting,
      it's possible for all the entire system to deadlock on the
      inaccessibility of permits, and in fact it appears to be a
      precondition for successful operation of this system that nothing ever
      tries to obtain two permits simultaneously.
      
      The precondition is difficult to check or track, but fortunately
      debugging it is relatively straightforward when the
      --max-concurrent-jobs is set low (which is a more stringent situation
      that places more demand on permits, in the presence of the same number
      of clients). This is the rationale for including this code, despite
      its invasive nature.
      1d2d9375
    • Edmund Smith's avatar
      Throttle the rate of LAVA API requests · 26faced0
      Edmund Smith authored
      We need to avoid saturating a Lava instance with more requests than it
      can handle when we have many jobs to monitor. To do so, we limit the
      number of concurrent API endpoints that are active to 20. This could
      easily be made configurable in future.
      
      To achieve this, we wrap the Lava instance in a ThrottledLava shell
      that requires a permit to allow each API call to be made. The permits
      are passed on to the return values and need to be held until the API
      call finishes.
      
      The permits themselves are granted by a Throttler, which has an
      underlying semaphore with limited capacity. Many endpoints are handled
      with a simple Throttled wrapper type, which is just a smart pointer
      that holds a Permit. The types that cannot be handled this way are the
      builders.
      26faced0
  3. Aug 12, 2022
    • Sjoerd Simons's avatar
      Support docker image tagging for tags · badf1fb2
      Sjoerd Simons authored
      
      Adjust the docker image generation to differentiate between tags and
      normal builds. For normal commmits it generates containers as
      <branch name>:<sha + pipeline id> as is the case now. For tags it the
      images will be placed under the lava-gitlab-runner name directly with
      the image tag matching the repository tag.
      
      The tagged images will also be protected from garbage collection to
      prevent users potentially missing images
      
      Signed-off-by: default avatarSjoerd Simons <sjoerd@collabora.com>
      v0.1.0
      badf1fb2
  4. Aug 11, 2022
    • Sjoerd Simons's avatar
      Merge branch 'eds/at8952' into 'main' · 6ea8e28a
      Sjoerd Simons authored
      Initial template transformation
      
      See merge request !3
      6ea8e28a
    • Edmund Smith's avatar
      Initial template transformation · 1e1cac97
      Edmund Smith authored
      This uses handlebars to substitute for job variables provided by
      gitlab, with the prefix of `job.`, so to substitute for
      `CI_JOB_TOKEN` you can use `{{ job.CI_JOB_TOKEN }}`.
      
      Note that this is the runner, and template transformations may already
      have been applied at an earlier generation step for the job
      (e.g. using jinja) which means the actual substitutions in this phase
      may need to have been escaped.
      1e1cac97
  5. Jul 18, 2022
  6. May 16, 2022
  7. Apr 13, 2022
  8. Mar 31, 2022
  9. Mar 22, 2022
  10. Mar 21, 2022
  11. Mar 05, 2022
  12. Oct 23, 2021
  13. Oct 15, 2021
  14. Oct 02, 2021
  15. Oct 01, 2021
Loading