Skip to content
  • epriestley's avatar
    Try to route cluster writes to nodes which won't need to synchronize first · 51073b97
    epriestley authored
    Summary:
    Ref T13109. Ref T13202. See PHI905. See PHI889. When we receive a write to a repository cluster, we currently send it to a random writable node.
    
    Instead, we can prefer:
    
      - the node currently holding the write lock; or
      - any node which is already up to date.
    
    These should simply be better nodes to take writes in all cases. The write lock is global for the repository, so there's no scaling benefit to spreading writes across different nodes, and these particular nodes will be able to accept the write more quickly.
    
    Test Plan:
      - This is observable by using `fprintf(STDERR, "%s\n", ...)` in the logic, then running `git push`. I'd like to pull this routing logic out of `PhabricatorRepository` at some point, probably into a dedicated `ClusterURIQuery` sort of class, but that is a larger change.
      - Added some `fprintf(...)` stuff around which nodes were being selected.
      - Added a `sleep(10)` after grabbing the write lock.
      - In one window, pushed. Then pushed in a second window.
        - Saw the second window select the lock holder as the write target based on it currently holding the lock.
        - Without a concurrent push, saw pushes select up-to-date nodes based on their up-to-date-ness.
    
    Reviewers: amckinley
    
    Reviewed By: amckinley
    
    Subscribers: joshuaspence, timhirsh
    
    Maniphest Tasks: T13202, T13109
    
    Differential Revision: https://secure.phabricator.com/D19734
    51073b97