1. 30 Jan, 2020 2 commits
    • epriestley's avatar
      Fix an issue where the last line of block-based diffs could be incorrectly hidden · ccf28a81
      epriestley authored
      Summary:
      Fixes T13468. See that task for discussion. The older source-rendering code mixes "line number" / "1-based" lists with "block number" / "0-based" lists and then has other bugs which cancel this out.
      
      For block-based diffs, build an explicit block-based mask with only block numbers. This sort of sidesteps the whole issue.
      
      Test Plan: Viewed the diff with the original reproduction case, plus various other block-based diffs, including one-block image diffs, in unified and side-by-side mode. Didn't spot any oddities.
      
      Maniphest Tasks: T13468
      
      Differential Revision: https://secure.phabricator.com/D20959
      ccf28a81
    • epriestley's avatar
      When issuing a "no-op" MFA token because no MFA is configured, don't give the... · 12c33709
      epriestley authored
      When issuing a "no-op" MFA token because no MFA is configured, don't give the timeline story a badge
      
      Summary:
      Fixes T13475. Sometimes, we issue a "no op" / "default permit" / "unchallenged" MFA token, when a user with no MFA configured does something which is configured to attempt (but not strictly require) MFA.
      
      An example of this kind of action is changing a username: usernames may be changed even if MFA is not set up.
      
      (Some other operations, notably "Sign With MFA", strictly require that MFA actually be set up.)
      
      When a user with no MFA configured takes a "try MFA" action, we see that they have no factors configured and issue a token so they can continue. This is correct. However, this token causes the assocaited timeline story to get an MFA badge.
      
      This badge is incorrect or at least wildly misleading, since the technical assertion it currently makes ("the user answered any configured MFA challenge to do this, if one exists") isn't explained properly and isn't useful anyway.
      
      Instead, only badge the story if the user actually has MFA and actually responded to some kind of MFA challege. The badge now asserts "this user responded to an MFA challenge", which is expected/desired.
      
      Test Plan:
        - As a user with no MFA, renamed a user. Before patch: badged story. After patch: no badge.
        - As a user with MFA, renamed a user. Got badged stories in both cases.
      
      Maniphest Tasks: T13475
      
      Differential Revision: https://secure.phabricator.com/D20958
      12c33709
  2. 29 Jan, 2020 7 commits
    • epriestley's avatar
      Add "Author's Packages" and "Committer's Packages" Herald rules for Commits and Hooks · c99485e8
      epriestley authored
      Summary: Fixes T13480. Adds the remaining missing Owners package rules for Herald commit adapters.
      
      Test Plan: Created hooks which care about these fields, pushed commits, saw sensible transcript values.
      
      Maniphest Tasks: T13480
      
      Differential Revision: https://secure.phabricator.com/D20957
      c99485e8
    • epriestley's avatar
      In Herald "Commit" rules, use repository identities to identify authors and committers · 6628cd2b
      epriestley authored
      Summary: Ref T13480. The Herald "Commit" rules still use raw commit data properties to identify authors and committers. Instead, use repository identities.
      
      Test Plan: Wrote a Herald rule using all four fields, ran it against various commits with and without known authors. Checked transcript for sensible field values.
      
      Maniphest Tasks: T13480
      
      Differential Revision: https://secure.phabricator.com/D20955
      6628cd2b
    • epriestley's avatar
      Respect repository identities when figuring out authors/committers in Herald pre-commit hook rules · 41f143f7
      epriestley authored
      Summary:
      Ref T13480. Currently, Herald commit hook rules use a raw address resolution query to identify the author and committer for a commit. This will get the wrong answer when the raw identity string has been explicitly bound to some non-default user (most often, it will fail to identify an author when one exists).
      
      Instead, use the "IdentityEngine" to properly resolve identities.
      
      Test Plan: Authored a commit as `X <y@example.com>`, a raw identity with no "natural" matches to users (e.g., no user with that email or username). Bound the identity to a particular user in Diffusion. Wrote a Herald pre-commit content rule, pushed the commit. Saw Herald recognize the correct user when evaluating rules.
      
      Maniphest Tasks: T13480
      
      Differential Revision: https://secure.phabricator.com/D20953
      41f143f7
    • epriestley's avatar
      In Herald transcripts, render some field values in a more readable way · a0a346be
      epriestley authored
      Summary:
      Ref T13480. Currently, some Herald field types are rendered in an unfriendly way on transcripts. Particularly, PHID lists are rendered as raw PHIDs.
      
      Improve this by delegating rendering to Value objects and letting "PHID List" value objects render more sensible handle lists. Also improve "bool" fields a bit and make more fields render an explicit "None" / empty value rather than just rendering nothing.
      
      Test Plan: Viewed various transcripts, including transcripts covering boolean values, the "Always" condition, large blocks of text, and PHID lists.
      
      Maniphest Tasks: T13480
      
      Differential Revision: https://secure.phabricator.com/D20951
      a0a346be
    • epriestley's avatar
      In Herald transcript rendering, don't store display labels in keys · 19662e33
      epriestley authored
      Summary:
      Ref T13480. Currently, when Herald renders a transcript, it puts display labels into array keys. This is a bad pattern for several reasons, notably that the values must be scalar (so you can't add icons or other markup later) and the values must be unique (which is easily violated because many values are translated).
      
      Instead, keep values as list items.
      
      Test Plan: Viewed Herald transcripts, saw no (meaningful) change in rendering output.
      
      Maniphest Tasks: T13480
      
      Differential Revision: https://secure.phabricator.com/D20949
      19662e33
    • epriestley's avatar
      Remove legacy pre-loading of handles from Herald rendering · a5a9a5e0
      epriestley authored
      Summary: Ref T13480. When Herald renders rules, it partly uses a very old handle pre-loading mechanism where PHIDs are extracted and loaded upfront. This was obsoleted a long time ago and was pretty shaky even when it worked. Get rid of it to simplify the code a little.
      
      Test Plan: Viewed Herald rules rendered into static text with PHID list actions, saw handles. Grepped for all affected methods.
      
      Maniphest Tasks: T13480
      
      Differential Revision: https://secure.phabricator.com/D20948
      a5a9a5e0
    • epriestley's avatar
      Don't use "line-through" style for completed items in remarkup checklists · 7a1681b8
      epriestley authored
      Summary: Fixes T13482. Although this style makes physical sense by relationship to a written checklist, it seems to do more harm than good in practice.
      
      Test Plan: Wrote a checklist with a checked-off item in remarkup, saw no more line-through.
      
      Maniphest Tasks: T13482
      
      Differential Revision: https://secure.phabricator.com/D20954
      7a1681b8
  3. 23 Jan, 2020 2 commits
  4. 16 Jan, 2020 1 commit
    • epriestley's avatar
      Update "git rev-parse" invocation to work in Git 2.25.0 · 6ccb6a64
      epriestley authored
      Summary:
      Fixes T13479. The behavior of "git rev-parse --show-toplevel" has changed in Git 2.25.0, and it now fails in bare repositories.
      
      Instead, use "git rev-parse --git-dir" to sanity-check the working copy. This appears to have more stable behavior across Git versions, although it's a little more complicated for our purposes.
      
      Test Plan:
        - Ran `bin/repository update ...` on an observed, bare repository.
        - ...on an observed, non-bare ("legacy") repository.
        - ...on a hosted, bare repository.
      
      Maniphest Tasks: T13479
      
      Differential Revision: https://secure.phabricator.com/D20945
      6ccb6a64
  5. 15 Jan, 2020 2 commits
  6. 14 Jan, 2020 3 commits
  7. 13 Dec, 2019 1 commit
    • epriestley's avatar
      Fix an XSS issue with certain high-priority remarkup rules embedded inside... · 54bcbdab
      epriestley authored
      Fix an XSS issue with certain high-priority remarkup rules embedded inside lower-priority link rules
      
      Summary:
      See <https://hackerone.com/reports/758002>. The link rules don't test that their parameters are flat text before using them in unsafe contexts.
      
      Since almost all rules are lower-priority than these link rules, this behavior isn't obvious. However, two rules have broadly higher priority (monospaced text, and one variation of link rules has higher priority than the other), and the latter can be used to perform an XSS attack with input in the general form `()[ [[ ... | ... ]] ]` so that the inner link rule is evaluated first, then the outer link rule uses non-flat text in an unsafe way.
      
      Test Plan:
      Tested examples in HackerOne report. A simple example of broken (but not unsafe) behavior is:
      
      ```
      [[ `x` | `y` ]]
      ```
      
      Differential Revision: https://secure.phabricator.com/D20937
      54bcbdab
  8. 10 Dec, 2019 1 commit
  9. 26 Nov, 2019 1 commit
    • epriestley's avatar
      Extend Config to full-width · 33c534f9
      epriestley authored
      Summary:
      Ref T13362. Some applications moved to fixed-width a while ago but I was generally unsatisfied with where they ended up and have been pushing them back to full-width.
      
      Push Config back to full-width. Some of the subpages end up a little weird, but this provides more space to work with to make some improvements, like makign `maniphest.statuses` more legible in the UI>
      
      Test Plan: Grepped for `setFixed(`, updated each page in `/config/`. Browsed each controller, saw workable full-width UIs.
      
      Maniphest Tasks: T13362
      
      Differential Revision: https://secure.phabricator.com/D20925
      33c534f9
  10. 25 Nov, 2019 1 commit
    • epriestley's avatar
      Implement "PolicyInterface" on "UserEmail" so "EmailQuery" can load them properly · 1667acfa
      epriestley authored
      Summary:
      See PHI1558. Ref T11860. Ref T13444. I partly implemented PHIDs for "UserEmail" objects, but they can't load on their own so you can't directly `bin/remove destroy` them yet.
      
      Allow them to actually load by implementing "PolicyInterface".
      
      Addresses are viewable and editable by the associated user, unless they are a bot/list address, in which case they are viewable and editable by administrators (in preparation for T11860). This has no real impact on anything today.
      
      Test Plan:
        - Used `bin/remove destroy <phid>` to destroy an individual email address.
        - Before: error while loading the object by PHID in the query policy layer.
        - After: clean load and destroy.
      
      Maniphest Tasks: T13444, T11860
      
      Differential Revision: https://secure.phabricator.com/D20927
      1667acfa
  11. 22 Nov, 2019 1 commit
  12. 21 Nov, 2019 1 commit
  13. 19 Nov, 2019 14 commits
    • epriestley's avatar
      Add a "--dry-run" flag to "bin/repository rebuild-identities" · 374f8b10
      epriestley authored
      Summary: Ref T13444. Allow the effects of performing an identity rebuild to be previewed without committing to any changes.
      
      Test Plan: Ran "bin/repository rebuild-identities --all-identities" with and without "--dry-run".
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20922
      374f8b10
    • epriestley's avatar
      Improve use of keys when iterating over commits in "bin/audit delete" and... · 63d84e0b
      epriestley authored
      Improve use of keys when iterating over commits in "bin/audit delete" and "bin/repository rebuild-identities"
      
      Summary:
      Fixes T13457. Ref T13444. When we iterate over commits in a particular repository, the default iteration strategy can't effectively use the keys on the table.
      
      Tweak the ordering so the "<repositoryID, epoch, [id]>" key can be used.
      
      Test Plan:
        - Ran `bin/audit delete --repository X` and `bin/repository rebuild-identities --repository X` before and after changes.
          - With just the key changes, performance was slightly better. My local data isn't large enough to really emphasize the key changes.
          - With the page size changes, performance was a bit better (~30%, but on 1-3 second run durations).
        - Used `--trace` and ran `EXPLAIN ...` on the new queries, saw them select the "<repositoryID, epoch, [id]>" key and report a bare "Using index condition" in the "Extra" column.
      
      Maniphest Tasks: T13457, T13444
      
      Differential Revision: https://secure.phabricator.com/D20921
      63d84e0b
    • epriestley's avatar
      Update repository identities after all mutations to users and email addresses · a7aca500
      epriestley authored
      Summary:
      Ref T13444. Currently, many mutations to users and email addresses (particularly: user creation; and user and address destruction) do not propagate properly to repository identities.
      
      Add hooks to all mutation workflows so repository identities get rebuilt properly when users are created, email addresses are removed, users or email addresses are destroyed, or email addresses are reassigned.
      
      Test Plan:
      - Added random email address to account, removed it.
      - Added unassociated email address to account, saw identity update (and associate).
        - Removed it, saw identity update (and disassociate).
      - Registered an account with an unassociated email address, saw identity update (and associate).
        - Destroyed the account, saw identity update (and disassociate).
      - Added address X to account A, unverified.
        - Invited address X.
        - Clicked invite link as account B.
        - Confirmed desire to steal address.
        - Saw identity update and reassociate.
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20914
      a7aca500
    • epriestley's avatar
      Give "PhabricatorUserEmail" a PHID · 89dcf979
      epriestley authored
      Summary:
      Ref T13444. To interact meaningfully with "DestructionEngine", objects need a PHID. The "UserEmail" object currently does not have one (or a real "Query").
      
      Provide basic PHID support so "DestructionEngine" can interact with the object more powerfully.
      
      Test Plan:
        - Ran migrations, checked data in database, saw sensible PHIDs assigned.
        - Added a new email address to my account, saw it get a PHID.
      
      Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20913
      89dcf979
    • epriestley's avatar
      Use DestructionEngine to destroy UserEmail objects · d69a7360
      epriestley authored
      Summary: Ref T13444. Prepare to hook identity updates when user email addreses are destroyed.
      
      Test Plan:
        - Destroyed a user with `bin/remove destroy ... --trace`, saw email deleted.
        - Destroyed an email from the web UI, saw email deleted.
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20912
      d69a7360
    • epriestley's avatar
      Add additional flags to "bin/repository rebuild-identities" to improve flexibility · 18da3469
      epriestley authored
      Summary:
      Ref T13444. Repository identities have, at a minimum, some bugs where they do not update relationships properly after many types of email address changes.
      
      It is currently very difficult to fix this once the damage is done since there's no good way to inspect or rebuild them.
      
      Take some steps toward improving observability and providing repair tools: allow `bin/repository rebuild-identities` to effect more repairs and operate on identities more surgically.
      
      Test Plan: Ran `bin/repository rebuild-identities` with all new flags, saw what looked like reasonable rebuilds occur.
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20911
      18da3469
    • epriestley's avatar
      Consolidate repository identity resolution and detection code · 0014d040
      epriestley authored
      Summary: Ref T13444. Send all repository identity/detection through a new "DiffusionRepositoryIdentityEngine" which handles resolution and detection updates in one place.
      
      Test Plan:
        - Ran `bin/repository reparse --message ...`, saw author/committer identity updates.
        - Added "goose@example.com" to my email addresses, ran daemons, saw the identity relationship get picked up.
        - Ran `bin/repository rebuild-identities ...`, saw sensible rebuilds.
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20910
      0014d040
    • epriestley's avatar
      Remove "PhabricatorEventType::TYPE_DIFFUSION_LOOKUPUSER" event · 6afbb610
      epriestley authored
      Summary: Ref T13444. This is an ancient event and part of the old event system. It is not likely to be in use anymore, and repository identities should generally replace it nowadays anyway.
      
      Test Plan: Grepped for constant and related methods, no longer found any hits.
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20909
      6afbb610
    • epriestley's avatar
      Distinguish between "Assigned" and "Effective" identity PHIDs more clearly and consistently · a2b2c391
      epriestley authored
      Summary:
      Ref T13444. You can currently explicitly unassign an identity (useful if the matching algorithm is misfiring). However, this populates the "currentEffectiveUserPHID" with the "unassigned()" token, which mostly makes things more difficult.
      
      When an identity is explicitly unassigned, convert that into an explicit `null` in the effective user PHID.
      
      Then, realign "assigned" / "effective" language a bit. Previously, `withAssigneePHIDs(...)` actualy queried effective users, which was misleading. Finally, bulk up the list view a little bit to make testing slightly easier.
      
      Test Plan:
        - Unassigned an identity, ran migration, saw `currentEffectiveUserPHID` become `NULL` for the identity.
        - Unassigned a fresh identity, saw NULL.
        - Queried for various identities under the modified constraints.
      
      Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20908
      a2b2c391
    • epriestley's avatar
      Make repository identity email address association case-insensitive · df0f5c6c
      epriestley authored
      Summary:
      Ref T13444. Currently, identities for a particular email address are queried with "LIKE" against a binary column, which makes the query case-sensitive.
      
        - Extract the email address into a separate "sort255" column.
        - Add a key for it.
        - Make the query a standard "IN (%Ls)" query.
        - Deal with weird cases where an email address is 10000 bytes long or full of binary junk.
      
      Test Plan:
        - Ran migration, inspected database for general sanity.
        - Ran query script in T13444, saw it return the same hits for "git@" and "GIT@".
      
      Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
      
      Maniphest Tasks: T13444
      
      Differential Revision: https://secure.phabricator.com/D20907
      df0f5c6c
    • epriestley's avatar
      When predicting project membership during edits, predict milestones will have parent membership · d58eddcf
      epriestley authored
      Summary:
      Depends on D20919. Ref T13462. When editing milestones, we currently predict they will have no members for policy evaluation purposes. This isn't the right rule.
      
      Instead, predict that their membership will be the same as the parent project's membership, and pass this hint to the policy layer.
      
      See T13462 for additional context and discussion.
      
      Test Plan:
        - Set project A's edit policy to "Project Members".
        - Joined project A.
        - Tried to create a milestone of project A.
          - Before: policy exception that the edit policy excludes me.
          - After: clean milestone creation.
        - As a non-member, tried to create a milestone. Received appropriate policy error.
      
      Maniphest Tasks: T13462
      
      Differential Revision: https://secure.phabricator.com/D20920
      d58eddcf
    • epriestley's avatar
      When predicting object policies for project milestones, adjust objects so they... · 959504a4
      epriestley authored
      When predicting object policies for project milestones, adjust objects so they behave like milestones
      
      Summary:
      Ref T13462. Currently, when testing milestone edit policies during creation, the project object does not behave like a milestone:
      
        - it doesn't have a milestone number yet, so it doesn't try to access the parent project; and
        - the parent project isn't attached yet.
      
      Instead: attach the parent project sooner (which "should" be harmless, although it's possible this has weird side effects); and give the adjusted policy object a dummy milestone number if it doesn't have one yet. This forces it to act like a milestone when emitting policies.
      
      Test Plan:
        - Set "Projects" application default edit policy to "No One".
        - Created a milestone I had permission to create.
          - Before: failed with a policy error, because the project behaved like a non-milestone and returned "No One" as the effective edit policy.
          - After: worked properly, correctly evaluting the parent project edit policy as the effective edit policy.
        - Tried to create a milestone I did not have permission to create (no edit permission on parent project).
          - Got an appropriate edit policy error.
      
      Maniphest Tasks: T13462
      
      Differential Revision: https://secure.phabricator.com/D20919
      959504a4
    • epriestley's avatar
      Remove "stronger/weaker" policy color hints from object headers · de66a8ec
      epriestley authored
      Summary:
      Fixes T13461. Some applications provide hints about policy strength in the header, but these hints are inconsistent and somewhat confusing. They don't make much sense for modern objects with Custom Forms, which don't have a single "default" policy.
      
      Remove this feature since it seems to be confusing things more than illuminating them.
      
      Test Plan:
        - Viewed various objects, no longer saw colored policy hints.
        - Grepped for all removed symbols.
      
      Maniphest Tasks: T13461
      
      Differential Revision: https://secure.phabricator.com/D20918
      de66a8ec
    • epriestley's avatar
      Update the "owner can always view/edit" policy exception rule · 1996b0cd
      epriestley authored
      Summary: Fixes T13460. This rule vanished from the UI in D20165; update things so it returns to the UI.
      
      Test Plan: {F7035134}
      
      Maniphest Tasks: T13460
      
      Differential Revision: https://secure.phabricator.com/D20917
      1996b0cd
  14. 13 Nov, 2019 2 commits
    • epriestley's avatar
      Surface edits to "Text" panels on dashboards as remarkup edits · e86aae99
      epriestley authored
      Summary: Fixes T13456. These edits are remarkup edits and should attach files, trigger mentions, and so on.
      
      Test Plan: Created a text panel, dropped a file in. After changes, saw the file attach properly.
      
      Maniphest Tasks: T13456
      
      Differential Revision: https://secure.phabricator.com/D20906
      e86aae99
    • epriestley's avatar
      Correctly identify more SSH private key problems as "formatting" or "passphrase" related · 2adc36ba
      epriestley authored
      Summary:
      Ref T13454. Fixes T13006. When a user provide us with an SSH private key and (possibly) a passphrase:
      
        # Try to verify that they're correct by extracting the public key.
        # If that fails, try to figure out why it didn't work.
      
      Our success in step (2) will vary depending on what the problem is, and we may end up falling through to a very generic error, but the outcome should generally be better than the old approach.
      
      Previously, we had a very unsophisticated test for the text "ENCRYPTED" in the key body and questionable handling of the results: for example, providing a passphrase when a key did not require one did not raise an error.
      
      Test Plan:
      Created and edited credentials with:
      
        - Valid, passphrase-free keys.
        - Valid, passphrased keys with the right passphrase.
        - Valid, passphrase-free keys with a passphrase ("surplus passphrase" error).
        - Valid, passphrased keys with no passphrase ("missing passphrase" error).
        - Valid, passphrased keys with an invalid passphrase ("invalid passphrase" error).
        - Invalid keys ("format" error).
      
      The precision of these errors will vary depending on how helpful "ssh-keygen" is.
      
      Maniphest Tasks: T13454, T13006
      
      Differential Revision: https://secure.phabricator.com/D20905
      2adc36ba
  15. 09 Nov, 2019 1 commit
    • epriestley's avatar
      Improve recovery from panel action rendering exceptions, and mark "Changeset"... · 72f82abe
      epriestley authored
      Improve recovery from panel action rendering exceptions, and mark "Changeset" queries as not suitable for panel generation
      
      Summary:
      Fixes T13443. When a panel raises an exception during edit action generation, it currently escapes to top level. Instead, catch it more narrowly.
      
      Additionally, mark "ChangesetSearchEngine" as not being a suitable search engine for use in query panels. There's no list view or search URI so it can't generate a sensible panel.
      
      Test Plan:
        - Added a changeset panel to a dashboard.
        - Before: entire dashboard fataled.
        - After: panel fataled narrowly, menu fatals narrowly, dashboard no longer permits creation of another Changeset query panel.
      
      Maniphest Tasks: T13443
      
      Differential Revision: https://secure.phabricator.com/D20902
      72f82abe