update.sh: update to latest Homebrew/brew tag. #998

MikeMcQuaid · 2016-09-17T18:46:49Z

Have you followed the guidelines in our Contributing document?
Have you checked to ensure there aren't other open Pull Requests for the same change?
Have you added an explanation of what your changes do and why you'd like us to include them?
Have you written new tests for your changes? Here's an example.
Have you successfully run brew tests with your changes locally?

Rather than following every change on master let’s have non-developer users (i.e. those who have never run a dev-cmd or set HOMEBREW_DEVELOPER) update between tags.

This provides a fairly natural beta (the master branch) and stable (the tags) approach without restricting us to any particular way of managing our tags.

Also, ensure we always fetch tags and allow forcing tag update functionality by setting
HOMEBREW_UPDATE_TO_TAG.

This shouldn't be merged until after we tag 1.0.0 as 0.9.9 is too old to have users on for now.

chdiza · 2016-09-18T06:06:57Z

I'm not sure I understand what this involves. If this means that the "non-developer" who does brew update won't receive an update unless a new tag has been made since their last update, then I don't think this is a good idea.

Reason: Suppose a user files an issue and some fix is made in the brew repo. But no new tag is created. Then we can't just tell users "do brew update and try again"; we'd have to say "set HOMEBREW_DEVELOPER and then do brew update and try again" (or "run a dev-cmd and then brew update and then try again). Besides being windy, that's just asking for user confusion. If to avoid this new tags are created frequently, this also seems like not a good idea. (Just wait until some tagged commit has to be reverted, etc.)

Apologies if I've misunderstood what this PR does.

zmwangx · 2016-09-18T06:15:23Z

If this means that the "non-developer" who does brew update won't receive an update unless a new tag has been made since their last update, then I don't think this is a good idea.

We've had a couple quite widespread breakages recently (including the linked kegs migration debacle today of which we're still dealing with the fallout) we definitely need to be more careful at upgrading all users to master.

Adding a homebrew_developer? check to every feature then remove it after a while doesn't scale.

We didn't say how often tags will be made by the way.

we'd have to say "set HOMEBREW_DEVELOPER and then do brew update and try again"

$ HOMEBREW_DEVELOPER=1 brew update

What's so difficult about that.

that's just asking for user confusion.

Slight user confusion when communicating to a single user or a few users is better than "no-commands-run" for ~~thousands of~~ at least many users (which happened today, or yesterday depending on where you are).

(Just wait until some tagged commit has to be reverted, etc.)

Reverting a tagged commit has no effect on that tag. We don't force push here. I don't see how it's worse than reverting a commit on the master branch when users are updated to the master branch.

codecov-io · 2016-09-18T12:46:48Z

Current coverage is 83.25% (diff: 100%)

Merging #998 into master will increase coverage by 7.99%

@@             master       #998   diff @@
==========================================
  Files           360        340     -20   
  Lines         15543      14051   -1492   
  Methods           0          0           
  Messages          0          0           
  Branches          0          0           
==========================================
+ Hits          11697      11698      +1   
+ Misses         3846       2353   -1493   
  Partials          0          0

Powered by Codecov. Last update f4a8d28...d1c3793

MikeMcQuaid · 2016-09-18T12:51:55Z

Then we can't just tell users "do brew update and try again"; we'd have to say "set HOMEBREW_DEVELOPER and then do brew update and try again" (or "run a dev-cmd and then brew update and then try again). Besides being windy, that's just asking for user confusion.

Really this is a question/debate about rolling release vs. tagged release software development approaches. I agree the tag approach would not make sense for taps (for now, at least) but except for when we've broken something with another change it's extremely rare that there are user fixes that we need to rapidly make changes to fix. In those cases, we can push them out more quickly to HOMEBREW_DEVELOPERs and, if things look good after a few days, make a tag.

chdiza · 2016-09-18T16:07:53Z

I like the idea of "tagged release" for brew internals. Very much so. I just was doubtful about the implementation of putting it behind HOMEBREW_DEVELOPER.

Instead of having to tell users to do HOMEBREW_DEVELOPER=1 brew update, there should perhaps be a new flag for update. I don't know what the name would be, but something like brew update --rolling or something.

What's so difficult about that.

It's longer and it looks weird to the sort of newbies who are prone to getting confused.

Slight user confusion when communicating to a single user or a few users is better than "no-commands-run" for thousands of at least many users (which happened today, or yesterday depending on where you are).

There aren't just these two possibilities.

MikeMcQuaid · 2016-09-18T16:10:31Z

@chdiza It's behind HOMEBREW_DEVELOPER or if you've ever run a dev-cmd. We may iterate on the approach but this will be the one we start with.

MikeMcQuaid · 2016-09-20T07:07:31Z

@jawshooah @reitermarkus A heads up that this will be happening in the next few days as it'll affect you too.

zmwangx · 2016-09-20T11:02:11Z

Library/Homebrew/cmd/update.sh

On a regular install tags aren't fetched, so UPSTREAM_TAG would be empty here.

Handled by #1046

Further tweaked the approach to handle if this is empty.

zmwangx · 2016-09-20T11:04:50Z

As it currently stands this branch seems problematic due to lack of tags locally. Reproducer on clean VM:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install hub
cd /usr/local/Homebrew
hub checkout https://github.com/Homebrew/brew/pull/998
brew update

and that last brew update produces

Checking out v in /usr/local/Homebrew...
To checkout MikeMcQuaid-update-between-tags in /usr/local/Homebrew run:
  'cd /usr/local/Homebrew && git checkout MikeMcQuaid-update-between-tags
fatal: Cannot update paths and switch to branch 'v' at the same time.
Did you intend to checkout 'refs/tags/' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/
Already up-to-date.

It's trying to check out v because UPSTREAM_TAG is empty.

MikeMcQuaid · 2016-09-20T11:23:04Z

@zmwangx Well noticed. We'll need to do a forced-git fetch if there's no tags.

Rather than following every change on `master` let’s have non-developer users (i.e. those who have never run a `dev-cmd` or set `HOMEBREW_DEVELOPER`) update between tags. This provides a fairly natural beta (the `master` branch`) and stable (the tags) approach without restricting us to any particular way of managing our tags.

To test the tag update functionality allow setting `HOMEBREW_UPDATE_TO_TAG`.

mgol · 2016-09-21T13:57:49Z

This broke my setup. I had updated to d5b6ecf about 5-6 hours ago via brew update && brew upgrade. I installed the s3cmd package 30 minutes ago with the following output:

$ brew install s3cmd
Checking out v1.0.0 in /usr/local/Homebrew...
To checkout master in /usr/local/Homebrew run:
  'cd /usr/local/Homebrew && git checkout master
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
==> Downloading https://homebrew.bintray.com/bottles/s3cmd-1.6.1.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring s3cmd-1.6.1.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/s3cmd/1.6.1: 59 files, 784.4K

These messages worried me so I invoked brew update:

$  brew update
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Already up-to-date.
+ brew upgrade --all
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0

This happened every time so I went into /usr/local/Homebrew and invoked git prune & git gc & git fsck and repeated brew update:

$ Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Already up-to-date.
+ brew upgrade --all
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0

MikeMcQuaid · 2016-09-21T14:03:44Z

@mgol Can I see the output of cd $(brew --repo) && git config homebrew.devcmdrun? Also, can you try to brew pull https://github.com/Homebrew/brew/pull/1060 && brew update --force and see how that behaves? Thanks!

MikeMcQuaid · 2016-09-21T14:05:04Z

@mgol and also, actually, echo HOMEBREW_DEVELOPER: $HOMEBREW_DEVELOPER. Thanks!

mgol · 2016-09-21T14:14:23Z

This:

cd $(brew --repo) && git config homebrew.devcmdrun

cds to /usr/local/Homebrew and prints nothing.

This:

brew pull https://github.com/Homebrew/brew/pull/1060 && brew update --force

resulted in:

rew pull https://github.com/Homebrew/brew/pull/1060 && brew update --force
Warning: Current branch is v1.0.0: do you need to pull inside master?
==> Fetching patch 
Patch: https://github.com/Homebrew/brew/pull/1060.patch
==> Applying patch
Applying: update.sh: tweak tag handling.
==> Patch closes issue #1060
==> Patch changed:
 Library/Homebrew/cmd/update.sh | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Updated Homebrew from d5b6ecf to 09e0b09.
Updated 2 taps (caskroom/cask, homebrew/core).
==> Updated Formulae
antigen               fzf                   gupnp                 ipfs                  offlineimap           sonar-runner        
aws-sdk-cpp           global                imagemagick           khard                 reposurgeon

And now brew update && brew upgrade behaves normally.

mgol · 2016-09-21T14:14:54Z

About the last bit:

$ echo HOMEBREW_DEVELOPER: $HOMEBREW_DEVELOPER
HOMEBREW_DEVELOPER:

So it's empty.

MikeMcQuaid · 2016-09-21T14:24:50Z

@mgol Thanks, will 🚢 #1060 to fix this shortly.

hkdobrev · 2016-09-21T18:12:02Z

Just my 2 cents on tag handling - Composer did a very nice thing when going stable. They have 3 update channels:

stable is the default and updates only to stable releases (following semver)
preview updates to any tag including alpha, beta and rc tags according to semver
snapshot updates to latest master.

The other nice thing is that once you update with a specific channel it keeps you on that channel and it explicitly stays what channel it uses the next time you update. This is really nicely working for Composer, not sure if it'd be good for Homebrew.

In short I really like the use of non-stable and stable tags which Composer does and hope Homebrew does something like that.

PS:

jtsom · 2016-09-21T18:54:14Z

So how do I get Homebrew to update from 1.0.0 to 1.0.1? brew update just says I'm up to date.

I did get the fatal: Cannot update paths and switch to branch 'v1.0.1' at the same time. message, but I can't get 1.0.1 now. Short of nuking the whole thing (which I don't want to have to reinstall all my packages), what's the solutions?

holybit · 2016-09-21T20:13:53Z

In my case it seems as if Homebrew updated correctly to 1.0.1 but the brew command doesn't seem to be showing the correct version?

$ brew update
Checking out v1.0.1 in /usr/local/Homebrew...
To checkout v1.0.1 in /usr/local/Homebrew run:
  'cd /usr/local/Homebrew && git checkout v1.0.1
Already up-to-date.

$ cd /usr/local/Homebrew && git checkout v1.0.1
Already on 'v1.0.1'

$ brew --version
Homebrew 1.0.0 (git revision a381b; last commit 2016-09-21)
Homebrew/homebrew-core (git revision 48e0; last commit 2016-09-21)

$ cd /usr/local/Homebrew
$ git show-ref --tags | tail -2
d5b6ecfc5078041ddf5f61b259c57f81d5c50fcc refs/tags/1.0.0
a381b597d3ebf40887b50657df269d867a209eb7 refs/tags/1.0.1

MikeMcQuaid · 2016-09-22T07:08:42Z

but the brew command doesn't seem to be showing the correct version?

Ignore this, it was me being an idiot and not remembering to bump the version number in the output.

MikeMcQuaid added the do not merge label Sep 17, 2016

BrewTestBot added the in progress Maintainers are working on this label Sep 17, 2016

MikeMcQuaid added this to the 1.0.0 milestone Sep 17, 2016

MikeMcQuaid mentioned this pull request Sep 18, 2016

update.sh: turn origin/branch into a variable. #1007

Merged

5 tasks

zmwangx reviewed Sep 20, 2016

View reviewed changes

MikeMcQuaid mentioned this pull request Sep 20, 2016

Update: ensure tags are fetched #1046

Merged

5 tasks

MikeMcQuaid added 2 commits September 21, 2016 09:03

update.sh: allow forcing tag update.

e8ebcb8

To test the tag update functionality allow setting `HOMEBREW_UPDATE_TO_TAG`.

MikeMcQuaid merged commit d5b6ecf into Homebrew:master Sep 21, 2016

MikeMcQuaid deleted the update-between-tags branch September 21, 2016 08:26

zmwangx removed do not merge in progress Maintainers are working on this labels Sep 21, 2016

MikeMcQuaid mentioned this pull request Sep 21, 2016

update.sh: tweak tag handling. #1060

Merged

5 tasks

zmwangx mentioned this pull request Sep 21, 2016

Cannot brew update anymore #1067

Closed

3 tasks

Homebrew locked and limited conversation to collaborators May 3, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

update.sh: update to latest Homebrew/brew tag. #998

update.sh: update to latest Homebrew/brew tag. #998

Uh oh!

MikeMcQuaid commented Sep 17, 2016 •

edited by apjanke

Loading

chdiza commented Sep 18, 2016

zmwangx commented Sep 18, 2016 •

edited

Loading

codecov-io commented Sep 18, 2016 •

edited

Loading

MikeMcQuaid commented Sep 18, 2016

chdiza commented Sep 18, 2016

MikeMcQuaid commented Sep 18, 2016

MikeMcQuaid commented Sep 20, 2016

zmwangx Sep 20, 2016 •

edited

Loading

MikeMcQuaid Sep 20, 2016

MikeMcQuaid Sep 21, 2016

zmwangx commented Sep 20, 2016

MikeMcQuaid commented Sep 20, 2016

mgol commented Sep 21, 2016

MikeMcQuaid commented Sep 21, 2016

MikeMcQuaid commented Sep 21, 2016

mgol commented Sep 21, 2016

mgol commented Sep 21, 2016

MikeMcQuaid commented Sep 21, 2016

hkdobrev commented Sep 21, 2016 •

edited

Loading

jtsom commented Sep 21, 2016

holybit commented Sep 21, 2016 •

edited

Loading

MikeMcQuaid commented Sep 22, 2016

Uh oh!

update.sh: update to latest Homebrew/brew tag. #998

update.sh: update to latest Homebrew/brew tag. #998

Uh oh!

Conversation

MikeMcQuaid commented Sep 17, 2016 • edited by apjanke Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

chdiza commented Sep 18, 2016

zmwangx commented Sep 18, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

codecov-io commented Sep 18, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current coverage is 83.25% (diff: 100%)

MikeMcQuaid commented Sep 18, 2016

chdiza commented Sep 18, 2016

MikeMcQuaid commented Sep 18, 2016

MikeMcQuaid commented Sep 20, 2016

zmwangx Sep 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

MikeMcQuaid Sep 20, 2016

Choose a reason for hiding this comment

MikeMcQuaid Sep 21, 2016

Choose a reason for hiding this comment

zmwangx commented Sep 20, 2016

MikeMcQuaid commented Sep 20, 2016

mgol commented Sep 21, 2016

MikeMcQuaid commented Sep 21, 2016

MikeMcQuaid commented Sep 21, 2016

mgol commented Sep 21, 2016

mgol commented Sep 21, 2016

MikeMcQuaid commented Sep 21, 2016

hkdobrev commented Sep 21, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

jtsom commented Sep 21, 2016

holybit commented Sep 21, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MikeMcQuaid commented Sep 22, 2016

MikeMcQuaid commented Sep 17, 2016 •

edited by apjanke

Loading

zmwangx commented Sep 18, 2016 •

edited

Loading

codecov-io commented Sep 18, 2016 •

edited

Loading

zmwangx Sep 20, 2016 •

edited

Loading

hkdobrev commented Sep 21, 2016 •

edited

Loading

holybit commented Sep 21, 2016 •

edited

Loading