Search does not suggest homebrew-cask formulae when untapped

[dwijnand]

brew config output

HOMEBREW_VERSION: 3.3.7
ORIGIN: https://github.com/Homebrew/brew
HEAD: dbec9de77ebc65a4845a5e672950219aa19b35b8
Last commit: 7 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 9cec8a98224cf1cdfd1f21567306e83a86e096fb
Core tap last commit: 17 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_MAKE_JOBS: 10
Homebrew Ruby: 2.6.8 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: 10-core 64-bit arm_firestorm_icestorm
Clang: 13.0.0 build 1300
Git: 2.30.1 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.77.0 => /usr/bin/curl
macOS: 12.0.1-arm64
CLT: 13.1.0.0.1.1633545042
Xcode: N/A
Rosetta 2: false

brew doctor output

Your system is ready to brew.

Verification

• [X] I ran brew update and am still able to reproduce my issue.
• [X] I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

On a fresh install, I was searching for the name of the iterm2 formula, so I ran brew search iterm2.

What happened (include all command output)?

dnw@192 ~ % brew search iterm2
==> Formulae
term

==> Casks
homebrew/cask-versions/iterm2-beta                                     homebrew/cask-versions/iterm2-legacy                                   homebrew/cask-versions/iterm2-nightly

What did you expect to happen?

I expected to see the regular iterm2 formula, but it wasn't in the output.

Seeing the naming convention I gave brew install iterm2 a try, which forced the tap of homebrew-cask, where the iterm2 formula is:

dnw@192 ~ % brew install iterm2
==> Homebrew is run entirely by unpaid volunteers. Please consider donating:
  https://github.com/Homebrew/brew#donations
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> New Formulae
brigade-cli
Running `brew update --preinstall`...

==> Tapping homebrew/cask
Cloning into '/opt/homebrew/Library/Taps/homebrew/homebrew-cask'...
remote: Enumerating objects: 609514, done.
remote: Counting objects: 100% (24/24), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 609514 (delta 10), reused 19 (delta 7), pack-reused 609490
Receiving objects: 100% (609514/609514), 274.36 MiB | 6.91 MiB/s, done.
Resolving deltas: 100% (430989/430989), done.
Tapped 3938 casks (4,018 files, 293.9MB).
==> Downloading https://iterm2.com/downloads/stable/iTerm2-3_4_14.zip
######################################################################## 100.0%
==> Installing Cask iterm2
==> Moving App 'iTerm.app' to '/Applications/iTerm.app'
🍺  iterm2 was successfully installed!

Looking at brew tap I see that cask-versions isn't tapped, so I don't see why the cask-versions formulae are suggested but not the regular cask formulae. Is it because "iterm2" is the whole name?

Step-by-step reproduction instructions (by running brew commands)

Looks like I can reproduce it by reverting some effects:


dnw@192 ~ % brew uninstall iterm2
b==> Uninstalling Cask iterm2
==> Backing App 'iTerm.app' up to '/opt/homebrew/Caskroom/iterm2/3.4.14/iTerm.app'
r==> Removing App '/Applications/iTerm.app'
==> Purging files for version 3.4.14 of Cask iterm2
dnw@192 ~ % brew untap homebrew/cask
Untapping homebrew/cask...
Untapped 3938 casks (4,019 files, 293.9MB).
dnw@192 ~ % brew search iterm2      
==> Formulae
term

==> Casks
homebrew/cask-versions/iterm2-beta                                     homebrew/cask-versions/iterm2-legacy                                   homebrew/cask-versions/iterm2-nightly

[dwijnand]

(I assume it's expected that I have to manually tap homebrew/cask the second time to be able to reinstall iterm2.)

[MikeMcQuaid]

I cannot reproduce this:

$ brew search iterm2
==> Formulae
term

==> Casks
iterm2-beta               iterm2-legacy             iterm2-nightly            iterm2

Leaving this open for a bit to see if any other maintainers can.

[dwijnand]

Is that without the cask repo tapped?

[MikeMcQuaid]

I have the cask repo tapped. I missed that this can only be reproduced with it untapped. How did it end up untapped in your case?

[dwijnand]

It was a fresh install.

[MikeMcQuaid]

Gotcha. Yeh, we should probably either:

• adjust the search logic to search homebrew-cask correctly

or:

• tap homebrew/cask by default on macOS in the installer (and/or consider doing so in search, too)

[Bo98]

Homebrew/cask and Homebrew/core code search is non-functional. This is a GitHub issue and has been a problem for a while.

The only real fix from our side would be to rewrite this to use our own JSON API instead.

[dwijnand]

Oh, so iterm2 isn't found in untapped Homebrew/cask because there are too many casks before the threshold cutoff, while casks-versions is small enough that it finds iterm2-beta, etc?

[Bo98]

I'm not 100% sure of the technical reason but repo size is probably a contributing factor.

Try code search the repo: https://github.com/Homebrew/homebrew-cask/search?q=cask. No matter the input, it doesn't work at all and brew search uses this exact search system when the repo is untapped.

cask-versions code search works: https://github.com/Homebrew/homebrew-cask-versions/search?q=cask

[MikeMcQuaid]

The only real fix from our side would be to rewrite this to use our own JSON API instead.

Well, auto-tapping homebrew-cask would be another option.

Try code search the repo: https://github.com/Homebrew/homebrew-cask/search?q=cask. No matter the input, it doesn't work at all and brew search uses this exact search system when the repo is untapped.

https://cs.github.com/Homebrew/homebrew-cask?q=cask works so it may be that this gets addressed in future. I don't think we should rely on that to happen soon, though.

[Bo98]

https://cs.github.com/Homebrew/homebrew-cask?q=cask works so it may be that this gets addressed in future.

I did not know that existed - that's useful to know. I'll sign up for that since code search not working in general has been a bit annoying at times.

[MikeMcQuaid]

Another option is we use something like the tree API instead: https://docs.github.com/en/rest/reference/git#get-a-tree https://api.github.com/repos/homebrew/homebrew-cask/git/trees/master?recursive=1

or the repo contents API: https://docs.github.com/en/rest/reference/repos#get-repository-content https://api.github.com/repos/homebrew/homebrew-cask/contents/Casks

[Rylan12]

Another option is we use something like the tree API instead: https://docs.github.com/en/rest/reference/git#get-a-tree https://api.github.com/repos/homebrew/homebrew-cask/git/trees/master?recursive=1

or the repo contents API: https://docs.github.com/en/rest/reference/repos#get-repository-content https://api.github.com/repos/homebrew/homebrew-cask/contents/Casks

Since we only search filenames anyway, this makes sense to me (although my guess is that its not faster, I'd rather get code searching to work on Homebrew/cask).

The only real fix from our side would be to rewrite this to use our own JSON API instead.

I believe your suggestion in #12350 will make this work (maybe not as-is but pretty effortlessly). If not, we should definitely make searching work with HOMEBREW_INSTALL_FROM_API (although that doesn't solve this problem in general)

[MikeMcQuaid]

Since we only search filenames anyway, this makes sense to me (although my guess is that its not faster, I'd rather get code searching to work on Homebrew/cask).

It'll be faster until the code search APIs are ported over to the beta code search. This is likely to be a little while and even then I'm not convinced they will be faster.

I believe your suggestion in #12350 will make this work (maybe not as-is but pretty effortlessly). If not, we should definitely make searching work with HOMEBREW_INSTALL_FROM_API (although that doesn't solve this problem in general)

👍🏻

[buckyster]

I noticed the same issue on a fresh install of brew on a clean machine. I had not added any taps manually. But searching for a package returned results from cask-versions but not cask (both untapped), which was inconsistent and confusing.

% /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

% brew --version
Homebrew 3.3.14

% brew search google-chrome

==> Formulae
googler

==> Casks
homebrew/cask-versions/google-chrome-beta
homebrew/cask-versions/google-chrome-canary
homebrew/cask-versions/google-chrome-dev

% brew install google-chrome

==> Tapping homebrew/cask
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask'...
.
.
.
🍺  google-chrome was successfully installed!

[kidonng]

Since Algolia DocSearch is already providing best-in-class search results on https://formulae.brew.sh, why not simply use that instead?

QUERY=swiss-army COUNT=5 \
      curl 'https://bh4d9od16a-dsn.algolia.net/1/indexes/*/queries?x-algolia-application-id=BH4D9OD16A&x-algolia-api-key=a57ef92bf2adfae863a201ee43d6b5a1' \
      -d '{"requests":[{"indexName":"brew_all","params":"query='$QUERY'&hitsPerPage='$COUNT'&facetFilters=%5B%22site%3A%20formulae%22%5D"}]}' |
      jq '.results[0].hits[].hierarchy | { type: .lvl0, name: .lvl1 }'

Response:

{
  "type": "Formulae",
  "name": "pandoc"
}
{
  "type": "Formulae",
  "name": "mailutils"
}
{
  "type": "Formulae",
  "name": "m-cli"
}
{
  "type": "Formulae",
  "name": "aptly"
}
{
  "type": "Formulae",
  "name": "bnd"
}

Easily create a terminal searcher with fzf:

FZF_DEFAULT_COMMAND=true fzf --disabled --bind 'change:reload:curl \'https://bh4d9od16a-dsn.algolia.net/1/indexes/*/queries?x-algolia-application-id=BH4D9OD16A&x-algolia-api-key=a57ef92bf2adfae863a201ee43d6b5a1\' -d \'{"requests":[{"indexName":"brew_all","params":"query={q}&hitsPerPage=25&facetFilters=%5B%22site%3A%20formulae%22%5D"}]}\' | jq -r \'.results[0].hits[] | .hierarchy.lvl0 + "/" + .hierarchy.lvl1 + if .content then ": " + .content else "" end\''

https://user-images.githubusercontent.com/44045911/159861748-f9b05f6f-a1c3-43fa-9fb4-020ae818e493.mov

[jchitel]

I just ran into this exact case when setting up a new mac. I was trying to install iTerm and I wasn't sure of the name of the cask, so I tried brew search --cask iterm and got no results. There wasn't even a ==> Casks section in the results. This issue appears to identify the core problem, which is that a fresh install does not provide the ability to search casks.

I think that this conversation is getting overrun with comments about improving search functionality/performance, but that seems unrelated to this actual issue. This comment by @MikeMcQuaid seems to address the actual problem:

Gotcha. Yeh, we should probably either:

adjust the search logic to search homebrew-cask correctly or:

tap homebrew/cask by default on macOS in the installer (and/or consider doing so in search, too)

The main point here is that after you tap the casks repo, you have access to cask search, and at that point everything works just fine. Whether the search works or performs well seems like a separate conversation.

Back when casks were a separate domain, I think it made sense that you would have to manually tap the casks repo in order to access search. But now that casks are a first-class concept under the default commands, I think it's a bit weird that a fresh install doesn't come with the casks repo already tapped.

I am in favor of either:

• Tapping the casks repo on install (this is ideal because you can see casks even if you aren't specifically requesting them)
• Tapping the casks repo the first time you try doing anything related to casks (e.g. brew search --cask)

[MikeMcQuaid]

• Tapping the casks repo on install (this is ideal because you can see casks even if you aren't specifically requesting them)
• Tapping the casks repo the first time you try doing anything related to casks (e.g. brew search --cask)

Either of these make sense to me 👍🏻

[Bo98]

Since Algolia DocSearch is already providing best-in-class search results on https://formulae.brew.sh/, why not simply use that instead?

This actually makes sense since it would be a single network operation for all official taps, like what we were doing before for code search, and we'd be offloading search effort to an actual search engine. Doesn't include cask-versions at the moment however.

[kidonng]

Doesn't include cask-versions at the moment however.

Probably because cask-versions isn't in the cask list, so DocSearch won't crawl them.

[MikeMcQuaid]

Good catch @kidonng.

IMO this is another reason we should think about consolidating all the Homebrew/homebrew-cask-* taps into Homebrew/homebrew-cask CC @Homebrew/cask.

[Bo98]

Even if we end up not consolidating repos, it wouldn't be a big task to add to formulae.brew.sh - we would just need to decide how that works in terms of API endpoints.

[vitorgalvao]

consolidating all the Homebrew/homebrew-cask-* taps

There’s four: main, versions, drivers, and fonts. I’d say it makes sense for fonts to remain separate. Versions and drivers could be merged into main, but drivers would need a cleanup beforehand. My suggestion would be to take the most popular ones and move them to main, dumping the rest. The reason that’s a separate repo is that otherwise the main one would be filled with highly-specific drivers.

[MikeMcQuaid]

Versions and drivers could be merged into main, but drivers would need a cleanup beforehand. My suggestion would be to take the most popular ones and move them to main, dumping the rest. The reason that’s a separate repo is that otherwise the main one would be filled with highly-specific drivers.

Sounds good, let's do it.

I’d say it makes sense for fonts to remain separate.

@vitorgalvao Can you elaborate on why? Relatedly, do you think it would matter if these e.g. didn't show up in search by default?

[kidonng]

I’d say it makes sense for fonts to remain separate.

@vitorgalvao Can you elaborate on why?

Probably because there are about 2000 font casks out there, mostly pulled from Google Fonts automatically. It would operate better standalone. This is just my observation, I can't speak for anyone.

[vitorgalvao]

Probably because there are about 2000 font casks out there, mostly pulled from Google Fonts automatically. It would operate better standalone.

Pretty much. The repo has specific requirements (SVN) and its own rules which make sense for its own niche goal of fonts but not apps (other casks).

Relatedly, do you think it would matter if these e.g. didn't show up in search by default?

It wouldn’t matter as much from the user’s perspective, no. But from the contributing side, it becomes harder to manage when a third of the repo (main has about 4000 casks) are casks you’re not supposed to touch. It’s clutter.

Most people don’t care about the vast majority of font casks. But a fonts resource which only has a handful of popular ones isn’t very useful either. The fonts repo is a dump. But because it’s automated and contained, it’s a useful dump. I ultimately feel that throwing all that on the main repo would make both situations worse.

[MikeMcQuaid]

The repo has specific requirements (SVN)

There's a PR open to make this go away sooner rather than later 😄

But from the contributing side, it becomes harder to manage when a third of the repo (main has about 4000 casks) are casks you’re not supposed to touch. It’s clutter. But because it’s automated and contained, it’s a useful dump. I ultimately feel that throwing all that on the main repo would make both situations worse.

Fair 👍🏻

[fharper]

To fix this, I immediately brew tap homebrew/cask after a fresh Homebrew install.

[MikeMcQuaid]

https://github.com/Homebrew/brew/pull/14407 should address this.