One of my favorite moments in HN history was watching the authors of the various search tools decide on a common ".ignore" file as opposed to each having their own: https://news.ycombinator.com/item?id=12568245
Such a good read. I actually went back though it the other day to steal the searching for the least common byte idea out to speed up my search tool https://github.com/boyter/cs which when coupled with the simd upper lower search technique from fzf cut the wall clock runtime by a third.
There was this post from cursor https://cursor.com/blog/fast-regex-search today about building an index for agents due to them hitting a limit on ripgrep, but I’m not sure what codebase they are hitting that warrants it. Especially since they would have to be at 100-200 GB to be getting to 15s of runtime. Unless it’s all matches that is.
Yeah, that Cursor blog post is a bit iffy since they just brush over the "ripgrep is slow on large monorepos", move on to techniques they used, and then completely ignore the fact that you have to build and maintain the index.
On a mid-size codebase, I fzf- and rg-ed through the code almost instantly, while watching my coworker's computer slow down to a crawl when Pycharm started reindexing the project.
I’ve read this multiple times over the years and this post is still the most interesting and informative piece describing the problem of making a fast grep-like tool. I love that it doesn’t just describe how ripgrep works but also how all the other tools work and then compares the various techniques. It’s simultaneously a tutorial and an expert deep dive. Just a beautiful piece of writing. In a perfect world, all code would be similarly documented.
Is IRIX experiencing a hobbyist revival or something? This is the second IRIX reference I’ve seen on here in the past two days, and there was a submission a day or two ago (c.f. a Voodoo video card?) as well. I haven’t personally encountered IRIX in the wild since a company I worked at in 2003. I suppose SGI has always had a cool factor but it’s unusual seeing it come up in a cluster of mentions like this.
SGUG tried hard to port newer packages for IRIX for several years but hit a wall with ABI mismatches leading to GOT corruption. This prevented a lot of larger packages from working or even building.
I picked up the effort again after wondering if LLMs would help. I ran into the ABI problems pretty quickly. This time though, I had Claude use Ghidra to RE the IRIX runtime linker daemon, which gave the LLM enough to understand that the memory structures I’d been using in LLVM were all wrong. See https://github.com/unxmaal/mogrix/blob/main/rules/methods/ir... .
After cracking that mystery I was able to quickly get “impossible” packages building, like WebKit, QT5, and even small bits of Go and Rust.
I’m optimistic that we’ll see more useful applications built for this cool old OS.
That is pretty neat. I guess this sort of unlocking and unblocking effort is exactly what’s needed for a revival.
I’m sort of thinking of AmigaOS/Workbench as well although, perhaps because of what I would assume was always a much larger user base than SGI had, it maybe never went away like SGI and IRIX did.
It is great seeing these old platforms get a new lease of life.
Ooh that's super interesting. I assume you shared the recipe with the irix community? I remember keeping Netscape up to date on my Indy was already a struggle in 2002.
I was using ripgrep once and it had a bug that led me downa terrifying rabbit hole - I can't recall what it was but it involved not being able to find text that absolutely should have been there.
Eventually I was considering rebuilding the machine completely but for some reason after a very long time digging deep into the rabbit hole I tried plain old grep and there was the data exactly where it should have been.
So it's such a vague story but it was a while back - I don't remember the specifics but I sure recall the panic.
Was the file in a .gitignore by any chance? I've got my home folder in git to keep track of dot/config files and that always catches me out. Really dislike it defaulting to that ignoring files that are ignored by git.
Sometimes I forget that some of the config files I have for CI in a project are under a dot directory, and therefore ignored by rg by default, so I have to repeat the search giving the path to that config files subdirectory if I want to see the results that are under that one (or use some extra flags for rg to not ignore dot directories other than .git)
I had that happen too recently… Basically rg x would show nothing but grep -r x showed the lines for any x. Tried multiple times with different x, then I kept using grep -r at that time. After a few days, I started using rg again and it worked fine but now I tend to use grep -r occasionally too to make sure.
I use "grep" to search files (it should never skip any unless I tell it to do otherwise) and "git grep" to be a programmer searching a codebase (where it should only look at code files unless I tell it to do otherwise). Two different hats.
I wouldn't want to use tools that straddle the two, unless they had a nice clear way of picking one or the other. ripgrep does have "--no-ignore", though I would prefer -a / --all (one could make their own with alias rga='rg --no-ignore')
Next time that happens try looking at the paths, adding a pair of -u, or running with --debug: by default rg will ignore files which are hidden (dotfiles) or excluded by ignore files (.gitignore, .ignore, …).
And burntsushi is one of us: he's regularly here on HN. Big thanks to him. As soon as rg came out I was building it on Linux. Now it ships stocks with Debian (since Bookworm? Don't remember): thanks, thanks and more thanks.
> > IMO, as long as the time differences remain small, I'm totally okay with ripgrep being slower by default on smaller corpora if it means being a lot faster by default on bigger corpora.
Note that this is the author of ripgrep replying to a third party commenter asking whether rg isn’t already lightweight, and comparing the two under various possible definitions of “lightweight”.
I don’t understand when people typeset some name in verbatim, lowercase, but then have another name for the actual command. That’s confusing to me.
Programmers are too enarmored with lower-case names. Why not Ripgrep? Then I can surmise that there might not be some program ripgrep(1) (there might be a shorter version), since using capital letters is not traditional for CLI programs.
> Stacked Git, StGit for short, is an application for managing Git commits as a stack of patches.
> ... The `stg` command line tool ...
Now, I’ve been puzzled in the past when inputing `stgit` doesn’t work. But here they call it StGit for short and the actual command is typeset in verbatim (stg(1) would have also worked).
How would you capitalise it? RipGrep? RIPGrep? You’d need to pick a side and lose the pun. (And of course grep itself would need to be GReP if we took it all the way)
Because we are constantly writing variables that are lowercase. Coming up with a name that is both short but immediately understandable is what we live for. Variables are our shrine, we stare at them everyday and are used to their beauty and simplicity.
The fun part is it is pretty easy to “rewrite” ripgrep in rust, because burntsushi wrote it as a ton of crates which you can reuse. So you can reuse this to build your own with blackjack and hookers.
It seems to me that `rg` is the number one most important part that enables LLMs to be smart agents in a codebase. Who would have thought that a code search tool would enable AGI?
There was this post from cursor https://cursor.com/blog/fast-regex-search today about building an index for agents due to them hitting a limit on ripgrep, but I’m not sure what codebase they are hitting that warrants it. Especially since they would have to be at 100-200 GB to be getting to 15s of runtime. Unless it’s all matches that is.
On a mid-size codebase, I fzf- and rg-ed through the code almost instantly, while watching my coworker's computer slow down to a crawl when Pycharm started reindexing the project.
It’s fast even on a 300mhz Octane.
SGUG tried hard to port newer packages for IRIX for several years but hit a wall with ABI mismatches leading to GOT corruption. This prevented a lot of larger packages from working or even building.
I picked up the effort again after wondering if LLMs would help. I ran into the ABI problems pretty quickly. This time though, I had Claude use Ghidra to RE the IRIX runtime linker daemon, which gave the LLM enough to understand that the memory structures I’d been using in LLVM were all wrong. See https://github.com/unxmaal/mogrix/blob/main/rules/methods/ir... .
After cracking that mystery I was able to quickly get “impossible” packages building, like WebKit, QT5, and even small bits of Go and Rust.
I’m optimistic that we’ll see more useful applications built for this cool old OS.
I’m sort of thinking of AmigaOS/Workbench as well although, perhaps because of what I would assume was always a much larger user base than SGI had, it maybe never went away like SGI and IRIX did.
It is great seeing these old platforms get a new lease of life.
Eventually I was considering rebuilding the machine completely but for some reason after a very long time digging deep into the rabbit hole I tried plain old grep and there was the data exactly where it should have been.
So it's such a vague story but it was a while back - I don't remember the specifics but I sure recall the panic.
Sometimes I forget that some of the config files I have for CI in a project are under a dot directory, and therefore ignored by rg by default, so I have to repeat the search giving the path to that config files subdirectory if I want to see the results that are under that one (or use some extra flags for rg to not ignore dot directories other than .git)
I still use it but Ive never trusted it fully since then I double check.
I wouldn't want to use tools that straddle the two, unless they had a nice clear way of picking one or the other. ripgrep does have "--no-ignore", though I would prefer -a / --all (one could make their own with alias rga='rg --no-ignore')
See https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#a... for the details.
I think riggrep will not search UTF-16 files by default. I had some such issue once at least.
https://reddit.com/r/rust/comments/1fvzfnb/gg_a_fast_more_li...
Also something-something about dependencies (a Rust staple): https://www.reddit.com/r/rust/comments/1fvzfnb/gg_a_fast_mor...
I don’t understand when people typeset some name in verbatim, lowercase, but then have another name for the actual command. That’s confusing to me.
Programmers are too enarmored with lower-case names. Why not Ripgrep? Then I can surmise that there might not be some program ripgrep(1) (there might be a shorter version), since using capital letters is not traditional for CLI programs.
Look at Stacked Git:
https://stacked-git.github.io/
> Stacked Git, StGit for short, is an application for managing Git commits as a stack of patches.
> ... The `stg` command line tool ...
Now, I’ve been puzzled in the past when inputing `stgit` doesn’t work. But here they call it StGit for short and the actual command is typeset in verbatim (stg(1) would have also worked).
You may be able to download ripgrep, and execute it (!), but god forbid you can create an alias in your shell in a persistant manner.
Really? "most" even? What CAN you do if you can't edit files in your own $HOME?
The TUI is great, and approximate matches are insanely useful.
Someone kinda did