I've over 17GB of data, downloaded a website, al of the content is .txt and .html.
I want to search inside all of these files.
What is the best tool to do that? any command or some software which can index so it'll be fast?

also is there a way to search inside file content or atleast files, in .rar archives without extracting them

ripgrep is rly fast

@tosensei i'm searching for a string inside file contents in a .rar file which contains thousands of .txt and .html.

now, i've extracted the archive.

but the search is slow, using ripgrep makes it faster. but is there a way to index the content of the .txt and .html files for much faster search.

TextCrawler is good on Windows: https://digitalvolcano.co.uk/tcdown... (also perf wise)

With metalsmith js (https://metalsmith.io) or a similar tool you can create custom processing pipelines without too much effort

The closest you'd be to "reading an archive's contents would be to extract them to a temp folder and remove them after the search operation has completed

VS Code also has a surprisingly performant and advanced search

Are you looking to search only once or many times? If the former you probably won't benefit from indexing

How about elasticsearch?

open in intellij. search for filename, particular text, everything is pretty fast + gui

for searching inside a zip, i think winrar is good for searching filenames as it provides a search bar. for non window platform, i am not sure.

if they are uncompressed, then again, opening in intellij will work just fine

Just use plain notepad++.

You can search entire folders. Lot's of plugins too

I'd give Sublime Texts "find in files" feature a go, you can use regex if you want.

Fd for Linux is a good tool

@awsmprog
Only back up

rm -rf *

He is doing a SPAM extractor which will look for email adrtesses.

@NoToJavaScript nope. i've downloaded over 17gb archive of erotic stories . wanted to find some interesting stories