Ranter
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Comments
-
How to create darknet:
1) create website
2) setup robots.txt to forbid all indexing
Welcome to the darknet. It's a shitty, boring place. It's the same as the real internet except everything is really freaking hard to find because there is no Google. Because that's the point. -
@SecFreak @deadPix3l Really? But there ain't no law that says a crawler has to follow what's inside the robots.txt file, it's just something it may choose to honor, right?
-
@ScriptCoded right. Robots.txt is more of a "please don't crawl here" which is why it's great for hackers cause you're essentially saying "this is where the good shit is" but they don't have to obey.
Google, Yahoo, and the like are just nice enough. I think there's an old search engine out there that doesn't respect it and oh man it's fun to search. (Don't ask, can't remember) -
@ScriptCoded honestly when I'm on a pen test, it's one of the first places I look. So many admin panels. Always.
Related Rants
How does the darkweb _actually_ work?
Edit: oops, wrong category. Sawy!
rant
darknet
darkweb