Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
I wrote a node + vue web app that consumes bing api and lets you block specific hosts with a click, and I have some thoughts I need to post somewhere.
My main motivation for this it is that the search results I've been getting with the big search engines are lacking a lot of quality. The SEO situation right now is very complex but the bottom line is that there is a lot of white hat SEO abuse.
Commercial companies are fucking up the internet very hard. Search results have become way too profit oriented thus unneutral. Personal blogs are becoming very rare. Information is losing quality and sites are losing identity. The internet is consollidating.
So, I decided to write something to help me give this situation the middle finger.
I wrote this because I consider the ability to block specific sites a basic universal right. If you were ripped off by a website or you just don't like it, then you should be able to block said site from your search results. It's not rocket science.
Google used to have this feature integrated but they removed it in 2013. They also had an extension that did this client side, but they removed it in 2018 too. We're years past the time where Google forgot their "Don't be evil" motto.
AFAIK, the only search engine on earth that lets you block sites is millionshort.com, but if you block too many sites, the performance degrades. And the company that runs it is a for profit too.
There is a third party extension that blocks sites called uBlacklist. The problem is that it only works on google. I wrote my app so as to escape google's tracking clutches, ads and their annoying products showing up in between my results.
But aside uBlacklist does the same thing as my app, including the limitation that this isn't an actual search engine, it's just filtering search results after they are generated.
This is far from ideal because filter results before the results are generated would be much more preferred.
But developing a search engine is prohibitively expensive to both index and rank pages for a single person. Which is sad, but can't do much about it.
I'm also thinking of implementing the ability promote certain sites, the opposite to blocking, so these promoted sites would get more priority within the results.
I guess I would have to move the promoted sites between all pages I fetched to the first page/s, but client side.
But this is suboptimal compared to having actual access to the rank algorithm, where you could promote sites in a smarter way, but again, I can't build a search engine by myself.
I'm using mongo to cache the results, so with a click of a button I can retrieve the results of a previous query without hitting bing. So far a couple of queries don't seem to bring much performance or space issues.
On using bing: bing is basically the only realiable API option I could find that was hobby cost worthy. Most microsoft products are usually my last choice.
Bing is giving me a 7 day free trial of their search API until I register a CC. They offer a free tier, but I'm not sure if that's only for these 7 days. Otherwise, I'm gonna need to pay like 5$.
Paying or not, having to use a CC to use this software I wrote sucks balls.
So far the usage of this app has resulted in me becoming more critical of sites and finding sites of better quality. I think overall it helps me to become a better programmer, all the while having better protection of my privacy.
One not upside is that I'm the only one curating myself, whereas I could benefit from other people that I trust own block/promote lists.
I will git push it somewhere at some point, but it does require some more work:
I would want to add a docker-compose script to make it easy to start, and I didn't write any tests unfortunately (I did use eslint for both apps, though).
The performance is not excellent (the app has not experienced blocks so far, but it does make the coolers spin after a bit) because the algorithms I wrote were very POC.
But it took me some time to write it, and I need to catch some breath.
There are other more open efforts that seem to be more ethical, but they are usually hard to use or just incomplete.
commoncrawl.org is a free index of the web. one problem I found is that it doesn't seem to index everything (for example, it doesn't seem to index the blog of a friend I know that has been writing for years and is indexed by google).
it also requires knowledge on reading warc files, which will surely require some time investment to learn.
it also seems kinda slow for responses,
it is also generated only once a month, and I would still have little idea on how to implement a pagerank algorithm, let alone code it.
4 -
One of main reason why I use Google as a search engine.
Me: Heads - Then I'll not use it.
Me: Okay google, toss a coin
GA: Sure,.... it's a tails.
Well.... fuck.3 -
This is a long post and if someone comments without reading carefully I don't care about that person's opinion.
I have 3 accounts here, and that is a must have for me. Let me explain:
Let's think of people and who they are in layers.
The innermost layers are made of private and intimate things: fears, dreams, shames, basically things that are mostly shared with very close people, like family, best friends, and specially significant others.
On the other hand, outermost layers are the public persona, who you are as a citizen, who you are in your profesion, and so on.
So, you wouldn't normally tell your boss about your favorite sex positions.
Let's also say there can be layers in the middle, and all the layers sometimes overlap, but let's not get too deep into this as I think I got the point across.
Here on I explain the original thesis.
I am a developer, and as such I want to fulfill my needs on dev communities, one of them being devrant.
I wish to learn from other devs, I expose my (sometimes controversial) points of view. I rant about annoying shit in the workplace.
But also, at some level, I wish to be taken seriously as a developer, I wish to build a reputation, and I wish to be accepted, even in a shallow social level. There is a social factor to what we do and it's totally normal.
Now, the problem is that I also would want to express my inner self.
So what I do is I don't use my main account for that, I use another, in fact 2 other accounts.
There are several reasons for that:
* I want to hide intimate shit from trolls.
Imagine I griefpost about a loved one that died, then later found myself in a heated discussion about some language, and then some troll comments something like "I'm glad your x died". i wouldn't react very well.
* I want to keep my posts consistent.
If people become interested in what I post as a dev, then they are going to expect dev related stuff from me. If I start posting like controversial points of view, that's not very cool because I'd be doing like a bait n switch on them.
* I want to maintain a reputation, and I want to not get banned on the main account
Reputation as a profesional is a real thing, and it shouldn't be affected by your personal shit.
Also sometimes you argue, and things get heated, and sometimes you get suspended or banned.
You try your hardest to be respectful, but in some communities, some mods are trigger happy.
By restricting this on your alt account, you're in a way promising that you'll have the upmost behaviour on your dev account because that means being professional.
Now, I said I had 2 other accounts.
The reason for having 2 is because I separate two layers:
In the 2nd account I am open and direct regarding my points of view, and more argumentative, but still trying to be relatively civil. I would also post things that might be controversial or not popular. I try to be real basically.
You can conclude that the 2nd account is the one posting this, since this post could trigger some people.
In the 3rd account, I talk about intimate shit like traumas, fears, emotional pain, things I know I'll get support for (the same support I give others when in need) and are not controversial in any way.
This way I can vent painful things and avoid trolls.
Cool people appreciate it when you're transparent about your shortcoming and dark thoughts.
But it takes one asshole in a high horse to judge you. And sometimes you need to give that asshole the middle finger without being afraid of ruining your reputation
or getting banned,
or being scared of that asshole laughing about your intimate shit (again, I use this account for that)
I know it sounds like I have multiple personalities but I swear I'm ok, and hopefully what I said makes sense. People might say "don't use alt accounts, go to another site", but I find that devrant has some interesting people.
The obvious downside is that you end up knowing people more than what they assume, because you interact with them through different accounts.
This is kinda shady, but I'm not interested in taking advantage of others anyway so...27 -
Hello and welcome, to a presentation in which I will tell you my thoughts on the shortcomings of modern day computers and programming practices.
Computers are based on a very fundamental and old idea, folders, and files, a file is basically a concrete amount of data, whereas a folder is a group of files, and it comes from the real life concept of files and folders, now it might be quite obvious already that using a concept invented in 1898 by a guy called Edwin G. Seibels, might not be the best way for computers to function in the year 2020, but alas, it is.
Unless of course, you step into the world of a programmer.
A programmer’s world is much different, they use this idea of a data structure, or in simpler terms, an object. An Object is just like what you would think of as an object in your head, something with different properties that you can think about in different ways, for example your mobile phone, it has a battery percentage, it has a screen size, it has free space available. Programmers use these data structures to analyse data very quickly, like finding all phones with a screen size bigger than a certain size for example.
The problem is that programmers still use files and folders to create the programs that use these objects.
Consider this example.
Let’s say you want to create a virtual version of a drink bottle, consider what properties it will have, colour, volume, height, width, depth, material, etc..
As a programmer, you can leverage programming features and change the properties of a drink bottle directly, if you wanted to change the colour, you just say, drink bottle “dot” colour, equals blue, or red.
But if the drink bottle was represented as a file, all the drink bottles data would be inside the one file, so you would have to open the whole file, find the line or section of the file that has the colour data of the drink bottle, and select it, highlight it, delete what’s there, and type in your new value.
One way to explain this better is to imagine a folder that now represents the drink bottle, imagine adding a new file into that folder that represents each property I described before, colour, volume, etc.., well now, you could just open that folder, find the file for colour, either by looking with your eyes or you could do a file search in the folder for a file called colour, open it, and edit the value inside. This way of editing objects is the one that more closely represents the way programmers and a program itself interacts with objects inside a running programming language.
But the thing is, programmers don’t use the folder/file way of creating objects and putting them into programs, because it would be too cumbersome, they just create 1 file for an object, or have lots of objects in a file, and create all the objects in 1 file, and then run the program which creates the objects, then when they stop the program, it deletes the objects. So there is no actual link between the object in a file and the object that the program creates by reading the data from that file, if you change the object in your program, it does not get saved to the file.
So programmers created databases to house these objects, but there is still a flaw in databases, they are hard to interface with, and mostly databases are just used to send data or retrieve data from, programmatically, you can’t really browse a database the way you can browse the files on your computer. You can, but database interfaces are not made to be easily navigated the way files and folders are.
As it stands, there is no way to store objects instead of files on your computer and interact with them in complex ways the way programmers can inside the programs they create.
If the idea of an object became standard the way a file and folder is standard, I think it would empower human’s a great deal to express things far more easily and fluidly than they can today.
Thanks for reading.8 -
WARNING - a lot of text.
I am open for questions and discussions :)
I am not an education program specialist and I can't decide what's best for everyone. It is hard process of managing the prigram which is going through a lot of instances.
Computer Science.
Speaking about schools: regular schools does not prepare computer scientists. I have a lot of thoughts abouth whether we need or do NOT need such amount of knowledge in some subjects, but that's completely different story. Back to cs.
The main problem is that IT sphere evolves exceedingly fast (compared to others) and education system adaptation is honestly too slow.
SC studies in schools needs to be reformed almost every year to accept updates and corrections, but education system in most countries does not support that, thats the main problem. In basic course, which is for everyone I'd suggest to tell about brief computer usage, like office, OS basics, etc. But not only MS stuff... Linux is no more that nerdy stuff from 90', it's evolved and ready to use OS for everyone. So basic OS tour, like wtf is MAC, Linux (you can show Ubuntu/Mint, etc - the easy stuff) would be great... Also, show students cloud technologies. Like, you have an option to do *that* in your browser! And, yeah, classy stuff like what's USB and what's MB/GB and other basic stuff.. not digging into it for 6 months, but just brief overview wuth some useful info... Everyone had seen a PC by the time they are studying cs anyway.. and somewhere at the end we can introduce programming, what you can do with it and maybe hello world in whatever language, but no more.. 'cause it's still class for everyone, no need to explain stars there.
For last years, where shit's getting serious, like where you can choose: study cs or not - there we can teach programming. In my country it's 2 years. It's possible to cover OOP principles of +/- modern language (Java or C++ is not bad too, maybe even GO, whatever, that's not me who will decide it. Point that it's not from 70') + VCS + sime real world app like simplified, but still functional bookstore managing app.
That's about schools.
Speaking about universities - logic isbthe same. It needs to be modern and accept corrections and updates every year. And now it depends on what you're studying there. Are you going to have software engineering diploma or business system analyst...
Generally speaking, for developers - we need more real world scenarios and I guess, some technologies and frameworks. Ofc, theory too, but not that stuff from 1980. Come-on, nowadays nobody specifies 1 functional requirement in several pages and, generally, nobody is writing that specification for 2 years. Product becomes obsolete and it's haven't even started yet.
Everything changes, whether it is how we write specification documents, or literally anything else in IT.
Once more, morale: update CS program yearly, goddammit
How to do it - it's the whole another topic.
Thank you for reading.3 -
!rant !dev
So, following up my last rant.
https://devrant.com/rants/2433162
I quit on Friday, this is what I said to my bosses.
"In the last week I had, 2 panic attacks, and I have 2 theories for this, one is that I have underlying psychological problems, the other theory is that we are under an impossible task, I choose to say now that I have to quit because I have psychological issues, but if you are willing to hear my other theory, that involves saying that meeting the deadline is not viable, then I can tell you that, so do want to listen that part?.
Bosses: No, we heard enough, we are going to have your contract terminated in order, and we will let you know when you can come and pick your paycheck."
So, that's them. Now about me and how I re-discovered GTD, or more precisely how I organized my whole weekend using taskwarrior with GTD, and why I think is going to be useful as a freelancer.
Before I feel good about telling you about my weekend I have to tell you a few things about myself.
I am a very impulsive person, I have a lot of energy in short surges, so I have to be able to maximize my activity when I'm in a surge, and I have to maximize my rest when I am not.
That's hard to do, it requires a balanced lifestyle, I am also very prone to being neurotic, and overwhelmed by the amount of stuff that I want to do.
And on top of that, when I am resting, I have surges of things that I want to have, do, or implement, it could be software related, as "Doing an app that will be the Uber of home services", to house improvements like, "I have to fix that leaking roof", and all the sort of stuff that happens in between hardware and software. That surge of consciousness doesn't allow me to have the proper rest that I need before I engage with activities again.
Because of this I have a very cyclic rhythm, with whole weeks burning my energy into doing stuff, and weeks resting doing very little and thinking too much.
Now about my weekend. Friday night I was browsing the web, and a thought came to my head. "The way you use your terminal, says a lot about your personality", and I got curious, so I searched for, "Show me your terminal", and found a post in dev.to to see all kind of nice terminal setups, from the very minimalist to very feature rich oh-my-zsh themes with plugins for git, aws and what not. One of these pictures really got my attention, a guy had set up his terminal to show him, how many task has he done in the day, and how many cups of coffee has he had.
So by investigating how he set up his terminal to show in the prompt the number of successfully completed tasks in the day, I found out that he was using taskwarrior, he was also kind enough to share the source code of his prompt setup, which I bookmarked to later incorporate that into my oh-my-zsh config.
After reading about taskwarrior, I also got a reference to GTD, I don't remember if this was one of those thoughts that I have and follow immediately, or if I read something that led me to a YouTube video summarizing GTD.
In the end, after watching that GTD video, I decided to give it a try to organize my life, and help me find a remote job, keep my house in order, plan my social activities as "hang out with friends", "visit mom and dad", and give the proper amount of attention to my GF, with whom I am deeply in love, and willing to spend the remaining of my years with her.
So my fist task was.
task add Ask for GF's parents blessing.
Which of course I have no intention of doing right now, but is one of the things that I will eventually have to do.
Then it started, I started adding tasks, and things to do, and go through the whole Capture phase of GTD.
Now it is a good time to write a small summary of what I think GTD is.
GTD is a life habit of organizing your life in todo-lists. And it was a very specific core method, that in the video summary that I watched was called CPR.
Capture, Process and Review.
Capture:
When you capture you just add your tasks to a bucket list.
So I took a notebook and started writing down everything that I wanted to have done. I also started to capture ideas as they came up to me, I did this by writing a telegram saved message in my phone, or directly adding it as a task in TW.
Process:
I read my telegram messages and put them into my task warrior list, then I started to organize my tasks into projects, breaking down every task that was not an atomic unit.
* And different projects started to emerge from this. One of them was project:Housekeeping.
And here's my screenshot of what I did this weekend, also the number of projects that I have, and all the things that I have to do in order to have what I think would be a very balanced, fun, and productive life.
You'll be able to see in the screenshot, that there's a blocked task, yes, tw allows you to organize dependencies too, so one task is delegated, and blocked by the delegation task.
1 -
Upgrading my tech skills.. Once again I feel my personal my personal dev environment and told are much more up-to-date than what I use at work.... Though the book Kim reading is on TDD and was written 3 years ago.
Maybe I should read another on in cloud services and ML... but don't have any motivation for these topics.
I need TDD for work because now we're emphasizing unit test coverage...
I usually only use manual functional tests to verify the final outputs as either the testing framework is broken (JS) or I don't have time to relearn the frameworks for the particular language...
Anyway got off topic... So questions after:
1. Do you ever feel your technologically always more ahead than what you do at work and essentially you bring skills to the job but you don't learn much out of it?
2. How do you test? I actually got into a bit of a argument/discussion with my colleagues about how to implement unit tests. Apparently there are 2 ways to test? Black box vs WhiteBox. She said she tests only Public methods using mock inputs, dependencies. She read online and seems there is an opinion that should only test public functions and if you can't then your app is designed incorrectly, not separated enough.
For me I test the private functions individually (WhiteBox/Java reflection) because the public one is like generateReport and as a whole is like a Pachinko machine, too many unique paths that would need a test case for.
So thoughts? Yes sorry for turning it into a remake I guess...24
Top Tags
Weekly Rant
View