PHP sucks balls,
It takes forever to do anything, it is so messy it feels like walking through a massive pile of shit!

Ok good I have your attention and that ++ 😇

But no this is not that kind of rant, quite the opposite.

In 70 lines of php shit as some people would call it, I am currently scrapping GitHub pages with ebook collections and with some minor regex pulling PDFs out and saving them to file.

About two years ago I get roped into a something when someone was requesting an $8000 laptop to run an "program" that they wrote in Excel to pull data from our mainframe.

In reality they are using our normal application that interacts with the mainframe and screen scrapping it to populate several Excel spreadsheets.

So this guy kept saying that he needed the expensive laptop because he needed the extra RAM and processing power for his application. At the time we only supported 32 bit Windows 7 so even though I told him ten times that the OS wouldn't recognize more than 3.5 GB of RAM he kept saying that increasing the RAM would fix his problem. I also explained that even if we installed the 64 bit OS we didn't have approval for the 64 bit applications.

So we looked at the code and we found that rather than reusing the same workbook he was opening a new instance of a workbook during each iteration of his loop and then not closing or disposing of them. So he was running out of memory due to never disposing of anything.

Even better than all of that, he wanted a faster processor to speed up the processing, but he had about 5 seconds of thread sleeps in each loop so that the place he was screen scrapping from would have time to load. So it wouldn't matter how fast the processor was, in the end there were sleeps and waits in there hard coded to slow down the app. And the guy didn't understand that a faster processor wouldn't have made a difference.

The worst thing is a "dev" that thinks they know what they are doing but they don't have a clue.

Laptop battery hack.
9 more lipos for moi :D

Fuck public transit. If I see on Google Maps that there's gonna be a bus at that place, at that time, there better be a goddamn fucking bus AT THAT PLACE, AT THAT FUCKING TIME!!! No instead let's scrap some shitty lines!

HOW ABOUT WE START SCRAPPING SERVICES JUST BECAUSE WE FEEL LIKE IT, HUH?! Back to postal mail and newspapers you go! You know what, for such fuckers let's just cut their entire internet access. Fucking pieces of shit!!!

1. Apply to as mant jobs as possible daily on dice/linkedin/indeed
using keyword resumes customized by scrapping

2. Filter out low-effort crap companies and filter out recruiters.

3. Post "dice/indeed/linkedin daily decrapified."

Tada! Fewer time-wasters during the job hunt.

4. Bonus: turn into a search engine.

5. Daily double round: turn crap listings and quality listings into AI training sets. Incorporate into search engine.

If industry can use bullshit hiring filters, we can use application filters!

Boss"So, we need to get some data about the users using the APIs from this list of sites."
Me"Alright, sounds feasible enough"

Navigating to first site.

M"Hold on, where's the API?"
B"What do you mean? You're looking at it."
M"This is a website with a search bar, not an API"
B"Same thing. Get to scrapping that data."
M"I-It's written in a JS framework to be reactive in a half-assed way."
B"We need that data"
M"The data is not even consistent!"
B"That's why we need to join it with all these different sources."

The API was a lie. None of the sites had anything remotely similar to an API.

Having to use bloody selenium with chrome driver to scrap all the information because of course, it has to be done programatically every week from now on.

I just hope no captcha of any kind is installed before I finish this project.

I feel like I'm the only person in the world who likes Visual Code from Microsoft as my default editor, I'm considering scrapping Atom for it.

Dev Diary Entry #56
Dear diary, the part of the website that allows users to post their own articles - based on an robust rights system - through a rich text editor, is done! It has a revision system and everything. Now to work on a secure way for them to upload images and use these in their articles, as I don't allow links to external images on the site.

Dev Diary Entry #57
Dear diary, today I finally finished the image uploading feature for my website, and I have secured it as well as I can.

First, I check filesize and filetype client-side (for user convenience), then I check the same things serverside, and only allow images in certain formats to be uploaded.

Next, I completely disregard the original filename (and extension) of the image and generate UUIDs for them instead, and use fileinfo/mimetype to determine extension. I then recreate the image serverside, either in original dimensions or downsized if too large, and store the new image (and its thumbnail) in a non-shared, private folder outside the webpage root, inaccessible to other users, and add an image entry in my database that contains the file path, user who uploaded it, all that jazz.

I then serve the image to the users through a server-side script instead of allowing them direct access to the image. Great success. What could possibly go horribly wrong?

Dev Diary Entry #58
Dear diary, I am contemplating scrapping the idea of allowing users to upload images, text, comments or any other contents to the website, since I do not have the capacity to implement the copyright-filter that will probably soon become a requirement in the EU... :(

Wat to do, wat to do...

Scrapping a classified ads website to start his own.

Hey guys.
So, I have a new printer for parts.
Question, what type of glass does scanner uses? Does it have some special treatment or is it normal glass? Thanks

For all things, for all men, that a man compliments a thing does not imply that this man at least attempts to understand this thing. However, for all men, that a man criticises a thing implies that this man at least attempts to understand this thing.

For all computer programs, that a computer program is terrible implies that scrapping the current implementation of this computer program and beginning anew may be the best method of fixing this computer program.

With few exceptions, for all programming languages $l$, given sufficient effort, $l$ source code can be human-readable.

The UNIX philosophy never became outdated.

For all computer programs $p$, $p$ should be written sufficiently well that the author of $p$ can be prideful of $p$.

For all computer programs $p$, a specification for $p$ should be written before $p$ is created.

For all good computer programs, a good computer program can run on terrible hardware.

Every clock cycle is valuable.

I’m not a web programmer; I’m an application and SQL developer. So when I’m tasked with scrapping a web site for an ETL feed, I thought it would just be a ton of substring and Post/Get calls.

Nope! There is this garbage called JBOSS.A4J where the page isn’t a page but a bunch of files that are merged together and then it isn’t “real” but like a bunch of Photoshop layers that “look” like a page. JavaScript functions based on key press and things like Select/Option that looks like an element but Selenium/PhantomJS (C#) can’t find it. Or my Google-Fu isn’t working.

Made a mistake in scrapping project, realized it AFTER it spent 3 days running, now I relaunched it (corrected) with 4 times more workers to meet the deadline

How to multi task :
Filling the pool;
Scrapping the shit up;
Getting vitamine D and fresh air;

When did you scrap your project and start over, and why?

I'm working on a second project at my company.

But as the project got longer and initially unplanned features started to pile up, the software has become a total mess and a pile of conversion layers for all sorts of I/O.

I'm starting to believe that it'd be better to learn from our mistakes and start over with a better plan, if it means we'll be able to achieve performance and stability improvements.

So, what was it like scrapping your project and starting over?

Okay new Rant
INSERT TRIGGER WARNING HERE

OSX still sucks I have been using the bloody darn thing for last 8 months still I found things that are obnoxiously trivial missing.
Latest incident I was trying to plug in my android phone(soft bricked) in recovery mode and I had to push a file with ADB (i save this mutherfuker for another day). So back to the original topic now I plug it in and but turns out it doesn't recognize my device now as a preliminary check I decide to check my USB cable and my DONGLE both seem to be working fine now I try rebooting back into recovery. Now after scrapping the internet for a few hours I find that this problem is caused because sometimes due to a recurrent bug in OSX the operating system sometimes fails to recognize the difference in between directories "Adam"(just an example) and "adam" which in turn can interfere with some of the flags used while checking if a device might be connected.

I mean this is fucked why the fuck can you not simply use your device as an external storage that would have made the process easier by a fucking lot.

I think the people at Apple are going the destroy a UNIX powerhouse just to make their OS more CUPCAKE friendly.
And all of this is in addition to the problems with AFS.

I just wish I had not bought mac for development

In the perfect future machines would do all the work... Everyone would have enough food, a house, a vehicle and lots of hobbies (cause there isn't work to keep us busy). In truth because machines are programmed by humans and humans are self destructive the rich will survive and live a good, work free live, the rest will live below the poverty line, scrapping anything to get something to eat... Remember Elisium?

When your new upgrade process looks like it will save you a few hours of time. After 18 hours and scrapping the improved process we've finally got the green light.

When you try to scrap without sleep function! :v

I am working on a pub-sub based protocol (like MQTT) with some added features. I am developing a python based server for my protocol which can be run on distributed architecture with load balancing without any tweaks. I am planning to make this server and the protocol open source.

The whole thing is getting so complex that I think about scrapping this project sometimes. I need your inspiration guys. Really, I need it. I know this protocol will be good enough to help people working on IoT, chat or any pub-sub based application if I can complete it. Cheer me up, please.

Can anyone suggest me good practice for scrapping website using js?

What do we all think about Microsoft’s strategy regarding vs code ? Are they actually incorporating their intellisense engine into the open source ??? Are they scrapping vs ? Seems uncharacteristic of their previous models.