Networking Professor: Alright, how many of you disagree with the way Google collects and uses data?

*basically everyone in the lecture hall raises their hands*

Professor: Okay, now if you were offered a job at Google, how many of you would still take it?

*everyone raises their hands.*

The new Dutch mass surveillance law goes into action on the first of May. I'll of course have a good security setup ready but that does not stop the bulk data collection.

I just setup a website which (still in English at the moment) requests a random search result from bing, google or DuckDuckGo every 3 seconds.

Will work on making it more 'real' :)

If stopping the surveillance isn't an option, let's add more data to filter out for them!

So, as everyone knows on here by now (or, a lot of ranters), I am a fervid privacy person.

Appearantly a new surveillance law in my country is about to extend mass surveillance/hacking a lot. So here a rundown of what they are about to be allowed to do (stuff that is not okay imo and this is the reason I am so pro-privacy):

- Mass Data Gathering: The intelligence agency over here (lets call it IA from now) can pretty much record everything send through the country.

- Extra Protection: If they want to conduct surveillance on journalists/lawyers, they have to go through extra channels first at least.

- Data/survaillance sharing: The IA is allowed to share their raw/filtered data with foreign intelligence agencies without limits. Also, they're allowed to conduct surveillance based on foreign requests.

- Secret DNA database: A secret DNA database will be created which can store the DNA profiles of any person who has commited any kind of crime. These profiles are allowed to be stored for a maximum of 30 years. This database is allowed to be shared with any foreign intelligence agency.

- Hacking: Unlimited power to hack any device deemed neccesary to hack in relation to crime. From computers to smartphones and so on. Also, it's allowed to use zero-days without reporting them to the vendor (we have seen what can go wrong with that through the ShadowBrokers scandal).

- Automatic Database Collection: They are allowed to directly tap into any database they see required (banks, healthcare, messaging services and so on). Practically this can lead to backdoors being build in because if you don't cooperate, you can go to prison. (mother of god I am not using anything closed source anymore if possible).

So yeah, this is pretty much the reason why I am so privacy consious. This country is fucked.

All data collection things should take an example of Debian:

Setting the (anonymous) data collection consent to 'no' by fucking default.

Well, here's the OS rant I promised. Also apologies for no blog posts the past few weeks, working on one but I want to have all the information correct and time isn't my best friend right now :/

Anyways, let's talk about operating systems. They serve a purpose which is the goal which the user has.
So, as everyone says (or, loads of people), every system is good for a purpose and you can't call the mainstream systems shit because they all have their use.

Last part is true (that they all have their use) but defining a good system is up to an individual. So, a system which I'd be able to call good, had at least the following 'features':
- it gives the user freedom. If someone just wants to use it for emailing and webbrowsing, fair enough. If someone wants to produce music on it, fair enough. If someone wants to rebuild the entire system to suit their needs, fair enough. If someone wants to check the source code to see what's actually running on their hardware, fair enough. It should be up to the user to decide what they want to/can do and not up to the maker of that system.
- it tries it's best to keep the security/privacy of its users protected. Meaning, by default, no calling home, no integrating users within mass surveillance programs and no unnecessary data collection.
- Open. Especially in an age of mass surveillance, it's very important that one has the option to check the underlying code for vulnerabilities/backdoors. Can everyone do that, nope. But that doesn't mean that the option shouldn't be there because it's also about transparency so you don't HAVE to trust a software vendor on their blue eyes.
- stability. A system should be stable enough for home users to use. For people who like to tweak around? Also, but tweaking *can* lead to instability and crashes, that's not the systems' responsibility.

Especially the security and privacy AND open parts are why I wouldn't ever voluntarily (if my job would depend on it, sure, I kinda need money to stay alive so I'll take that) use windows or macos. Sure, apple seems to care about user privacy way more than other vendors but as long as nobody can verify that through source code, no offense, I won't believe a thing they say about that because no one can technically verify it anyways.

Some people have told me that Linux is hard to use for new/(highly) a-technical people but looking at my own family and friends who adapted fast as hell and don't want to go back to windows now (and mac, for that matter), I highly doubt that. Sure, they'll have to learn something new. But that was also the case when they started to use any other system for the first time. Possibly try a different distro if one doesn't fit?

Problems - sometimes hard to solve on Linux, no doubt about that. But, at least its open. Meaning that someone can dive in as deep as possible/necessary to solve the problem. That's something which is very difficult with closed systems.

The best example in this case for me (don't remember how I did it by the way) was when I mounted a network drive at boot on windows and Linux (two systems using the same webDav drive). I changed the authentication and both systems weren't in for booting anymore. Hours of searching how to unfuck this on windows - I ended up reinstalling it because I just couldn't find a solution.
On linux, i found some article quite quickly telling to remove the entry for the webdav thingy from fstab. Booted into a root recovery shell, chrooted to the harddrive, removed the entry in fstab and rebooted. BAM. Everything worked again.

So yeah, that's my view on this, I guess ;P

With the wake of some rants shouting at Linuxers who express their opinion in a considered to be very not good way, I decided to make such a rant. Not to be annoying but because, although I get that fanboyism in that way isn't even good in MY opinion, I do think that one should be able to express their opinion.

But, If you'd like to express your opinion, I think you at least should do that with some good arguments. Not everyone might agree with those arguments but hey, that's the point of opinions sometimes :)

I don't hate windows/mac for being windows or mac. Nope.

I hate the systems for not giving the user freedom to do what they wish with the system but more importantly, for integrating their users in worlds biggest mass surveillance program AND on top of fucking that, not giving peoples the option to look at the source code aka at what's ACTUALLY going on in the system. Next to that, Windows 10's data collection is officially not legal in the netherlands so don't even try justifying their fucking data slurping.

Of course there's a chance that they don't contain any bad stuffs but since the Snowden revelations I don't trust those commercial companies anymore on their 'blue' eyes.

Yeah, I've ranted about this before, I know, felt like doing it again in combination with my reason above. I also know that I will probs receive hate for this but oh well, i'm used to that by now.

So yeah, windows and osx: go fuck yourself.

Alright, I just wanna start off by saying that I'm a huge privacy guy. I hate all kinds of data collection companies like Google/Facebook/Amazon and all that. Yet I'm forced to have a Google account for certain reasons.

But the thing that I want to say is that I often talk to people to express my concerns about privacy. And most people agree and say that "yeah sure that's some scary shit" but don't actually do much about it.

And the thing is. If you just browse through all of the options in your Google account. You can turn off and remove almost all data collection/ad serving and identification options entirely.

And yet purple complain about that Google sees what you buy and shit. Turn everything off. Get and adblocker and get noscript. The single best browser add-on out there. It's almost that easy to get at least acceptable amounts of Internet privacy.

Please, don't ever ignore the significance of in Internet privacy. And the potential issue of net neutrality. Don't be ignorant. Don't be the client.

After work I wanted to come home and work on a project. I have a few ideas for a few things I want to do, so I started a Trello board with the ideas to start mapping things out. But there were guys redoing the kitchen tile and it was noisy as fuck. So I packed up and headed to the library.

So I get all set up, and start plugging away. Currently working on a database design for a project that is a form for some user data collection for my dad, for an internal company thing. I am not contracted for this - I just know the details so I am using it as a learning exercise. Anyway...

I'm fucking about in a VM in MySQL and I feel someone behind me. So I turn and it's this girl looking over my shoulder. She asks what I am doing, and it turned into a 2 hour conversation. She is only a few years older than me (21) but she was brilliant. She (unintentionally) made me feel SO stupid with her scope of knowledge and giant brain. I learned quite a bit from talking to her and she offered to help me further, if I liked.

And she was really cute. We exchanged phone numbers...

I just implemented parallelism in one of my major data collection tools for work, and cut the tool's run duration by 90%. That seems like as good a time as any to go to sleep.

!rant
The change log from notepad++ update. The last paragraph is the cream!

" The issue of a hijacked DLL concerns scilexer.dll (needed by Notepad++) on a compromised PC, which is replaced by a modified scilexer.dll built by the CIA. When Notepad++ is launched, the modified scilexer.dll is loaded instead of the original one.
It doesn't mean that CIA is interested in your coding skill or in your sex message content typed in Notepad++, but rather it prevents raising any red flags while the DLL does data collection in the background.

It's not a vulnerability/security issue in Notepad++, but for remedying this issue, from this release (v7.3.3) forward, notepad++.exe checks the certificate validation in scilexer.dll before loading it. If the certificate is missing or invalid, then it just won't be loaded, and Notepad++ will fail to launch.

Checking the certificate of DLL makes it harder to hack. Note that once users’ PCs are compromised, the hackers can do anything on the PCs. This solution only prevents from Notepad++ loading a CIA homemade DLL. It doesn't prevent your original notepad++.exe from being replaced by modified notepad++.exe while the CIA is controlling your PC.

Just like knowing the lock is useless for people who are willing to go into my house, I still shut the door and lock it every morning when I leave home. We are in a f**king corrupted world, unfortunately. "

Manual Data Entry: Most boring job
This reminds me of one conversation with one of my faculty..
Faculty: Why not try some Machine Learning Project?
Me: Cool. Any ideas you have already thought
Faculty: Comes up with a really noble idea
Me: Awesome idea. But we need data
Faculty: Don't worry. I will get it. Just help me setup Hadoop (see the irony.. no data yet, and he wants big data setup)
Me: But we don't have data. Let's focus of data collection, Sir
Faculty: I will get it. Don't worry. Trust me.
( I did setup for him twice coz he formatted the system on which I did the setup first time)
After 6 months,
Me: (same question) Sir, Data??
Faculty: I got it.
Me: Great. Give me, I can start looking into it from today.
Faculty: Actually, it's in a register written manually in a different language (which even I can't understand) I will hire data entry guys to convert it into English digital contents.
Me: *facepalm*
Road to Manual data entry to Big Data
Dedicating this pencil to the individuals keeping the register up to date and Sir in hopes of converting it into big data..
Long way to go..

I had to open the desktop app to write this because I could never write a rant this long on the app.

This will be a well-informed rebuttal to the "arrays start at 1 in Lua" complaint. If you have ever said or thought that, I guarantee you will learn a lot from this rant and probably enjoy it quite a bit as well.

Just a tiny bit of background information on me: I have a very intimate understanding of Lua and its c API. I have used this language for years and love it dearly.

[START RANT]

"arrays start at 1 in Lua" is factually incorrect because Lua does not have arrays. From their documentation, section 11.1 ("Arrays"), "We implement arrays in Lua simply by indexing tables with integers."

From chapter 2 of the Lua docs, we know there are only 8 types of data in Lua: nil, boolean, number, string, userdata, function, thread, and table

The only unfamiliar thing here might be userdata. "A userdatum offers a raw memory area with no predefined operations in Lua" (section 26.1). Essentially, it's for the API to interact with Lua scripts. The point is, this isn't a fancy term for array.

The misinformation comes from the table type. Let's first explore, at a low level, what an array is. An array, in programming, is a collection of data items all in a line in memory (The OS may not actually put them in a line, but they act as if they are). In most syntaxes, you access an array element similar to:

array[index]

Let's look at c, so we have some solid reference. "array" would be the name of the array, but what it really does is keep track of the starting location in memory of the array. Memory in computers acts like a number. In a very basic sense, the first sector of your RAM is memory location (referred to as an address) 0. "array" would be, for example, address 543745. This is where your data starts. Arrays can only be made up of one type, this is so that each element in that array is EXACTLY the same size. So, this is how indexing an array works. If you know where your array starts, and you know how large each element is, you can find the 6th element by starting at the start of they array and adding 6 times the size of the data in that array.

Tables are incredibly different. The elements of a table are NOT in a line in memory; they're all over the place depending on when you created them (and a lot of other things). Therefore, an array-style index is useless, because you cannot apply the above formula. In the case of a table, you need to perform a lookup: search through all of the elements in the table to find the right one. In Lua, you can do:

a = {1, 5, 9};
a["hello_world"] = "whatever";

a is a table with the length of 4 (the 4th element is "hello_world" with value "whatever"), but a[4] is nil because even though there are 4 items in the table, it looks for something "named" 4, not the 4th element of the table.

This is the difference between indexing and lookups. But you may say,

"Algo! If I do this:

a = {"first", "second", "third"};
print(a[1]);

...then "first" appears in my console!"

Yes, that's correct, in terms of computer science. Lua, because it is a nice language, makes keys in tables optional by automatically giving them an integer value key. This starts at 1. Why? Lets look at that formula for arrays again:

Given array "arr", size of data type "sz", and index "i", find the desired element ("el"):

el = arr + (sz * i)

This NEEDS to start at 0 and not 1 because otherwise, "sz" would always be added to the start address of the array and the first element would ALWAYS be skipped. But in tables, this is not the case, because tables do not have a defined data type size, and this formula is never used. This is why actual arrays are incredibly performant no matter the size, and the larger a table gets, the slower it is.

That felt good to get off my chest. Yes, Lua could start the auto-key at 0, but that might confuse people into thinking tables are arrays... well, I guess there's no avoiding that either way.

Writing more infrastructure than product.

Look, my application requests and transforms data from a single external API endpoint, it's just one GET request...

But I made an intelligent response caching middleware to prevent downtime when the parent API goes down, I made mocks and tests for everything, the documentation is directly generated from the code and automatically hosted for every git branch using hooks, responses are translated into JSONschema notation which automatically generate integration tests on commit, and the transformations are set up as a modular collection of composable higher order lenses!

Boss: Please use less amphetamine.

WIP - Scarlet Witch project
AKA me collecting/cleaning/processing data while someone else will get the glory 😂

"Thank you for choosing Microsoft!"

No Microsoft, I really didn't choose you. This crappy hardware made you the inevitable, not a choice.

And like hell do I want to run your crappy shit OS. I tried to reset my PC, got all my programs removed (because that's obviously where the errors are, not the OS, right? Certified motherfuckers). Yet the shit still didn't get resolved even after a reset. Installing Windows freshly again, because "I chose this".

Give me a break, Microshaft. If it wasn't for your crappy OS, I would've gone to sleep hours ago. Yet me disabling your shitty telemetry brought this shit upon me, by disabling me to get Insider updates just because I added a registry key and disabled a service. Just how much are you going to force data collection out of your "nothing to hide, nothing to fear" users, Microsoft?

Honestly, at this point I think that Microsoft under Ballmer might've been better. Because while Linux was apparently cancer back then, at least this shitty data collection for "a free OS" wasn't yet a thing back then.

My mother still runs Vista, an OS that has since a few months ago reached EOL. Last time she visited me I recommended her to switch to Windows 7, because it looks the same but is better in terms of performance and is still supported. She refused, because it might damage her configurations. Granted, that's probably full of malware but at this point I'm glad she did.

Even Windows 7 has telemetry forcibly enabled at this point. Vista may be unsupported, but at least it didn't fall victim to the current status quo - data mining on every Microshaft OS that's still supported.

Microsoft may have been shady ever since they pursued manufacturers into defaulting to their OS, and GPU manufacturers will probably also have been lobbied into supporting Windows exclusively. But this data mining shit? Not even the Ballmer era was as horrible as this. My mother may not realize it, but she unknowingly avoided it.

Sorry for being late, stuffs came inbetween!

I have done a few privacy rants/posts before but why not another one. @tahnik did one a few days ago so I thought I'd do a new one myself based on his rant.

So, online privacy. Some people say it's entirely dead, that's bullshit. It's up to an individual, though, how far they want to go as for protecting it.

I personally want to retain as much control over my data as possible (this seems to be a weird thing these days for unknown reasons...). That's why I spend quite some time/effort to take precautions, read myself into how to protect my data more and so on.

'Everyone should have the choice of what services they use' - fully agreed, no doubt about that.

I just find one thing problematic. Some services/companies handle data in a way or have certain business models which takes the control which some people want/have over their data away when you communicate with someone using that service.

Some people (like me) don't want anything to do with google but even when I want to email my best fucking friend, I lose the control over that email data since he uses gmail.

So, when someone chooses to use gmail and I *HAVE* to email them, my choice is gone.

TO BE VERY CLEAR: I'm not blaming that on the users, I'm blaming that on the company/service.

Then for example, google analytics. It's a very good/powerful when you're solely looking at its functions.
I just don't want to be part of their data collection as I don't want to get any data into the google engine.

There's a solution for that: installing an addon in order to opt out.

I'm sorry, WHAT?! --> I <-- have to install an addon in order to opt out of something that is happening on my own motherfucking computer?! What the actual fuck, I don't call that a fucking solution. I'll use Privacy Badger + hosts files to block that instead.

Google vs 'privacy' friendly search engines - I don't trust DDG completely because their backend is closed/not available to the public but I'd rather use them then a search engine which is known to be integrated into PRISM/other surveillance engines by default.

I don't mind the existence of certain services, as long as they don't integrated you with data hungry companies/mass surveillance without you even using their services.

Now lets see how fast the comment section explodes!

The nightmare continues.

Currently dealing with a code review from a “principal” dev (one step above senior), who is unironically called a “legendary dev” by some coworkers. It’s painfully obvious he didn’t read the code, and just started complaining and nitpicking.

It’s full of requests to do things that make absolutely no sense, and would make the code an unmaintainable mess.
• Ex: moving the logic and data collection from the module’s many callers into the module instead of just passing in the data.
• Ex: hiding api endpoint declarations by placing them in the module itself, and using magic instance variables to pass data to it. Basically: using global functions and variables instead of explicit declarations and calls.
• Ex: moving the logic to determine which api endpoint to use, for all callers, into the view.

More comments about methods being “too complex” (barely holds water) right next to comments saying “why are these separate? merge them together!”

Incredulously asking how many times I’m checking permissions and how ridiculous it all is. (The answer? Twice.)

Conflating my “permissions” param and method names with a supposedly forthcoming permissions system overhaul, and saying I shouldn’t use permissions because my code will all have to get rewritten. Even if that were true, and it’s likely not, the ticket still needs to use the current permissions. I can’t just ignore them because they might be rewritten someday.

Requests to revert some code cleanup because the reviewer thought the previous heavily-nested and uncommented versions (with code duplication) were easier to read. Unsurprisingly, he wrote them.

On the same ticket, my boss wants me to remove all styling and clientside validation, debouncing, and error messages from a form. Says “success” and “connection failed” messages are good enough. The form in question sends SMS and email using arbitrary user input for addresses. He also says it shouldn’t be denounced on the server, and doesn’t want me to bother checking permissions. Hello, spam!

Related: the legendary dev reviewer says he can’t think of a reason why we would want to disable the feature for consumers, so I should remove the consumer feature flag.

You can’t make this stuff up.

Chill guys, data collection is just archaeology in advance.

Paranoid Developers - It's a long one
Backstory: I was a freelance web developer when I managed to land a place on a cyber security program with who I consider to be the world leaders in the field (details deliberately withheld; who's paranoid now?). Other than the basic security practices of web dev, my experience with Cyber was limited to the OU introduction course, so I was wholly unprepared for the level of, occasionally hysterical, paranoia that my fellow cohort seemed to perpetually live in. The following is a collection of stories from several of these people, because if I only wrote about one they would accuse me of providing too much data allowing an attacker to aggregate and steal their identity. They do use devrant so if you're reading this, know that I love you and that something is wrong with you.

That time when...

He wrote a social media network with end-to-end encryption before it was cool.

He wrote custom 64kb encryption for his academic HDD.

He removed the 3 HDD from his desktop and stored them in a safe, whenever he left the house.

He set up a pfsense virtualbox with a firewall policy to block the port the student monitoring software used (effectively rendering it useless and definitely in breach of the IT policy).

He used only hashes of passwords as passwords (which isn't actually good).

He kept a drill on the desk ready to destroy his HDD at a moments notice.

He started developing a device to drill through his HDD when he pushed a button. May or may not have finished it.

He set up a new email account for each individual online service.

He hosted a website from his own home server so he didn't have to host the files elsewhere (which is just awful for home network security).

He unplugged the home router and began scanning his devices and manually searching through the process list when his music stopped playing on the laptop several times (turns out he had a wobbly spacebar and the shaking washing machine provided enough jittering for a button press).

He brought his own privacy screen to work (remember, this is a security place, with like background checks and all sorts).

He gave his C programming coursework (a simple messaging program) 2048 bit encryption, which was not required.

He wrote a custom encryption for his other C programming coursework as well as writing out the enigma encryption because there was no library, again not required.

He bought a burner phone to visit the capital city.

He bought a burner phone whenever he left his hometown come to think of it.

He bought a smartphone online, wiped it and installed new firmware (it was Chinese; I'm not saying anything about the Chinese, you're the one thinking it).

He bought a smartphone and installed Kali Linux NetHunter so he could test WiFi networks he connected to before using them on his personal device.

(You might be noticing it's all he's. Maybe it is, maybe it isn't).

He ate a sim card.

He brought a balaclava to pentesting training (it was pretty meme).

He printed out his source code as a manual read-only method.

He made a rule on his academic email to block incoming mail from the academic body (to be fair this is a good spam policy).

He withdraws money from a different cashpoint everytime to avoid patterns in his behaviour (the irony).

He reported someone for hacking the centre's network when they built their own website for practice using XAMMP.

I'm going to stop there. I could tell you so many more stories about these guys, some about them being paranoid and some about the stupid antics Cyber Security and Information Assurance students get up to. Well done for making it this far. Hope you enjoyed it.

Somewhat unpopular opinion time. I don’t 100% hate Facebook.

I do not support their data collection, but my biggest problem with FB is the users. My connections are inherently idiots because I’m in high school with people who are way too narcissistic and full of drama. I left FB because of the toxicity, and the data protection as a result was a bonus.

I support the original mission of Facebook, to connect people. I just wish it was still about that.

The day I discovered Schrödinger's lesser known paradox of simultaneously being fired and not fired.

This isn't really much of a dev story, but I figured I'd share it anyway.

About two minutes into signing into all my stuff, I suddenly was kicked out of everything. I tried logging in a few more times, and then suddenly started getting the error, "Your account has been disabled for security reasons." I couldn't sign into chat, and co-workers confirmed that I was missing from the company directory. My manager didn't come in for another two hours, and we couldn't get anyone else to answer what the hell was going on. So I was kinda panicking.

Eventually, we found out from one of our coordinators that someone else with the same name as me was leaving the company, and they had deactivated the wrong person.

It ended up getting a lot better. They told me that it could take up to 48 hours to restore my access (it took longer), so I found stuff to do so I could maintain my paycheck. One of those things was assisting someone with data collection and processing, where I eventually said, "Dude, I could totally automate this," and now that's what I'm getting paid to do.

I just got denied access to a metro station because I use the old BuzzyPazz over the new Mobib card, that digitally authorizes you and knows where you are, and when. I refuse to use it over privacy concerns and because my BuzzyPazz is indefinitely valid. LONG LIVE FUCKING TECHNOLOGY, DESIGNED NOT FOR LEGALLY GETTING ON A FUCKING METRO, BUT FUCKING DATA COLLECTION!!!!

Installed Miami Street earlier today.. some random free Shaftgame.
Late at night now, I figured "let's try this out".

> Logging in...
*crashes*

*goes to the settings for this crap game*
> *crashes*

Fucking worthless piece of Microshit.. yet another data collection hook that REQUIRES your shit to log in just to fucking work? Fucking Shaftfuckers, 5GB of internet traffic I spent on this?! Just to see it be a worthless data-hungry paperweight?!! Luckily my residential connection is unmetered and has some decent speeds.. but still, FUCK YOU MICROSHAFT!!!

Coincidentally, keyboard input completely broke when I wanted to do a minor edit to the drafted rant. Microshit can't even design a decent keyboard driver anymore, huh.. I DIDN'T WANT TO HAVE TO REWRITE THIS SHIT FOR A SECOND TIME, FUCKING REDMOND MICROCUNTSUCKERS!!!!

I am much too tired to go into details, probably because I left the office at 11:15pm, but I finally finished a feature. It doesn't even sound like a particularly large or complicated feature. It sounds like a simple, 1-2 day feature until you look at it closely.

It took me an entire fucking week. and all the while I was coaching a junior dev who had just picked up Rails and was building something very similar.

It's the model, controller, and UI for creating a parent object along with 0-n child objects, with default children suggestions, a fancy ui including the ability to dynamically add/remove children via buttons. and have the entire happy family save nicely and atomically on the backend. Plus a detailed-but-simple listing for non-technicals including some absolutely nontrivial css acrobatics.

After getting about 90% of everything built and working and beautiful, I learned that Rails does quite a bit of this for you, through `accepts_nested_params_for :collection`. But that requires very specific form input namespacing, and building that out correctly is flipping difficult. It's not like I could find good examples anywhere, either. I looked for hours. I finally found a rails tutorial vide linked from a comment on a SO answer from five years ago, and mashed its oversimplified and dated examples with the newer documentation, and worked around the issues that of course arose from that disasterous paring.

like.
I needed to store a template of the child object markup somewhere, yeah? The video had me trying to store all of the markup in a `data-fields=" "` attrib. wth? I tried storing it as a string and injecting it into javascript, but that didn't work either. parsing errors! yay! good job, you two.

So I ended up storing the markup (rendered from a rails partial) in an html comment of all things, and pulling the markup out of the comment and gsubbing its IDs on document load. This has the annoying effect of preventing me from using html comments in that partial (not that i really use them anyway, but.)

Just.
Every step of the way on building this was another mountain climb.
* singular vs plural naming and routing, and named routes. and dealing with issues arising from existing incorrect pluralization.
* reverse polymorphic relation (child -> x parent)
* The testing suite is incompatible with the new rails6. There is no fix. None. I checked. Nope. Not happening.
* Rails6 randomly and constantly crashes and/or caches random things (including arbitrary code changes) in development mode (and only development mode) when working with multiple databases.
* nested form builders
* styling a fucking checkbox
* Making that checkbox (rather, its label and container div) into a sexy animated slider
* passing data and locals to and between partials
* misleading documentation
* building the partials to be self-contained and reusable
* coercing form builders into namespacing nested html inputs the way Rails expects
* input namespacing redux, now with nested form builders too!
* Figuring out how to generate markup for an empty child when I'm no longer rendering the children myself
* Figuring out where the fuck to put the blank child template markup so it's accessible, has the right namespacing, and is not submitted with everything else
* Figuring out how the fuck to read an html comment with JS
* nested strong params
* nested strong params
* nested fucking strong params
* caching parsed children's data on parent when the whole thing is bloody atomic.
* Converting datetimes from/to milliseconds on save/load
* CSS and bootstrap collisions
* CSS and bootstrap stupidity
* Reinventing the entire multi-child / nested params / atomic creating/updating/deleting feature on my own before discovering Rails can do that for you.

Just.
I am so glad it's working.
I don't even feel relieved. I just feel exhausted.

But it's done.
finally.

and it's done well. It's all self-contained and reusable, it's easy to read, has separate styling and reusable partials, etc. It's a two line copy/paste drop-in for any other model that needs it. Two lines and it just works, and even tells you if you screwed up.

I'm incredibly proud of everything that went into this.
But mostly I'm just incredibly tired.

Time for some well-deserved sleep.

Hell no?

This rant has been one that I've been wanting to rant about for a while now. Me being drunk as fuck right now (mind, stay awake!) doesn't really help, but meh.
At least Wanblows was able to install its "features" properly... Except it wasn't, being the featureful ShitOS it is.

I want to rant about privacy. Not about "nothing to hide, nothing to fear". That's been ranted about plenty by the MIcroshaft-loving folks as well as the privacy-aware opposition. Rather, I'd like to rant about the privacy-concious.

I am a privacy-concious-person, with his current status quo being that he doesn't yet know a privacy-concious solution to every data-intrusive "common solution" out there. So I tend to value privacy next to De Lijn while sharing location data to Google with Google Maps. Point is, I do not know privacy-concious solutions to everything out there yet. So I use the convenient over the privacy-aware.
(after review while drunk I was unable to make sense of this)

In the privacy-aware circles I tend to see that it seems to be black and white. You share your data with Google, yet you oppose data collection by local institutes? WRONG!!! YOU MUST BE A TINFOIL HAT!!!
No, seriously I don't want to share my data with Google. Just that they're the only realtime navigation platform with decent UI out there that I know of right now.

Privacy isn't all black and white here. I block any intrusion that I'm able to, anything else I abide to, while awaiting a good alternative that does respect my privacy which I would gladly use instead. That does not imply that "I have nothing to hide". I do, and I have a lot to hide.

So that makes up the black and white nature of privacy, which is a fallacy. Another one is the whole idea of "I have nothing to hide" to begin with.

If you have nothing to hide, would you be comfortable with sharing your location data (IP address, habits, common trends, etc) with me? To share your information with me, to have your contacts share your info with me, without your consent? Of course you wouldn't. But that's what's happening right now.

There was a computer programming teacher in my 1st semester who taught C. He used to have this conventional way of teaching C like other Engineering subjects which was going to more theories before writing actual codes.
These are the conversations with him.
(First day, a guy asks him some questions.)
Guy: Sir, why do we need to learn C? There are other languages used extensively for other tasks like python,etc. Why bother with this boring C?
Teacher: C is used to learn other languages. After learning C, you can easily learn other languages.
Guy: Sir, where is C's application? Where is it used?
Teacher: It is used in academics to lay foundation for students to learn other languages which are used to build softwares.
(Fucking Hilarious)

(A month after he was asking some questions to students.)
Teacher: What is an array? What is an array-name?
Student 1: Array, is this collection of data that can be stored in a single type.
Teacher: Then what is an array-name?
Student 1: I don't know.
Teacher: (angrily) Array-name is a definition itself.
(We were supposed to answer that. It was a standard definition.)

Ok, a few days ago I recommended contabo.com as a good and cheap VPS, because our company uses it and it is blazingly fast. BUT! I am definitely NOT doing this!

"False or fraudulent orders are commonplace on the Internet. We have to take measures to prevent such false or fraudulent orders. Our system has identified your order as a possible false order. We must now prove your order as a valid order, otherwise we cannot provide you with the services you bought. We need your help with this. Please send us a copy of your passport or national identity card or something similar which corresponds to the data you have provided to us. Alternatively, you can send us your telephone or electricity bill if it contains your address. The address must match the address you provided on our homepage. It is sufficient if you simply take a photo of the document. The only purpose of this is to prove your order as authentic. This is why it does not matter which of the mentioned documents you send us and it does not matter if you make a scan or take a photo. A scammer could not provide any of these documents, this means one document is already enough to prove that your order is authentic and valid."

WARNING: There is a dangerous malware out in the wild, and chances are, you have it installed on your computer.

It's called Windows Update, and it is marketed as a software that "delivers security patches to your PC". Wrong. What it actually does is hard-reboot your computer at randomly picked time intervals without asking for your consent, or even showing any type of warning, basically deleting all unsaved progress that you've made in your programs or games. It also deletes/undoes all registry tweaks that you might have made (e.g. to the context menu), it deletes your nvidia display configurations, uninstalls any custom themes that you might have installed, possibly even downloads another malware disguised as "Microsoft Edge" and shoves it in your face on next boot without giving a possibility to close it. Oh and it might also make your computer unbootable so you have to go to the advanced recovery settings to fix it manually.

Yes, everything I just mentioned above happened to me about an hour ago. This LITERALLY classifies the software as a malware (Google: "software that is specifically designed to disrupt, damage, or gain unauthorized access to a computer system"). If we throw in all the data collection that happens without your consent, Microsoft actually manages to check not only one, but all three boxes in the "malware" definition.

Pleas, stop using microshit, and switch to linux as soon as possible if you can.

I had spent the last year working on a online store power by woocommerce with over 100k products from various suppliers. This online store utilized a custom API that would take the various formats that suppliers offer their inventory in and made them consistent. Now everything was going swimmingly initially, but then I began adding more and more products using a plug-in called WP all import. I reached around 100k products and the site would take up to an entire minute to load sometimes timing out. I got desperate so I installed several caching plugins, but to no avail this did not help me. The site was originally only supposed to take three to four months but ended up taking an entire year. Then, just yesterday I found out what went wrong and why this woocommerce website with all of these optimizations was still taking anywhere from 60 to 90 seconds to load, or just timing out entirely. I had initially thought that I needed a beefier server so I moved it to a high CPU digitalocean VM. While this did help a little bit, the site was still very slow and now I had very high CPU usage RAM usage and high disk IO. I was seriously stumped the Apache process was using a high amount of CPU and IO along with MYSQL as well. It wasn't until I started digging deeper into the database that I actually found out what the issue was. As I was loading the site I would run 'show process list' in the SQL terminal, I began to notice a very significant load time for one of the tables, so I went to go and check it out. What I did was I ran a select all query on that particular table just to see how full it was and SQL returned a error saying that I had exceeded the maximum packet size. So I was like okay what the fuck...
So I exited my SQL and re-entered it this time with a higher packet size. I ran a query that would count how many rows were in this particular table and the number came out to being in the millions. I was surprised, and what's worse is that this table belong to a plugin that I had attempted to use early in the development process to cache the site. The plugin was deactivated but apparently it had left PHP files within the wp content directory outside of the actual plugin directory, so it's still executing scripts even though the plugin itself was disabled. Basically every time I would change anything on the site, it would recache the whole thing, and it didn't delete any old records. So 100k+ products caching on saves with no garbage collection... You do the math, it's gonna be a heavy ass database. Not only that but it was serialized data, so when it did pull this metric shit ton of spaghetti from the database, PHP then had to deserialize it. Hence the high ass CPU load. I had caching enabled on the MySQL end of things so that ate the ram. I was really desperate to get this thing running.
Honest to God the main reason why this website took so long was because the load times made it miserable to work on. I just thought that the hardware that I had the site on was inadequate. I had initially started the development on a small Linux VM which apparently wasn't enough, which is why I moved it to digitalocean which also seemed to not be enough, so from there I moved to a dedicated server which still didn't seem to be enough. I was probably a few more 60-second wait times or timeouts from recommending a server cluster to my client who I know would not be willing to purchase it. The client who I promised this site to have completed in 3 months and has waited a year. Seriously, I would tell people the struggles that I would go through with this particular site and they would just tell me to just drop the site; just take the money, just take the loss. I refused to, this was really the only thing that was kicking my ass. I present myself as this high-and-mighty developer like I'm just really good at what I do but then I have this WordPress site that's just beating the shit out of me for a year. It was a very big learning experience and it was also very humbling as well, it made me realize that I really don't know as much as I think I might. It was evidence that there is still so much more to learn out there, I did learn a lot from that experience especially about optimizing websites the different types of methods to do that particular lonely on the server side and I'll be able to utilize this knowledge in the future.
I guess the moral of the story is, never really give up. Ultimately things might get so bad that you're running on hopes and dreams. Those experiences are generally the most humbling. Now I can finally present the site that I am basically a year late on to the client who will be so happy that I did not give up on the project entirely. I'll have experienced this feeling of pure euphoria, and help the small business significantly grow their revenue. Helping others is very fulfilling for me, even at my own expense.

Anyways, gonna stop ranting. Running out of characters. If you're still here... Ty for reading :')

Using mongodb for one product

A colleague as experimenting with elastic search (I think it was).

It installed a proxy around the collection to get all events for the external search storage.

Worked well, but it was just a test so once done we removed it

But thats where it got scary.

When we removed the proxy through the search dashboard it dropped the underlying collection of live data!!!

A collection it did not create.

Hows that for bad UI.

Always experiment on a separate db server.

Oh sweet mother of god.
What is the NBN Connection i have attached to my router?

It goes "give me all your data... NOW"
I'm not used to such speed in my house, excuse me while I watch my porn collection in 4K!

!rant
I am on vacation from my full time job this week. I wanted to use this week to write a PoC for a potential customer of my side business. really interesting project for me.

potential customer is a window and door manufacturer and needs an application to manage their racks.

their ERP system already has a simple rack management but it is only useable in house.

they want the drivers to be able to scan racks they deliver to a customer with a native app and they want to have a webapp for the customers to see racks that are assigned to them as well as reporting a rack ready for collection. And that all needs to be in sync with their local ERP system.

as i am a .net guy i decided to go with the abp framework (because it got recommended to me) and xamarin for the native app part (because i have experience in this).

i have now spent 4 days implementing this and it has been so rewarding. the framework is so powerful and it's template saved me endless hours.

i even wrote a very basic connector service which synchronizes data between my app and the clients ERP system. Just one way until now because of time issue, but i learned to scaffold an ef core with db first. It is noticable that the ERP system is 2-tiered - meaning the clients directly talk to the db.

Tomorrow i will implement the xamarin client.

4 days just coding what i want to. choosi g my own velocity and making my own priorities without any interruptions or discussions and a bunch of new things to learn.

Probably wasted half a day because of stupidy (implemented some bugs) but fixing and learning is part of the journey and i lime that part, too.

i am so relaxed right now 😁 just wanted to share this without a real reason :P

Thoughts prior to feedback meeting, about how it's gonna go.

---------------------------------------------------------
Scenario one:
Supervisor: The shit is this? You call this a research work? Get the fuck out of here! You're fired and even your unborn kids are banned from coming into this institute ever again!

Me: *walks out sobbing* (dunno how one can walk out of a zoom meeting, but this is imaginary so who's counting?)

Scenario two:
S: Umm, good work. I just don't think it's presentable. Maybe come back in like a few weeks when you actually polish this into a "real scientific work".

Me: *sobs after meeting. Starts preping for seppuku cuz no idea where I'm headed with this work any further*

Scenario three:
S: nah man. This is no good. Let's start from the bottom. Like, start data collection from the beginning or something.

Me: *sobs and commits seppuku on the meeting.* (I just have a pen tho. Hope it has the same effect as a sword)

---------------------------------------------------------

There are other scenarios, but they all end up in me sobbing and/or committing seppuku in/after the meeting so yeah the drama is running high right now.

It was my thesis defense and I've made a pretty complex algorithm of sorting out data to their respective tables. One panelist told me that it is not an algorithm but just a collection of for loops and if statements. An algorithm should contain mathematical equations he said. I facepalmed during the course of interrogation.

Saw a reddit thread earlier asking about the most unsettling shit that people have found out Google has on them by downloading their data. I saw a bunch of comments about people finding voice recordings that Google had taken. After reading these, I was wondering what I could find from downloading my data. Decided to download my data, and on the page for it I saw that apparently I had disabled location history, audio activity, and device information.

Knowing companies like Google, I wouldn't be surprised if they didn't stop recording that stuff, just that they're not providing it to me. There were zero voice recordings, but there was location history up until about the beginning of 2017.

Another thing they have is all the pictures from all of my hangouts chats. Apparently there had a good amount of older pictures of myself in there. Going back to probably 2-3 years ago, when I had my emo hair. Just a bit of a throwback. One picture I saw was from last January, when my hair was reaching my chest. Made me really miss my hair.

Other than that, nothing that interesting. Just something I thought I'd share.

One of the reasons their extensive data collection is becoming annoying for not just the privacy freaks, but everyone is when it starts suggesting me dumb stuff like no thanks I learnt coding by spending two years on messed up projects and wrecked half a dozen servers before I had an idea on how to host a website go feed this zuckershit to some other n00🅱

So yeah XML is still not solved in year 2018. Or so did I realize the last days.
I use jackson to serialize generic data to JSON.

Now I also want to provide serialization to XML. Easy right? Jackson also provides XML serialization facitlity similar to JAXB.

Works out of the box (more or less). Wait what? *rubbing eyes*

<User>
<pk>234235</pk>
<groups typeCode="usergroup">
<pk>6356679041773291286</pk>
</groups>
<groups typeCode="usergroup">
<pk>1095682275514732543</pk>
</groups>
</User>

Why is my groups property (java.util.Set) rendered as two separate elements? Who the fuck every though this is the way to go?

So OK *reading the docs* there is a way to create a collection wrapper. That must be it, I thought ...

<User typeCode="user">
<pk>2540591810712846915</pk>
<groups>
<groups typeCode="usergroup">
<pk>6356679041773291286</pk>
</groups>
<groups typeCode="usergroup">
<pk>1095682275514732543</pk>
</groups>
</groups>
</User>

What the fuck is this now? This is still not right!!!

I know XML offers a lot of flexibility on how to represent your data. But this is just wrong ...

The only logical way to display that data is:
<User typeCode="user">
<pk>2540591810712846915</pk>
<groups>
<groupsEntry typeCode="usergroup">
<pk>6356679041773291286</pk>
</groupsEntry>
<groupsEntry typeCode="usergroup">
<pk>1095682275514732543</pk>
</groupsEntry>
</groups>
</User>

It would be better if the individual entries would be just called "group" but I guess implementing such a logic would be pretty hard (finding a singular of an arbitrary word?).

So yeah theres a way for that * implementing a custom collection serializer* ... wait is that really the way to go? I mean common, am I the only one who just whants this fucking shit just work as expected, with the least amount of suprise?
Why do I have to customize that ...

So ok it renders fine now ... *writes test for it+
FUCK FUCK FUCK. why can't jackson not deserialize it properly anymore? The two groups are just not being picked up anymore ...

SO WHY, WHY WHY are you guys over at jackson, JAXB and the like not able to implement that in the right manner. AND NOT THERE IS ONLY ONE RIGHT WAY TO DO IT!

*looks at an apple PLIST file* *scratches head* OK, gues I'll stick to the jackson defaults, at least it's not as broken as the fucking apple XML:

<plist version="1.0">
<dict>
<key>PayloadOrganization</key>
<string>Example Inc.</string>
<key>PayloadDisplayName</key>
<string>Profile Service</string>
<key>PayloadVersion</key>
<integer>1</integer>
</dict>
</plist

I really wonder who at apple has this briliant idea ...

Proudest bug squash? Probably the time I fixed a few bugs by accident when I was just trying to clean up an ex-coworker's messy code.

So I used to work with a guy who was not a very good programmer. It's hard to explain exactly why other than to say that he never really grew out of the college mindset. He never really learned the importance of critical thinking and problem-solving. He did everything "by the book" to a point where if he ran into an issue that had no textbook solution, he would spin his wheels for weeks while constantly lying to us about his progress until one of us would finally notice and take the problem off his plate. His code was technically functional, but still very bad.

Quick Background: Our team is responsible for deploying and maintaining cloud resources in AWS and Azure. We do this with Terraform, a domain-specific language that lets us define all our infrastructure as code and automate everything.

After he left, I took on the work to modify some of the Terraform code he'd written. In the process, I discovered what I like to call "The Übervariable", a map of at least 80 items, many of them completely unrelated to each other, which were all referenced exactly once in his code and never modified. Basically it was a dynamic collection variable holding 80+ constants. Some of these constants were only used in mathematical expressions with multiple other constants from the same data structure, resulting in a new value that would also be a constant. Some of the constants were identical values that could never possibly differ, but were still stored as separate values in the map.

After I made the modification I was supposed to make, I decided I was so bothered by his shitty code that I would spend some extra time fixing and optimizing it. The end result: one week of work, 800 lines of code deleted, 30 lines added, and a massive increase in efficiency. I deleted the Übervariable and hardcoded most of the values it contained since there was no possible reason for any of them to change in the future. In the process, I accidentally fixed three bugs that had been printing ominous-sounding warnings to the console whenever the code was run.

I have a lot of stories about this guy. I should post some more of them eventually.

We had 1 Android app to be developed for charity org for data collection for ground water level increase competition among villages.

Initial scope was very small & feasible. Around 10 forms with 3-4 fields in each to be developed in 2 months (1 for dev, 1 for testing). There was a prod version which had similar forms with no validations etc.

We had received prod source, which was total junk. No KT was given.

In existing source, spelling mistakes were there in the era of spell/grammar checking tools.

There were rural names of classes, variables in regional language in English letters & that regional language is somewhat known to some developers but even they don't know those rural names' meanings. This costed us at great length in visualizing data flow between entities. Even Google translate wasn't reliable for this language due to low Internet penetration in that language region.

OOP wasn't followed, so at 10 places exact same code exists. If error or bug needed to be fixed it had to be fixed at all those 10 places.

No foreign key relationships was there in database while actually there were logical relations among different entites.

No created, updated timestamps in records at app side to have audit trail.

Small part of that existing source was quite good with Fragments, MVP etc. while other part was ancient Activities with business logic.

We have to support Android 4.0 to 9.0 of many screen sizes & resolutions without any target devices issued to us by the client.

Then Corona lockdown happened & during that suddenly client side professionals became over efficient.

Client started adding requirements like very complex validation which has inter-entity dependencies. Then they started filing bugs from prod version on us.

Let's come to the developers' expertise,
2 developers with 8+ years of experience & they're not knowing how to resolve conflicts in git merge which were created by them only due to not following git best practice for coding like only appending new implementation in existing classes for easy auto merge etc.

They are thinking like handling click events is called development.

They don't want to think about OOP, well structured code. They don't want to re-use code mostly & when they copy paste, they think it's called re-use.

They wanted to follow old school Java development in memory scarce Android app life cycle in end user phone. They don't understand memory leaks, even though it's pin pointed by memory leak detection tools (Leak canary etc.).

Now 3.5 months are over, that competition was called off for this year due to Corona & development is still ongoing.

We are nowhere close to completion even for initial internal QA round.

On top of this, nothing is billable so it's like financial suicide.

Remember whatever said here is only 10% of what is faced.

- An Engineering lead in a half billion dollar company.

Started a job as a full stack developer. My first task was shocking! Do these small edits on this backend script that collects stuff from one database and edits the entries in another... piece of cake so far!

Here is the project on the TFS...
HOLD ON! IS THIS VISUAL BASIC?!!
I came here to do .Net framework development and .Net Standard... I wasn’t told that there will be VB, I have never used vb.net before.

Now... that I’m going to maintain this script in the future, I decided to rewrite it in C#, few things I learned on my journey of doing this:
1- There is an access modifier in VB called Friend
2- There is a data structure/type called Collection, it’s a value,key pair! Not key value pair... Value first, then key!!
3- Do you know how null is null everywhere?!! In VB they call it Nothing! Yes, as in...
if(myVar == nothing)
{
//stuff
}
Asking the guy responsible for that choice... he thinks VB is easier to read than C#
I DONT WANT YOU TO READ IT, I WANT IT TO MAKE SENSE AND WORK WITH THE REST OF THE C# CODE WE HAVE!!

My gods, fuck WordPress in the backend!
Why did my company decide on writing a data collection api with many dynamic pages on top of WordPress. It's a blog, not a wonder tool you dumbwits.
Never have I had more fuck this moments per hour..

Today we start working on a app that learns biometric data from the user for extra security, so if some one else uses my account... The system would know and shuts the bad user out. Although we use an api for the biometric data collection, it's still epic! 😀😀😀

Only bad thing is that the deadline is next week

The application that was working fine on local host failed to do one task which is to read users data from excel and save it on mongo db, in some collection document's embedded document. My last night and today morning was tensed due to this. So I decided to re deploy the same application on another instance that I have. Worked completely fine
😐😐😐

I've had a Xiaomi Mi 8 for a few months now. Although I'm impressed by what I got for the amount I paid (a phone that cost about $250 for 6GB RAM, Snapdragon 845, Android 9 and premium build quality is quite a steal), it definitely comes with a consequence.

MIUI (specifically MIUI 11) is godawful. It is single-handedly the worst Android ROM I've ever used since my shitty Android 2.2 phone back around 2010. If you're gonna buy a Xiaomi phone, plan to install Lineage OS on it (but even that's a pain which I'll explain why later).

- Navigation buttons don't hide while watching a video.

Why? God only knows. The ONLY way to bypass without root this is to use its garbage fullscreen mode with gestures, which is annoying as all hell.

- 2 app info pages?

Yeah, the first one you can access just by going to its disaster of a settings app, apps, manage apps and tap on any one.

The 2nd one you can access through the app info button in any 3rd party launcher. Try this: Download Nova launcher, go to the app drawer, hold on any app and tap "app info", and you'll see the 2nd one.

Basically, instead of modifying Android's FOSS source code, they made a shitty overlay. These people are really ahead of their time.

- Can only set lock screen wallpapers using the stock Gallery app

It's not that big an issue, until it is, when whatever wallpaper app you're using only allows you to set the wallpaper and not download them. I think this is both a fuckup on Xiaomi and (insert wallpaper app name here), but why Xiaomi can't include this basic essential feature that every other Android ROM ever made has is beyond me.

- Theming on MIUI 11 is broken

Why do they even bother having a section to customize the boot animation and status bar when there's not one goddamn theme that supports it? At this point you're only changing the wallpaper and icon pack which you can do on any Android phone ever. Why even bother?

They really, REALLY want to be Apple.

Just look at their phones. They're well designed and got good specs, but they don't even care anymore about being original. The notch and lack of a headphone jack aren't features, they're tremendous fuckups by the dead rotting horse known as Apple that died when Steve Jobs did.

Xiaomi tries to build a walled garden around an inherently customizable OS, and the end result is a warzone of an Android ROM that begs for mercy from its creator. Launchers integrate horribly (Does any power user actually use anything that isn't Nova or Microsoft launcher?), 3rd party themes and customization apps need workarounds, some apps don't work at all. People buy from Xiaomi to get a high end budget Android phone at the price of some ads and data collection, not a shitter iOS wannabe.

They really, REALLY want you to have a sim card

If you don't have a sim card and you're using your phone for dev stuff, you're a 2nd class citizen to Xiaomi. Without one, you can't:

- Install adb through adb
- Write to secure settings
- Unlock your bootloader and get away from this trash Android ROM

What's the point? Are they gonna shadow ban you? Does anyone contact them to unlock their bootloader saying "yeah I wanna use a custom rom to pirate lizard porn and buy drugs"? They made this 1000000000x harder than it needs to be for no reason whatsoever. Oh yeah and you gotta wait like a week or something for them to unlock it. How they fucked up this bad is beyond me.

So yeah. Xiaomi. Great phones, atrocious OS.

PouchDB.

It promised full-blown CRDT functionality. So I decided to adopt it.

Disappointment number one: you have to use CouchDB, so your data model is under strict regulations now. Okay.

Disappointment number two: absolutely messed up hack required to restrict users from accessing other users’ data, otherwise you have to store all the user data in single collection. Not the most performant solution.

Disappointment number three: pagination is utter mess. Server-side timestamps are utter mess. ANY server-side logic is utter mess.

Just to set it to work, you need PouchDB itself, websocket adapter (otherwise only three simultaneous syncs), auth adapter (doesn’t work via sockets), which came out fucking large pile of bullshit at the frontend.

Disappointment number four, the final one: auth somehow works but it doesn’t set cookie. I don’t know how to get access.

GitHub user named Wohali, number one CouchDB specialist over there, doesn’t know that either.

It also doesn’t work at Incognito mode, doesn’t work at Firefox at all.

So, if you want to use PouchDB, bear that in mind:
1. CouchDB only
2. No server-side logic
3. Authorization is a mess
4. Error logs are mess too: “ERROR 83929629 broken pipe” means “out of disk space” in Erlang, the CouchDB language.
5. No hosting solutions. No backup solutions, no infrastructure around that at all. You are tied to bare metal VPS and Ansible.
6. Huge pile of bullshit at frontend. Doesn’t work at Incognito mode, doesn’t work at Firefox.

Told to sanitize a large collection of PowerPoints of customer data. Found one resolvable IP address and about 200 typos and other mistakes. Deleted the IP address.. mission successful!

I grabbed 30 random DOS malware samples from my collection, rolled via urand over Python list, and tried to figure out how they work.

Results:

1x zipped EICAR
4x working but effectively useless ("yeah you wiped the first 100 sectors of the drive... but you wrote their prior contents. Literally nothing's changed...")
10x CPU hang
10x crashdump back to DOS
5x crashdump back to DOS but ERRORLEVEL=0 so normal termination despite real errors being given?

also make sure SOURCER is disassembling using 486 or Pentium opcodes or it misses some 286/386 opcodes and will count half the program as data.

A guy that I know from uni (used to be my TA), asked me to help him with data collection for his thesis.
The thing is, there are plenty of other people that he is better acquainted with (and are my friends), and he did not ask them.
So I was wondering, DS/DL/ML people, is this how you make a move on a girl? Ask her to collect data with you?

In the last few days I've been collecting data for our lidar-based people detector, and it made me wonder if foot fetish is also extended to lidar scans 😂

Google. Once I thought they where kind of cool. Then I woke up. Data collection and privacy issues among other things.

Why recruiters want me to fill up the Application form and I dont get any reply from them since. Is this a data collection thing?

Can anyone recommend good resources for learning how to design NoSQL (document) data models?

I'm interested in stuff that talks about how to make the choices about distributing data across collections, etc.

When to have a single collection, when to split data across different collections, when to duplication data, etc,

48 hours.
We had 3 weeks of "manual data collection": pencil, paper and a dozen of people around all the offices of the company with the task to collect serial numbers of every piece of equipment used.
Then we had 3 weeks of data entry, a dozen of people copying all handwritten data to a custom made VB form.

And then there was me, the guy that was in charge of verifying, zipping and sending the data to the client. I spent 48h non stop to go through everything, finding, fixing or delete unusable data.
I had to delete at least 25% of the data because incomplete or completely unusable (serial numbers too short or too long, for example).
48h in the office.

The data was then delivered to the customer. 2 days after, when I finally woke up, everyone was in panic because:
- serial numbers were not matching
- addresses were wrong
- the number of delivered records was smaller than expected

What did I learn from this experience?
When your deadline is tomorrow, and you need 4 weeks to complete your work, ignore the deadline and inform everyone at any level that you are ignoring the deadline. And then resign and find a better job.

Ah, yes, pencils and paper are powerful tools, but rat poison too. You just need to use them in the right place. The only data collection that can be trusted when done with a pencil is the one involving checkboxes.

markdown is not good enough! the tools aren't there for non-devs and there's no concordance on moving forward *compatibly* for anything other than headers and __possibly__ lists.

md has been around for years and still no consensus on comments, meta data, css, data imports, etc.

i could never in good faith recommend to a non-dev to use markdown, even though every academic and professional writer from legal to journalism should exclusively be using markdown to write and store their documents. the data portability and ease of search, retrieval, collection, distribution, etc of markdown compared to pdf or docx is enormous. markdown is the hex format of text, the perfect layer of data and visual so that the user and the computer can both operate on text as blocks of data rather than weirdly styled paragraphs that need to be reformatted BY HAND for citation-style or journal format, or paper size. FOR EACH SUBMISSION. Academics literally rewrite their 100-page papers to accommodate up to 10 different submission requirements.

They could be clicking MLA vs Chicago and/or using a journal's stylesheet to recompile for its styles.

Today there is some support from zotero et al to take away some of the pain, but it makes ZERO SENSE for writers to have to keep and store and keep up to date, multiple versions of the same document. Git pull does not exist for them. But the worst part is that git isnt the solution to their problem. They need a compiler more than they need version control. But they also desperately need vcs. They ALL literally have a million files named "dumdum.dumFINAL-3084_lastversion \2020, this one.dum".

They dont have git or anything like it, because they need a line-by-line solution like markdown for git to become effective.

All of writing is basically mired in the fact that people cant even roll up their paragraphs and see what the fuck it is theyre saying. Most writing reads like a long scroll through some nonsense that goes nowhere. Like this rant. but the point is that markdown and line-by line editing actually produces more logically sound writing. You start to think in terms of defining ideas in blocks, ... like code.

I work at company that uses Drupal for everything. And i mean EVERYTHING. Our dumb CTO once even wanted us to join tender for flight data collection system... of course it would run on fucking drupal...

Yeah i can see its advantages but it has learning curve the shape of the snail shell and if you want it to do something new you either find module for it or drupal will start crying, shits itself and tell you to go fuck yourself.. also it is full of surprises to make your day as miserable as possible, like you send variable as $content['varname'] to user template and it returns as $user_profile['varname']['value']... and yes user template has $content array for content but why use it for storing content that i want to render.. it is used for other content to render... because in drupal content != content...

I started using laravel for my freelance projects and it took my less than 2 week to get up to speed and start working and is incredible fast to work in... You know.. its fun when you want to just add feature you just code that feature into your app.. and not spend 2 fucking years crawling through retarded preprocess functions...

Whenever i try to suggest we use other frameworks.. "Muh drupal has MODULES".. yeah because drupal is the only thing in universe that has modules.. When client has only need for simple site with simple template why use wordpress and have it done in 2 days when you can use drupal have 10 000 unnecessary DB queries that drupal does on every page load to load page title and make that site in a week.. or why use laravel for e-shop with specific functionality requested by client that would take 2 weeks to add in laravel when you can spend 2 months modifing uber-cart or drupal commerce modules only to hit some Drupal core surprise that wont allow for that feature to be implemented...

New excuse for not doing work:
"taking data"

My dev system is tied up taking data for an hour to store data for a replay system I intend to build. I need raw data from the system. I need to know how much data I will collect in an hour as well. I also need to collect said data. For about an hour. This data will be fed back into the system sans the data collection. This will allow me to replay whole jobs that our field techs perform. If they get me data I can reprocess the data and fix my positioning algorithms. Kinda fun!

So the excuses are now up to 2:
"compiling"
and
"taking data"

Hey just brainstorming a business/ startup idea I may try out sometime down the line. I wanted to put it in writing available to my peers for review. If that sounds boring, sorry.

So I've had an idea and I know it's a million dollar idea because it's absolutely boring as fuck.

Recently I have been learning about NoSQL and it has gotten me pretty excited about unstructured data.

Now the first thing you should know about me is I like to make business software. I don't like games or social networks or blah blah blah, I like business stuff. One dream I have always had is to make THE business solution. I've noticed so many specific business solutions for very specific areas of work. Specific software for car washes, which is separate from the software for car maintenance, which is separate from the point-of-sales software, which is separate from the [...]

One of the problems with this is the inconsistency. Modular is good, but only if the modules are compatible. They aren't. Training needs to be provided for each individual system since they are all vastly different. And worst of all, since all of these different applications reach their own niche market, they charge out the butt for things that are usually very simple "POST a form over http(s)" machines.

I mean let's not get too dreamy here. My solution is an over-complicated form-builder. But it would be a game-changer for small and medium-sized businesses. Allowing users to build their own front-end and back-end disguised as a drag-and-drop form builder would be THE alternative, because they could bring all of their solutions into a single solution (one bill!) and since THEY are the ones that build what they need, they can have custom business software for the price of a spreadsheet program.

The price difference we could offer would be IMMENSE. Not only would we be able to offer "cookie-cutter" pricing as opposed to "custom" pricing, but since this generic solution could be used for essentially all of their systems, we aren't just decreasing one bill. We're decreasing one bill, and eliminating the rest entirely. We could devastate competition.

"BUT ALGO", you scream in despair, "USERS AREN'T SMART ENOUGH TO DRAG AND DROP FORM PARTS TO MAKE A FORM"

I mean ya true. But you say that like it's a bad thing. For one, we can just offer a huge library of templates. And for another, which is part of the business plan, we can charge people support dollars to help them drag and drop their stupid fucking forms!! Think of the MONEEYYYY YOU COULD MAKEE BY EXPLAINING HOW TO COLLECT FIRST AND LAST NAMEEE. Fuck.

The controls library would be extensible of course. You would be able to download different, more specialized controls if you need them. But the goal would be to satsify those needs with the standard collection of controls (Including interesting ones line barcode scanner and signature input and all that). But if all else fails, maybe someone made an open source control for you to implement and ignore that stupid donation button. We all do.

This could PURGE the world of overpriced and junky specialized business software, and best of all, it's aimed at smaller businesses. With smaller businesses making more profit, they will stay afloat better and may start to compete with their larger foes. Greater for the entire economy.

Anyways, I'm sure it's full of holes. Everything always is. But I still think it's something I'll try before I die.

So I cant resize the window and I can't scroll down anymore... Where the fuck is "below" so I can turn "off" data collection 😤

Our app has web scale data. So let me take all my tables from a relational database and map them one to one to a collection in a document database.

MongoDB database with really relational data. One main collection that had refs to four other collections, all of those references necessary to populate data for a page view. Complicated aggregate to populate all the necessary data and then filter based on criteria selected by the user. And then the client decides that he wants the information to be sortable by column. Some of those columns are fields on the main model, no problem. Others are fields on the refs, which is more of a problem. Especially given that these refs aren’t one single object. They’re arrays of objects.

The revelation was that I could just write an aggregate function to flat map the main collection, returning only the fields necessary for the search, and output it to a new collection and instead use that new collection for displaying and filtering/sorting search results.

But you can’t run the aggregate all the time, you surely say. If anything changes in the main collection, it won’t be reflected in the search results!

Mongoose post(‘findOneAndUpdate’) hooks, my friends. Mongoose post(‘findOneAndUpdate’) hooks.

Never been so happy to have a thing working properly in my life.

#justathought

Since a country is considered to be the strongest definition of a group of unified people, consider the following scenario:

Every country makes it mandatory that any foreign company can only sell their software products as free to use softwares, or one time cost software, or a membership software.

These foreign companies are NOT allowed to generate revenue via Advertisement services or data collection/sharing/analysis AT ALL.

The sole right to share/collect data and use Advertisments as a revenue model remains in the hands of domestic companies born, registered and working in that country only.

This would generate an equal chance for both domestic and international companies to grow(domestic companies getting a chance to grow with a better revenue model, while international companies getting a chance to grow by monetising their tools/algorithms and investing in domestic companies) , resulting in countries getting a chance to grow themselves.
Open source will still win, as open source majorly contains tools and technologies for general public use.
Premium tools and frameworks would become even more valuable, and would he shared among countries like the way they share space researches and resources on nuclear technology today.

Privacy will win, as the data of a particular country stayed within a country. Domestic Companies sharing data with other companies (or even their foreign parent companies) would be held against the respective country's laws only, and government would be more involved in protecting its citizens from data theft.

Is it feasible?

!rant

Rant from my previous work as a consultant Data Engineer (wish I had known this site back then).

During my stay at the place, we have a big client whose contact with us was an incompetent stressful fellow.

I single-handedly build a humongous automated data pipeline using Airflow. I am very proud of my baby as my first massive project and check it obsessively for every possible flaw, especially when writing down documentation for the poor soul that would take my place.

Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.

After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.

"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."

Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.

I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.

Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.

Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm.

Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.

After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.

"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."

Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.

I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.

Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.

Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm.

Probably working for a scientific research firm and building in house tools for collection and analysis of data...

feeling like shit at work because I'm not productive at all.... I'm a fullstack web dev and was assigned to create a java data importer with multiple sources, multiple scenarios and using various data types... What makes this difficult is that I'm not used to strictly typed languages, because I'm used to swapping variable types and nulling them down/whatever I need to do with them whenever I want. In java I need to assign the correct variable types, there are no asociative arrays . I've been fixing one issue this whole day. Litteraly one fucking issue. Maneged to fuck javas garbage collection even though it's supposed to be automatic. Fuck. I feel like I need to stay late, and program on the weekends to achieve anything with this assigment because right now I feel like I make 0 progress. Boss leaves for vacation next week for a week, and he's the other dev that theoretically should be working with me...

Anyone out there a Scala fan? I am! Seems like most of the stories here are quite negative but positively is my thing so here's my 2 cents:

Scala is an amazing data processing language. It's a functional language with a lot of really great things like a consistent collection library api, case classes, brilliant async library's like Akka Actors, and plenty of solid learning resources like Twitter school and Martin Odersky's online course.

Make an Async task (Java) and...

DONT use a loop to iterate though a time series collection. Don't linear search that shit.

DO use a queue and pop() it like its hot. Check that shit to see when it needs to be used instead of searching.

DO assert that your time series data is in order (Predication mother fucker).

DO throw an exception that you data is all fucked when it's all fuck up.

Stay sexy and use a fucking queue man.

When you search about disabling windows data collection, anti privacy settings and end up on frogs REEEEING, you know there's something fishy

tldr; selenium-java (my newest learned tool) vs beautifulsoup4 (my most experience with) or scrapy(average experience, mediocre ability with). Which should I use if allowed to use any for web scrapeing assignment

We were explicitly told we can use anything we know from class or self study (slight bonus for self study implementations) for the group project, but would it be OK/fair for me to use beautifulsoup4 or scrapy to pull the data from the assigned site rather than the selenium-java we were taught in class

If I did use bs4 or scrapy my group wouldn't be able to edit if needed but the data collection is only a small (if immensely important) part of the assignment and I'd have the bs4 script done a lot quicker than with selenium which I have learned more recently (for class) and have less experience with

I was talking with my friend on call who was grocery shopping and was confused which icecream to buy. I suggested him to get Chapman’s Frosty. I started seeing advertisements of Chapman’s all over Instagram after. Kinda scary.

In a MNC company
Manager - where is the data collection located

Employee - in your fucking butt

So, I rather like data, what do you think I should collect to have my own little offline library?

so I offered to help a guy with his startup idea which is in the making for a year at least. (A telemetry data collection device, mind you). It's an unpaid internship with no strings attached, no contract, mind you.

I am nowhere near the programming pro like most of you guys here, so that's what I want to focus on improving.

First thing he tells me to do is a market research on competitors.... I thought I left that bullshit behind when I left business school..

If I want to work without getting paid, I'd rather stay at home and learn c++ and opencl, and work on the MSc thesis full time, thanks.

Do you think it's the right thing to do or should I give it time?

One of my preferred functions is Collections.unmodifiableList(List).

What a relief was the introduction of the collection FW. And the Function above changed the usage of lists (also sets, maps) a lot. You could now just expose your internal list without worrying that somebody messes with the data.

I wrote a script, modified + tested it and that worked fine 6 months ago. I expected to use it then but didn't but all good so i kept a special branch for my script which generates data from old db, formats it to new db structure and dumps it into a collection. In fact, i ran one of the scripts last month (360 records) and it worked fine. Today, when i need to migrate 14,000 records, script refuses to run. I am ready to fight the universe because wtf.

As much as the build quality and features of Google products are appreciated, how many of you are concerned about the data they collect about users ? Do you guys take steps to avoid such data collection ? Who wins eventually .. convenience or privacy ?

Hey guys, any WPF developers here?

I'm having lotsa trouble getting WPF XAML data bindings to work. Disclaimer - I'm new to OOP and thhe syntax of OOP is so damn confusing I'm never sure anything is the "right" way.

The task is to create test data for certain classes and output it in WPF. The code I have is a public static class that generates test data for certain classes and stores these objects inside a static List<Object> depending on the object. I couldn't figure out any other way to store all these objects to later be able to output them.

Then I found out that you can use ObservableCollection to automate a lot of the CRUD stuff. So I tried to change the Lists to static ObservableCollections. It mostly works and I even got it to output the data in XAML by using DataGrid.ItemsSource = TestDataCreationClass.authors in the MainWindow.xaml.cs. However, I can not for the life of me figure out how to do the bind through XAML only using the ItemsSource property. No matter what I do, it cannot find the Collection.

I googled for quite a while and every example seems completely different from mine so I'm at a loss.

If you need any more info or code snippets I'd be glad to provide them.

Any kind of help is appreciated.
Thanks in advance!

I am using MERN to develop a poll app. I have 6 collections in my database. I have an API for each collection, 4 of those API returns the data whereas the other 2 return 'undefined'. What is the issue I am facing? Is MongoDB not allowing me to use more than 4 collections?