A super creepy webcrawler I built with a friend in Haskell. It uses social media, various reverse image searches from images and strategically picked video/gif frames, image EXIF data, user names, location data, etc to cross reference everything there is to know about someone. It builds weighted graphs in a database over time, trying to verify information through multiple pathways — although most searches are completed in seconds.

I originally built it for two reasons: Manager walks into the office for a meeting, and during the meeting I could ask him how his ski holiday with his wife and kids was, or casually mention how much I would like to learn his favorite hobby.

The other reason was porn of course.

I put further development in the freezer because it's already too creepy. I'd run it on some porn gif, and after a long search it had built a graph pointing to a residence in rural Russia with pictures of a local volleyball club.

To imagine that intelligence agencies probably have much better gathering tools is so insane to think about.

Good Morning!, its time for practiseSafeHex's most incompetent co-worker!

Todays contestant is a very special one.

*sitcom audience: WHY?*

Glad you asked, you see if you were to look at his linkedin profile, you would see a job title unlike any you've seen before.

*sitcom audience oooooooohhhhhh*

were not talking software developer, engineer, tech lead, designer, CTO, CEO or anything like that, No No our new entrant "G" surpasses all of those with the title ..... "Software extraordinaire".

*sitcom audience laughs hysterically*

I KNOW!, wtf does that even mean! as a previous dev-ranter pointed out does this mean he IS quality code? I'd say he's more like a trash can ... where his code belongs

*ba dum tsssss*

Ok ok, lets get on with the show, heres some reasons why "G" is on the show:

One of G's tasks was to build an analytics gathering library for iOS, similar to google analytics where you track pages and events (we couldn't use google's). G was SO good at this job he implemented 2 features we didn't even ask for:

- If the library was unable to load its config file (for any reason) it would throw an uncatchable system integrity error, crashing the app.
- If anything was passed into any of the functions that wasn't expected (null, empty array etc.) it would crash the app as it was "more efficient" to not do any sanity checks inside the library.

This caused a lot of issues as some of the data needed to come from the clients server. The day we launched the app, within the first 3 hours we had over 40k crash logs and a VERY angry client.

Now, what makes this story important is not the bugs themselves, come on how many times have we all done something stupid? No the issue here was G defended all of this as the right thing to do!

.. and no he wasn't stoned or drunk!

G claimed if he couldn't get the right settings / params he wouldn't be able to track the event and then our CEO wouldn't have our usage data. To which I replied:

"So your solution was to not give the client an app instead? ... which also doesn't give the CEO his data".

He got very angry and asked me "what would you do then?". I offered a solution something like why not have a default tag for "error" or "unknown" where if theres an issue, we send up whatever we have, plus the file name and store it somewhere else. I was told I was being ridiculous as it wasn't built to track anything like that and that would never work ... his solution? ... pull the library out of the app and forget it.

... once again giving everyone no data.

G later moved onto another cross-platform style project. Backend team were particularly unhappy as they got no spec of what needed to be done. All they knew was it was a single endpoint dealing with very complex model. There was no Java classes, super classes, abstract classes or even interfaces, just this huge chunk of mocked data. So myself and the lead sat down with him, and asked where the interfaces for the backend where, or designs / architecture for them etc.

His response, to this day frightens me ... not makes me angry, not bewilders me ... scares the living shit out of me that people like this exist in the world and have successful careers.

G: "hhhmmm, I know how to build an interface, but i've never understood them ... Like lets say I have an interface, what now? how does that help me in any way? I can't physically use it, does it not just use up time building it for no reason?"

us: "... ... how are the backend team suppose to understand the model, its types, integrate it into the other systems?"

G: "Can I not just tell them and they can write it down?"

**
I'll just pause here for a moment, as you'll likely need to read that again out of sheer disbelief
**

I've never seen someone die inside the way the lead did. He started a syllable and his face just dropped, eyes glazed over and he instantly lost all the will to live. He replied:

" wel ............... it doesn't matter ... its not important ... I have to go, good luck with the project"

*killed the screen share and left the room*

now I know you are all dying in suspense to know what happened to that project, I can drop the shocking bombshell that it was in fact cancelled. Thankfully only ~350 man hours were spent on it

... yep, not a typo.

G's crowning achievement however will go down in history. VERY long story short, backend got deployed to the server and EVERYTHING broke. Lead investigated, found mistakes and config issues on every second line, load balancer wasn't even starting up. When asked had this been tested before it was deployed:

G: "Yeah I tested it on my machine, it worked fine"

lead: "... and on the server?"

G: "no, my machine will do the same thing"

lead: "do you have a load balancer and multiple VM's?"

G: "no, but Java is Java"

... and with that its time to end todays episode. Will G be our most incompetent? ... maybe.

Tune in later for more practiceSafeHex's most incompetent co-worker!!!

So this was a couple years ago now. Aside from doing software development, I also do nearly all the other IT related stuff for the company, as well as specialize in the installation and implementation of electrical data acquisition systems - primarily amperage and voltage meters. I also wrote the software that communicates with this equipment and monitors the incoming and outgoing voltage and current and alerts various people if there's a problem.

Anyway, all of this equipment is installed into a trailer that goes onto a semi-truck as it's a portable power distribution system.

One time, the computer in one of these systems (we'll call it system 5) had gotten fried and needed replaced. It was a very busy week for me, so I had pulled the fried computer out without immediately replacing it with a working system. A few days later, system 5 leaves to go work on one of our biggest shows of the year - the Academy Awards. We make well over a million dollars from just this one show.

Come the morning of show day, the CEO of the company is in system 5 (it was on a Sunday, my day off) and went to set up the data acquisition software to get the system ready to go, and finds there is no computer. I promptly get a phone call with lots of swearing and threats to my job. Let me tell you, I was sweating bullets.

After the phone call, I decided I needed to try and save my job. The CEO hadn't told me to do anything, but I went to work, grabbed an old Windows XP laptop that was gathering dust and installed my software on it. I then had to build the configuration file that is specific to system 5 from memory. Each meter speaks the ModBus over TCP/IP protocol, and thus each meter as a different bus id. Fortunately, I'm pretty anal about this and tend to follow a specific method of id numbering.

Once I got the configuration file done and tested the software to see if it would even run properly on Windows XP (it did!), I called the CEO back and told him I had a laptop ready to go for system 5. I drove out to Hollywood and the CFO (who was there with the CEO) had to walk about a mile out of the security zone to meet me and pick up the laptop.

I told her I put a fresh install of the data acquisition software on the laptop and it's already configured for system 5 - it *should* just work once you plug it in.

I didn't get any phone calls after dropping off the laptop, so I called the CFO once I got home and asked her if everything was working okay. She told me it worked flawlessly - it was Plug 'n Play so to speak. She even said she was impressed, she thought she'd have to call me to iron out one or two configuration issues to get it talking to the meters.

All in all, crisis averted! At work on Monday, my supervisor told me that my name was Mud that day (by the CEO), but I still work here!

Here's a picture of the inside of system 8 (similar to system 5 - same hardware)

"full stack" means "you'll be doing everything from gathering client requirements through data architecture up to the UI design and of course implementing all of it"

"backend" means "you'll be coding everything from database through server-side code and client-side code including html and css"

"we need you on-site all day every day" means "we have no idea how and why we should use repositories with remote access despite being a company developing an internet app, and we don't trust that you would be working anyway"

"interesting challenging projects" means "the same boring crap as every other company, running on an incredibly botched and dezorganized codebase".

"competitive pay" means "actual pay is around 1.5 times the minimum allowed pay, and everything else is being siphoned off into (stupid and useless) 'benefits' like massage and fitness discount coupons"

"friendly collective having fun at numerous company events each years" means "it is mandatory for you to participate on our weekend drinking retreats but you'll only find out when we fire you because you're 'not a team player' after you refused to participate on those"

So, as everyone knows on here by now (or, a lot of ranters), I am a fervid privacy person.

Appearantly a new surveillance law in my country is about to extend mass surveillance/hacking a lot. So here a rundown of what they are about to be allowed to do (stuff that is not okay imo and this is the reason I am so pro-privacy):

- Mass Data Gathering: The intelligence agency over here (lets call it IA from now) can pretty much record everything send through the country.

- Extra Protection: If they want to conduct surveillance on journalists/lawyers, they have to go through extra channels first at least.

- Data/survaillance sharing: The IA is allowed to share their raw/filtered data with foreign intelligence agencies without limits. Also, they're allowed to conduct surveillance based on foreign requests.

- Secret DNA database: A secret DNA database will be created which can store the DNA profiles of any person who has commited any kind of crime. These profiles are allowed to be stored for a maximum of 30 years. This database is allowed to be shared with any foreign intelligence agency.

- Hacking: Unlimited power to hack any device deemed neccesary to hack in relation to crime. From computers to smartphones and so on. Also, it's allowed to use zero-days without reporting them to the vendor (we have seen what can go wrong with that through the ShadowBrokers scandal).

- Automatic Database Collection: They are allowed to directly tap into any database they see required (banks, healthcare, messaging services and so on). Practically this can lead to backdoors being build in because if you don't cooperate, you can go to prison. (mother of god I am not using anything closed source anymore if possible).

So yeah, this is pretty much the reason why I am so privacy consious. This country is fucked.

Had an internet/network outage and the web site started logging thousands of errors and I see they purposely created a custom exception class just to avoid/get around our standard logging+data gathering (on SqlExceptions, we gather+log all the necessary details to Splunk so our DBAs can troubleshoot the problem).

If we didn't already know what the problem was, WTF would anyone do with 'There was a SQL exception, Query'? OK, what was the exception? A timeout? A syntax error? Value out of range? What was the target server? Which database? Our web developers live in a different world. I don't understand em.

I recently tried to apply the same data analytics rationale that I use at work to my personal life. This is not a rant, it is more like an data storytelling of an actual use case I would like some input on.

I set a goal - gotta thin up a bit and calm down my ticker - and got a (almost unreasonably expensive) field expert consultant to yell at me about it for a couple hours.

I unravel the metrics - there is like a million weight-related KPIs and most say nothing at all. I have never seen an non-infrastructure measurable subject that could not be resumed to 2-5 performance metrics. I got overall weight, how well my nine-years-old business suit fits me, heart rate, and day-after relative muscle pain (it will make sense soon).

Then its data-pipeline time. I bought a cheap weight scale and smartwatch, and every morning I input the data in an app. Yes, I try to put on the suit every morning. It still does not fit.

After establishing a baseline, I tried to fit different approaches. Doing equipment-free exercises, going to the gym, dieting. None was actually feasible in the long run, but trying different approaches does highlight the impacts and the handling profile of each method.

Looking at the now-gathered data, one thing was obvious - can't do dieting because it is not doable to have a shopping list and meals for me and another for the family.
Gym is also off the table - too much overhead. I spend more time on the trip there and back than actually there.
And home exercise equipment is either super crappy or very expensive. But it is also the most reasonable approach.

So it is solutions time. I got a nice exercise bycicle (not a peloton), an yoga mat (the wife already had that one) and an exercise program that uses only those two resources. Not as efficient without dieting, not as measurable and broad as the gym, but it fits my workflow. Deploy to production!

A few months pass and the dataset grows. The signal is subtle but has support - it works! The handling, however, needs improvement, since I cannot often enough get with the exercise program. Some mornings are just after some hard days.

I start thinking about what else I can improve in the program, but it is already pretty lean and full of compromises.
So I pull an engineer and start thinking about the support systems and draft profile. What else could be draining my willpower and morning time?

Chores. Getting the kids ready for school, firing up the moka pot, setting the off-brand roomba, folding the overnight-dried clothes, cooking breakfast, doing the dishes, cleaning the toilets. All part of my morning routine. It might benefit from some automation.

Last month I got that machine our elders call "wasteful" and "useless crap lazy entitled Americans invented because they feel oh-so-insulted for simply doing something by hand like everyone always did" - a "dish-washer".
Heh, I remember how hard was to convince my mother-in-law that an remote-controled electric garage door would not make she look like an spoiled brat.
Still to early to call, but I think that the dishwasher just saved me about 25 mins every morning. It might be enough to save willpower for me to do more exercise.

This is all so reflective of all data analytics cases really are out in the wild - the analytics phase seems so small compared to the gathering and practical problem-solving all around. And yet d.a. is what tells you that you are doing the wrong thing all along. Or on what you should work next.

So I am currently making an app for a retail franchise and that retail franchise is getting a website done by a company. Since the recipes that the guys want are going to be on the website, the web dev company made a mysql db that has all the recipes.

I thought since the people at the head of the retail franchise have spent time gathering these recipes, I'll just get the data for the app from the db.

I called the project manager for the website up (That's the only contact I got) and asked if they could given me access to the db to use for the app or if I could make a script that would get data from it.

Now this is the part where I tell you I'm only 14 years old and these guys know that because of the head of the retail company.

He puts it on speaker and asks me to say it again and I hear quite a bit of people laughing. I knew what was happening. He asks do you know anything about databases, I explained to him what a db was and how I was going to get the data from it and etc. Half way through me telling him that it would be beneficial for both me and the retail guys, he hung up the call and blocked me.

I asked the head of the retail company if he could ask them about it and he said that he didn't know much about tech so he couldn't ask and if I could find an alternative option.

You might be thinking that the company didn't want to give me permission to access the db which is respectable but they have done this previously as well.
I gave them advice on putting a rewards card feature on the site so the customers could track their points on the site as well as on the app and the PM said "We don't want advice from you."

It has now been 3 weeks on and off because of school where I had to code a ui for the lady at the company to enter all her recipes for the app and waste a lot of time communicating with different people to get all the data.

I hate being disrespected because of my inexperience when I can truly do some extraordinary things with software. 😡😤😣

Its also very hard to find a job being 14.

People think Machine Learning is all about using Super Complex Prediction Models...
But turns out to devote most time in data gathering and data cleaning(preprocessing).

It's been a while DevRant!

Straight back into it with a rant that no doubt many of us have experienced.

I've been in my current job for a year and a half & accepted the role on lower pay than I normally would as it's in my home town, and jobs in development are scarce.

My background is in Full Stack Development & have a wealth of AWS experience, secure SaaS stacks etc.

My current role is a PHP Systems Developer, a step down from a senior role I was in, but a much bigger company, closer to home, with seemingly a lot more career progression.

My job role/descriptions states the following as desired:

PHP, T-SQL, MySQL, HTML, CSS, JavaScript, Jquery, XML

I am also well versed in various JS frameworks, PHP Frameworks, JAVA, C# as well as other things such as:

Xamarin, Unity3D, Vue, React, Ionic, S3, Cognito, ECS, EBS, EC2, RDS, DynamoDB etc etc.

A couple of months in, I took on all of the external web sites/apps, which historically sit with our Marketing department.

This was all over the place, and I brought it into some sort of control. The previous marketing developer hadn't left and AWS access key, so our GitLabs instance was buggered... that's one example of many many many that I had to work out and piece together, above and beyond my job role.

Done with a smile.

Did a handover to the new Marketing Dev, who still avoid certain work, meaning it gets put onto me. I have had a many a conversation with my line manager about how this is above and beyond what I was hired for and he agrees.

For the last 9 months, I have been working on a JAVA application with ML on the back end, completely separate from what the colleagues in my team do daily (tickets, reports, BI, MI etc.) and in a multi-threaded languages doing much more complicated work.

This is a prototype, been in development for 2 years before I go my hands on it. I needed to redo the entire UI, as well as add in soo many new features it was untrue (in 2 years there was no proper requirements gathering).

I was tasked initially with optimising the original code which utilised a single model & controller :o then after the first discussion with the product owner, it was clear they wanted a lot more features adding in, and that no requirement gathering had every been done effectively.

Throughout the last 9 month, arbitrary deadlines have been set, and I have pulled out all the stops, often doing work in my own time without compensation to meet deadlines set by our director (who is under the C-Suite, CEO, CTO etc.)

During this time, it became apparent that they want to take this product to market, and make it as a SaaS solution, so, given my experience, I was excited for this, and have developed quite a robust but high level view of the infrastructure we need, the Lambda / serverless functions/services we would want to set up, how we would use an API gateway and Cognito with custom claims etc etc etc.

Tomorrow, I go to London to speak with a major cloud company (one of the big ones) to discuss potential approaches & ways to stream the data we require etc.

I love this type of work, however, it is 100% so far above my current job role, and the current level (junior/mid level PHP dev at best) of pay we are given is no where near suitable for what I am doing, and have been doing for all this time, proven, consistent work.

Every conversation I have had with my line manager he tells me how I'm his best employee and how he doesn't want to lose me, and how I am worth the pay rise, (carrot dangling maybe?).

Generally I do believe him, as I too have lived in the culture of this company and there is ALOT of technical debt. Especially so with our Director who has no technical background at all.

Appraisal/review time comes around, I put in a request for a pay rise, along with market rates, lots of details, rates sources from multiple places.

As well that, I also had a job offer, and I rejected it despite it being on a lot more money for the same role as my job description (I rejected due to certain things that didn't sit well with me during the interview).

I used this in my review, and stated I had already rejected it as this is where I want to be, but wanted to use this offer as part of my research for market rates for the role I am employed to do, not the one I am doing.

My pay rise, which was only a small one really (5k, we bring in millions) to bring me in line with what is more suitable for my skills in the job I was employed to do alone.

This was rejected due to a period of sickness, despite, having made up ALL that time without compensation as mentioned.

I'm now unsure what to do, as this was rejected by my director, after my line manager agreed it, before it got to the COO etc.

Even though he sits behind me, sees all the work I put in, creates the arbitrary deadlines that I do work without compensation for, because I was sick, I'm not allowed a pay rise (doctors notes etc supplied).

What would you do in this situation?

I am not sure if devrant is the best place to post this so sorry if it is not.

How far do employers/recruiters go when searching online information about their applicants?

Do they simply check your fb? simple google search your name? advanced queries with multiple search engines? data gathering software like maltego? or really check and link leaked db dumps and pastebins?

If anyone has any knowledge or experience with this I would love to hear.

Thanks in advance

Google acquired two interesting products companies last week.
One is making customizable phone apps from spreadsheets the other is gathering sales data from local shops.

appsheet and pointy

At this point I think they’re still missing code editor. Microsoft have visual studio and amazon as always was first and acquired c9.io when vscode was one year old.

How the fuck they missed the code that would run remotely on multiple machines should have ability to connect to one node with debugger after they fucked docker with their k8s.

Situation: I have a love hate relationship with python due to the lack of types as I have in more established languages such as C#, Java and shit even TypeScript

Situation (cont): A rather large codebase that i have developed for multiple processes at work run on Python.

I don't hate it, I just don't absolutely love it, there is a lot of things to like about Python, but man I do have some conflicts with it, I have been facing out to use other solutions that feel scripty, such as the newer versions of C# with .net, but I would say that about 80% of our codebase runs on Python, the rest is PHP.

I am somewhat traditional in the way my programs run, I started with C++ and Java, then for whatever reason (I blame codecademy at the time) switched over to Ruby and Javascript, mostly Javascript. I do not remember how I found Python, I do remember learning it with an online tutorial, shit was easy to get started with.

My codebase running on Python is huge, and they do a lot from automation scripts, to data gathering and database management, never had I been bitten with the "oh noes is so slow" bug since my code is not Google level big, for everything else Python seems rather fast imho

I dunno, big time love hate relationship

I was studying a lot the last year, i learned a lot about Machine Learning/Deep Learning, Data Gathering, Data Analysis, ETL, Model Architecture Design, Training, Fine Tuning, Backend Development, DataBases, API Development, ORMs, Rest, GraphQL, OAuth, CI/CD, Docker, Deployment to Production environments like Heroku, Git and more stuff i dont remember while writing this. I built and keep adding stuff to my Github Portafolio.

Im not able to get a job. I started looking for jobs as Data Scientists, no response never. I take a look at freelancer sites, nothing seems to fit my skills. And when there is a minimal fit, they always want a Full Stack Web Developer, i dont know Frontend Development, i dont like do it.

Dont know what to do or how to land any job.

My options aeems to be:

1.Learn Frontend Dev and work as Full Stack in underpaying freelance jobs

2.Keep applying to Remote-Only startups, but they still wants people with 3+ years of experience.

i cant work in my city, here are not any company startup hiring no one, we are 30 years in the past here.

What you do in my place?

Someone should make a movie about three ghosts that haunt a BLOODY CROOK who makes his employees and coworkers burn the midnight oil in the bloody CHISTMAS EVE because the fucker haven't finished something that should have been ready TWO FUCKING WEEKS AGO.
The ghost of Christmas past shows the fucker that he was a bloody LAZY KID who made his elderly relatives cook, host, clean, wash the dishes and everything else all by themselves during family-gathering season.
The ghost of Christmas present shows him his employees' children teary eyed that daddy doesn't get to watch cartoons with them before bedtime (we're not Christians but just because my house is a steak-free zone it doesn't mean my kids don't expect gifts from santa, like most kids in their school!)
The ghost of Christmas future shows a Netflix documentary on how the fucker got arrested for being a BLOODY CROOK that gets played by some actor who is a hollywood-level jerk who beats his wife. And the show gets a 3% on rotten tomatoes, just to salt the wound. Oh, and a voiceover says the real BLOODY CROOK hanged himself in prison or something and his family is happy he did it.

Fuck, I hate, for real hate, people whose tardiness bleeds out on honestly-working people. I had to wake up one of my devs to fix the SHIT that the bloody crook higher-up shat on us.
My guy is getting a raise as soon as I can scream at the bean counters and my boss will be getting some loooooong, data-rich report on how the bloody crook's department is pissing in our soup.
Fuck everything.

Hey guys (and girls (and people who dont identify as either of them)),

As some people here might know I am currently in the planning phase of an open source alternative for Thingiverse. After gathering information on multiple platforms Ive heard people really wanting it to be decentralized (a bit like Mastodon).

I really like that idea and thus want to implement it, does any of you have experience or knowledge of decentralized systems programming? I have trouble getting information on how this works. If someone is willing to help me explain a few things like "How does one instance retrieve data from other instances? How does it know a new instance is running somewhere?"

So if anyone knows a great source for information or can and wants to help me via a chat session please respond below

personal projects, of course, but let's count the only one that could actually be considered finished and released.

which was a local social network site. i was making and running it for about three years as a replacement for a site that its original admin took down without warning because he got fed up with the community. i loved the community and missed it, so that was my motivation to learn web stack (html, css, php, mysql, js).

first version was done and up in a week, single flat php file, no oop, just ifs. was about 5k lines long and was missing 90% of features, but i got it out and by word of mouth/mail is started gathering the community back.

right as i put it up, i learned about include directive, so i started re-coding it from scratch, and "this time properly", separated into one file per page.

that took about a month, got to about 10k lines of code, with about 30% of planned functionality.

i put it up, and then i learned that php can do objects, so i started another rewrite from scratch. two or three months later, about 15k lines of code, and 60% of the intended functionality.

i put it up, and learned about ajax (which was a pretty new thing since this was 2006), so i started another rewrite, this time not completely from scratch i think.

three months later, final length about 30k lines of code, and 120% of originally intended functionality (since i got some new features ideas along the way).

put it up, was very happy with it, and since i gathered quite a lot of user-generated data already through all of that time, i started seeing patterns, and started to think about some crazy stuff like auto-tagging posts based on their content (tags like positive, negative, angry, sad, family issues, health issues, etc), rewarding users based on auto-detection whether their comments stirred more (and good) discussion, or stifled it, tracking user's mental health and life situation (scale of great to horrible, something like that) based on the analysis of the texts of their posts...

... never got around to that though, missed two months hosting payments and in that time the admin of the original site put it back up, so i just told people to move back there.

awesome experience, though. worth every second.

to this day probably the project i'm most proud of (which is sad, i suppose) - the final version had its own builtin forum section with proper topics, reply threads, wysiwyg post editor, personal diaries where people could set per-post visibility (everyone, only logged in users, only my friends), mental health questionnaires that tracked user's results in time and showed them in a cool flash charts, questionnaire editor where users could make their own tests/quizzes, article section, like/dislike voting on everything, page-global ajax chat of all users that would stay open in bottom right corner, hangouts-style, private messages, even a "pointer" system where sending special commands to the chat aimed at a specific user would cause page elements to highlight on their client, meaning if someone asked "how do i do this thing on the page?", i could send that command and the button to the subpage would get highlighted, after they clicked it and the subpage loaded, the next step in the process would get highlighted, with a custom explanation text, etc...

dammit, now i got seriously nostalgic. it was an awesome piece of work, if i may say so. and i wasn't the only one thinking that, since showing the page off landed me my first two or three programming jobs, right out of highschool. 10 minutes of smalltalk, then they asked about my knowledge, i whipped up that site and gave a short walkthrough talking a bit about how the most interesting pieces were implemented, done, hired XD

those were good times, when I still felt like the programmer whiz kid =D

as i said, worth every second, every drop of sweat, every torn hair, several times over, even though "actual net financial profit" was around minus two hundred euro paid for those two or three years of hosting.

My work product: Or why I learned to get twitchy around Java...

I maintain a Java based test system, that tests a raster image processor. The client is a Java swing project that contains CORBA bindings to the internal API of the raster image processor. It also has custom written UI elements and duplicated functionality that became available in later versions of Java, but because some of the third party tools we use don't work with later versions of Java for some reason, it's not possible to upgrade Java to gain things as simple as recursive directory deletion, yes the version of Java we have to use does not support something as simple as that and custom code had to be written to support it.

Because of the requirement to build the API bindings along with the client the whole application must be built with the raster image processor build chain, which is a heavily customised jam build system. So an ant task calls out to execute a jam task and jam does about 90% of the heavy lifting.

In addition to the Java code there's code for interpreting PostScript files, as these can be used to alter the behaviour of the raster image processor during testing.

As if that weren't enough, there's a beanshell interface to allow users to script the test system, but none of the users know Java well enough to feel confident writing interpreted Java scripts (and that's too close to JavaScript for my comfort). I once tried swapping this out for the Rhino JavaScript interpreter and got all the verbal support in the world but no developer time to design an API that'd work for all the departments.

The server isn't much better though. It's a tomcat based application that was written by someone who had never built a tomcat application before, or any web application for that matter and uses raw SQL strings instead of an orm, it doesn't use MVC in any way, and insane amount of functionality is dumped into the jsp files.

It too interacts with a raster image processor to create difference masks of the output, running PostScript as needed. It spawns off multiple threads and can spend days processing hundreds of gigabytes of image output (depending on the size of the tests).

We're stuck on Tomcat seven because we can't upgrade beyond Java 6, which brings a whole manner of security issues, but that eager little Java updated will break the tool chain if it gets its way.

Between these two components we have the Java RMI server (sometimes) working to help generate image data on the client side before all images are pulled across a UNC network path onto the server that processes test jobs (in PDF format), by reading into the xref table of said PDF, finding the embedded image data (for our server consumed test files are just flate encoded TIFF files wrapped around just enough PDF to make them valid) and uses a tool to create a difference mask of two images.

This tool is very error prone, it can't difference images of different sizes, colour spaces, orientations or pixel depths, but it's the best we have.

The tool is installed in both the client and server if the client can generate images it'll query from the server which ones it needs to and if it can't the server will use the tool itself.

Our shells have custom profiles for linking to a whole manner of third party tools and libraries, including a link to visual studio 2005 (more indirectly related build dependencies), the whole profile has to ensure that absolutely no operating system pollution gets into the shell, most of our apps are installed in our home directories and we have to ensure our paths are correct for every single application we add.

And... Fucking and!

Most of the tools are stored as source bundles in a version control system... Not got or mercurial, not perforce or svn, not even CVS... They use a custom built version control system that is built on top of RCS, it keeps a central database of locked files (using soft and hard locks along with write protecting the files in the file system) to ensure users can't get merge conflicts by preventing other users from writing to the files at all.

Branching is heavy weight and can take the best part of a day to create a new branch and populate the history.

Gathering the tools alone to build the Dev environment to build my project takes the best part of a week.

What should be a joy come hardware refresh year becomes a curse ("Well fuck, now I loose a week spending it setting up the Dev environment on ANOTHER machine").

Needless to say, I enjoy NOT working with Java. A lot of this isn't Javas fault, but there's a lot of things that Java (specifically the Java 6 version we're stuck on) does not make easy.

This is why I prefer to build my web apps in python or node, hell, I'd even take Lua... Just... Compiling web pages into executable Java classes, why? I mean I understand the implementation of how this happens, but why did my predecessor have to choose this? Why?

The perks of being an AI researcher/Data Engineer:

Having to explain what I do for a living at every single family gathering.

What is the most detailed and spy-ish web analytics u know?
I want to do some testing and want to know if anything besides GoogleAnalytics is worth looking..

So I believe in replicated effort.
I like the idea of multiple groups of people creating or engaging in the very same endeavor at any moment as long as intermediaries between these people visit on each project and determine its merits.

Examples could be data gathering, or development or research.

Example maybe being multiple people operating methods of altitude heightmap gathers or canvas imaging and publishing the data.

Or canvassing neighborhoods for new construction.

Or as in my case creating a nice handy dandy file types database for use as lookup table values in my application.

You know what I hate however ?

HAVING TO DO IT ALL THE HELL OVER AGAIN NO MATTER HOW QUICK OF A TASK ONCE I'VE DONE THE TASK ITS LIKELY NOT GOING TO GET MUCH BETTER THE SECOND OR THIRD GODDAMN TIME AROUND !!!!

Hi im writing my research paper and im a bit lost.

My title is:
"File sharing over LAN: An Assessment Study for Future Implimentations"

Im lost at the statement of the problem. Besides the problem of would file transfer be possible (ik it is, but for the sake of the paper it is asked) can you guys think of any other problems?

This is my research paper and i have to make a survey out of it. Everything's done except for this part. Which im lost. Thanks in advance for any future help. :)

Edit:
Im looking for the problem of my research, which is to say; "what problem am i going to solve"

As well as survey questions i could write for when i start data gathering.

You know when I think back to the ideas I've had and the things I've worked on. I'm having difficulty, with the exception of certain far out projects that were like unattainable, in thinking of anything I've done or thought of that does not involve: data visualization, data gathering, encryption/obfuscation, inventory/storage and/or communication.

am I just unimaginative ?

I did have an idea for a code translator and how it would work and what interfaces you'd have to adopt and how you'd attack implementing things that don't translate well like c++ to js for example. or c++ to c# for that matter ! but I never got far into it. though that would have been attainable as long as you had easy ways of generating bindings.

i mean pathing and navigation were things I thought of too but... that would pretty much be implementing someone else's stuff

I have a project in need of machine learning. It takes an image and turns it into text. How do I begin acquiring the data needed to feed the machine? Should I just start taking pictures of this particular item on many different devices and get as many friends to do the same? How do I begin gathering my data is the question?

@theNSA do you have anything to do with this?

For some reason it always bothers me...