Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "parsing"
-
Got this from a recruiter:
We are looking for a **Senior Android Developer/Lead** at Philadelphia PA
Hiring Mode: Contract
Must have skills:
· 10-12 years mobile experience in developing Android applications
· Solid understanding of Android SDK on frameworks such as: UIKit, CoreData, CoreFoundation, Network Programming, etc.
· Good Knowledge on REST Ful API and JSON Parsing
· Good knowledge on multi-threaded environment and grand central dispatch
· Advanced object-oriented programming and knowledge of design patterns
· Ability to write clean, well-documented, object-oriented code
· Ability to work independently
· Experience with Agile Driven Development
· Up to date with the latest mobile technology and development trends
· Passion for software development- embracing every challenge with a drive to solve it
· Engaging communication skills
My response:
I am terribly sorry but I am completely not interested in working for anyone who might think that this is a job description for an Android engineer.
1. Android was released in September 2008 so finding anyone with 10 years experience now would have to be a Google engineer.
2. UIKit, CoreData, CoreFoundation are all iOS frameworks
3. Grand Central Dispatch is an iOS mechanism for multithreading and is not in Android
4. There are JSON parsing frameworks, no one does that by hand anymore
Please delete me from your emailing list.49 -
Oh, man, I just realized I haven't ranted one of my best stories on here!
So, here goes!
A few years back the company I work for was contacted by an older client regarding a new project.
The guy was now pitching to build the website for the Parliament of another country (not gonna name it, NDAs and stuff), and was planning on outsourcing the development, as he had no team and he was only aiming on taking care of the client service/project management side of the project.
Out of principle (and also to preserve our mental integrity), we have purposely avoided working with government bodies of any kind, in any country, but he was a friend of our CEO and pleaded until we singed on board.
Now, the project itself was way bigger than we expected, as the wanted more of an internal CRM, centralized document archive, event management, internal planning, multiple interfaced, role based access restricted monster of an administration interface, complete with regular user website, also packed with all kind of features, dashboards and so on.
Long story short, a lot bigger than what we were expecting based on the initial brief.
The development period was hell. New features were coming in on a weekly basis. Already implemented functionality was constantly being changed or redefined. No requests we ever made about clarifications and/or materials or information were ever answered on time.
They also somehow bullied the guy that brought us the project into also including the data migration from the old website into the new one we were building and we somehow ended up having to extract meaningful, formatted, sanitized content parsing static HTML files and connecting them to download-able files (almost every page in the old website had files available to download) we needed to also include in a sane way.
Now, don't think the files were simple URL paths we can trace to a folder/file path, oh no!!! The links were some form of hash combination that had to be exploded and tested against some king of database relationship tables that only had hashed indexes relating to other tables, that also only had hashed indexes relating to some other tables that kept a database of the website pages HTML file naming. So what we had to do is identify the files based on a combination of hashed indexes and re-hashed HTML file names that in the end would give us a filename for a real file that we had to then search for inside a list of over 20 folders not related to one another.
So we did this. Created a script that processed the hell out of over 10000 HTML files, database entries and files and re-indexed and re-named all this shit into a meaningful database of sane data and well organized files.
So, with this we were nearing the finish line for the project, which by now exceeded the estimated time by over to times.
We test everything, retest it all again for good measure, pack everything up for deployment, simulate on a staging environment, give the final client access to the staging version, get them to accept that all requirements are met, finish writing the documentation for the codebase, write detailed deployment procedure, include some automation and testing tools also for good measure, recommend production setup, hardware specs, software versions, server side optimization like caching, load balancing and all that we could think would ever be useful, all with more documentation and instructions.
As the project was built on PHP/MySQL (as requested), we recommended a Linux environment for production. Oh, I forgot to tell you that over the development period they kept asking us to also include steps for Windows procedures along with our regular documentation. Was a bit strange, but we added it in there just so we can finish and close the damn project.
So, we send them all the above and go get drunk as fuck in celebration of getting rid of them once and for all...
Next day: hung over, I get to the office, open my laptop and see on new email. I only had the one new mail, so I open it to see what it's about.
Lo and behold! The fuckers over in the other country that called themselves "IT guys", and were the ones making all the changes and additions to our requirements, were not capable enough to follow step by step instructions in order to deploy the project on their servers!!!
[Continues in the comments]26 -
Dev: what do I call this file ?
Me: just name it something meaningful so other dev's know what it is
Two days pass
Me: time to do code review .. oh look a new file ..
Git comment : new file for sax parsing , architecture gave the ok.
File name : SomethingMeaningful.java11 -
Started being a Teaching Assistant for Intro to Programming at the uni I study at a while ago and, although it's not entirely my piece of cake, here are some "highlights":
* students were asked to use functions, so someone was ingenious (laughed my ass off for this one):
def all_lines(input):
all_lines =input
return all_lines
* "you need to use functions" part 2
*moves the whole code from main to a function*
* for Math-related coding assignments, someone was always reading the input as a string and parsing it, instead of reading it as numbers, and was incredibly surprised that he can do the latter "I always thought you can't read numbers! Technology has gone so far!"
* for an assignment requiring a class with 3 private variables, someone actually declared each variable needed as a vector and was handling all these 3 vectors as 3D matrices
* because the lecturer specified that the length of the program does not matter, as long as it does its job and is well-written, someone wrote a 100-lines program on one single line
* someone was spamming me with emails to tell me that the grade I gave them was unfair (on the reason that it was directly crashing when run), because it was running on their machine (they included pictures), but was not running on mine, because "my Python version was expired". They sent at least 20 emails in less than 2h
* "But if it works, why do I still have to make it look better and more understandable?"
* "can't we assume the input is always going to be correct? Who'd want to type in garbage?"
* *writes 10 if-statements that could be basically replaced by one for-loop*
"okay, here, you can use a for-loop"
*writes the for loop, includes all the if-statements from before, one for each of the 10 values the for-loop variable gets*
* this picture
N.B.: depending on how many others I remember, I may include them in the comments afterwards19 -
this.title = "gg Microsoft"
this.metadata = {
rant: true,
long: true,
super_long: true,
has_summary: true
}
// Also:
let microsoft = "dead" // please?
tl;dr: Windows' MAX_PATH is the devil, and it basically does not allow you to copy files with paths that exceed this length. No matter what. Even with official fixes and workarounds.
Long story:
So, I haven't had actual gainful employ in quite awhile. I've been earning just enough to get behind on bills and go without all but basic groceries. Because of this, our electronics have been ... in need of upgrading for quite awhile. In particular, we've needed new drives. (We've been down a server for two years now because its drive died!)
Anyway, I originally bought my external drive just for backup, but due to the above, I eventually began using it for everyday things. including Steam. over USB. Terrible, right? So, I decided to mount it as an internal drive to lower the read/write times. Finding SATA cables was difficult, the motherboard's SATA plugs are in a terrible spot, and my tiny case (and 2yo) made everything soo much worse. It was a miserable experience, but I finally got it installed.
However! It turns out the Seagate external drives use some custom drive header, or custom driver to access the drive, so Windows couldn't read the bare drive. ffs. So, I took it out again (joy) and put it back in the enclosure, and began copying the files off.
The drive I'm copying it to is smaller, so I enabled compression to allow storing a bit more of the data, and excluded a couple of directories so I could copy those elsewhere. I (barely) managed to fit everything with some pretty tight shuffling.
but. that external drive is connected via USB, remember? and for some reason, even over USB3, I was only getting ~20mb/s transfer rate, so the process took 20some hours! In the interim, I worked on some projects, watched netflix, etc., then locked my computer, and went to bed. (I also made sure to turn my monitors and keyboard light off so it wouldn't be enticing to my 2yo.) Cue dramatic music ~
Come morning, I go to check on the progress... and find that the computer is off! What the hell! I turn it on and check the logs... and found that it lost power around 9:16am. aslkjdfhaslkjashdasfjhasd. My 2yo had apparently been playing with the power strip and its enticing glowing red on/off switch. So. It didn't finish copying.
aslkjdfhaslkjashdasfjhasd x2
Anyway, finding the missing files was easy, but what about any that didn't finish? Filesizes don't match, so writing a script to check doesn't work. and using a visual utility like windirstat won't work either because of the excluded folders. Friggin' hell.
Also -- and rather the point of this rant:
It turns out that some of the files (70 in total, as I eventually found out) have paths exceeding Windows' MAX_PATH length (260 chars). So I couldn't copy those.
After some research, I learned that there's a Microsoft hotfix that patches this specific issue! for my specific version! woo! It's like. totally perfect. So, I installed that, restarted as per its wishes... tried again (via both drag and `copy`)... and Lo! It did not work.
After installing the hotfix. to fix this specific issue. on my specific os. the issue remained. gg Microsoft?
Further research.
I then learned (well, learned more about) the unicode path prefix `\\?\`, which bypasses Windows kernel's path parsing, and passes the path directly to ntfslib, thereby indirectly allowing ~32k path lengths. I tried this with the native `copy` command; no luck. I tried this with `robocopy` and cygwin's `cp`; they likewise failed. I tried it with cygwin's `rsync`, but it sees `\\?\` as denoting a remote path, and therefore fails.
However, `dir \\?\C:\` works just fine?
So, apparently, Microsoft's own workaround for long pathnames doesn't work with its own utilities. unless the paths are shorter than MAX_PATH? gg Microsoft.
At this point, I was sorely tempted to write my own copy utility that calls the internal Windows APIs that support unicode paths. but as I lack a C compiler, and haven't coded in C in like 15 years, I figured I'd try a few last desperate ideas first.
For the hell of it, I tried making an archive of the offending files with winRAR. Unsurprisingly, it failed to access the files.
... and for completeness's sake -- mostly to say I tried it -- I did the same with 7zip. I took one of the offending files and made a 7z archive of it in the destination folder -- and, much to my surprise, it worked perfectly! I could even extract the file! Hell, I could even work with paths >340 characters!
So... I'm going through all of the 70 missing files and copying them. with 7zip. because it's the only bloody thing that works. ffs
Third-party utilities work better than Microsoft's official fixes. gg.
...
On a related note, I totally feel like that person from http://xkcd.com/763 right now ;;21 -
I'm convinced code addiction is a real problem and can lead to mental illness.
Dev: "Thanks for helping me with the splunk API. Already spent two weeks and was spinning my wheels."
Me: "I sent you the example over a month ago, I guess you could have used it to save time."
Dev: "I didn't understand it. I tried getting help from NetworkAdmin-Dan, SystemAdmin-Jake, they didn't understand what you sent me either."
Me: "I thought it was pretty simple. Pass it a query, get results back. That's it"
Dev: "The results were not in a standard JSON format. I was so confused."
Me: "Yea, it's sort-of JSON. Splunk streams the result as individual JSON records. You only have to deserialize each record into your object. I sent you the code sample."
Dev: "Your code didn't work. Dan and Jake were confused too. The data I have to process uses a very different result set. I guess I could have used it if you wrote the class more generically and had unit tests."
<oh frack...he's been going behind my back and telling people smack about my code again>
Me: "My code wouldn't have worked for you, because I'm serializing the objects I need and I do have unit tests, but they are only for the internal logic."
Dev:"I don't know, it confused me. Once I figured out the JSON problem and wrote unit tests, I really started to make progress. I used a tuple for this ... functional parameters for that...added a custom event for ... Took me a few weeks, but it's all covered by unit tests."
Me: "Wow. The way you explained the project was; get data from splunk and populate data in SQLServer. With the code I sent you, sounded like a 15 minute project."
Dev: "Oooh nooo...its waaay more complicated than that. I have this very complex splunk query, which I don't understand, and then I have to perform all this parsing, update a database...which I have no idea how it works. Its really...really complicated."
Me: "The splunk query returns what..4 fields...and DBA-Joe provided the upsert stored procedure..sounds like a 15 minute project."
Dev: "Maybe for you...we're all not super geniuses that crank out code. I hope to be at your level some day."
<frack you ... condescending a-hole ...you've got the same seniority here as I do>
Me: "No seriously, the code I sent would have got you 90% done. Write your deserializer for those 4 fields, execute the stored procedure, and call it a day. I don't think the effort justifies the outcome. Isn't the data for a report they'll only run every few months?"
Dev: "Yea, but Mgr-Nick wanted unit tests and I have to follow orders. I tried to explain the situation, but you know how he is."
<fracking liar..Nick doesn't know the difference between a unit test and breathalyzer test. I know exactly what you told Nick>
Dev: "Thanks again for your help. Gotta get back to it. I put a due date of April for this project and time's running out."
APRIL?!! Good Lord he's going to drag this intern-level project for another month!
After he left, I dug around and found the splunk query, the upsert stored proc, and yep, in about 15 minutes I was done.1 -
The time my Java EE technology stack disappointed me most was when I noticed some embarrassing OutOfMemoryError in the log of a server which was already in production. When I analyzed the garbage collector logs I got really scared seeing the heap usage was constantly increasing. After some days of debugging I discovered that the terrible memory leak was caused by a bug inside one of the Java EE core libraries (Jersey Client), while parsing a stupid XML response. The library was shipped with the application server, so it couldn't be replaced (unless installing a different server). I rewrote my code using the Restlet Client API and the memory leak disapperead. What a terrible week!2
-
PineScript is absolute garbage.
It's TradingView's scripting language. It works, but it's worse than any language I have ever seen for shoddy parsing. Its naming conventions are pretty terrible, too:
transparency? no, "transp"
sum? no, cum. seriously. cum(array) is its "cumulative sum."
There are other terrible names, but the parser is what really pisses me off.
1) If you break up a long line for readability (e.g. a chained ternary), each fragment needs to be indented by more than its parent... but never by a multiple of 4 spaces because then it isn't a fragment anymore, but its own statement.
2) line fragments also cannot end in comments because comments are considered to be separate lines.
3) Lambdas can only be global. They're just fancy function declarations. Someone really liked the "blah(x,y,z) =>" syntax
4) blocks to `if`s must be on separate lines, meaning `if (x) y:=z` is illegal. And no, there are no curly braces, only whitespace.
There are plenty more, but the one that really got me furious is:
98) You cannot call `plot()`, `plotshape()`, etc. if they're indented! So if you're using non-trivial logic to optionally plot things like indicators, fuck you.
Whoever wrote this language and/or parser needs to commit seppuku.rant or python? pinescript or fucking euphoria? or ruby? why can't they just use lua? or javascript? tradingview17 -
I send a PR to your GitHub repo.
You close it without a word.
I tell you that your lib crashes because you're trying to parse JavaScript with a (bad) regex, but you keep insisting that no, there exist no problem, and even if you barely know what "parsing" means, you keep denying in front of the evidence.
Well fuck you and your shitty project. I'll keep using my fucking fork.
And if you're reading this, well, fuck you twice. Moron.10 -
Waisting some times on codewars.com
~~~~
3 kyu challenge:
Given a string with mathematical operations like this: ‘3+5*7*(10-45)’, compute the result
~~~~~
*Does a quick and easy one liner in python using eval()*
*sees people actually writing some 100 lines parsing the string and calculating using priority of operation*
Poor them...
(Btw, passed to lvl 4 kyu thx to this)14 -
Less a rant, more just a sad story.
Our company recently acquired its sister company, and everyone has been focused on improving and migrating their projects over to our stack.
There's a ton of material there, but this one little story summarizes the whole very accurately, I think. (Edit: two stories. I couldn't resist.)
There's a 3-reel novelty slot machine game with cards instead of the usual symbols, and winnings based on poker-like rules (straights and/or flushes, 2-3 of a kind, etc.) The machine is over a hundred times slower than the other slot machines because on every spin it runs each payline against a winnings table that exhastively lists every winning possibility, and I really do mean exhaustively. It lists every type of win, for every card, every segment for straights, in every order, of every suit. Absolutely everything.
And this logic has been totally acceptable for just. so. long. When I saw someone complaining in dev chat about how much slower it is, i made the bloody obvious suggestion of parsing the cards and applying some minimal logic to see if it's a winning combination. Nobody cared.
Ten minutes later, someone from the original project was like "Hey, I have an idea, why don't we do it algorithmically to not have a 4k line rewards table?"
He seriously tried stealing a really bloody obvious idea -- that he hadn't had for years prior -- and passing it off as his own. In the same chat. Eight messages below mine. What a derpballoon.
I called him out on it, and he was like "Oh, is that what you meant by parsing?" 🙄
Someone else leaped in to defend the ~128x slower approach, saying: "That's the tech we had." You really didn't have a for loop and a handful of if statements? Oh wait, you did, because that's how you're checking your exhaustive list. gfj. Abysmal decisions like this is exactly why most of you got fired. (Seriously: these same people were making devops decisions. They were hemorrhaging money.)
But regardless, the quality of bloody everything from that sister company is like this. One of the other fiascos involved pulling data from Facebook -- which they didn't ever even use -- and instead of failing on error/unexpected data, it just instantly repeated. So when Facebook changed permissions on friends context... you can see where this is going. Instead of their baseline of like 1400 errors per day, which is amazingly high, it spiked to EIGHTEEN BLOODY MILLION PER DAY. And they didn't even care until they noticed (like four days later) that it was killing their other online features because quite literally no other request could make it out. More reasons they got fired. I'm not even kidding: no single api request ever left the users' devices apart from the facebook checks.
So.
That's absolutely amazing.8 -
Imagine, you get employed to restart a software project. They tell you, but first we should get this old software running. It's 'almost finished'.
A WPF application running on a soc ... with a 10" touchscreen on win10, a embedded solution, to control a machine, which has been already sold to customers. You think, 'ok, WTF, why is this happening'?
You open the old software - it crashes immediately.
You open it again but now you are so clever to copy an xml file manually to the root folder and see all of it's beauty for the first time (after waiting for the freezed GUI to become responsive):
* a static logo of the company, taking about 1/5 of the screen horizontally
* circle buttons
* and a navigation interface made in the early 90's from a child
So you click a button and - it crashes.
You restart the software.
You type something like 'abc' in a 'numberfield' - it crashes.
OK ... now you start the application again and try to navigate to another view - and? of course it crashes again.
You are excited to finally open the source code of this masterpiece.
Thank you jesus, the 'dev' who did this, didn't forget to write every business logic in the code behind of the views.
He even managed to put 6 views into one and put all their logig in the code behind!
He doesn't know what binding is or a pattern like MVVM.
But hey, there is also no validation of anything, not even checks for null.
He was so clever to use the GUI as his place to save data and there is a lot of parsing going on here, every time a value changes.
A thread must be something he never heard about - so thats why the GUI always freezes.
You tell them: It would be faster to rewrite the whole thing, because you wouldn't call it even an alpha. Nobody listenes.
Time passes by, new features must be implemented in this abomination, you try to make the cripple walk and everyone keeps asking: 'When we can start the new software?' and the guy who wrote this piece of shit in the first place, tries to give you good advice in coding and is telling you again: 'It was almost finished.' *facepalm*
And you? You would like to do him and humanity a big favour by hiting him hard in the face and breaking his hands, so he can never lay a hand on any keyboard again, to produce something no one serious would ever call code.4 -
Ok, rant incoming.
Dates. Frigging dates. Apparently we as a species are so bloody incompetent we cannot even decide on a one format for how to write today. No, instead we have one for every language and framework, because every moron thinks they know better how to write the date. All of them equivalent and all of them different enough to make me start lactating out of frustration trying to parse this garbage... And when you finally manage to parse it on one platform it turns out that your ORM just decided to use the less common version of the date, and have fun converting one to the other. I hope that ever time someone comes up with a new date format will be hit in a face with a red hot frying pan untill they give up programming in favour of growing cactuses.12 -
When I found out my JSON didn't parse because I used single instead of double quotes after two hours8
-
Interviewer : So what frameworks and library you usually use?
Me : i use volley for networking, gson for parsing, livedata/architecture components for architecture and observability , room for database and java for app development
I : ok so make this sample app using retrofit for networking, moshi for parsing, mvrx for architecture , rx for observability , sqldelight for db, dagger2 and kotlin for app dev. You have 8 hours
Me :(wtf?) But i never used those libs or language!
I : we just want to check how easily you adapt to different surroundings.
Me : -_-
Honestly i don't know of it was a great experience or a bad one . I was stressed the whole time but was able to adapt to almost all of those libraries and frameworks.
At the end i got selected but decided not to go for those ppl. That was just a lucrative opening of a venus fly trap, they would have stressed the hell out of me11 -
Wrote a rant yesterday (or recently?) in which I explained that I needed to rewrite the core of something I'm writing in order to make it more extendable/flexible/modular.
Finished the rewrite yesterday and started to write a module (it exists our of modules and one can write a specific module for a specific kind of task) for another use case that has shown itself in the past few weeks.
Fun thing is that part of the core stayed the same and I hardly made changes to the libraries which the core uses a lot but the modules are, except for a few similarities (like one default invoke methods), completely different but do use the libraries to make sure they've got all functions needed to properly fulfill their task.
Ran a rule (what I call something in the project) hoping that everything works together the way it's supposed to and that the config files are interpreted well by the parsing 'engine' (pretty much switch cases and if-elses 😅).
FUCKING BAM IT FUCKING WORKS 😍2 -
Recently saw a piece of code left by a former coworker where he converted an int to a long, by converting it to a string, then parsing the string.5
-
No comments allowed in JSON pisses me off so much.
Sure, I get all the arguments of "it's supposed to be a data-only format for machines", "there are alternatives which support comments", and "you can add comments and then minify the file before parsing"
But right now, when I just need to put a quick note inside a super confusing legacy package manager config about why certain dependencies to be frozen at a specific version, IT FUCKING PISSES ME OFF THAT I CAN'T JUST ADD A FUCKING COMMENT.18 -
Lead dev: Hey boss, you really do like Python right?
Me: No
Lead dev: Well it's cuz I was think....wait what? WTF do you mean no, you have automated a fuckload of BS with Python and we are still using it, why tf would you use Python if you don't like it?
Me: I like it enough for the automation scripts that we have and for parsing documents or generating glue scripts, its already installed in every server that we have, so testing bs in dev and then using them in prod is cake, it doesn't mean I LOVE python, I like it for what we use it.
Lead dev: Well ain't already bash and perl installed as well?
Me: Do you know bash and or perl?
Lead dev: No, don't you?....
Me: No......
L Dev: (using a Jim Carrey impersonation) WELLL ALLRIGTHY THEN! What is the other language that you used for X project?
Me: Clojure, do you remember that one?
* he said paren paren paren paren yes paren i space paren do close paren close paren etc etc
L Dev: (((((((yes (i (do)))))))) and nevermind, I'll get back to working more with Python
Me: das what I fucking thought esse6 -
full stack web development in 2017: mvvm model, api for backend, parsing on frontend
me: <?php echo "<div>hello".$name."</div>" ?>9 -
I fucking hate it when developers don’t respect user locale. My phone language is UK English, my Discord app language is UK English and my region is UK.
Then why the fuck is Discord showing me MM/DD/YYYY date format? How hard is it to pass locale when parsing time?19 -
TLDR; Wrote an awesome piece of code, but there's no one capable of understanding how hard it is.
I spent an entire night building an insanely complicated automation script, that picks out certain configuration files (in javascript fml), does some crazy parsing to pick out strings, passes them into a free translation API, translates them, and does more insane parsing to insert those strings as javascript objects.
Spent 3 hours on the bloody parsing algorithm alone.
Manager: Oh this language is really nice. Good of you to discover it can do that.
I didn't "DISCOVER" it ffs! Its a product of my head! I built the damn algo from scratch.
Seriously, screw non devs who trivialize the complexities of writing a good program. Its NOT as simple as opening notepad and typing in
import {insanelyComplexSolution} from 'daveOnTheInternet'3 -
the first program ive written in my own programming language
schreibe
{
hallo welt
}
i didnt know about parsing back then, so everything was its own line4 -
Dear external developer dumbass from hell.
We bought your company under the assumption you had a borderline functioning product and/or dev team. Ideally both
For future reference expect "file path" arguments can contain backslashes and perhaps even the '.' character. It ain't that hard. Maybe try using the damn built in path parsing capabilities every halfway decent programming environment has had since before you figured out how to smash your head against the keyboard hard enough for your shitty excuse of a compiler stops arguing and gives in.
I am fixing your shit by completely removing it with one line of code calling the framework and you better not reject this.
This is not a pull request ITS A GOD DAMN PULL COMMAND.
- Is what i would _like_ to say right now... you know if i wouldn't be promptly fired for doing so :p
How's you guys friday going?8 -
Super excited to get my first ML project on, been working on this on - off for a couple of years. Now that I had sometime to spare could get it completed :) And along the way learned a lot of things as well :)
Let me know what you guys think.
https://github.com/Prabandham/...
https://github.com/Prabandham/...
A lot of thanks to this Repo that helped in parsing the image to string.
https://github.com/BenNG/...
Of course PR's and suggestions really welcome and appreciated.9 -
A lot of engineering fads go in circle.
Architecture in the 80s: Mainframe and clients.
Architecture in the 90s: Software systems connected by an ESB.
Architecture in the 2000s: Big central service and everyone connects to it for everything
Architecture in the 2010s: Decentralized microservices that communicate with queues.
Current: RabbitMQ and Kafka.
... Can't we just go back to the 90s?
I hate fads.
I hate when I have to get some data, and it's scattered on 20 different servers, and to load a fucking account page, a convoluted network of 40 apps have to be activated, some in PHP, others in JS, others on Java, that are developed by different teams, connected to different tiny ass DBs, all on huge clusters of tiny ass virtual machines that get 30% load at peak hours, 90% of which comes from serializing and parsing messages. 40 people maintaining this nightmare, that could've been just 7 people making a small monolithic system that easily handles this workload on a 4-core server with 32GB of RAM.
Tripple it, put it behind a load balancer, proper DB replication (use fucking CockroachDB if you really want survivability), and you've got zero downtime at a fraction of the cost.
Just because something's cool now, doesn't mean that everybody has to blindly follow it for fucks sake!
Same rant goes for functional vs OOP and all that crap. Going blindly with any of these is just a stupid fad, and the main reason why companies need refactoring of legacy code.12 -
Do you wanna build a Program?
Come on it can be in C
Doesn't compile anymore
bugs to report
Perhaps a parsing tree
We used to be peer buddies
And now we're not
I wish you would tell me why!-
Do you wanna build a Program?
It doesn't have to be in C.
Stackoverflow, Anna
Okay, bye2 -
Woke up at 5am with the realization that I could use regular expressions to parse the string representation of regular expressions to build this program to parse regular expressions into more human-readable English.
I am so tired.7 -
A dev team has been spending the past couple of weeks working on a 'generic rule engine' to validate a marketing process. The “Buy 5, get 10% off” kind of promotions.
The UI has all the great bits, drop-downs, various data lookups, etc etc..
What the dev is storing the database is the actual string representation FieldA=“Buy 5, get 10% off” that is “built” from the UI.
Might be OK, but now they want to apply that string to an actual order. Extract ‘5’, the word ‘Buy’ to apply to the purchase quantity rule, ‘10%’ and the word ‘off’ to subtract from the total.
Dev asked me:
Dev: “How can I use reflection to parse the string and determine what are integers, decimals, and percents?”
Me: “That sounds complicated. Why would you do that?”
Dev: “It’s only a string. Parsing it was easy. First we need to know how to extract numbers and be able to compare them.”
Me: “I’ve seen the data structures, wouldn’t it be easier to serialize the objects to JSON and store the string in the database? When you deserialize, you won’t have to parse or do any kind of reflection. You should try to keep the rule behavior as simple as possible. Developing your own tokenizer that relies on reflection and hoping the UI doesn’t change isn’t going to be reliable.”
Dev: “Tokens!...yea…tokens…that’s what we want. I’ll come up with a tokenizing algorithm that can utilize recursion and reflection to extract all the comparable data structures.”
Me: “Wow…uh…no, don’t do that. The UI already has to map the data, just make it easy on yourself and serialize that object. It’s like one line of code to serialize and deserialize.”
Dev: “I don’t know…sounds like magic. Using tokens seems like the more straightforward O-O approach. Thanks anyway.”
I probably getting too old to keep up with these kids, I have no idea what the frack he was talking about. Not sure if they are too smart or I’m too stupid/lazy. Either way, I keeping my name as far away from that project as possible.4 -
I’m on this ticket, right? It’s adding some functionality to some payment file parser. The code is atrocious, but it’s getting replaced with a microservice definitely-not-soon-enough, so i don’t need to rewrite it or anything, but looking at this monstrosity of mental diarrhea … fucking UGH. The code stink is noxious.
The damn thing reads each line of a csv file, keeping track of some metadata (blah blah) and the line number (which somehow has TWO off-by-one errors, so it starts on fucking 2 — and yes, the goddamn column headers on line #0 is recorded as line #2), does the same setup shit on every goddamned iteration, then calls a *second* parser on that line. That second parser in turn stores its line state, the line number, the batch number (…which is actually a huge object…), and a whole host of other large objects on itself, and uses exception throwing to communicate, catches and re-raises those exceptions as needed (instead of using, you know, if blocks to skip like 5 lines), and then writes the results of parsing that one single line to the database, and returns. The original calling parser then reads the data BACK OUT OF THE DATABASE, branches on that, and does more shit before reading the next line out of the file and calling that line-parser again.
JESUS CHRIST WHAT THE FUCK
And that’s not including the lesser crimes like duplicated code, misleading var names, and shit like defining class instance constants but … first checking to see if they’re defined yet? They obviously aren’t because they aren’t anywhere else in the fucking file!
Whoever wrote this pile of fetid muck must have been retroactively aborted for their previous crimes against intelligence, somehow survived the attempt, and is now worse off and re-offending.
Just.
Asdkfljasdklfhgasdfdah28 -
Definitely Rust, and a bit Haskell.
Rust has made me much more conscious of data ownership through a program, to the point that any C/C++ function I wrote that takes a pointer nowadays gets a comment on ownership.
I wish it was a bit less pedantic about generics sometimes, which is why I've started working on a "less pedantic rust", where generics are done through multiple dispatch à la Julia, but still monomorphising everything I can. I've only started this week, but I already have a tokenizer and most of the type inference system (an SLD tree) ready. Next up is the borrow checker and parsing the tokenized input to whatever the type inference and borrow checker need to work with, and of course actual code generation...
Haskell is my first FP language, and introduced me to some FP patterns which, turns out, are super useful even with less FP languages. -
I am much too tired to go into details, probably because I left the office at 11:15pm, but I finally finished a feature. It doesn't even sound like a particularly large or complicated feature. It sounds like a simple, 1-2 day feature until you look at it closely.
It took me an entire fucking week. and all the while I was coaching a junior dev who had just picked up Rails and was building something very similar.
It's the model, controller, and UI for creating a parent object along with 0-n child objects, with default children suggestions, a fancy ui including the ability to dynamically add/remove children via buttons. and have the entire happy family save nicely and atomically on the backend. Plus a detailed-but-simple listing for non-technicals including some absolutely nontrivial css acrobatics.
After getting about 90% of everything built and working and beautiful, I learned that Rails does quite a bit of this for you, through `accepts_nested_params_for :collection`. But that requires very specific form input namespacing, and building that out correctly is flipping difficult. It's not like I could find good examples anywhere, either. I looked for hours. I finally found a rails tutorial vide linked from a comment on a SO answer from five years ago, and mashed its oversimplified and dated examples with the newer documentation, and worked around the issues that of course arose from that disasterous paring.
like.
I needed to store a template of the child object markup somewhere, yeah? The video had me trying to store all of the markup in a `data-fields=" "` attrib. wth? I tried storing it as a string and injecting it into javascript, but that didn't work either. parsing errors! yay! good job, you two.
So I ended up storing the markup (rendered from a rails partial) in an html comment of all things, and pulling the markup out of the comment and gsubbing its IDs on document load. This has the annoying effect of preventing me from using html comments in that partial (not that i really use them anyway, but.)
Just.
Every step of the way on building this was another mountain climb.
* singular vs plural naming and routing, and named routes. and dealing with issues arising from existing incorrect pluralization.
* reverse polymorphic relation (child -> x parent)
* The testing suite is incompatible with the new rails6. There is no fix. None. I checked. Nope. Not happening.
* Rails6 randomly and constantly crashes and/or caches random things (including arbitrary code changes) in development mode (and only development mode) when working with multiple databases.
* nested form builders
* styling a fucking checkbox
* Making that checkbox (rather, its label and container div) into a sexy animated slider
* passing data and locals to and between partials
* misleading documentation
* building the partials to be self-contained and reusable
* coercing form builders into namespacing nested html inputs the way Rails expects
* input namespacing redux, now with nested form builders too!
* Figuring out how to generate markup for an empty child when I'm no longer rendering the children myself
* Figuring out where the fuck to put the blank child template markup so it's accessible, has the right namespacing, and is not submitted with everything else
* Figuring out how the fuck to read an html comment with JS
* nested strong params
* nested strong params
* nested fucking strong params
* caching parsed children's data on parent when the whole thing is bloody atomic.
* Converting datetimes from/to milliseconds on save/load
* CSS and bootstrap collisions
* CSS and bootstrap stupidity
* Reinventing the entire multi-child / nested params / atomic creating/updating/deleting feature on my own before discovering Rails can do that for you.
Just.
I am so glad it's working.
I don't even feel relieved. I just feel exhausted.
But it's done.
finally.
and it's done well. It's all self-contained and reusable, it's easy to read, has separate styling and reusable partials, etc. It's a two line copy/paste drop-in for any other model that needs it. Two lines and it just works, and even tells you if you screwed up.
I'm incredibly proud of everything that went into this.
But mostly I'm just incredibly tired.
Time for some well-deserved sleep.7 -
[Begin Rant] When you show your senior manager your REST Web Service and he says "Oh no nooo... I don't wanna see no code"... Me: Code?? That ain't code you fat silly fucker it's the command line output data which I spent a week parsing, batch processing, and storing into the database! [End Rant] :[4
-
When you're parsing SO's questions from your cellphone, you find one that's easy, you answer it, then when it's done, you read the question again to realized you totally missed the point.
When you realize that the StackExchange app doesn't allow you to delete your own comment, so you have to rush to the nearest computer to delete your answer before anyone downvotes it.
When you sit back again to resume your breakfast, the adrenaline rush slowly fading away.2 -
My Unreal project right now:
FVector
std::vector
Eigen::Vector
KDL::Vector
....
*sigh*
And people ask me why I like Lisp. A Vector is a Vector goddamit! Half my project is now just parsing the same variable to the different functions. sigh....I hate this.6 -
Should I actually look into getting a dev job..?
*I have a high school diploma (graduated three years early)
*College dropout (3-4 months, Computer Science - Personal Reasons)
*No prior work experience.
*Good textural communication skills, poor verbal communication skills.
*Currentally unemployed. (NEET :P)
*I have extensive personal experience with Java, and Python. Some Lua. Knowledge of data generation, parsing, Linux, Windows, Terminal(cmd & bash), & Encryption(Ciphers).
*Math, but very little algebra/geometry (though, could easily improve these).
*Work best under preasure.
Remote only.
Think anyone would hire me..?13 -
Imagine saving Integers and Floats in a MySQL table as strings containing locale based thousand sepatators...
man... fickt das hart!
Wait, there's more!
Imagine storing a field containing list of object data as a CSV in a single table column instead of using JSON format or a separate DB table.... and later parsing it by splitting the CSV string on ";"...7 -
I spent over a decade of my life working with Ada. I've spent almost the same amount of time working with C# and VisualBasic. And I've spent almost six years now with F#. I consider all of these great languages for various reasons, each with their respective problems. As these are mostly mature languages some of the problems were only knowable in hindsight. But Ada was always sort of my baby. I don't really mind extra typing, as at least what I do, reading happens much more than writing, and tab completion has most things only being 3-4 key presses irl. But I'm no zealot, and have been fully aware of deficiencies in the language, just like any language would have. I've had similar feelings of all languages I've worked with, and the .NET/C#/VB/F# guys are excellent with taking suggestions and feedback.
This is not the case with Ada, and this will be my story, since I've no longer decided anonymity is necessary.
First few years learning the language I did what anyone does: you write shit that already exists just to learn. Kept refining it over time, sometimes needing to do entire rewrites. Eventually a few of these wound up being good. Not novel, just good stuff that already existed. Outperforming the leading Ada company in benchmarks kind of good. At the time I was really gung-ho about the language. Would have loved to make Ada development a career. Eventually build up enough of this, as well as a working, but very bad performing compiler, and decide to try to apply for a job at this company. I wasn't worried about the quality of the compiler, as anyone who's seriously worked with Ada knows, the language is remarkably complex with some bizarre rules in dark corners, so a compiler which passes the standards test indicates a very intimate knowledge of the language few can attest to.
I get told they didn't think I would be a good fit for the job, and that they didn't think I should be doing development.
A few months of rapid cycling between hatred and self loathing passes, and then a suicide attempt. I've got past problems which contributed more so than the actual job denial.
So I get better and start working even harder on my shit. Get the performance of my stuff up even better. Don't bother even trying to fix up the compiler, and start researching about text parsing. Do tons of small programs to test things, and wind up learning a lot. I'm starting to notice a lot of languages really surpassing Ada in _quality of life_, with things package managers and repositories for those, as well as social media presence and exhaustive tutorials from the community.
At the time I didn't really get programming language specific package managers (I do now), but I still brought this up to the community. Don't do that. They don't like new ideas. Odd for a language which at the time was so innovative. But social media presence did eventually happen with a Twitter account that is most definitely run by a specific Ada company masquerading as a general Ada advocate. It did occasionally draw interest to neat things from the community, so that's cool.
Since I've been using both VisualStudio and an IDE this Ada company provides, I saw a very jarring quality difference over the years. I'm not gonna say VS is perfect, it's not. But this piece of shit made VS look like a polished streamlined bug free race car designed by expert UX people. It. Was. Bad. Very little features, with little added over the years. Fast forwarding several years, I can find about ten bugs in five minutes each update, and I can't find bugs in the video games I play, so I'm no bug finder. It's just that bad. This from a company providing software for "highly reliable systems"...
So I decide to take a crack at writing an editor extension for VS Code, which I had never even used. It actually went well, and as of this writing it has over 24k downloads, and I've received some great comments from some people over on Twitter about how detailed the highlighting is. Plenty of bespoke advertising the entire time in development, of course.
Never a single word from the community about me.
Around this time I had also started a YouTube channel to provide educational content about the language, since there's very little, except large textbooks which aren't right for everyone. Now keep in mind I had written a compiler which at least was passing the language standards test, so I definitely know the language very well. This is a standard the programmers at these companies will admit very few people understand. YouTube channel met with hate from the community, and overwhelming thanks from newcomers. Never a shout out from the "community" Twitter account. The hate went as far as things like how nothing I say should be listened to because I'm a degenerate Irishman, to things like how the world would have been a better place if I was successful in killing myself (I don't talk much about my mental illness, but it shows up).
I'm strictly a .NET developer now. All code ported.5 -
Many of you who have a Windows computer may be familiar with robocopy, xcopy, or move.
These functions? Programs? Whatever they may be, were interesting to me because they were the first things that got me really into batch scripting in the first place.
What was really interesting to me was how I could run multiples of these scripts at a time.
<storytime>
It was warm Spring day in the year of 2007, and my Science teacher at the time needed a way to get files from the school computer to the hard-drive faster. The amount of time that the computer was suggesting was 2 hours. Far too long for her. I told her I’d build her something that could work faster than that. And so started the program would take up more of my time than the AI I had created back in 2009.
</storytime>
This program would scan the entirety of the computer's file system, and create an xcopy batch file for each of these directories. After parsing these files, it would then run all the batch files at once. Multithreading as it were? Looking back on it, the throughput probably wasn't any better than the default copying program windows already had, but the amount of time that it took was less. Instead of 2 hours to finish the task it took 45 minutes. My thought for justifying this program was that; instead of giving one man to do paperwork split the paperwork among many men. So, while a large file is being copied, many smaller files could be copied during that time.
After that day I really couldn't keep my hands off this program. As my knowledge of programming increased, so did my likelihood of editing a piece of the code in this program.
The surmountable amount of updates that this program has gone through is amazing. At version 6.25 it now sits as a standalone batch file. It used to consist of 6 files and however many xcopy batch files that it created for the file migration, now it's just 1 file and dirt simple to run, (well front-end, anyways, the back-end is a masterpiece of weirdness, honestly) it automates adding all the necessary directories and files. Oh, and the name is Latin for Imitate, figured it's a reasonable name for a copying program.
I was 14, so my creativity lacked in the naming department >_<1 -
Let's talk about input forms.
Please don't do that!
Setting time with the arrow buttons in the UI.
Each time change causes the page to re-fetch all the records. This is fetching, parsing, rendering tens of thousands of entries on every single click. And I want to set a very specific time, so there's gonna be a lot of gigs of traffic wasted for /dev/null.
Do I hear you say "just type the date manually you dumbass!"? I would indeed be a dumbass if I didn't try that. You know what? Typing the date in manually does nothing. Apparently, the handler is not triggered if I type it in manually/remove the focus/hit enter/try to jump on 1 leg/draw a blue triangle in my notebook/pray3 -
I am currently in a bit of a (well-deserved) lull at work, both of my projects are finishing up/ finished, so tomorrow should be pretty light, as the latter half of today was.
And I have really gotten interested in the HTTP protocol. It's so interesting learning how it all works under the hood.
So I think I'm going to be researching/ messing around with creating a cpp project that essentially implements cURL from the ground up, creating sockets, reading from them, parsing the HTTP requests... all that. I don't expect to actually get it done, but it should be an immense learning experience. I have a clear goal: implement this function:
std::string get(const std::string&);
Once I'm able to just GET as simple as that, I know I have achieved my goal!3 -
I made a functional parsing layer for an API that cleans http body json. The functions return insights about the received object and the result of the parse attempt. Then I wrote validation in the controller to determine if we will reject or accept. If we reject, parse and validation information is included on the error response so that the API consumer knows exactly why it was rejected. The code was super simple to read and maintain.
I demoed to the team and there was one hold out that couldn’t understand my decision to separate parse and validate. He decided to rewrite the two layers plus both the controller and service into one spaghetti layer. The team lead avoided conflict at all cost and told me that even though it was far worse code to “give him this”. We still struggle with the spaghetti code he wrote to this day.
When sugar-coating someone’s engineering inadequacies is more important than good engineering I think about quitting. He was literally the only one on the team that didn’t get it.2 -
Today's project was answering the question: "Can I update tables in a Microsoft Word document programmatically?"
(spoiler: YES)
My coworker got the ball rolling by showing that the docx file is just a zip archive with a bunch of XML in it.
The thing I needed to update were a pair of tables. Not knowing anything about Word's XML schema, I investigated things like:
- what tag is the table declared with?
- is the table paginated within the table?
- where is the cell background color specified?
Fortunately this wasn't too cumbersome.
For the data, CSV was the obvious choice. And I quickly confirmed that I could use OpenCSV easily within gradle.
The Word XML segments were far too verbose to put into constants, so I made a series of templates with tokens to use for replacement.
In creating the templates, I had to analyze the word xml to see what changed between cells (thankfully, very little). This then informed the design of the CSV parsing loops to make sure the dynamic stuff got injected properly.
I got my proof of concept working in less than a day. Have some more polishing to do, but I'm pretty happy with the initial results!6 -
"I don't have any issues with that (OS, code push, software, etc), so I'm not sure why you are having issues." I love reading comments like that, as if that solves the issue. The fuck does it matter if you don't have issues, they are having issues you moron. For a place for developers you'd think they'd be able to think logically, maybe they're in the wrong line of business. Maybe that software had an error parsing some file because some bit got flipped when writing to the HDD because there's an issue with the drive and the ECC failed for some reason, who cares. There's an issue and you saying it works for you makes you sound like a fucking moron... There's an error/crash, it happens, that's software.4
-
If you didn't think NodeJS dependency hell was that bad, you should try sequentially parsing a graph that's stored as an array of nodes and their references, where processing of said nodes forces you to use some async functions that depend on other async functions.
What should have been 20 lines of code written in 30 minutes has turned into 3 hours of horror, reading about babel, realizing that it's just adding more problems without solving one, assessing the effort of modification of async libraries to include sync methods as well, trying out asyncwait, async, and everything else there is, trying to rethink the recursive algorithm, rewriting it several times, cursing and hating myself for not choosing to use Python or .NET Core, screaming senselessly at my wife in a language as familiar to her as Klingon, crying in the bathroom, re-assessing my life choices, thinking whether it was a mistake to dedicate 10 years to this career, maybe I'm just not cut out for it since I can't handle this simple task, watching noose tying tutorials on youtube, thinking about my naked empty RPI that won't connect to the server any time soon.
Seriously. Why is it SO BAD?! Or is it just me?5 -
"So Alecx, how did you solve the issues with the data provided to you by hr for <X> application?"
Said the VP of my institution in charge of my department.
"It was complex sir, I could not figure out much of the general ideas of the data schema since it came from a bunch of people not trained in I.T (HR) and as such I had to do some experiments in the data to find the relationships with the data, this brought about 4 different relations in the data, the program determined them for me based on the most common type of data, the model deemed it a "user", from that I just extracted the information that I needed, and generated the tables through Golang's gorm"
VP nodding and listening intently...."how did you make those relationships?" me "I started a simple pattern recognition module through supervised mach..." VP: Machine learning, that sounds like A.I
Me: "Yes sir, it was, but the problem was fairly easy for the schema to determ.." VP: A.I, at our institution, back in my day it was a dream to have such technology, you are the director of web tech, what is it to you to know of this?"
Me: "I just like to experiment with new stuff, it was the easiest rout to determine these things, I just felt that i should use it if I can"
VP: "This is amazing, I'll go by your office later"
Dude speaks wonders of me. The idea was simple, read through the CSV that was provided to me, have the parsing done in a notebook, make it determine the relationships in the data and spout out a bunch of JSON that I could use. Hook it up to a simple gorm golang script and generate the tables for that. Much simpler than the bullshit that we have in php. I used this to create a new database since the previous application had issues. The app will still have a php frontend and backend, but now I don't leave the parsing of the data to php, which quite frankly, php sucks for imho. The Python codebase will then create the json files through the predictive modeling (98% accuaracy) and then the go program will populate the db for me.
There are also some node scripts that help test the data since the data is json.
All in all a good day of work. The VP seems scared since he knows no one on this side of town knows about this kind of tech. Me? I am just happy I get to experiment. Y'all should have seen his face when I showed him a rather large app written in Clojure, the man just went 0.0 when he saw Lisp code.
I think I scare him.12 -
Writing a function to take a string of delimited entities, parse each character to find the separators, capture the characters in between separators, and return an array of entities.
I used this for about a year before I learned about String.split()
Yeah.1 -
I have to write an xml configuration parser for an in-house data acquisition system that I've been tasked with developing.
I hate doing string parsing in C++... Blegh!16 -
Me, the only iOS dev at work one day, and colleague (who we'll call AndroidBoy), the only Android dev at work that same day (he's been working with us for less than two months). There was a change in one of the jsons we received from the server: instead of receiving a list, we now received a dictionary with strings as keys and lists as values. My iOS colleague had already made this modification on our parse function the day before.
AndroidBoy: "Hey what happened with the json?"
Me: "Oh, well instead of parsing a list, we'll parse a dictionary and get the list from each key. You basically have to do the same thing, only this time the lists are organized into categories."
AndroidBoy: "Oh, ok. But I don't know how to parse a dictionary while using Retrofit." (Context: Retrofit is a framework for request handling - correct me if I am mistaken, that's just what I've been told)
Me: "Sucks, dude, can't help ya. I've never worked with that and don't have that much exp. with Android."
I go out for a cigarette break. When I return, AndroidBoy is nowhere to be seen and suddenly I can't seem to get that data in my app. AndroidBoy comes in from the room where the backend colleagues work.
AndroidBoy: "Solved it!"
Me: "Solved what?"
AndroidBoy: "I told them to change back to a list and just put the key inside the objects of the list."
... he used the precious time of the backend colleagues to change the thing back hust because he was too lazy to search how to parse a dictionary. I was so amazed by his answer, that I didn't know whether to laugh, scream at him or punch him in the face. Not to mention the fact that now I had to revert just so he could avoid that extra work.5 -
I'm not much a fan of JavaScript. In fact, I am not very fond of any dynamic language, but JavaScript is one of my least favorites.
But this isn't about that. I use NodeJS for all of my web serving. Why would I do that? Am I a masochist? Yes.
But this isn't about that. I use NodeJS because having the same language on client and server side is something that web has never really seen before, not in this scale. Something I really really love with NodeJS is socket connections. There's no JSON parsing, no annoying conversion of data types. You can get network data and use it AS IS. If you transmit over socket using JSON, as soon as that data arrives on the server, it is available to use. It gets me so hard.
JavaScript is built to be single-threaded, and this is rooted deep into the language. NodeJS knows this isn't gonna work. And while there's still no way to multi-thread, they still try their best and allow certain operations (Usually IO) to run async as if you were using ajax.
With modern versions of the language, the server and client side can share scripts! With the inclusion of the import keyword, for the first time I have ever seen, client and server can use the same fucking code. That is mindblowing.
Syntax is still fluffy and data types are still mushy but the ability to use the same language on both sides is respectable. Can't wait for WebASM to go mainstream and open this opportunity up to more languages!10 -
need advice on integrating .t3d file support in c++ engine
hey, been stuck trying to figure out how to get .t3d file support working in our custom game engine, which is all written in c++. it's turning out to be a lot trickier than expected and i'm kinda hitting a wall here. the documentation on .t3d itself is pretty much nonexistent and what's out there is either outdated or too vague to be of any real use. i've tried a few things based on general file parsing logic in c++ but keep running into issues with either reading the files correctly or integrating the data into the engine in a way that actually works. it feels like i'm missing something fundamental but can't pin down what it is. has anyone here gone through this process before or has any experience with .t3d files? i could really use some advice on how to approach this, or even just some resources that explain the format in detail. also, if there are any common pitfalls or things to watch out for when adding new file format support to a game engine, i'd appreciate the heads up. thanks in advance for any help, feeling pretty stuck and any guidance would be a huge help.14 -
Kinda all other devs translate incompetent with a lack of knowledge
i would go with not able to recognize his lack of knowledge
Story 1:
once we had a developer, whom was given the task to try out a REST/Json API using Java
after a week he presented his solution,
2 Classes with actual code and a micro-framework for parsing and generating JSON
so i asked him, why he didn't use a framework like jackson or gson, while this presentation he felt pretty offended by this question
a couple of weeks later i met him and he was full of thanks for me, because i showed him, that there are frameworks like that, and even said sorry for feeling offended
- no incompentence here -
Story 2:
once i had a lead dev, who was so self-confident, he refactored (for no reason but refactoring itself) half the app and commited without trying to compile/run test
but not only once, but on a regular basis
as you may imagine, he broke the application multiple times and blamed the other devs
- incompentence warning-
Story 3:
once i had a dev, which wanted to stay up with the latest versions of his libraries
npm update && commit without trying to compile/testing multiple times
- incompentence warning-
Story 4:
once i had a cto
* thought email-marketing is cutting edge
* removed test-systems completely to reduce costs
* liked wordpress
* sets vm to sleep without letting anyone know
- i guess incompetent alert -2 -
A cache - related bug that gets triggered only at high loads, 10k parallel sessions or so.
Parsing 30GB of logs, trying to find something to work with....
yippee......5 -
I spent four days doing a rewrite for a possible performance boost that yielded nothing.
I spent an hour this morning implementing something that boosted parsing of massive files by 22% and eliminated memory allocations during parsing.
Work effort does not translate into gains.17 -
So I just spent the last few hours trying to get an intro of given Wikipedia articles into my Telegram bot. It turns out that Wikipedia does have an API! But unfortunately it's born as a retard.
First I looked at https://www.mediawiki.org/wiki/API and almost thought that that was a Wikipedia article about API's. I almost skipped right over it on the search results (and it turns out that I should've). Upon opening and reading that, I found a shitload of endpoints that frankly I didn't give a shit about. Come on Wikipedia, just give me the fucking data to read out.
Ctrl-F in that page and I find a tiny little link to https://mediawiki.org/wiki/... which is basically what I needed. There's an example that.. gets the data in XML form. Because JSON is clearly too much to ask for. Are you fucking braindead Wikipedia? If my application was able to parse XML/HTML/whatevers, that would be called a browser. With all due respect but I'm not gonna embed a fucking web browser in a bot. I'll leave that to the Electron "devs" that prefer raping my RAM instead.
OK so after that I found on third-party documentation (always a good sign when that's more useful, isn't it) that it does support JSON. Retardpedia just doesn't use it by default. In fact in the example query that was a parameter that wasn't even in there. Not including something crucial like that surely is a good way to let people know the feature is there. Massive kudos to you Wikipedia.. but not really. But a parameter that was in there - for fucking CORS - that was in there by default and broke the whole goddamn thing unless I REMOVED it. Yeah because CORS is so useful in a goddamn fucking API.
So I finally get to a functioning JSON response, now all that's left is parsing it. Again, I only care about the content on the page. So I curl the endpoint and trim off the bits I don't need with jq... I was left with this monstrosity.
curl "https://en.wikipedia.org/w/api.php/...=*" | jq -r '.query.pages[0].revisions[0].slots.main.content'
Just how far can you nest your JSON Wikipedia? Are you trying to find the limits of jq or something here?!
And THEN.. as an icing on the cake, the result doesn't quite look like JSON, nor does it really look like XML, but it has elements of both. I had no idea what to make of this, especially before I had a chance to look at the exact structured output of that command above (if you just pipe into jq without arguments it's much less readable).
Then a friend of mine mentioned Wikitext. Turns out that Wikipedia's API is not only retarded, even the goddamn output is. What the fuck is Wikitext even? It's the Apple of wikis apparently. Only Wikipedia uses it.
And apparently I'm not the only one who found Wikipedia's API.. irritating to say the least. See e.g. https://utcc.utoronto.ca/~cks/...
Needless to say, my bot will not be getting Wikipedia integration at this point. I've seen enough. How about you make your API not retarded first Wikipedia? And hopefully this rant saves someone else the time required to wade through this clusterfuck.12 -
I have come across the most frustrating error i have ever dealt with.
Im trying to parse an XML doc and I keep getting UnauthorizedAccessException when trying to load the doc. I have full permissions to the directory and file, its not read only, i cant see anything immediately wrong as to why i wouldnt be able to access the file.
I searched around for hours yesterday trying a bunch of different solutions that helped other people, none of them working for me.
I post my issue on StackOverflow yesterday with some details, hoping for some help or a "youre an idiot, Its because of this" type of comment but NO.
No answers.
This is the first time Ive really needed help with something, and the first time i havent gotten any response to a post.
Do i keep trying to fix this before the deadline on Sunday? Do i say fuck it and rewrite the xml in C# to meet my needs? Is there another option that i dont even know about yet?
I need a dev duck of some sort :/39 -
Am i the only one who doesnt see how graphql is useful? Yeah you can cut back on data transfer and let the server do the parsing but like, you can do the same if you spend 3 minutes properly designing the backend?6
-
I have quite a few of these so I'm doing a series.
(2 of 3) Flexi Lexi
A backend developer was tired of building data for the templates. So he created a macro/filter for our in house template lexer. This filter allowed the web designers (didn't really call them frond end devs yet back then) could just at an SQL statement in the templates.
The macro had no safe argument parsing and the designers knew basic SQL but did not know about SQL Injection and used string concatination to insert all kinds of user and request data in the queries.
Two months after this novel feature was introduced we had SQL injections all over the place when some piece of input was missing but worse the whole product was riddled with SQLi vulnerabilities.2 -
This is meant as a follow-up on my story about how I'm no longer and Ada developer and everything leading up to that. The tldr is that despite over a decade of FOSS work, code that could regularly outperform a leading Ada vendor, and much needed educational media, I was rejected from a job at that vendor, as well as a testing company centered around Ada, as well as regularly met with hostility from the community.
The past few months I have been working on a "pattern combinator" engine for text parsing, that works in C#, VB, and F#. I won't explain it here, but the performance is wonderful and there's substantial advantages.
From there, I've started a small project to write a domain specific language for easily defining grammars and parsing it using this engine.
Microsoft's VisualStudio team has reached out and offered help and advice for implementing the extensions and other integrations I want.
That Ada vendor regularly copied things I had worked on, "introducing" seven things after I had originally been working on them.
In the almost as long experience with .NET I've rarely encountered hostility, and the closest thing to a problem I've had has been a few, resolved, misunderstandings.
Microsoft is a pretty damn good company. And it's great to actually be welcomed/included.2 -
Should I be excited or concerned?
Newbie dev(babydev) who just learned string vs int and the word "boolean", is SUPER into data parsing, extrapolation and recursion... without knowing what any of those terms.
2 ½ hrs later. still nothing... assuming he was confused, I set up a 'quick' call...near 3 hrs later I think he got that it was only meant so I could see if/where he didnt understand... not dive into building extensive data arch... hopefully.
So, we need some basic af PHP forms for some public-provided input into a mySQL db. I figured I'd have him look up mySQL variables/fields, teach him a bit about proper db/field setup and give him something to practice on his currently untouched linux container I just set up so he could have a static ipv4 and cli on our new block (yea... he's spoiled, but has no clue).
I asked him to list some traits of X that he thinks could be relevant. Then to essentially briefly explain the logic to deciding/returning the values/how to store in the db... essentially basic conditionals and for loops... which is also quite new to him.
I love databases; I know I'm not in the majority... I assumed he'd get a couple traits in his mind and exhaust himself breaking them down. I was wrong. He was/likely is in his sleep now, over complicating something that was just meant as a basic af.
Fyi, the company is currently weighted towards more autistics (him and myself included) than neurotypicals.
I know I was(still am) extremely abnormal, especially when it comes to things like data.
So, should I be concerned/have him focus elsewhere for a bit?... I dont want to have him burnout before he even gets to installing mySQL44 -
>About to create Helperclass for JSON parsing and writing
>Realises there's the GSON-library by Google
>shamefully and silently like a fart deletes 1 hour of work
>repeatedly bangs head against desktop4 -
Fuck my life! I have been given a task to extract text (with proper formatting) from Docx files.
They look good on the outside but it is absolute hell parsing these files, add to these shitty XML human error and you get a dev's worst nightmare.
I wrote a simple function to extract text written in 'heading(0-9)' paragraph style and got all sorts of shit.
One guy used a table with borders colored white to write text so that he didn't have to use tabs. It is absolute bullshit.2 -
Something I hate in .NET: It wants to take care of different decimal seperators in different countries.
No .NET, I would handle this myself anyway :(2 -
That feeling when you’re scraping a website to build an API and your script downloads 4049/6170 pages before failing and you have to rewrite it so that puppeteer hits the next button 4049 times before executing the script. 😅
This database is so frustrating.
I hate this website (the one I’m scraping).
It’s going to be so satisfying when this is finished.6 -
I had to implement an internal tool in C++ which parses a file and converts the content into another format.
It did take hundreds of lines of code to get it working, file handling and parsing data in C/C++ is terrible.
I'd rather done it with some scripting language, and additionally implemented it in python as a side-project (in less than an hour and < 20 lines of code, BTW) but it should be C++ "Because that is how we do it here".
At the end the tool was only used for a few weeks, because someone had an idea how to completely avoid the need for that converted data.3 -
Is your code green?
I've been thinking a lot about this for the past year. There was recently an article on this on slashdot.
I like optimising things to a reasonable degree and avoid bloat. What are some signs of code that isn't green?
* Use of technology that says its fast without real expert review and measurement. Lots of tech out their claims to be fast but actually isn't or is doing so by saturation resources while being inefficient.
* It uses caching. Many might find that counter intuitive. In technology it is surprisingly common to see people scale or cache rather than directly fixing the thing that's watt expensive which is compounded when the cache has weak coverage.
* It uses scaling. Originally scaling was a last resort. The reason is simple, it introduces excessive complexity. Today it's common to see people scale things rather than make them efficient. You end up needing ten instances when a bit of skill could bring you down to one which could scale as well but likely wont need to.
* It uses a non-trivial framework. Frameworks are rarely fast. Most will fall in the range of ten to a thousand times slower in terms of CPU usage. Memory bloat may also force the need for more instances. Frameworks written on already slow high level languages may be especially bad.
* Lacks optimisations for obvious bottlenecks.
* It runs slowly.
* It lacks even basic resource usage measurement.
Unfortunately smells are not enough on their own but are a start. Real measurement and expert review is always the only way to get an idea of if your code is reasonably green.
I find it not uncommon to see things require tens to hundreds to thousands of resources than needed if not more.
In terms of cycles that can be the difference between needing a single core and a thousand cores.
This is common in the industry but it's not because people didn't write everything in assembly. It's usually leaning toward the extreme opposite.
Optimisations are often easy and don't require writing code in binary. In fact the resulting code is often simpler. Excess complexity and inefficient code tend to go hand in hand. Sometimes a code cleaning service is all you need to enhance your green.
I once rewrote a data parsing library that had to parse a hundred MB and was a performance hotspot into C from an interpreted language. I measured it and the results were good. It had been optimised as much as possible in the interpreted version but way still 50 times faster minimum in C.
I recently stumbled upon someone's attempt to do the same and I was able to optimise the interpreted version in five minutes to be twice as fast as the C++ version.
I see opportunity to optimise everywhere in software. A billion KG CO2 could be saved easy if a few green code shops popped up. It's also often a net win. Faster software, lower costs, lower management burden... I'm thinking of starting a consultancy.
The problem is after witnessing the likes of Greta Thunberg then if that's what the next generation has in store then as far as I'm concerned the world can fucking burn and her generation along with it.6 -
Fuck Workday! Fuck any company that uses Workday! Fucking stop parsing resume if you’re bad at it! Fucking stop asking to fill the same information in the resume I just uploaded!4
-
For some project, I wrote this algoritem in Java to parse a lot of XML files and save data to database. It needs to have some tricks and optimisations to run in acceptable time. I did it and in average, it run for 8-10 minutes.
After I left company they got new guy. And he didn't know Java so they switched to PHP and rewrote the whole project. He did algorithm his way. After rewrite it run around 8 hours.
I was really proud of myself and shocked they consider it acceptable.4 -
Sorry, need to vent.
In my current project I'm using two main libraries [slack client and k8s client], both official. And they both suck!
Okay, okay, their code doesn't really suck [apart from k8s severely violating Liskov's principle!]. The sucky part is not really their fault. It's the commonly used 3rd-party library that's fucked up.
Okhttp3
yeah yeah, here come all the booos. Let them all out.
1. In websockets it hard-caps frame size to 16mb w/o an ability to change it. So.. Forget about unchunked file transfers there... What's even worse - they close the websocket if the frame size exceeds that limit. Yep, instead of failing to send it kills the conn.
2. In websockets they are writing data completely async. Without any control handles.. No clue when the write starts, completes or fails. No callbacks, no promises, no nothing other feedback
3. In http requests they are splitting my request into multiple buffers. This fucks up the slack cluent, as I cannot post messages over 4050 chars in size . Thanks to the okhttp these long texts get split into multiple messages. Which effectively fucks up formatting [bold, italic, codeblocks, links,...], as the formatted blocks get torn apart. [didn't investigate this deeper: it's friday evening and it's kotlin, not java, so I saved myself from the trouble of parsing yet unknown syntax]
yes, okhttp is probably a good library for the most of it. Yes, people like it, but hell, these corner cases and weird design decisions drive me mad!
And it's not like I could swap it with anynother lib.. I don't depend on it -- other libs I need do! -
Had a bad day at work :( They gave me this code for some obscure streaming job and asked me to complete it. Only after 3 days did I realize that the LLD given to me was incorrect as the data model was updated. Another 2 more days, I was able to debug the code and run it successfully— I was able to parse the tables and generate the required frame but not able to stream it back to the output topic as per the LLD. That’s where I needed help but none of my emails/messages were replied to. The main guy who is pretty technical scheduled a code review session with me— I expected that I would run the code and he would spot it something I might’ve missed and why my streaming function isn’t working. Instead, what happened was that he grilled me on each and every line of the code (which had some obscure tables queried) and then got super mad at me saying “Why are we having this code review session if your code is not complete?”. I’m like bruh, you asked for it, and yes, the main parsing logic is done and I’m just having this issue in the last part. And he’s like “Why didn’t you tell me earlier?”. Wtf?! I left at least 5 emails and a dozen messages. He’s like this has to go live on Monday, and I’m like Ok, I’ll work in the weekend. And he’s like “Don’t tell me all these things! You’re not doing me a favor by working on weekends! How am I to ask my colleagues to connect with you separately on Saturday/Sunday? You should have done the on the weekdays itself. What were you doing this whole week?”. Bruh, I was running the code multiple times and debugging it using print statements. All while you were ignoring my attempts to reach out to you. SMH 🤦♂️ I can go on and on about this whole saga.4
-
So I had been developing a real estate website and developing a MLS feed parser. I had only 1 year experience at that time and parsing a XML feed was already complex enough. On top of it, the client wanted to automate feed download from the MLS provider through HTTP authentication. Managed to do it. Everything worked for 15 days and on 16th day the property location markers stopped appearing on Google maps. Turned out that address to lat-long reverse geocoding was failing because API limit exhausted. My bad, I coded it on view instead of caching the lat-long in database. Fixed it in a day and viola!
-
A long time ago I started a project to make a devRant client with Python and Qt5.
I got far but got bored, or whatever. Was still in school, etc.
I have started from scratch again. Including a nogui mode. Sharing because I actually have made some pretty good progress in the past two days.
Plans: Besides the obvious fully-featured client: full support for plugins, custom themes, custom CLI commands. Multiple logins.
Considering a system that allows you to run a bot, and a bot framework (parsing arguments for you, marking notifs read, etc.)
And yes, it's called qtpy-rant (Pronounced cutie pie rant)3 -
Workarounds are great. I remember one time, I had a server that let anyone access any file as long as the knew the right path. I wanted to store data in a .txt (it wasnt secure passwords or anything, so calmyourtities), but then had access too it. Now, this server wasn't running anything except PHP, so I created a database.php, and within was just some php tags. I ended up modifying the database.php from other PHP scripts and storing all the data as PHP comment, then parsing thru it as I needed, so loading mydomain.biz/database.php wouldn't show the data. ex of my database.php (to all that might not understand because I'm bad at explaining):
<?php
//USER1:DATA1
//USER2:DATA2
?>2 -
While parsing nodes in a graph.
In terms of readability and variable naming, how wrong (if at all) is to use:
1. broNode (for sibling nodes)
2. papaNode / mamaNode (for parent nodes)
3. babyNode (for child nodes)
I sincerely don't know how to review this PR7 -
Golang's standard command line parsing library suuuucckkkkssss. But who the fuck writes command line apps any more? Everything has to be a fucking asynchronous daemon in a docker container orchestrated by kuberwhateverthefuck running in the ClOuD2
-
Microbiology, biochemistry, genetics.
Mostly reading some articles, parsing some ATGC, trying to understand topic from books.6 -
At what point can I claim to not be a script kiddie anymore?
Like, I've built compilers, and interpreters for an excel-like syntax, I refactored a pdf-parsing library from the ground up. I managed databases and wrote protocols for communicating with hardware.
But most of my experience is with python / nodejs / golang. It is only recently that I started playing with C and rust for actual efficient system code.8 -
[CMS Of Doom™]
Ah, yes, their built-in bullshit newsletter module just sent the n-th user n emails. Wonderful considering n=368.
The culprit? Better don't ask...
OK, anyway: So the mailer is running as a CRONjob, but nah, not as a console script call but by a public HTTP GET URL call, fucking obviously (it's the CMS Of Doom for a reason).
So these fucking imbeciles "implemented" an ob_start() callback where HTML links are - for whatever fucking reason - modified by some regex (obviously everybody knows parsing HTML by Regex is trivial). In this case the link was somehow modified to recall the mailer Cronjob...
This must have upset the pngoing mailing process thus spamming mails. Whyyyy
And I've thought I've seen it all after 6 months in this legacy hell...
This is why you don't run a company consisting of only beginners in PHP (in cluding their "CEO")! -
So, I've had a personal project going for a couple of years now. It's one of those "I think this could be the billion-dollar idea" things. But I suffer from the typical "it's not PERFECT, so let's start again!" mentality, and the "hmm, I'm not sure I like that technology choice, so let's start again!" mentality.
Or, at least, I DID until 3-4 months ago.
I made the decision that I was going to charge ahead with it even if I started having second thoughts along the way. But, at the same time, I made the decision that I was going to rely on as little external technology as possible. Simplicity was going to be the key guiding light and if I couldn't truly justify bringing a given technology into the mix, it'd stay out.
That means that when I built the front end, I would go with plain HTML/CSS/JS... you know, just like I did 20+ years ago... and when I built the back end, I'd minimize the libraries I used as much as possible (though I allowed myself a bit more flexibility on the back end because that seems to be where there's less issues generally). Similarly, any choice I made I wanted to have little to no additional tooling required.
So, given this is a webapp with a Node back-end, I had some decisions to make.
On the back end, I decided to go with Express. Previously, I had written all the server code myself from "first principles", so I effectively built my own version of Express in other words. And you know what? It worked fine! It wasn't particularly hard, the code wasn't especially bad, and it worked. So, I considered re-using that code from the previous iteration, but I ultimately decided that Express brings enough value - more specifically all the middleware available for it - to justify going with it. I also stuck with NeDB for my data storage needs since that was aces all along (though I did switch to nedb-promises instead of writing my own async/await wrapper around it as I had previously done).
What I DIDN'T do though is go with TypeScript. In previous versions, I had. And, hey, it worked fine. TS of course brings some value, but having to have a compile step in it goes against my "as little additional tooling as possible" mantra, and the value it brings I find to be dubious when there's just one developer. As it stands, my "tooling" amounts to a few very simple JS scripts run with NPM. It's very simple, and that was my big goal: simplicity.
On the front end, I of course had to choose a framework first. React is fine, Angular is horrid, Vue, Svelte, others are okay. But I didn't want to bother with any of that because I dislike the level of abstraction they bring. But I also didn't want to be building my own widget library. I've done that before and it takes a lot of time and effort to do it well. So, after looking at many different options, I settled on Webix. I'm a fan of that library because it has a JS-centric approach. There's no JSX-like intermediate format, no build step involved, it's just straight, simple JS, and it's powerful and looks pretty good. Perfect for my needs. For one specific capability I did allow myself to bring in AnimeJS and ThreeJS. That's it though, no other dependencies (well, at first, I was using Axios because it was comfortable, but I've since migrated to plain old fetch). And no Webpack, no bundling at all, in fact. I dynamically load resources, which effectively is code-splitting, and I have some NPM scripts to do minification for a production build, but otherwise the code that runs in the browser is what I actually wrote, unlike using a framework.
So, what's the point of this whole rant?
The point is that I've made more progress in these last few months than I did the previous several years, and the experience has been SO much better!
All the tools and dependencies we tend to use these days, by and large, I think get in the way. Oh, to be sure, they have their own benefits, I'm not denying that... but I'm not at all convinced those benefits outweighs the time lost configuring this tool or that, fixing breakages caused by dependency updates, dealing with obtuse errors spit out by code I didn't write, going from the code in the browser to the actual source code to get anywhere when debugging, parsing crappy documentation, and just generally having the project be so much more complex and difficult to reason about. It's cognitive overload.
I've been doing this professionaly for a LONG time, I've seen so many fads come and go. The one thing I think we've lost along the way is the idea that simplicity leads to the best outcomes, and simplicity doesn't automatically mean you write less code, doesn't mean you cede responsibility for various things to third parties. Those things aren't automatically bad, but they CAN be, and I think more than we realize. We get wrapped up in "what everyone else is doing", we don't stop to question the "best practices", we just blindly follow.
I'm done with that, and my project is better for it! -
Can we all just agree to stop actively imagining progressively harder to parse CSV formatting options? For fucks sake I’ve had to build in tolerance for quoted and unquoted data, combined data and split data, ways to split the data and recombine it, compare every data point, filter some data, only add data, only remove data, base data updates on non Boolean fields in the file, set end point matching based on arbitrary fields, column number matching, header matching, manipulate malformed urls and reassemble the file with proper ones, it goes the fuck on. CSV’s should just be simple and not hard to format. Why does everyone want to try so fucking hard to do bizarro shit?!
-
Client wants some CMS text to be automatically translated. So I checked and Google seems to have a solution for that. I thought to to be as simpel as doing a request and parsing the response. That's how API's work, right?
No. First I must create an account, that account must have a credit card, then I need to setup credentials, the default ones working with path variables, an API key... etc etc etc.
I feel so stupid for just not understanding their docs. I'm just a dude that installs a CMS and makes pretty CSS for it. I've worked with REST APIs before (Mollie, Carerix) but none of them ever demanded the level of knowledge and setup the Google Translate API demands.
Am I just a bad developer or is this shit just too complex for your average web developer?9 -
Best:
Got a role change to automation engineer, which is sort of a 'just fix problems' position, like with tooling, get rid of manual work, remove as many spreadsheet as we can.
I started looking into rust.
Worst:
People think we depend heavily in javascript because our products are extensions, our golden product is an extension, so a few members of my team insist in depending in our core team and use their javascript stuff, even for string parsing, even if we do have a python package that does rhe same thing that is officially maintained too.
I refuse.
The good again:
My boss let's me refuse, I am not forced into javascript, they let me use whatever I want as long as it is reasonable.2 -
I like my log messages to indicate automatically where in the code something happened, so that I can easily identify where a message originated from while tracking down problems.
In C/C++ this is nice and easy - write a logging routine, wrap it in macros for the different log levels and have that automatically output __FILE__, __LINE__ etc.
I wanted to do something similar in NodeJS, as I'd found myself manually writing the file name in the log message and then splitting functionality out into new files and it became a mess.
The only way I found to be able to do this was to create an "Error" object and access the "stack" member of it. This is a string containing a stack backtrace, suitable for writing to console/file. I just wanted the filename/line/routine.
So I ended up splitting the string into lines, then for each of the lines, trimming the surrounding spaces (or tabs?), and parsing them to see if the stack entry is inside my logger module. The first entry outside of that module must therefore be the thing that called it, so I then parse out the routine or object and method, filename and line number.
It's a lot of clumsy work but the output is pretty neat. I just wish it were simpler!2 -
Unreal Engine + third party library adventures part2:
now we gotta parse everything from unreal formats, to the library formats, than back to unreal again... sigh. Why can't it just use normal data formats like everybody else? I mean, come on, strings, ints, vectors etc. they all have standart libraries. why does unreal need to make their own shiet out of it. Just why...1 -
In my experience object oriented is very good for composing high level abstractions into a complete system. Functional is awesome for validation, parsing and massaging data in any way and imperative is tithe most useful paradigm to handle side effect dependent code that either manipulate the computers state ( read/write) or communicate with external systems.
The people acting as if one of them is the one true way are misleading you.3 -
Three days ago my focus was shifted from a development role to a support role. I was shifted to replace another support guy who had used fraud to get the position. I have no experience with this role but there was decent KT and I'm catching on fine. During onboarding and KT I'm serving as the first contact for new tickets and whatnot...
Today I got a ticket with an error on our production instance that no one had ever seen before. It prevented the guy from using our service entirely. I tried to reproduce it and... I couldn't use the service either. No one could. Everything was down. I could see the sweat building on my manager's forehead.
Thankfully another member on my team has done a bit of support before, so we collaborated with each other and other teams throughout the day to figure out what's wrong and how to fix it. I'm listening to them chat remotely as we speak - so far I've been working on it 9 hours straight.
This service is used by everyone - it's a business critical service with due dates on actions and escalations to managers... Imagine if the support ticketing service for your company crashed. That means a lot of people are asking what's wrong, requiring extensions, etc. I've been answering to managers and seniors in the business throughout the day.
The best part? We figured out why the server went down, and the reason is fantastic: someone updated the server's code without telling anyone, and all they had done was remove critical parsing code. Just took it right out, pushed, redeployed. We don't know who did it or who even has access to do that. I guess I have some detective work cut out for me after we've fixed everything that was broken by that.
I miss coding already.1 -
The feature was to parse a set of fairly complex xml files following a legacy schema. Problem was, the way this was done previously did not conform to the schema so it was a guideline at best, which over the course of many years snowballed into an anarchy where clients would send in whatever and it was continuously updated per case as needed. They wanted to start enforcing their new schema while phasing out the old method.
The good news is that parsing and serialization is very testable, so I rounded up what I could find of example files and got to work. Around the same time I asked our client if they had any more examples of typical cases we need to deal with, and sure enough a couple of days later I receive a zip with hundreds of files. They also point out that I should just disregard the entire old set since they decided to outright cut support for it after all if it makes things simpler. Nice.
I finish the feature in a decent amount of time. All my local tests pass, and the CD tests pass when I push my branches. Once we push to our QA env though and the integration tests run, we get a pass rate of less than 10%.
I spend a couple of days trying to figure out what's going on, and eventually narrow it down to some wires being crossed with the new vs. old xml formats. I'm at a loss. I keep trying to chip away at it until I'm left with a minimal example, and I have one of those lean-back moments where you're just "I don't get it". My tests pass locally, but in the QA environment they fail on the same files.
We're now 3 people around my workstation including the system architect, and I'm demonstrating to the others how baffling and black magic this is. I postulate that maybe something is cached in my local environment and it's not actually testing the new files. I even deleted the old ones.
"Are you sure you deleted the right files?"
"Duh of course -- but let me check..."1 -
Made parsers that are downloading and parsing 100 years of law data right now.
I am so happy I don’t have to read every document and just let computer do it for me.
That are great times we live in.2 -
I dug up my old ledger web app that I wrote when I was in my late twenties, as I realized with a tight budget toward the end of this year, I need to get a good view of future balances. The data was encrypted in gpg text files, but the site itself was unencrypted, with simple httpasswd auth. I dove into the code this week, and fixed a lot of crap that was all terrible practice, but all I knew when I wrote it in the mid-2000s. I grabbed a letsencrypt cert, and implemented cookies and session handling. I moved from the code opening and parsing a large gpg file to storing and retrieving all the data in a Redis backend, for a massive performance gain. Finally, I switched the UI from white to dark. It looks and works great, and most importantly, I have that future view that I needed.1
-
Which misanthropic, terrible, perverse excuse for a dogfucker decided that damned non breaking spaces (SPACES!) return false on isWhitespace? It's in the name, space, it's white, it's a fucking white space, a whitespace if you will so who do I have to kill for wasting two damned hours of my life trying to parse away those bastards?3
-
Fuck me, i spent the last 2 days trying to populate a database about the game satisfactory from the wikia, only to read on a subreddit they shipped a json file with api the data I want. I need to check that tomorrow, because I just want to sleep, but if that's true, just kill me.
I FUCKING HATE PARSING WEBSITES8 -
So, I'm looking into something and end up on Stack Overflow. Someone posted the question:
"Does minified javascript improve performance?"
This question was old as shit, all they way from 07/25/09, and about an Adobe Air application. (Remember that? Me neither...) It had a great, accepted, and still accurate answer, posted the same day the question was asked. Now, fast forward 8 years and on 12/08/17 (A mere 7 months ago...) the following answer was posted. I don't know what they were thinking, but here it is, complete and unabridged, with my comments in square brackets:
"I'd like to post this as a separate answer as it somewhat contrasts the accepted one: [Somewhat contrasts? More like completely contradicts...]
Yes, it does make a performance difference as it reduces parsing time - and that's often the critical thing. For me, it was even just simply linear in the size and I could get it from 12s to 4s parse time by minifying from 3MB to 1MB. [First off, your parse time should NEVER be THE critical thing, but secondly, and more importantly, WHO THE FUCK HAS 1MB OF MINIFIED JS ON A PAGE!!!]
It's not a big app either, it just has a couple of reasonable dependencies. [THERE IS ABSOFUCKINGLUTELY NOTHING REASONABLE ABOUT ANYTHING HE JUST SAID! What dependancies is he using?! You could use minified and not even gzipped jQuery, AngularJS, Vue, Ember, React, AND Dojo libraries on the SAME PAGE, AND have 118k of application code, AND STILL NOT HAVE HIT 1MB QUITE YET!!!]
So the moral of the story here is: Yes, minifying is important for performance - and not because of bandwidth, but because of parsing. [Javascript should NEVER take longer to parse then to download, even on a low powered device...]"
So, yeah, I'm at a loss for what this guy was thinking, but the thought the people like this exist, and that my browser might one day be subjected to their horrific nightmare of code terrifies me...2 -
Whoever designed how audio metadata is stored in gamemaker archives is now on my hitlist.
I think my comments on my parser sums up my mental state pretty well.
https://gist.github.com/BenjaminUrq...4 -
There are a couple:
A system that updates user accounts to connect them into our wifi system by parsing thousands of processing files written in Clojure. The project was short lived and mainly experimental, It has complete test cases and the jar generated from it is still purring silently on the main application. It was used to replace an $85k vendor application that made no fucking sense. The code has not been touched in 2 years and the jar is still there. The dba mentioned the solution to the vendor, the vendor tried buying it from me, but being that it belongs to the institution nothing was touched, still, it got the VP's attention that I can make programs that would be bought for that level, it caught his attention even more when I showed him the codebase and he recognized a Lisp variant (he is old, and was back in the day a Fortran and Cobol developer)
A small Python categorical ML program that determines certain attributes of user generated data and effectively places them on the proper categories on the main DB. The program generates estimates of the users and the predictions have a 95% correctness rate. The DBA still needs to double check the generated results before doing the db updates. I don't remember how I coded it because I was mostly drunk when I experiment on the scenario. It also got the attention of the VP and director since the web tech manager was apparently doing crazy ML shit that they were not expecting me to do, it made them paranoid that I would eventually leave for a ML role somewhere, still here, but I want more moneys!!
A program that generates PDF documentation from user data, written in Go, Python and Perl (yes Perl) I even got shit from the lead developer since I used languages outside of their current scope of work. Dude had no option but to follow along with it :P since I am his boss
Many more. I am normally proud of my work code. But my biggest moment is my current ntural language processing unit that I am trying to code for my home, but I don't have enough power to build it with my computers, currently, my AI is too stupid, but sometimes it does reply back to my commands and does the things I ask it to do (simple things, opening a browser, search for a song etc) but 7 times out of ten it wont work :P -
On a website, using var something = $.parseJSON('{"with": "perfectly valid javascript in here"}') when you could just have done var something = {"native": "javascript goes here", "with": "no parsing needed"};
-
I can't believe I have been so blind all this time! For years I have unquestionably accepted the lies of corporations and governments around the world, but I have been enlightened and it is glorious!
So thank you, JavaScript, you magnificent beast - for informing me of the misinformation that has resided in the public consciousness for thousands of years!
It is not the year 2016, it is in fact only the year 116 - wake up sheeple!2 -
I went down a rabbit hole of code changes to try and delete a stupid for loop with a break in it.
It was super stupid and I gave up and submitted to the fact that some battles are not worth the time and stress.
OK... But seriously, It was returning multiple entities from the database, but we only always want the first one. My logic is that we should just go in there and fix the LINQ so we are explicitly getting one entity out.
But fuck that logic. No I'll have to change fucking everything that's tied to that method and expects a list from it. Every fucking thing. That includes error handling, parsing, for loops..... Nevermind...
You can have your foreach and your break. I'm taking mine, now.rant break my back on this stupid code what do you want on the frontend last minute changes did this to me they couldn't decide1 -
Currently making a perfect sudoku webapp / plugin using native JS and html templates where I'm very enthousiast about.
It allows to select multiple cells and then put in a number and all selected have that number. It keeps state of every change, you can do unlimited redo's. Right click or double click someehere removes selection. Not built yet, but it will have a box where you can paste sudoku's you've found on the internet. I just parse 81 times [1-9] with regex. So all formats are supported including noisy ones as long the noise is not numbers. Making your own puzzle is very easy. Art is to make hard ones. I'm generating extra hard puzzles using C threading. For reference: there are 6,670,903,752,021,072,936,960 sudoku puzzles possible and from that I try to resolve the hard ones using simple human logging with brute forcing as fallback until it can use logic again. 30 million attempts to solve per secon. I should at some more logic. I don't do xwing or ywing, bs imho. You have to be a superhuman to spot xwing / ywing possibilities. I think i can imagine a better logic myself. We'll see.
And yes, that's a real screenshot. Puzzle is validated and it found issues. Marked with red font. Green is current selection by user11 -
Teach data structures by showing how they're used in real life situations. Don't make us do some nonsense puzzle shit. For example, a friend of mine is learning stacks/queues right now and his assignment is to build a simple HTML parsing algorithm to determine whether an HTML file is valid. This shows the student a practical use of the data structure and reinforces that this shit actually does get used in real life.
-
Haha, fuck you kotlin! and your null safety!
I was able to break it 😆
After reading about its syntax for over 2 weeks , i finally sat down to write a simple parsing app completely in kotlin. And now i don't know how, but i am able to store a null in a "val x:String" (i.e a non null variable)
I am not going to claim it as some miracle or discovery as some other ranters, it might be a mistake. I am just a 21 yo Android/java dev trying to re write my old ,tested java code to kotlin by myself, without any auto convert, in the middle of night when i am 99% asleep by brain.
I will try to raise an SO question with details, but all i used was a simple volley request returning heterogeneous data, a gson convertor and a single activity,
Right now i am buzzing off to my sleep11 -
Imagine a web way ahead of our time where its size goes beyond our imagination...
This is my first rant, and I'll cut to the chase! I don't like how web currently stands. Here's what makes me angry the most altough I know there's a myriad of solutions or workarounds:
- A gazillion credentials/accounts/services in your lifetime.
- Everyone tries to reinvent the wheel.
- There's no single source of truth.
- Why the fuck there's so much design in a vision that started as a network of documents? Why is it that we need to spend time and energy to absorb the page design before we can read what we are after?
- What's up with the JS front end frameworks?! MB's of code I need to download on every page I visit and the worse is the evaluation/parsing of it. Talk about acessibility and the energy bills. I don't freaking need a SPA just give a 20-50ms page load and I'm good to go!
- I understand that there's a whole market based on it but do we really need all that developer tools and services?
- Where's our privacy by the way? Why the fuck do I need ads? Can't I have a clue about what I wan't to buy?
Sticking with this points for now... Got plenty more to discuss though.
What I would like to see:
A unique account where i can subscribe services/forums/whatever. No credentials. Credentials should be on your hardware or OS. Desktop Browser and mobile versions sync everything seemlesly. Something like OpenID.
Each person has his account and a profile associated where I share only what I want with whom I want when I want to.
Sharing stuff individually with someone is easy and secure.
There's no more email system like we know. Email should be just email like it started to be. Why the hell are we allowing companies to send us so much freaking "look at me now, we are awesome", "hey hey buy from me".. Here's an idea, only humans should send emails. Any new email address that sends you an email automatically requests your "permission" to communicate with you. Like a friend request.
Oh by the way did I tell you that static mail is too old for us? What we need is dynamic email. Editing documents on the fly, together, realtime, on the freaking email. Better than mail, slack and google docs combined.
In order for that to work reasonably well, the individual "letter" communication would have to be revamped in a new modern approach.
What about the single source of truth I talked about? Well heres what we should do. Wikipedia (community) and Larry Page (concept) gave us tremendous help. We just need to do better now.
Take the spirit of wikipedia and the discoverability that a good search engine provides us and amp that to a bigger scale. A global encyclopedia about everything known to mankind. Content could be curated from us all just like a true a network.
In this new web, new browser or whatever needed to make this happen I could save whatever I want, notes, files, pictures... and have it as I left it from device to device.
Oh please make web simple again, not easy just simple and bigger.
I'm not old by the way and I don't see a problem with being older btw.
Those are just my stupid rants and ideas. They are worth nothing. What I know for sure is that I'll do something about or fail trying to.12 -
GREAT PROGRESS TODAY!
I already made the login part of the devRant rewrite and I can reliably log in and out of the app.
From here, it’s just API requests, JSON parsing and just fitting the data to templates.
...and the avatar system.
Fuck.2 -
I wasn't hired to do a dev's job (handled sales) but they asked me to help the non-HQ end with sorting transaction records (a country's worth) for an audit.
Asked HQ if they could send the data they took so I wouldn't need to request the data. We get told sure, you can have it. Waits for a month. Nothing. Apparently, they've forgotten.
Asks for data again. They churn it out in 24 hours. Badly Parsed. Apparently they just put a mask of a UI and stored all fields as one entire string (with no separators). The horror!
Ended up wasting most of a week simply fixing the parsing by brute force since we had no time.
Good news(?): We ended up training the front desk people to ending their fields with semi-colons to force backend into a possibly parsed state. -
A famous quote attributed to Jamie Zawinski:
"Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems."
I am living this nightmare. Joined a project where schmucks are parsing SQL using regular expressions. Talked to a friend, a compiler developer expert. He told me that regexes can't be used for checking if braces are matched. Pumping Lemma. Those fuckers should have used ANTLR or something. Anyways planning to leave this project.2 -
I had an interesting mystery the other day. I work in the UK, but I'm working remotely from the US for a while. First day, I made some changes, ran the tests and they failed. Weird part was the failing test was for a component I hadn't touched. I took a closer look, and realized it was a date off by several hours. The test was checking that a passed in date appears in the output. But it was creating the date by parsing a string. The library I was using defaults to local time, but the component uses UTC. So, I had inadvertently created a unit test that only passes when run from UTC. But I had never noticed before because my work is in that timezone. Yikes!
-
Man... I hate refactoring. After I had finished up an issue this morning, I had to refactor old sql queries and the parsing to the views.
I've worked on it all day and I still haven't finished! Still loving my job, tasks like these are unavoidable but they drain the life out of me.3 -
I started building an application for FIDAL (Italian Federation of Athletics) because why not: I was bored and wanted to learn Flutter.
There is no API, but I didn't even expect it. Parsing the HTML is easy enough.
BUT OH MY GOD THE ENTIRE WEBSITE IS SHIT. Take this page: http://www.fidal.it/graduatorie.php, it uses some useless jQuery plugin and uses a buttload of JavaScript that isn't even needed. BUT WAIT. Try entering an invalid "club code" (http://fidal.it/graduatorie.php/...), a FUCKING white page with 200, are you kidding?
I'd also like to mention that all pages that require form input won't load correctly if you don't include "submit=Invia" in the URL.
I am not giving up.3 -
DAILY LARAVEL PROBLEMS
I need to parse a JWT with some custom claims. There's a JWT library with Laravel; documentation really lacking, kinda hardcoded to work with Laravel but whatever; it's already installed, let's see what can I do with it.
It turns out I can't say something like "take this token, parse it, tell me it's valid". Let's see how that goes.
You need to build a parsing class with a manager, some auth stuff, a parser.
To build said manager you need a provider that implements a contract, a blacklist, a factory (of what?)
To build the factory (of what?) you need a claim factory and a payload validator
To build the claim factory you need a request
To build the blacklist you need a Storage
To build the storage you need a CacheContract
To build a CacheContract you need IDK it's a mess
To build the contract you need... IDK for real
WHY LARAVEL IS SHIT: 'cause only in this framework it seems reasonable to build this clusterfuck to parse a base64 encoded string, throw some json_decode and check a signature. And have it work only to authenticate a user.1 -
the one that exists (c#) seems underused compared to where it could (or even should) be used. and the place that uses it the most (enterprise) butchers and mangles its use, just as enterprise tends to do with everything.
the one that i'm designing... the fact that it doesn't exist yet, and that even as i'm zeroing in on syntax and philosophy that i'm very much starting to be proud of, i still don't have a proper idea of how to implement even the most basic parser/interpreter for it, not because it's in any way difficult or unusual, but just because... i've never done that before, so i get into weird circular thought paths that produce weird nonsensical code...
... on top of that, i still only have a very, very fuzzy idea of how will it (sometime in extremely distant future) actually implement the most interesting and core feature - event-based continuous (partial) re-parsing of the source code and the fact that traversing the tokens at the leaf level of the syntax tree should result in valid machine code (or at least assembly) that is the "compiled" program.
i *know* it's possible, i just don't yet know enough to have a contrete idea how exactly to achieve it.
but imagine - a programming language where interactive programming is basically the default way of working, and basically the same as normal programming in it, except the act of parsing is also the (in-memory) compilation at the same time, so it's running directly on the hardware instead of via interpretrer/vm/any of that overhead crap.
also then kinda open-source by definition.
and then to "only" write an OS in that, and voilá! a smalltalk-like environment with non-exotic, c-family syntax and actual native performance!
ahhh... <3
* a man can dream *2 -
I wonder what is going on in the minds of inexperienced developers. It must be very interesting.
I just read this bit of code
The task was to implement a certain schema into a database. They were given an ORM library, and several tutorials explaining what an ORM was and how it did it.
The result were these 3 models:
- A defaultUser with all of the defaults values for the User model. It wasn't even supposed to be instantiated, just accessed by `defaultUser.fields.username.default`
- another default table for another model.
- The "main" model, containing all other models in the form of JSON fields that would contain an array with other JSON objects that would represent the instances themselves.
I will say though, they made a home-brew ORM with (most of) all the logic a normal ORM has by parsing through the "main model", except, of course, common sense2 -
!rant
...so I started learning F#... again, second attempt, first one failed like 3 years ago because I had no real usecase appropriate for the language, so I didn't get it.
...but now i'm trying to make my own language, meaning also (wanting) parser/interpreter/compiler, and I found a lecture where dude shows off THREE he wrote (within 24 hours) for three different languages...
...and it showed me that doing my parser/interpreter/compiler in F#, using pattern matching, is going to be incredibly awesome, as opposed to doing it as string parsing in C#...6 -
People in this discussion will refer to my classmates doing their project we were allowed to choose like 3 months ago and back then I already told them they won't ever manage to make it work in time.
People: "yo could you help real fast"
Me: "aaah not again but okay why not"
People: "here's the code"
Me: "the fuck is this? What is it supposed to do?"
People: "ummm it should doooo eeeerrr... Like this and .... Yeah we don't know. The output should look like that but it doesn't"
Me: "the fuck did you even try to do. That'll never work. And I told you the same thing like 3 months ago when you just chose the project. Just go and use some random math lib you find"
People: "kkkk"
Their code seriously made no sense at all. Parsing substring within like 5 nested loops and like 20vars called "s1" and "theString" and what not. -
1. The power you have with it.
With that, I mean solving your own or someone else's issues.
Be it automation, parsing weird formats, or anything else, it can usually be achieved by coding something for the task.
2. The logic (when compared to humans).
Unlike humans, code does what it's told to do. Us humans, well, we often misinterpret things. Code, however, usually has a single meaning.
3. How things can be improved over time.
Finding a way do something faster and implementing it is really satisfying.4 -
Spend several months designing an alternative approach to string matching/parsing not based on parser combination or Regex. Benchmark and profile the ever living hell out of it. Find out the performance is highly competitive with FParsec and far easier on memory than either approach. Discover FParsec responds very badly when failing. Document all of this, do a presentation on the design. Upload video of presentation with links to it on a few sites. Presentation gets downvoted, and receive hatemail. Yay.18
-
Parsing logs to create conditional insert statements cause expert morons fucked up production database cluster.
Database is partially corrupted and cannot be used and they don’t know how to fix it.2 -
Ok the ionic datetime component horror ended. I got it working good enough but I'm not proud of my code. The whole thing is a giant hack parsing dates to and from strings, switching locales, setting months and days and using the month as a daypicker as this fucking component does not allow me display day names in it, only month names. Such a mess... at least now I can work on the stuff that matters.
Actually though about making this open source... I reconsidered.8 -
So, i'm trying to get linkr (a pretty cool short link service) to work in a docker container since 4 hours now to host it on my server. There is no official container because it needs a working database connection and stuff during installation which can only be done via console and (for whatever reason I couldn't find out yet) need to be done while building the container. The problem is, I can't connect it to the database while building the container so there is no database during installation to create tables and stuff and the build will fail. ARGH.
Why the hell would you do this????? Theyre actually saying in their readme there is no dockerfile because the config options are specific to your configuration...?!?!
The thing is entirely written in python, so reading and parsing configfiles on the fly should not really be a problem.
Of course I could ssh into the container and run the installation script but that's not the point.
Docker is not about being lazy.
It's about portability.
Maybe I don't want to bloat my server with your 39579372639 npm dependencies? Or I don't want to install a freakin apache, because I have every other site on nginx and therefore wouldn't work with apache.
AAAAAAAARRRRRRGGHHGGGGG
in the end, I'm probably going to modify the thing to install tables when running the container and giving the first user admin rights instead of prompting to enter credentials for a new admin user.
And yet I didn't even speak python. -
I'm having the dawning realization, reading the utf specification and thinking "parsing the data files isn't too hard..." that the little side project on a side project on a side project isn't going to be finished until like, Christmas.
Fuck.7 -
Spent way too long wondering why my changes in the parsing script wasn't showing up in the output file until I realised that I was in "text.txt" and not "textResult.txt".1
-
Is it possible to develop a wear app for android (devrant) as a third party app? I read that, normally, the main app should process data (like using the API, parsing it and getting the needed data together) and just sends them to the wear to process. Writing a wear app as a third party would mean that the wear app had to calculate the stuff, which is in Googles opinion back practise I guess....?2
-
Parsing my college's terrible classless html float left div mosaic timetables so I could hide lectures I wasn't taking.
-
TLDR;
Side project update.
Made simple nlp library in python and published it’s first version to open source.
Now I can feed it with parsed pdf text.
See rant https://devrant.com/rants/2192388/...
Why ?
Cause during reading book about nltk I couldn’t find simple extendible way to provide support for polish language and I wanted to abstract stemming, word normalization, tokenizer etc. so I can provide ex. different conditions for separate text files and don’t write much code what is an asset when you work solo.
It’s about 12GB of pdf public accessible law data I am trying to handle ( at first ) which is about 35000 files from last 90 years.
So far I automated downloading web pages and pdf documents from them. Extracting data from web pages and saving it to database. Extracting text from pdf files. I have about 5-6 projects to do all of it above maybe at the end I will put it to some workflow manager like Luigi or just run it by cronjob.
First thing for website version 1.0 part is find correlation between all documents inside law text using nlp library by building custom conditions. Then just generate directory structure and html files with links between documents.
Website version 2.0 is already in my mind but it will be creepy to make it and will take at least 1-2 months and I want to publish fast.
I have some pdfs with only images instead of text and tesseract worked quite good with them so maybe I will try to process them when everything go live.
Learned a lot about pdf as now I know that font in pdf is not always providing unicode characters ( stupid form of obfuscation) so when you extract text you need to build glyph vector to text map for every font.
Pdf is full vector representation - just like svg - what is logic if you think a bit and know that some printers are running using postscript.
Let’s hope next update will be about flutter mobile app which started all of shit above. It’s almost ready ( except getting data from api I am trying to do and logo for release version ). It’s last piece of puzzle.3 -
I'm rewriting the wrapper I've been using for a couple years to connect to Lord of the Rings Online, a windows app that runs great in wine/dxvk, but has a pretty labyrinthine set of configs to pull down from various endpoints to craft the actual connection command. The replacement I'm writing uses proper XML parsing rather than the existing spaghetti-farm of sed/grep/awk/etc. I'm enjoying it quite a bit.1
-
Daylight Savings Time.
Europe switches to DST this month and specs are broken because of Ruby's DateTime which isn't DST-aware so it generates invalid dates and can't do date math properly. Losing a couple days to refactoring horrendous code that uses *nested* Timecop.freeze, mixtures of Time, Date, and DateTime, math and parsing revolving around the end of the month.
Death to DateTime and death to DST.1 -
It's sort of two separate projects although they are very tightly related.
The first is a pattern combination library and parsing engine. It takes a superficially similar approach to Regex or parser combinators, but with some important underlying differences.
The second is a specialized (not turing complete) language for rapidly defining full language grammars and parsers/lexers for those languages. -
!rant
finally after months and months of just planning and doing boring stuff a piece of code that was really just fun to code and plan for some days:
i just wrote my first "real" parser for a simple DSL. so much fun! i just really can recommend that to everybody.
i've use a parser combinator. the concept of this parser combinator ist to combine simple parsers (like when it starts with a number or a "-" and continues with numbers then its an integer etc) into a big one. i've written it in c# and used "Sprache" first and after some time i switch to "Superpower". a really great lib, but lacks a bit of documentation. anyway, i've your're interested in these things and want learn how your "daily code" gets parsed i would recommend that to you! :)
greetings to all fellow devRanters and happy coding / parsing! :)1 -
So I see posts about an interview question/challenge of inverting a binary tree. I don't use trees very often (mainly file related or parsing server nodes), but I thought I would learn how to do this.
I saw a page that started talking about different ways to invert enough to understand that one type of inversion is swapping left and right nodes. So I stopped before they showed how.
Then I created a test program that has a tree structure and also can display a tree before and after modification. This was kind of fun.
So then I wrote the inversion function. It was less than 10 lines of code. Wtf? I thought it would be harder than this.
Then I started wondering where trees were used. So today I have been learning how they are used and why I might need one to solve a problem. One use I intuited was parsing regex or a language. Apparently it is useful there.
What I am learning is that a lot of these interview questions are really test to see if you can comprehend instructions when stressed. Or you will ask questions to clarify the task. It doesn't necessarily test your ability to solve hard problems.
One thing that perplexes me. If inverting a tree is swapping nodes left<->right, then why not leave data in place and just swap roles in the functions. Maybe I completely misunderstood what inversion means or why it would be done. I guess if this is not inverting I have the structure to try other methods now.2 -
by this point I just hate rust for being inconsistent
made a cargo command with clap
it works if you run it directly but if you install it with cargo globally then evidently it's for no reason sending in an extra argument into the program which messes everything up
of course clap has no documentation about this so I jank my way to just work around the issue by parsing the arguments before I give them to clap
and on cargo -- list my command doesn't have a description even though the thing I copied has a working description in that listing, and my cargo file has a description, and clap is happily inferring my description from said cargo file
chatGPT thinks I just need the description field filled in the cargo file and it'll work. well it doesn't.
how is a language this young and this much of a mess already
why the hell did they randomly decide to pass in extra arguments? argh.1 -
Yesterday I asked a question on stack overflow about what algorithm I should use in order to parse command line strings like in gnu getopt for example.. And I've got downvoted for no GOOD FUCKING REASON. On top of that, my question is on hold. WTF?! For some time now stack overflow is becoming more and more a community of fucking cunts, arse-holes and toxic people.
Title:
" What parser algorithm is best suited for command line parsing? [on hold] "
My question:
"I want to write my own command line library from scratch. What algorithm should I use in order to parse gnu style args like in getopt for example ? I mean what's the best way other than tokenizing and parse them in a naive way? Should I try to look at LR, LL algorithms or this is way too overkill?"
Their response:
"Your question sounds like "I want to do X. What's the best way to do it?". Too broad, you need to be more specific about what problem you're having. (And keep your question clean. No meta-stuff in there.)"
I mean, what more context-specific reason should I add you dense motherfucker!? I want an algorithm to parse your momma's cunt so hard 'till it blows the fuck up. This what you want? You fucking senseless piece of garbage. God, give me a car to run over their fucking internet cable and over their head, too.8 -
I feel like I may be done with dev... the imposter syndrome has been hitting hard lately. I really want to get into Natural Language Processing, I'm currently looking at skip-gram parsing a dictionary using Word2Vec, then I came across a paper called dic2vec which looks promising.. half the time, I just think I'm barking up the wrong tree, or that it's been done before. Most times I conclude that I have nothing new to offer and there's gotta be half a thousand people like me, striving in the same space. Possibly failing. Don't get me wrong, the state of consumer software at the moment NECESSITATES my involvement (I'm looking at you (epic games, windows) , every which way I look at it. I just don't know where or how to get going. Viva la revolution. A toast, to shitty software and exceptionally low moral *klink*6
-
Aka... How NOT to design a build system.
I must say that the winning award in that category goes without any question to SBT.
SBT is like trying to use a claymore mine to put some nails in a wall. It most likely will work somehow, but the collateral damage is extensive.
If you ask what build tool would possibly do this... It was probably SBT. Rant applies in general, but my arch nemesis is definitely SBT.
Let's start with the simplest thing: The data format you use to store.
Well. Data format. So use sth that can represent data or settings. Do *not* use a programming language, as this can neither be parsed / modified without an foreign interface or using the programming language itself...
Which is painful as fuck for automatisation, scripting and thus CI/CD.
Most important regarding the data format - keep it simple and stupid, yet precise and clean. Do not try to e.g. implement complex types - pain without gain. Plain old objects / structs, arrays, primitive types, simple as that.
No (severely) nested types, no lazy evaluation, just keep it as simple as possible. Build tools are complex enough, no need to feed the nightmare.
Data formats *must* have btw a proper encoding, looking at you Mr. XML. It should be standardized, so no crazy mfucking shit eating dev gets the idea to use whatever encoding they like.
Workflows. You know, things like
- update dependency
- compile stuff
- test run
- ...
Keep. Them. Simple.
Especially regarding settings and multiprojects.
http://lihaoyi.com/post/...
If you want to know how to absolutely never ever do it.
Again - keep. it. simple.
Make stuff configurable, allow the CLI tool used for building to pass this configuration in / allow setting of env variables. As simple as that.
Allow project settings - e.g. like repositories - to be set globally vs project wide.
Not simple are those tools who have...
- more knobs than documentation
- more layers than a wedding cake
- inheritance / merging of settings :(
- CLI and ENV have different names.
- CLI and ENV use different quoting
...
Which brings me to the CLI.
If your build tool has no CLI, it sucks. It just sucks. No discussion. It sucks, hmkay?
If your build tool has a CLI, but...
- it uses undocumented exit codes
- requires absurd or non-quoting (e.g. cannot parse quoted string)
- has unconfigurable logging
- output doesn't allow parsing
- CLI cannot be used for automatisation
It sucks, too... Again, no discussion.
Last point: Plugins and versioning.
I love plugins. And versioning.
Plugins can be a good choice to extend stuff, to scratch some specific itches.
Plugins are NOT an excuse to say: hey, we don't integrate any features or offer plugins by ourselves, go implement your own plugins for that.
That's just absurd.
(precondition: feature makes sense, like e.g. listing dependencies, checking for updates, etc - stuff that most likely anyone wants)
Versioning. Well. Here goes number one award to Node with it's broken concept of just installing multiple versions for the fuck of it.
Another award goes to tools without a locking file.
Another award goes to tools who do not support version ranges.
Yet another award goes to tools who do not support private repositories / mirrors via global configuration - makes fun bombing public mirrors to check for new versions available and getting rate limited to death.
In case someone has read so far and wonders why this rant came to be...
I've implemented a sort of on premise bot for updating dependencies for multiple build tools.
Won't be open sourced, as it is company property - but let me tell ya... Pain and pain are two different things. That was beyond pain.
That was getting your skin peeled off while being set on fire pain.
-.-5 -
I've been trying Flutter the past 2days. I liked how amazing Flutter framework is but I also hated how UGLY the Dart language is. First of all, I've been doing Java(Android) and JSX(React). So, after coming from these languages, Dart seems awful. Here are the things that I hated the most:
1. JSON parsing: The worst part of Flutter/Dart. No GSON/Jackson equivalent. not even possible to make one as described in the doc
2. Redux for flutter. I tried and I hated it. So, Tried some other state management libs
3. the way static functions are written in a class
4. Widgets hiarachy
5. Ambiguity - "this context is not the context we need, we need that one to make it work. so, pass that even when you don't have it" (if you have used flutter, you will know what I mean)4 -
I wish there’s a better engine for parsing HTML for relevant content from a URL than mercury for free.
I was making a class project in react and that is not getting it done to my thing. 😑😑 -
I'm sure this has been discovered before but I just realized that a lexer defined as a set of functions which tail-call each other with the leftover text to switch states can record the location of tokens from the back of the string, thereby eliminating a parameter from pretty much every function. The world is full of wonder.2
-
stop using arrow function everywhere!!!!!!!
what that is mean ?
fns.reduce( (prev, fn) => fn(prev), input)
Are this is `fns.reduce` with two parameters
Or arrow function that return `input` variable.
take your time to visual parsing this crap4 -
The Uniparser: a single, self-describing general parser capable of parsing/syntax highlighting/linting every language ever made.
-
So I have a question to anyone familiar with the General Transit Feed Specification...
Why is the data provided in text files? Is there not a way to format the data to allow for random access to it?
Like I'm currently writing a transit app for a school project, and as far as I can tell, the only way to get all specific stops for a route, is to first look up all trips in a route, then look up all the stopids that are associated with a trip in stoptimes.txt (while also filtering out duplicates since the goal is to get stop ids, not specifically stop times) and then look up those stop ids in the stops.txt file.
The stoptimes file alone is over 500000 lines long, unless there is a way better way to be parsing the data that I'm not aware of? Currently I'm just loading the entire stoptimes file into a data structure in memory because the extra bit of ram used seems negligible compared to the load times I'm saving...
Would it be faster if I just parsed all the data once and threw it into a database? (And then updated the database once a month when the new data comes in?)3 -
Okay, yes, modsecurity WAF is amazing and all, but... When one tries to implement its rules atop an existing app that wasn't developed in accordance to the rules... That hurts.
How tf am I supposed to parse and present a 6.5GB / 22M line audit log to the client?! Just parsing that monstrosity once takes *minutes*, let alone doing any sort of sorting / analysis!
I feel sick. This is exactly why I am a sysadmin and not a programmer, I don't like writing analysis stuff, or programs more complex than a few hundred lines of bash... :|5 -
Well, that's a lovely youtube ad, if I ever saw one
XML Parsing Error: no root element found
Location: https://secure.jdn.monster.com/rend...-...
Line Number 1, Column 1:
^
By the looks of it, it may have been an ad for monster.com . -
Question.. architecting a large system. I’ve broken it down to microservices for the DB and rest API / gateway
I want there to be some some processes that run continuously not event driven via rest. Say analytics for example what is the best way todo that? Just another service running on on a server? And said service has its own API? That when the other rest APIs are called could then hop and call the new service?
Or say we had a PDF upload via rest should that service then do the parsing before uploading to DB .. or should the rest api that does the uploading then call another rest api to another service dedicated todo the parsing and uploading to the db?
I think the bigger way to explain the question is the encapsulation between DAL.. data access layer which I have existing.. but then there’s the BLL .. buisness logic layer which I don’t know if it should have its own APIs via own microservices running in the background.10 -
I use the ICU format often for translation because it's simple enough and supported on many platforms. It's something of a standard so I can use the same translation string format and similar library functions everywhere.
ICU is like a really simple templating language, somewhere between printf and something like smarty or twig simplified and specifically intended for internationalisation.
I updated a library providing ICU compatible parsing and formatting for one of the platforms I'm using and find tests break. I assume that only thing to change is the API. ICU very rarely changes and if it did it would be unexpected for it to break the syntax in a major way without big news of a new syntax.
The main contributor of the library has changed since some time last year. Someone else picked up the project from previous contributors.
Though the library is heavily advertised as using ICU it has now switched to using a custom extended format that's not fully compatible and that is being driven by use case demand rather than standardisation.
Seems like a nice chap but has also decided for a major paradigm shift for the library.
The ICU format only parses ICU templates for string substitution and formatting. The new format tries to parse anything that looks XML like as well but with much more strict rules only supporting a tiny subset of XML and failing to preserve what would otherwise be string literals.
Has anyone else seen this happen after the handover of an opensource library where the paradigm shifts?3 -
So, do any of your poor fuckers have the opportunity - nay, PRIVILEGE of using the absolute clusterfuck piece of shit known as SQL Server Integration Services?
Why do I keep seeing articles about how "powerful" and "fast" it is? Why do people recommend it? Why do some think it's easy to use - or even useful?
It can't report an error to save its life. It's logging is fucked. It's not just that it swallows all exceptions and gives unhelpful error messages with no debugging information attached, its logging API is also fucked. For example, depending on where you want to log a message - it's a totally different API, with a billion parameters most of which you need to supply "-1" or "null" to just to get it do FUCKING DO SOMETHING. Also - you'll only see those messages if you run the job within the context of SQL FUCKING SERVER - good luck developing on your ACTUAL FUCKING MACHINE.
So apart from shitty logging, it has inherited Microsoft's insane need to make everything STATICALLY GODDAMN TYPED. For EVERY FUCKING COMPONENT you need to define the output fields, types and lengths - like this is 1994. Are you consuming a dynamic data structure, perhaps some EAV thing from a sales system? FUCK YOU. Oh - and you can't use any of the advances in .NET in the last 10 years - mainly, NuGet and modern C# language features.
Using a modern C# language feature REMOVES THE ABILITY TO FUCKING DEBUG ANYTHING. THE FUCKER WILL NOT STOP ON YOUR BREAKPOINTS. In addition - need a JSON parsing library? Want to import a SDK specific to what you're doing? Want to use a 3rd party date library? WELL FUCK YOU. YOU HAVE TO INDEPENDENTLY INSTALL THE ASSEMBLIES INTO THE GAC AND MAKE IT CONSISTENT ACROSS ALL YOUR ENVIRONMENTS.
While i'm at it - need to connect to anything? FUCK YOU, WE ONLY INCLUDE THE MOST BASIC DATABASE CONNECTORS. Need to transform anything? FUCK YOU, WRITE A SCRIPT TASK. Ok, i'd like to write a script task please. FUCK YOU IM GOING TO PAUSE FOR THE NEXT 10 MINUTES WHILE I FIRE UP A WHOLE FUCKING NEW INSTANCE OF VISUAL STUDIO JUST TO EDIT THE FUCKING SCRIPT. Heaven forbid you forget to click the "stop" button after running the package and open the script. Those changes you just made? HAHA FUCK YOU I DISCARDED THEM.
I honestly cant understand why anyone uses this shit. I guess I shouldn't really expect anything less from Microsoft - all of their products are average as fuck.
Why do I use this shit? I work for a bunch of fucks that are so far entrenched in Microsoft technologies that they literally cannot see outside of them (and indeed don't want to - because even a cursory look would force them to conclude that they fucked up, and if you're a manager thats something you can never do).
Ok, rant over. Also fuck you SSIS1 -
Come on, how hard can it be?
On every fucking TLV data structure I get to handle, the hobo who defined the structure obviously stopped reading the TLV specification after the second sentence.
Fucked up tags, misuse of length encoding, and as a result no real TLV parser can handle that crap. Workarounds and manual parsing all over the place for *every* *single* interface.
Get your shit together, and if you don't want to handle the complex parts, then at least make the simple types right. -
How come so many dev teams are working with blindfolds on?
We have two projects that communicate using endpoints. One of them throws a parsing error with some data. Cool, just give the calling project some debug references and attach a debugger right?
Apparently not. I haven't figured out why we can't do that, it seems like the project only works using nuget references so we never get any debug info for the other project.
Asked around how we usually solve issues like this. The answer: "idk the codegen always works, so we never solve issues like this".
What.
It "always works". Except now it doesn't. And you've never tried debugging it? Instead just working with blindfolds on trying random shit until it does?
This is far from the first time I've heard this on a team. That and "we don't need error codes, if something goes wrong we have to fix it either way". I'm losing faith in the dev world... -
Implemented a feature against a "restful" json api. The feature works, test-driven development ftw.
Yet on the run with the live api: certain important fields all only contain the value `0`.
Confused I asked around what's going on, expected a bug in the api. Now I've been told that those fields never worked and the relevant information has to be gathered by either querying against a (deprecated!) mysql database. Or use a different endpoint increasing the http request overhead by factor over 1000.
We call it team work. -
How do you guys work with forms on backend? Do you have framework in your language or do you work with raw body, parsing and validating everything with your own tools?7
-
My dumbass thinking it would be easy to get a string value of an exported symbol in a .so and now I'm manually parsing and applying symbol relocations3
-
Built the most generic file importer.
So a customer had his SAP system giving us some 5 million barcodes in a csv which we needed to parse. But as there could be different file types and I thought the handling would always include the same steps I made them configurable through function pointers. - Did not want to make it as spooky as the rest of the code base where the function pointers were buried deep in some shared memory configs, which might even change at run time, but rather I statically used the member functions of my class. Just to poke fun on the ugly C++ syntax of member function pointers. I still shudder at the thought some poor soul now has to maintain that code.
(For the actual parsing I actually used a one liner in awk which was churning through the records in one minute which was faster than the SAP guys seemed to be accustomed to.) -
I'm parsing the results from a hierarchical database query to make it compatible with another hierarchical database. Specifically, I'm parsing user created notes from a pipe-delimited CSV that was translated from the query result. Everything runs smoothly. I send my output to be processed through our batch system. Everything checks out as expected.
A couple blank notes throw exceptions, but I knew that was going to happen. Then suddenly, I read "Cannot create note with unprintable characters"
Okay?
Somehow this person had been able to type the Ash (æ) character and an accented A (à) this came from a system that didn't even support characters outside of alphanumerics. How? -
I was so happy after I managed to integrate some Python code into Java with Jython. Then I realized the application uses 1200Mb of RAM, it should run on a small home router with 1GB of RAM. Yeah... Giving up Jython and going back to old fashion command-line parsing!2
-
That feeling when you inherit a script to automate something that takes 10 seconds. Why would they even write this? It's not like the task is hard....
...
And why would they write it this way? I'm sure if I just move this part and ....
That feeling when you spend several hours improving and redesigning a perfectly functional script to automate a 10 second task for zero gain aside from cleaner code. "But the code for this quick-and-dirty script I'm never going to look at again looks so much better now!"
... If only it did a bunch of complicated parsing, regex matching, and error checking just so I can answer one less prompt.... Unless that parsing fails. Then it should still ask me for that prompt... And also validate that the answers I give are valid and correct....
That feeling when you spend a whole nother day starting from scratch to implement error checking and complex parsing logic knowing full well the original task takes 10 seconds to do manually and is needed at most twice a day (for a grand total of 20s a day)
WHY AM I LIKE THIS?!?!?!4 -
What’s your favorite method to deal with using and securely storing usernames and passwords?
Example: parsing information from a part of a website requiring a login
Everyone I talk to at school has their own different method and now I’m curiously looking for as many methods as people are willing to share13 -
Is there actually a proper way to do Math in Javascript, without parsing every value to avoid something like
"1" + 3 = 13 ?rant trying to learn proper math start to hate dynamic var types wtf is going on with that language javascript advice needed11 -
I despise whovere come up with VBA IDE. Why is all the code confined within one window when multimonitor setups existed even back in the 14'' CRT era? And what sort of a fuckhead thought it's a good idea to throw a pop-up (with a notification sound!) every time you want to move from partially written line to a different one?
But hey, I managed to optimise the data parsing from 97 down to 3 sec.3 -
Hey guys. I need your advice about writing a documentation. I want to make a flowchart with all processes of this client’s mobile app so I could see the UX logic and also all UI screens in the app. I also want to somehow add backend info in order to see which endpoints are being called in these screens and what type of responses are we parsing there.
Currently flowchart is done in draw.io, some of sketches are in zeplin, and there is 0 backend documentation just some implementation in our mobile app.
I would like to combine all of this into some useful document/overview so I could pass this doc to a junior dev and he could jump working into project without a problem.
Do you know any tools to do this?1 -
I'm currently working on a project that scrapes the SEC's EDGAR website for type 4 filings.
I currently have the required data in raw text format that somehow looks like xml, i really can't tell what it is but i'm trying to parse this data into json.
I've not parsed something as complex as this before and will appreciate any form of pointers as to how to go about this.
i have attached a screenshot of one sample.
this link fetches the data of a single filing in text format.
https://sec.gov/Archives/edgar/...5 -
Hello Devs!
I'm a huge fan of football, and I want to understand how apps like flashscore or livescore work ( feed sources, parsing data, real time notifications, etc.). Please, point me to the right direction.5 -
This is a rant, this is a FUC$%NG RANT:
Damn you Apple and making parsing iOS receipts such a pain!
Said while screaming for help 😱😱😱😱 -
the back-end now has spend 9 months to be able to upload phonebooks .... and it still doesn't work :'D
How... It's not even a complex feature. Just parsing a simple xslx file. I don't know how incompetent a senior developer can be.9 -
Nothing better than Rust and LALRPOP. I've been trying to play with Bison and C++ for the whole quarantine and nothing would compile. I just sat for two days with Rust and LALRPOP and I was able to make a small interpreter that can make new variables, calculate simple expressions and print stuff. Like this:
var = 5 + 3;
print var;
var = 2 * var + 4;
var2 = 3 * var + 3 * (var + 4);
print var2 * var;
And all this in just two days. I have no Rust experience except for toying with it on an online playground. I have no LALRPOP nor parsing experience. Two days.
Now, it's not like I wouldn't be able to do this in C++ too if somebody told me how to. But nobody has. And the documentation online is gruesome. None of the bison example I found online could compile. This is why documentation matters people! Honestly, if there's one reason I think old projects die, it's because they ether don't update themselves OR they don't update their documentations. Look at the US government now, looking for COBOL programmers.4 -
For a little background on the sort of stuff I'm dealing with, check out my last rant.
Anyways, I'm testing this pipeline at work and was just reminded of the fucktarded way a "software engineer", who had a bachelor's biology degree, decided to handle a json file.
The script is question is loading a json file containing an array of objects. The script is written in perl. There's a JSON module. Use that? Fuck no! Let's rather perform an in-place sed command on the file substituting the commas separating objects in the array with newlines, then proceed to read the file line-by-line and parse out the tokens manually. Mind you, in the process of adding the newlines he didn't keep the commas, so now all of these json files his bullshit handled are invalid json that cannot be parsed.
The dumb ass was lucky the data in the file is always output upstream as a single line and the tokens for each object are always in the same order, so that never led to problems. But now, months later after I fixed his stupidity I am being reminded of it again as I'm testing and debugging some old projects as part of regression testing new changes I'm making.
TL;DR Fuck dumbwit motherfuckers who can't even google search "parsing a json file" and doing literally anything that is less fucktarded than manually parsing a json file2 -
Retrofit + Gson is faster than Volley + my static helper class for parsing JSONArrays -.-
I feel defeated //android4 -
I wrote about Java sockets and networking a few weeks ago. As a follow up, how would you manage state in a scenario with a server and multiple clients? A few solutions I've thought of:
Making the client and the server share a common class that makes the both of them know what can and can not be done.
Having a "Message" class which will contain a static method for parsing a received payload and return a Message instance with intent and methods to resolve intent (i.e. sending data to the client).
Perhaps both concepts should be used, or perhaps none. I'd like to hear your ideas :)8 -
I like Firefox a lot.
But it isn't very nice with WPAs, an area of my interest, and downloads PDFs instead of showing them...
Plus I have seen Vivaldi is pretty good for quite some things, like tabs groups and tabs hibernation, has notes, a cool calendar...
But Chrome's console...
It's the only reason I stay with Firefox. (I not only use it at work, but I also use the command line as a pocket JS engine for little scripting and parsing.)
If only I could get selection bracket wrapping and a multiline editor... is it that hard?4 -
Okay I usually like Swift, but why did they have to make parsing JSON such a pain in the ass?? You have to loop through every child collection and should give up the type that it should be. Obviously you could work around this by writing some sort of wrapper but if I want to do a simple request to an API it results it so much unnecessary code.1
-
I've to say that javascript is no language I like because I'm more fastinated of building a nice and scalable backend then building a gui (in the browser).
The funny thing is that right now javascript helps me at my current project because many websites implement wide-opened apis for their js frontend and it just works like a charm to use them instead of parsing the whole html and do some XPath stuff to fetch information. -
Been debugging my angularjs rest request for about 7 hours, only to realize the problem was with the parsing of the returned object...
And no I don't know why I'm working with angularjs.1 -
Worst dev tech, it has to be MS Word. Apparently they follow the OOXML guidelines to create a Word doc but they have to put in their own shit-flavored ingredients to increase a developer's misery. Parsing that document is pure agony.
I was looking for a way to find the default font size of the document and found this gem https://social.msdn.microsoft.com/F... -
Client creates bug report for me (Android dev): "Customer 193796 gets a JSON parsing error when trying to access their order history."
...okay, what the hell am I expected to do about that? -
The tokenization of Orchid files depends on the exports of imported files, but in a way that never influences their exports or imports, so Orchid allows circular references.
I sometimes feel like my subconscious and conscious mind use the scope of my projects to annoy each other.2 -
Does anybody know if there's a tool for parsing protobuf using live Network capture? I basically want to be able to pass profiles into something like Wireshark and get a live request response cycle1
-
Just turned myself inside out why parsing data from serial didn't work. A full day, and 20 minutes before going home I realize the uart interrupt function was too slow. I just didn't get all chars...
-
I swear to god, getting Chumsky to do my bidding has almost taken longer than writing a parser by hand. I'm not looking for operator precedence, I'm not looking for complicated rules or anything, the main part of my language is literally just S-expressions, with some top level bells and whistles.
I don't even have a working lexer yet because I wanted to use this piece of shit library which usually matches the fewest possible characters to parse significant newlines but the Padded combinator takes as much whitespace at the end as it can find, and a host of other atomics don't actually adhere to the library's lazy principle in their procedural implementation. I've had enough. I'm going to bed, and tomorrow I'm writing tickets.
Actually, I'll probably also write PRs because I actually want the fixes to exist and not just complain about the problems, but I also really want to complain before I get started on that because I spent about two weeks just on this bullshit.3 -
Anyone using lirc (Linux infrared remote control) and got a Windows or Android program to control it through the network?
I did not find any that actually worked so I started my own yesterday.
The communication with lirc is surprisingly easy.
My next step is parsing the remote configuration files from lirc.
After that I need to get the GUI look better.1 -
There's a moment when you're looking at a reStructuredText parser, you have a basic idea of the spec, and you know with absolute certainty it's not parsing it correctly. It gets the basics right, sure. But now you know why the tools that use it have disclaimers like "Any para lists or other directives must be at the end of the string". And you're thinking "how hard would it *really* be to write one?". After all, I'm now extracting all the docstrings from the code... Is it really that much more? Shit...
-
A young new dev was working on his first ticket, about a bug during parsing of an uploaded excel file. Our issue was that if the file contained an empty line, all remaining rows were ignored. So the task included extending our tests to cover this case. After 2 weeks (!), his merge request comes in. His idea (without ever asking for help) was to parse the whole file (in some cases huge) in the production code a second time, just to count the rows (!!) and save the count in a public static int field, which was verified in his new test.2
-
[Prestashop question / rant]
Yes, it's not StackOverflow here, neither is it prestashop support forum - but I trust u people most :)
Proper solution for working with big(?) import of products from XML (2,5MB, ~8600 items) to MySQL(InnoDB) within prestashop backoffice module (OR standalone cronjob)
"solutions" I read about so far:
- Up php's max execution time/max memory limit to infinity and hope it's enough
- Run import as a cronjob
- Use MySQL XML parsing procedure and just supply raw xml file to it
- Convert to CSV and use prestashop import functionality (most unreliable so far)
- Instead of using ObjectModel, assemble raw sql queries for chunks of items
- Buy a pre-made module to just handle import (meh)
Maybe an expert on the topic could recommend something?3 -
A system to build note-taking systems. tatatap dot com.
It’s the most successful for a few reasons: it got launched, people find it useful, but most importantly it’s been fun and continues to be fun to work on.
I think the fun-to-make factor is massively underestimated as a success indicator. Working on the right product (whatever that means) that is unenjoyable is like using an amazing computer with a broken keyboard. It’s never going to work.
Sure, with any project there’s annoying stuff, but it’s the trend overall. Is the core functionality fun to work on?
In the case of Tap the core component is a notation parser, open sourced called sowhat, github dot com/tatatap-com/sowhat
That was super fun to make and learn about lexing and parsing. It’s pretty far along but there’s still a lot I’m planning to add. -
I am trying to extract data from the PubSub subscription and finally, once the data is extracted I want to do some transformation. Currently, it's in bytes format. I have tried multiple ways to extract the data in JSON format using custom schema it fails with an error
TypeError: __main__.MySchema() argument after ** must be a mapping, not str [while running 'Map to MySchema']
**readPubSub.py**
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
import json
import typing
class MySchema(typing.NamedTuple):
user_id:str
event_ts:str
create_ts:str
event_id:str
ifa:str
ifv:str
country:str
chip_balance:str
game:str
user_group:str
user_condition:str
device_type:str
device_model:str
user_name:str
fb_connect:bool
is_active_event:bool
event_payload:str
TOPIC_PATH = "projects/nectar-259905/topics/events"
def run(pubsub_topic):
options = PipelineOptions(
streaming=True
)
runner = 'DirectRunner'
print("I reached before pipeline")
with beam.Pipeline(runner, options=options) as pipeline:
message=(
pipeline
| "Read from Pub/Sub topic" >> beam.io.ReadFromPubSub(subscription='projects/triple-nectar-259905/subscriptions/bq_subscribe')#.with_output_types(bytes)
| 'UTF-8 bytes to string' >> beam.Map(lambda msg: msg.decode('utf-8'))
| 'Map to MySchema' >> beam.Map(lambda msg: MySchema(**msg)).with_output_types(MySchema)
| "Writing to console" >> beam.Map(print))
print("I reached after pipeline")
result = message.run()
result.wait_until_finish()
run(TOPIC_PATH)
If I use it directly below
message=(
pipeline
| "Read from Pub/Sub topic" >> beam.io.ReadFromPubSub(subscription='projects/triple-nectar-259905/subscriptions/bq_subscribe')#.with_output_types(bytes)
| 'UTF-8 bytes to string' >> beam.Map(lambda msg: msg.decode('utf-8'))
| "Writing to console" >> beam.Map(print))
I get output as
{
'user_id': '102105290400258488',
'event_ts': '2021-05-29 20:42:52.283 UTC',
'event_id': 'Game_Request_Declined',
'ifa': '6090a6c7-4422-49b5-8757-ccfdbad',
'ifv': '3fc6eb8b4d0cf096c47e2252f41',
'country': 'US',
'chip_balance': '9140',
'game': 'gru',
'user_group': '[1, 36, 529702]',
'user_condition': '[1, 36]',
'device_type': 'phone',
'device_model': 'TCL 5007Z',
'user_name': 'Minnie',
'fb_connect': True,
'event_payload': '{"competition_type":"normal","game_started_from":"result_flow_rematch","variant":"target"}',
'is_active_event': True
}
{
'user_id': '102105290400258488',
'event_ts': '2021-05-29 20:54:38.297 UTC',
'event_id': 'Decline_Game_Request',
'ifa': '6090a6c7-4422-49b5-8757-ccfdbad',
'ifv': '3fc6eb8b4d0cf096c47e2252f41',
'country': 'US',
'chip_balance': '9905',
'game': 'gru',
'user_group': '[1, 36, 529702]',
'user_condition': '[1, 36]',
'device_type': 'phone',
'device_model': 'TCL 5007Z',
'user_name': 'Minnie',
'fb_connect': True,
'event_payload': '{"competition_type":"normal","game_started_from":"result_flow_rematch","variant":"target"}',
'is_active_event': True
}
Please let me know if I m doing something wrong while parsing the data to JSON. Also, I am looking for examples to do data masking and run some SQL within Apache Beam4 -
#Suphle Rant 7: transphporm failure
In this issue, I'll be sharing observations about 3 topics.
First and most significant is that the brilliant SSR templating library I've eyed for so many years, even integrated as Suphle's presentation layer adapter, is virtually not functional. It only works for the trivial use case of outputting the value of a property in the dataset. For instance, when validation fails, preventing execution from reaching the controller, parsing fails without signifying what ordinance was being violated. I trim the stylesheet and it only works when outputting one of the values added by the validation handler. Meaning the missing keys it can't find from controller result is the culprit.
Even when I trimmed everything else for it to pass, the closing `</li>` tag seems to have been abducted.
I mail project owner explaining what I need his library for, no response. Chat one of the maintainers on Twitter, nothing. Since they have no forum, I find their Gitter chatroom, tag them and post my questions. Nothing. The only semblance of a documentation they have is the Github wiki. So, support is practically dead. Project last commit: 2020. It's disappointing that this is how my journey with them ends. There isn't even an alternative that shares the same philosophy. It's so sad to see how everybody is comfortable with PHP templating syntax and back end logic entagled within their markup.
Among all other templating libraries, Blade (which influenced my strong distaste for interspersing markup and PHP), seems to be the most popular. First admission: We're headed back to the Blade trenches, sadly.
2nd Topic: While writing tests yesterday, I had this weird feeling about something being off. I guess that's what code smell is. I was uncomfortable with the excessive amount of mocking wrappers I had to layer upon SUT before I can observe whether the HTML adapter receives expected markup file, when I can simply put a `var_dump` there. There's a black-box test for verifying the output but since the Transphporm headaches were causing it to fail, I tried going white-box. The mocking fixture was such a monstrosity, I imagined Sebastian Bergmann's ghost looking down in abhorrence over how much this Degenerate is perverting and butchering his creation.
I ultimately deleted the test travesty but it gave rise to the question of how properly designed system really is. Or, are certain things beyond testing white box? Are there still gaps in the testing knowledge of a supposed testing connoisseur? 2nd admission.
Lastly, randomly wanted to tweet an idea at Tomas Votruba. Visited his profile, only to see this https://twitter.com/PovilasKorop/.... Apparently, Laravel have implemented yet another feature previously only existing in Suphle (or at the libraries Arkitekt and Deptrac). I laughed mirthlessly as I watch them gain feature-parity under my nose, when Suphle is yet to be launched. I refuse to believe they're actually stalking Suphle3 -
I know it a strange thing to ask but I need help with English grammar ( I am not a English native)
I am doing an assignment on context free grammar program.
What the program need to do it "Produce a grammar tree based on the rule"
For example a sentence "Jame sat.", It will need to produce a grammar tree like this.
S
_____|___
NP VP
| |
N V
| |
holmes sat
The only thing I need to do it give a grammar rule to the program like this
S -> NP VP
NP -> N | Det N
VP -> V | NP
What the above rule said is that the sentence is composed of Noun Phrase and Verb Phrase and Noun Phrase is either is a Noun or a determiner and Noun.
Now here is the meat of the question.
I am having trouble parsing a grammar tree for this sentence "She never said a word until we were at the door here"
I have provided my grammar rule below. I think I forget to add some grammar rule or some of my grammar rule are incorrect. I am not a native English speaker so I am having trouble on it.
The search engine have fail me (I have look up various grammar tree) so I am asking for help on devrant. Grammar nazi are warmly welcome :)
S -> NP VP | S Conj VP | S Conj S
NP -> N | Det NP | N PP | Adj N | Det Adj N | Det N
PP | Det NOM | Det N PP
VP -> V | V NP | V NP PP | V PP | V NP PP Adv
VP -> Adv V NP | VP PP | VP AVP | V PP PP | V
ADJP
PP -> P NP
AVP -> Adv | Adv Adj | Adv AVP
NOM -> Adj Adj N | Adj N | Adj NP
ADJP -> Adj | Adj PP
**I am not sure about these two clause below**
VP -> V | Adv
NP -> Adj Adj N22 -
Once upon a time I was working with an engineer who loved sed and awk a bit too much. We had data stored in SharePoint that was retrievable via an RSS feed. Said engineer insisted on using curl to grab the feed and sed/awk to parse the HTML ...
I on the other hand suggested using libcurl (primarily for NTLM auth support) and parsing the RSS feed using libxml.
Which engineer do you think management decided supporting?
Hint: Reusability and maintainability were big requirements in this project.1 -
So i have been after this null exception for days now in my webhook my senior gave me the asp
And they told me like make a new project out of it i kept on passing my dialogueflow agents and kept getting null exception and today i finally figured out it was the code for v1 of dialogueflow and today i wrote a new json parsing code and voila it passed im so happy but i encountered new error just few lines ahead about that unexpected character encountered ugh I'm so tired1