Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "are we winning yet"
-
They keep training bigger language models (GPT et al). All the resear4chers appear to be doing this as a first step, and then running self-learning. The way they do this is train a smaller network, using the bigger network as a teacher. Another way of doing this is dropping some parameters and nodes and testing the performance of the network to see if the smaller version performs roughly the same, on the theory that there are some initialization and configurations that start out, just by happenstance, to be efficient (like finding a "winning lottery ticket").
My question is why aren't they running these two procedures *during* training and validation?
If [x] is a good initialization or larger network and [y] is a smaller network, then
after each training and validation, we run it against a potential [y]. If the result is acceptable and [y] is a good substitute, y becomes x, and we repeat the entire procedure.
The idea is not to look to optimize mere training and validation loss, but to bootstrap a sort of meta-loss that exists across the whole span of training, amortizing the loss function.
Anyone seen this in the wild yet?5 -
In reply to:
https://devrant.com/rants/3957914/...
Okay, we must first establish common ground here. What do we understand about "showing"? I understand you probably mean displaying/rendering, more abstractly: "obtaining". Good, now we move on.
What's the point of a front-end? Well, in the 90's that used to be an easy answer: to share information (not even in a user-friendly way, per se). Web 2.0 comes, interaction with the website. Uh-oh, suddenly we have to start minding the user. Web 3.0 comes, ouch, now the front-end is a mini-backend. Even tougher, more leaks etc. The ARPAnet was a solution, a front-end that they had built in order to facilitate research document-sharing between universities. Later, it became the inter(national) net(work).
First there was SGML to structure the data (it's a way of making it 'pretty' in a lexicographical way) and turn it into information (which is what information is: data with added semantics) and later there was HTML to structure it even further, yet we all know that its function was not prettification, but rather structure. Later came CSS, to make it pretty. With its growing popularity, the web started to be used as a publishing device.
source:
https://w3.org/Style/CSS20/...
If we are to solely display JSON data in a pretty way, we may be limiting ourselves to the scenario of rendering pretty web pages using aesthetic languages such as CSS. We must also understand that if we are only focusing on making a website pretty with little to moderate functionality, we aren't really winning. A good website has to be a winner in all aspects, which is why frameworks came into existence, but.. lmao, let's leave that to another discussion.
Now let me recall back my college days.. front-end.. front-end.. heck, even a headset can be a front-end to a pick-order backend. We must think back to the essence, to the abstract. All other things are just implementations of it (yes, the horrendous thousands of Javascript libraries, lol).
So, my college notes say:
"Presentation layer: this is the UI.
In this layer you ask the middle tier for information, which gets that information from a database, which then goes back to middle tier, back to presentation. In the case of the headset, the operators can confirm an order is ready. This is essentially the presentation tier again: you're getting information from the middle tier and 'presenting it' as it were.
The presentation layer is in essence the question: how do I bring my application data to my end users in a platform-and solution-independent way?"
What's JSON? A way to transport data between the middle tier and the presentation tier. Is that what frontend development is? Displaying it in a pretty way? I don't think it is, because 'pretty' is an extra feature of obtaining and displaying data. Do we always have to display data in a pretty way? Not necessarily. We could write a front-end script (in NodeJS perhaps) that periodically fetches certain information from a middle-tier is serves a more functional role rather than a rendering one.
The prettification of data was a historical consequence of the popularity of the web (which is a front-end) (see second paragraph with link). Since the essence of a front-end is to obtain information from the back-end (with stress on obtaining), its presentation is not necessarily a defining characteristic of it, but rather an optional and solution-dependent aspect, a facet.4 -
I'm on a remote contract (has no centraloffice at our company) and was hired to work remote.
New PM wants to reenergize culture. Everyone has to come in and no more flexible hours. Lack of space means no more dual monitors. Lack of desks means we push desks together to form a "conference table." More people working means slower internet. Three people have separate meetings? Someone can stay, someone can sit in reception, and someone is in the hall.
But hey... we can see each other now and we're all available to one another.2 -
Aka... How NOT to design a build system.
I must say that the winning award in that category goes without any question to SBT.
SBT is like trying to use a claymore mine to put some nails in a wall. It most likely will work somehow, but the collateral damage is extensive.
If you ask what build tool would possibly do this... It was probably SBT. Rant applies in general, but my arch nemesis is definitely SBT.
Let's start with the simplest thing: The data format you use to store.
Well. Data format. So use sth that can represent data or settings. Do *not* use a programming language, as this can neither be parsed / modified without an foreign interface or using the programming language itself...
Which is painful as fuck for automatisation, scripting and thus CI/CD.
Most important regarding the data format - keep it simple and stupid, yet precise and clean. Do not try to e.g. implement complex types - pain without gain. Plain old objects / structs, arrays, primitive types, simple as that.
No (severely) nested types, no lazy evaluation, just keep it as simple as possible. Build tools are complex enough, no need to feed the nightmare.
Data formats *must* have btw a proper encoding, looking at you Mr. XML. It should be standardized, so no crazy mfucking shit eating dev gets the idea to use whatever encoding they like.
Workflows. You know, things like
- update dependency
- compile stuff
- test run
- ...
Keep. Them. Simple.
Especially regarding settings and multiprojects.
http://lihaoyi.com/post/...
If you want to know how to absolutely never ever do it.
Again - keep. it. simple.
Make stuff configurable, allow the CLI tool used for building to pass this configuration in / allow setting of env variables. As simple as that.
Allow project settings - e.g. like repositories - to be set globally vs project wide.
Not simple are those tools who have...
- more knobs than documentation
- more layers than a wedding cake
- inheritance / merging of settings :(
- CLI and ENV have different names.
- CLI and ENV use different quoting
...
Which brings me to the CLI.
If your build tool has no CLI, it sucks. It just sucks. No discussion. It sucks, hmkay?
If your build tool has a CLI, but...
- it uses undocumented exit codes
- requires absurd or non-quoting (e.g. cannot parse quoted string)
- has unconfigurable logging
- output doesn't allow parsing
- CLI cannot be used for automatisation
It sucks, too... Again, no discussion.
Last point: Plugins and versioning.
I love plugins. And versioning.
Plugins can be a good choice to extend stuff, to scratch some specific itches.
Plugins are NOT an excuse to say: hey, we don't integrate any features or offer plugins by ourselves, go implement your own plugins for that.
That's just absurd.
(precondition: feature makes sense, like e.g. listing dependencies, checking for updates, etc - stuff that most likely anyone wants)
Versioning. Well. Here goes number one award to Node with it's broken concept of just installing multiple versions for the fuck of it.
Another award goes to tools without a locking file.
Another award goes to tools who do not support version ranges.
Yet another award goes to tools who do not support private repositories / mirrors via global configuration - makes fun bombing public mirrors to check for new versions available and getting rate limited to death.
In case someone has read so far and wonders why this rant came to be...
I've implemented a sort of on premise bot for updating dependencies for multiple build tools.
Won't be open sourced, as it is company property - but let me tell ya... Pain and pain are two different things. That was beyond pain.
That was getting your skin peeled off while being set on fire pain.
-.-5 -
random thought:
Life= tension. But you are only winning it if you don't let these tensions divert you from "what you gotta do"
Life will always find a way to give you some kind of problem. if you are not tensed about something, you ain't living a life.
But we often start ignoring our favourite habits/ aspirations/ goals in order to tackle these problems .
For eg I had made a habit of meditating for 15 mins followed by 45 mins of workout b/w 7-8pm last year. but since January 12th there hasn't been a day when i could achieve this habit simply because life kept throwing random tensions at me.
1. first my hand got fractured and i had to leave gym for a month. "no worries", i thought. "meditation doesn't require a working hand and i could do start walking as an excercise from next month"
2. then my office got wfo 3/5 days. i will get back to home after 7.30 and on wfh days my work won't finish till 8 on home days. "no worries" , i thought
"i will shift it to 9-10 every night next month"
3. then next month office got wfo 4/5 days annd family started steps for buying a new home. all time affter pfffice went into those steps/discussions "no worries ", i thought. "it wiill be over by the next month and i will be free"
this next month hasn't fome yet :/3