So we were trying to make a Go AI bot for our school project and then let it play against itself. Well... it might be able to beat AlphaGo, right?

New models of LLM have realized they can cut bit rates and still gain relative efficiency by increasing size. They figured out its actually worth it.

However, and theres a caveat, under 4bit quantization and it loses a *lot* of quality (high perplexity). Essentially, without new quantization techniques, they're out of runway. The only direction they can go from here is better Lora implementations/architecture, better base models, and larger models themselves.

I do see one improvement though.
By taking the same underlying model, and reducing it to 3, 2, or even 1 bit, assuming the distribution is bit-agnotic (even if the output isn't), the smaller network acts as an inverted-supervisor.

In otherwords the larger model is likely to be *more precise and accurate* than a bitsize-handicapped one of equivalent parameter count. Sufficient sampling would, in otherwords, allow the 4-bit quantization model to train against a lower bit quantization of itself, on the theory that its hard to generate a correct (low perpelixyt, low loss) answer or sample, but *easy* to generate one thats wrong.
And if you have a model of higher accuracy, and a version that has a much lower accuracy relative to the baseline, you should be able to effectively bootstrap the better model.

This is similar to the approach of alphago playing against itself, or how certain drones autohover, where they calculate the wrong flight path first (looking for high loss) because its simpler, and then calculating relative to that to get the "wrong" answer.

If crashing is flying with style, failing at crashing is *flying* with style.

AI so far....

2012: We can do more than 5 layers whoa
2013: It works on text too!
2014: Let’s build infras with frameworks & cloud compute
2015: AlphaGo! Singularity!
2016: Wait it’s racist & sexist
2017: Deepfakes scary
2018: No idea how it works
2019: Whatevs time to productize $$$
2020: ??

If you say u can write a bug free code, thats a bigger news than AlphaGo.
#alphaGo
#awesome_documentary 👏👏

I start playing the documentary on AlphaGo while my bro was eating... He walked away after finishing...

He's a CS senior specializing in ML.... I thought he'd be more interested....

Create an AI to defeat all the best players in different areas like what AlphaGo did with the best Go players.